sed sed

From Noah.org
Revision as of 12:23, 12 October 2015 by Root (Talk | contribs) (POSIX regular expression character classes)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


Pull values out of Linux kernel /proc/cmdline

This example retrieves the value of root device on the Linux kernel command-line.

sed -e 's/^.*root=\([^[:space:]]*\).*/\1/' /proc/cmdline

POSIX regular expression character classes

POSIX regex character classes are used inside regex chracter classes, so you often see them with double square brackets. For example, [[:digit:]].

[:digit:] 	digits 0 through 9.
[:xdigit:] 	hex digits 0 through f, equivalent to [0-9a-fA-F].
[:alpha:] 	alpha characters [a-zA-Z] (may include other letters depending on locale language).
[:upper:] 	alpha characters, upper case [A-Z].
[:lower:] 	alpha characters, lower case [a-z].
[:alnum:] 	alphanumeric charactera [0-9a-zA-Z] (may include other letters depending on locale language).
[:space:] 	all whitespace characters (space, tab, NL, FF, VT, CR), equivalent to [\s]. Compare to [:blank:].
[:blank:] 	space and TAB characters, equivalent to [\W]. Compare to [:space:].
[:punct:] 	punctuation  . , " ' ? ! ; : # $ % & ( ) * + - / < > = @ [ ] \ ^ _ { } | ~
[:print:] 	printable characters, including whitespace.
[:graph:] 	printable characters, excluding whitespace.
[:cntrl:] 	control characters (NL CR LF TAB VT FF NUL SOH STX EXT EOT ENQ ACK
                    SO SI DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC IS1 IS2 IS3 IS4 DEL)

This finds a line in a config file and sets a value to 1 even if that line is commented out.

sed -e 's/^[#]\?[[:blank:]]*noswap[[:blank:]]*=.*/noswap = 1/' /etc/xen-tools/xen-tools.conf

So lines line the following:

# noswap = 0
noswap = 0
noswap=1

...will be replaced by the following:

noswap = 1

sed add new-lines in inserted or appended text

If you want to add a short section to a file you can't use \n in replacement text to indicate new-lines. Some version of sed may allow this, but it is not portable. One trick is to use capturing regex expressions to capture the line-feed from a donor line. You can then refer to this capture group as many times as you need in the replacement text.

The following example adds a short section to deflate.conf to enable compression on JSON content. Notice that the search pattern captures everything not a line-feed into group 1 and then captures the new-line into group 2. In the replacement text the donor line is restored with \1\2 and then our added section begins using group 2 as needed for new-lines.

sed -e '/<IfModule mod_deflate.c>/{G;s/\([^\n]*\)\(\n\)/\1\2# custom section begin\2 AddOutputFilterByType DEFLTE application\/json\2# custom section end\2/;}' /etc/apache2/mods-available/deflate.conf

get the default gateway

This will set the variable 'GATEWAY_DEFAULT' to the IP address of the default gateway.

GATEWAY_DEFAULT=$(ip route list 0.0.0.0/0 | sed -n -e "s/^default.*[[:space:]]\([[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\).*/\1/p")

find part of a line, substitute, keep rest of line intact

sed -i -e "s/^#DatabaseDirectory \(.*\)/DatabaseDirectory \\1/" myfile.txt

Insert a line before the first line in a file

This inserts search example.com at the beginning of resolv.conf.

sed -i -e '1 i search example.com' /etc/resolv.conf

find pattern in a line, insert new line before it

This finds line that begins with exit(ignore leading spaces), then insert 'authdarmond start' before it.

sed -i -e "/^\\s*exit/i authdaemond start" /etc/rc.local

find pattern and append line

This will add QMAILQUEUE environment variable to qmail-smtpd/run. Notice how the replace pattern spans multiple lines using backslash to escape.

sed -i -e "/#\!\/bin\/sh/a\                                                     
QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl\" ; export QMAILQUEUE" /var/qmail/supervise/qmail-smtpd/run

print everything between two patterns

Use ranges for this. A range can take a line number or a pattern.

sed -n "/begin_pattern/,/end_pattern/p" myfile.txt

search and replace in multiple files

This will perform a search and replace on all files in a directory tree:

find ./ -type f -exec sed -i -e "s/find_this_text/replace_with_this_text/g" '{}' \;

sed documentation

This is one of the best sed documents I found: Grymoire Sed