From RootdevWiki
Contents |
Using Sed
Sed is an extremely powerful tool in the linux arsenal, but its somewhat arcane syntax can be daunting to new users. It is well worth studying some examples to get a feel for how it works. The "official recipes" can be seen here:
Below are some examples from my own experience
Command Line Usage
Using sed with Find
Find all directories in the current directory and use sed to strip off the trailing slash
find ./ -maxdepth 1 -mindepth 1 -type d| sed 's/.\///'
ps memory monitor
Get the %memory usage and command instance from ps: In this instance I want to compare the memory usage of Tomcat1 and Tomcat4
ps wax -o %mem,command | grep [t]omcat[14] | sed \ 's/\/.*\(tomcat[14]\).*/\1/'
Pretty this up a bit and make sure we're only looking at the tomcat and not at anyone editing config files or tailing the logs:
ps wax -o %mem,command |grep java |grep [t]omcat[14] \
|sed 's/\/.*tomcat\([14]\).*/TC\1/'| \
awk '{print $2":",$1}'
Change filenames in a directory
For wrongly titled filenames, you can correct them in place with the following one liner:
for x in `ls 10.32.2.10_gi1_0_101*`; \ do mv $x `echo $x | sed 's/101/124/'`; done
In this example our filenames are all titled with _101* in the filename, it should be _124*
NB: The -i switch won't work here as you're using the sed operation as the argument to determine the $y variable. Obviously you can use cp instead of mv depending on your level of paranoia.
Alternatively
Use rename:
rename 's/0_101/0_124/' 10.32.2.10_gi1_0_101*
Remove whitespace from a single filename
This method only works for a single filename so the first will barf if the find returns multiple results
x=`find *.mp3 -print0|xargs -0 -I{} ls {}`; \
y=$(echo $x |sed 's/\s/_/g'); mv "$x" $y
or more specific:
x=`ls filename\ with\ spaces\ in\ 1.txt`; \ y=$(echo $x |sed 's/\s/_/g'); echo $y; mv "$x" $y
Remove whitespace from multiple filenames
for i in *; do mv "$i" `echo $i| sed 's/ /_/g'`; done
Or, just for the hell of it - using tr:
for x in *.txt; do mv "$x" `echo $x|tr " " "_"`; done
Remove Version Numbers from a list of installed RPMs
Assuming you have saved your list of RPMs in some manner equivalent to:
rpm -qa | sort > installed
This is a slightly inefficient way of doing it as you have to run it twice:
sed -i.bak 's/\(.*\)-[0-9].*/\1/g' installed
Should be tidied up but it does the trick
Alternatively, don't do that at all, do this:
rpm -qa --qf %{NAME}\\n
Remove Messages from Mailqueue
Based on a set of arbitrary search criteria, in this case: to remove all nagios alerts that are not destined for me (because I happen to know that any other messages are destined for three other people):
for i in `grep -l nagios qf*`; \ do for j in `egrep -l "peter|paul|mary" $i`; \ do k=`echo $j|sed 's/^q/d/'`; rm -f $j $k; done; done
Manipulating files with sed
sed is capable of editing files in place with the -i switch. If you're not confident that your sed expression won't destroy your file in some way you can automatically back it up by adding an extension to the switch:
sed -i.bak 's/foo/bar/g' file
Insert line at Start of File
I'm sure there are other ways to do this but if you wanted to insert a line at the start of a file then try this:
sed -i '1s/\(^.*$\)/--\n\1/' foo
Insert line at End of File
This is fairly straightforward:
sed -i '$a\Text to add to end of file' foo
$ dollar denotes the last line a\ means append.
Insert Line in Fixed Length File
If you want to keep a file at a fixed number of lines whilst adding to it here's a way to remove the first line and add a line at the end at the same time:
For our example, create a 5 line file:
$ for i in `seq 1 5`; do echo $i >> foo; done $ cat foo 1 2 3 4 5
Then add the number 6 to the end and remove the first line:
$ sed -i '1d;$a\6' foo $ cat foo 2 3 4 5 6
Variable expansion in sed
Say you wanted to script the above example such that the last line to be inserted is a variable:
$NUM
How do you do this in sed when both $ and \ are sed metacharacters? If you enclose it in single quotes the shell will fail to expand $NUM:
$ sed -i '1d;\$a\$NUM' foo $ cat foo 2 3 4 5 $NUM
If you enclose it in double quotes the shell will try to expand $a and escape the $ on $NUM:
$ sed -i "1d;$a\$NUM" foo sed: -e expression #1, char 6: extra characters after command
The trick is to enclose it in double quotes and escape the sed metacharacters instead. Simple, but one that trips lots of people up.
sed -i "1d;\$a\\$NUM" foo
