Sunday, March 16, 2008

Accessing external variable in AWK and SED


$ echo "unix scripting"
unix scripting

In SED:

This is a general substitution. I am trying to replace "unix" with "BASH", so "unix scripting" will become "BASH scripting"
$ echo "unix scripting" | sed 's/unix/BASH/'
BASH scripting

Suppose, the text "BASH" is assigned to a variable called "var", now if I try to replace "unix" with "$var" in sed single quote notation, its not going to work as SED can't expand external variable in single quotes.
$ var="BASH"; echo "unix scripting" | sed 's/unix/$var/'
$var scripting

Try the same above with double quotes, this will work.
$ var="BASH"; echo "unix scripting" | sed "s/unix/$var/"
BASH scripting

In AWK

General substitution of "unix" with "BASH", will work. "unix scripting" will become "BASH scripting"
$ echo "unix scripting" | awk '{gsub(/unix/,"BASH")}; 1'
BASH scripting

"BASH" is assigned in variable "var". So the following substitution is not going to work.
$ var="BASH"; echo "unix scripting" | awk '{gsub(/unix/,"$var")}; 1'
$var scripting

Method1: See the "'" (double quote-single quote-double quote) before and after the variable var.
$ var="BASH"; echo "unix scripting" | awk '{gsub(/unix/,"'"$var"'")}; 1'
BASH scripting

Method2: Use awk -v flag this way.
$ var="BASH"; echo "unix scripting" | awk -v v="$var" '{sub(/unix/,v)}1'
BASH scripting

13 comments:

Anonymous said...

That example for AWK was very useful! Googled a bit and found your piece of code. Thanks a lot!! It solved a harrasing problem.... :-)

Unknown said...

Thanks for your In SED explanation. I was just what I was looking for.

P Chapman said...

dude - the bit on awk was wicked helpful. thanks for the spot.

halhuntley said...

That double quote, single quote, double quote saved me a lot of time. Thanks.

Jitesh said...

thanx buddy !!! you give me solution .... :)

r-duh said...

you just helped me write a neat little bash script with your tip on awk. thanks!

r-duh said...

you just helped me write a neat little bash script with your tip on awk. thanks!

Neosapien said...

this is seriously some of the hottest awesomesauce i've found on the interwebs in years. thanks!

Unknown said...

Thanks everyone, very pleased with the comments. Thanks again, its inspiring.

linuxworld said...

This exactly what i am looking for...
Thanks a Lot...:)

Unknown said...

Hi, how do you denote tab in this double quote context?

I'm trying to convert 'a|b' to 'a b' using sed and a variable for |

$ cat file.txt
a|b

$ pipe="|"
sed "s/$pipe/\t/g" file.txt

gives me
atb

thanks!

Unknown said...

@Es pacey, thanks for the question.

Works for me on my Ubuntu 10.04.4 having GNU sed version 4.2.1
$ cat file.txt
a|b
$ pipe="|"
$ sed "s/$pipe/\t/g" file.txt
a b

Can you try this ?
$ sed 's/'"$pipe"'/\t/g' file.txt
a b

Also which OS are you in ?

Unknown said...

We have to be careful about the contents of $var before plugging it into the sed code.

For example, if $var were to contain a slash "/" then the innocuous looking substitution
sed "s/$var/something/"
would blow up.

Another thing to keep in mind is whether the $var is being utilized on the LHS or the RHS of the sed substitution. Because characters are interpreted specially on the LHS & RHS in sed will need escaping.

© Jadu Saikia www.UNIXCL.com