Saturday, May 16, 2009

Conditional append of field using awk


Graphically my requirement was like this :


I had to append a field before each line based on the 1st field (ID field), i.e. for the first line for a specific unique ID, the field with text "Agg line" should be appended and for the successive lines for that ID, field with text "sub-line" has to be appended.

So my input file was in a csv format like this:


$ cat details.txt
ID5,17.95,107.0,Y
ID5,6.56,12.3,Y
ID5,7.36,22.5,Y
ID5,4.03,72.2,Y
ID6,282.8,134.1,Y
ID6,111.56,61.7,Y
ID6,171.24,72.4,Y
ID7,125.6,89,Y

Output required:

Agg line,ID5,17.95,107.0,Y
sub-line,ID5,6.56,12.3,Y
sub-line,ID5,7.36,22.5,Y
sub-line,ID5,4.03,72.2,Y
Agg line,ID6,282.8,134.1,Y
sub-line,ID6,111.56,61.7,Y
sub-line,ID6,171.24,72.4,Y
Agg line,ID7,125.6,89,Y

Awk solution:

$ awk '
BEGIN {FS=OFS=","}
!_[$1]++{print "Agg line",$0;next}
{print "sub-line",$0}
' details.txt

Related posts:

- Replace field other than the first occurrence using awk

3 comments:

Nathan said...

first of all , Jadu , thanks for your kindly comments.
and again , as usual ((-: , my non awk way :

#!/bin/bash

declare -i FIELD_ID=-1
declare CUR_FID
declare NEW_LINE

function getFieldID() {

local F_ID=$(head -n1 <<<"$1" )
F_ID=${F_ID%%,*}
F_ID=$(sed -nr 's/[A-Z]//g;p' <<<"$F_ID")

echo "$F_ID"
}

while read LINE
do

CUR_FID=$(getFieldID "$LINE")

if [[ $CUR_FID -ne $FIELD_ID ]]
then

echo "Agg line,${LINE}"
FIELD_ID=$CUR_FID

else

echo "sub-line,${LINE}"

fi


done < details.txt

Jadu Saikia said...

@Nathan, really amazing. You use the parameter substitutions so well in bash. Also the arrays.
Thanks for this useful script; keep posting; you are always welcome.

Jadu Saikia said...

The solution using python is here http://pythonstarter.blogspot.com/2009/05/python-append-field-based-on-condition.html

© Jadu Saikia www.UNIXCL.com