Friday, July 26, 2013

Unix - merge multiple consecutive lines


Input file:
$ cat infile.txt 
aid=33
pw=3
nn=90
aid=32
pw=30
nn=70
aid=56
pw=3
nn=93
Required:
Combine or merge every three consecutive lines of the above file so that the output becomes:
aid=33,pw=3,nn=90
aid=32,pw=30,nn=70
aid=56,pw=3,nn=93
Awk solution: If line number is divisible by 3 then put a new line(\n) else put a comma(,) i.e.
$ awk '{printf("%s%s", $0, (NR%3 ? "," : "\n"))}' infile.txt 
aid=33,pw=3,nn=90
aid=32,pw=30,nn=70
aid=56,pw=3,nn=93
Another way using Awk:
$ awk 'NR%3{printf $0",";next;}1' infile.txt 
aid=33,pw=3,nn=90
aid=32,pw=30,nn=70
aid=56,pw=3,nn=93
Using UNIX paste command:
$ paste -d"," - - - < infile.txt 
aid=33,pw=3,nn=90
aid=32,pw=30,nn=70
aid=56,pw=3,nn=93
A bash command line solution:
$ while read line1; do read line2; read line3; echo "$line1,$line2,$line3"; done < infile.txt 
aid=33,pw=3,nn=90
aid=32,pw=30,nn=70
aid=56,pw=3,nn=93
Related posts:
  1. Join multiple lines using Awk
  2. Combine related consecutive lines using Awk
  3. Merging lines in UNIX

Wednesday, May 8, 2013

Unix - Append 0 to single digit date

Input file file.txt has dates in month/day/year format.
$ cat file.txt 
3/4/2013
3/10/2013
10/4/2013
12/10/2012
Required: Add prefix 0 to first and second field if its a single digit.

Awk solution:
$ awk 'BEGIN {FS=OFS="/"} 
    { 
 if (length($1) == 1) $1="0"$1
 if (length($2) == 1) $2="0"$2
        { print }
}' file.txt
Output:
03/04/2013
03/10/2013
10/04/2013
12/10/2012
Related posts:
- A newbie tutorial on Unix Awk
- Awk if else
- Convert date format in unix using awk and sed

Thursday, February 28, 2013

UNIX time command output redirect to file

As you know the 'time' command run programs and summarize system resource usage. Here is a way how you can redirect 'time' command output to file. I would recomend you to go through this page to understand more about 'time' command.

e.g. We are executing 'prog.sh' along with 'time' command.
$ time ./prog.sh 
Initiating merge
Merge completed

real	0m8.803s
user	0m0.010s
sys	0m0.000s
#Trying to redirect the output to a file
$ time ./prog.sh > out.txt

real	0m8.804s
user	0m0.020s
sys	0m0.000s
#out.txt content:
$ cat out.txt 
Initiating merge
Merge completed
#So it only redirected the STDOUT of the script executed, but the 'time' command outut is not redirected.
#This is becuase the command time sends it's output to STDERR (instead of STDOUT)
#To capture output of 'time' command:
$ { time ./prog.sh ; } 2> out.txt
Initiating merge
Merge completed
#Now out.txt content:
$ cat out.txt 

real	0m8.303s
user	0m0.010s
sys	0m0.000s
#And to capture output of script as well as time command:
$ { time ./prog.sh ; } &> out.txt
#'out.txt' now has both the outputs.
$ cat out.txt 
Initiating merge
Merge completed

real	0m8.303s
user	0m0.020s
sys	0m0.000s
As I have mentioned in my earlier post on UNIX 'time' command, there's two types of time command available:
1) Shell's in-build time: Gives only scheduler information
2) /usr/bin/time: Gives more information, also allows formatting the output

The second (/usr/bin/time) one accepts output redirection without code block:
$ /usr/bin/time ./prog.sh &> newout.txt

$ cat newout.txt 
Initiating merge
Merge completed
0.00user 0.01system 0:08.30elapsed 0%CPU (0avgtext+0avgdata 5728maxresident)k
0inputs+8outputs (0major+719minor)pagefaults 0swaps
Related posts:
- UNIX redirect man pages to file
- Redirect UNIX top command output to file
- UNIX redirect both stderr and stdout to file
- Run UNIX bash loop with nohup command

© Jadu Saikia www.UNIXCL.com