Saturday, October 31, 2009

Extract range of lines using sed awk bash


Below are few different ways to print or extract a section of a file based on line numbers.

Lets try to extract lines between line number 27 and line number 99 of input file 'file.txt'

Using sed editor:

$ sed -n '27,99 p' file.txt > /tmp/file1

Which is same as:

$ sed '27,99 !d' file.txt > /tmp/file2

Awk alternative : you can make use of awk NR variable

$ awk 'NR >= 27 && NR <= 99' file.txt > /tmp/file3

Using Linux/UNIX 'head' and 'tail' command:

$ head -99 file.txt | tail -73 > /tmp/file4

Which is basically:

$ head -99 file.txt | tail -$(((99-27)+1)) > /tmp/file5

In vi editor, we can use the following command in ex mode (open the main file 'file.txt' in vi):

:27,99 w! /tmp/file6

i.e. Write lines between line number 27 and line number 99 of main file 'file.txt' to file '/tmp/file6'

Perl alternative would be:

$ perl -ne 'print if 27..99' file.txt > /tmp/file7

And the solution using python:

$ python
Python 2.5.2 (r252:60911, Jul 22 2009, 15:35:03)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2

>>> fp = open("/tmp/file8","w")
>>> for i,line in enumerate(open("file.txt")):
... if i >= 26 and i < 99 :
... fp.write(line)
...
>>>

So the contents of all the output files produced (i.e /tmp/file[1-8]) will be the same (i.e. line number 27 to line number 99 of 'file.txt')

5 comments:

ps axu -o user | grep -i root said...

python can also do one-liner for this one:

python -c "fp=open('/tmp/file8', 'w'); fp.write(''.join(open('file.txt', 'r').readlines()[25:99]))"

or if you want to use sheel redirection:

python -c "print(''.join(open('file.txt', 'r').readlines()[25:99]))" > /tmp/file8

Unknown said...

@ps axu : thanks for your comment. Its useful.

Seifer said...

Hi, first of all, thanks a lot, I'm finding so many things I didn't know how to do.
Just one question on this regard.
In the example:

sed -n '27,99 p' file > anotherfile

what would be the sintaxis to use variables instead of numbers?
I've tried a lot of ways, but nothing seems to work.
Thanks again, and good luck.

Unknown said...

@Seifer, thanks for your question. To use variable use double quotes instead of single quotes, something like this:

m=27; n=99; sed -n "$m,$n p" file

A related post here:

http://unstableme.blogspot.com/2008/03/accessing-external-variable-in-awk-and.html

Keep in touch. Thanks

Unknown said...

I am trying to recursively extract line 14 using sed from all conf files in a directory but I am not sure what the correct command is.

I've tried this but it only prints line 14 from the first file:
sed -n 14p *.conf

© Jadu Saikia www.UNIXCL.com