Tuesday, March 17, 2009

Changing temporary directory for sort - bash


I was sorting a very big file using Linux sort command and unfortunately the sort failed as there was not enough space on my /tmp directory.

$ sort -t "|" -k5 ka.log.32323112.out > ka.log.32323112.out
sort: /tmp/sort1928700448: write error: No space left on device

just to mention, sort by default uses /tmp for temporaries.

So, how you can tell sort to use some other directory for temporaries ?

From man pages of sort(1)

-T, --temporary-directory=DIR
use DIR for temporaries, not $TMPDIR or /tmp; multiple options specify multiple directories

$ sort -T /home/jadu/ -t "|" -k5 ka.log.32323112.out > ka.log.32323112.out

It worked :-)

Another way would be : export env variable TMPDIR to some directory which have sufficient space.

e.g.

$ export TMPDIR=/path/to/other/directory
$ sort -t "|" -k5 ka.log.32323112.out > ka.log.32323112.out

It worked too :-)

1 comment:

shavenwarthog said...

Your solution leaves TMPDIR remaining in the environment. I suggest:

env TMPDIR=/tmp sort file > file.out

Or, for multiple commands:

(export TMPDIR=/tmp; sort; sort)

Thanks for the blog! Us Shell Firsters have to stick together ;)

© Jadu Saikia www.UNIXCL.com