Remove x number of lines from beginning of file

Jon Gale donjuanica at gmail.com
Thu Oct 25 16:24:18 MDT 2007


PLUGers,

What would be the most efficient way to remove x number of lines from a very
large (many gigabyte) file?

Preliminary tests show that sed is not a good option:

[root at linuxboy root]# time sed -i -e '1,4d' test2.txt

real    0m0.016s
user    0m0.000s
sys     0m0.010s
[root at linuxboy root]# time sed -i -e '1,4d' test.txt

real    1m13.596s
user    0m18.020s
sys     0m52.930s

test2.txt is only 100 lines.  test.txt is 10,000,000 lines.

Is there a better way to do this that won't necessitate scanning the entire
file just to remove the first x number of lines?

Thanks,

Jon

-- 
What's this?  Another BLOG?  http://www.jtoo.net/



More information about the PLUG mailing list