Remove x number of lines from beginning of file
shane at hathawaymix.org
Thu Oct 25 16:54:22 MDT 2007
Shane Hathaway wrote:
> Jon Gale wrote:
>> [root at linuxboy root]# time sed -i -e '1,4d' test.txt
>> real 1m13.596s
>> user 0m18.020s
>> sys 0m52.930s
>> test2.txt is only 100 lines. test.txt is 10,000,000 lines.
>> Is there a better way to do this that won't necessitate scanning the entire
>> file just to remove the first x number of lines?
> The majority of that time is spent in I/O, not computation. The best
> way to accomplish this would be to write the output to a different hard
> drive with similar specs, enabling you to use the combined throughput of
> both drives.
Of course that's assuming you don't want to use lower-level hacks.
Nicholas suggested a loopback device, which is interesting. A FUSE
filesystem could do a similar trick, with more flexibility but probably
More information about the PLUG