Remove x number of lines from beginning of file
jefferya at programmerq.net
Thu Oct 25 16:40:24 MDT 2007
The only way that I would know of to accomplish something like that
would be to move/change the file header on disk to reflect the new state
of the file. It would mark the bits representing the first 10 lines as
I am not aware of a utility that does this, but if you were to do this
operation several times a second, the time involved in writing, testing,
and debugging a low level utility might be worth it.
Jon Gale wrote:
> What would be the most efficient way to remove x number of lines from a very
> large (many gigabyte) file?
> Preliminary tests show that sed is not a good option:
> [root at linuxboy root]# time sed -i -e '1,4d' test2.txt
> real 0m0.016s
> user 0m0.000s
> sys 0m0.010s
> [root at linuxboy root]# time sed -i -e '1,4d' test.txt
> real 1m13.596s
> user 0m18.020s
> sys 0m52.930s
> test2.txt is only 100 lines. test.txt is 10,000,000 lines.
> Is there a better way to do this that won't necessitate scanning the entire
> file just to remove the first x number of lines?
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://plug.org/pipermail/plug/attachments/20071025/7c637d9f/attachment.bin
More information about the PLUG