Remove x number of lines from beginning of file

Jeff Anderson jefferya at
Thu Oct 25 16:40:24 MDT 2007

The only way that I would know of to accomplish something like that
would be to move/change the file header on disk to reflect the new state
of the file. It would mark the bits representing the first 10 lines as
free space.

I am not aware of a utility that does this, but if you were to do this
operation several times a second, the time involved in writing, testing,
and debugging a low level utility might be worth it.

Jeff Anderson

Jon Gale wrote:
> PLUGers,
> What would be the most efficient way to remove x number of lines from a very
> large (many gigabyte) file?
> Preliminary tests show that sed is not a good option:
> [root at linuxboy root]# time sed -i -e '1,4d' test2.txt
> real    0m0.016s
> user    0m0.000s
> sys     0m0.010s
> [root at linuxboy root]# time sed -i -e '1,4d' test.txt
> real    1m13.596s
> user    0m18.020s
> sys     0m52.930s
> test2.txt is only 100 lines.  test.txt is 10,000,000 lines.
> Is there a better way to do this that won't necessitate scanning the entire
> file just to remove the first x number of lines?
> Thanks,
> Jon

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : 

More information about the PLUG mailing list