tcpslice fails on large files
Setup
I have a network trace in tcpdump format that is around 30GB in size. tcpslice was pulled from a daily source snapshot from tcpdump.org's daily snapshots, and built as listed in a related issue. The 32-bit machine is running Gentoo Linux, kernel version 2.6.10 (patched by gentoo-sources package). glibc is version 2.3.4.
Symptoms
When I try to save a slice which is greater than 2GB in size, tcpslice writes exactly 2GB to the file, then fails with the following error: File size limit exceeded
Background
C's standard I/O functions in stdio.h often do not support files larger than 2GiB by default. There are a lot of details that I've dug up while researching this issue, but the bottom line is that any applications (and libraries on which those apps depend) must specify compiler flags to be able to support files larger than 2GiB.
Resolution
Even though tcpslice had been built with the compiler flags -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE (see the related issue), libpcap had not. Therefore, I fetched libpcap-0.8.3, untar'd it to a temporary directory, did a ./configure, and added the aforementioned compiler flags to the DEFS line in the generated Makefile. I then did a make, make install (as root), and did the same in the tcpslice directory.
One tool which helped me resolve this issue is strace. I used the following command:
strace tcpslice start_time end_time huge-trace-file.dump -w output-slice.dump 2| grep open\(
When I noticed that the O_LARGEFILE flag was not passed to the open() call for output-slice.dump, I knew there was a problem. After doing the recompile detailed above, I see that O_LARGEFILE is in the output.