Can you explain how you used dd here? Ive never seen it used this way, curious
back
2 comments
Sure! I've created a gist so you can see for yourself but the basic idea is as described. Read a chunk, find the first date in it and then decide if you want to read further forward or back in the file.
https://gist.github.com/aidos/5a6a3fa887f41f156b282d72e1b79f...
For anyone else, here's the awk for combining lines in the log files for making them greppable too: https://gist.github.com/aidos/44a9dfce3c16626e9e7834a83aed91...
dd lets you specify an offset to start reading the file at, with `skip`. This would let you perform a binary search by picking an offset in the file, reading a small chunk (say, a kilobyte), and scanning for the date/time string within it. Each read should be O(1) in terms of the size of the file, so a O(log(n)) for the binary search, whereas a grep-based approach is O(n).
(The datetime in the log message is presumably sorted, or nearly so).