I also found it useful to do some one-off batch processing of a big file without thinking of chunking and doing too complicated things. Imagine something like a 60GB file in a custom format that need to be deduplicated. It's easier to just load it all up in memory and do run the algorithm 2-3x slower using a 50GB swapfile than spend hours to complicate the algorithm.