in_batches_of
Rails is nice. Rails is REAL nice, but, you would expect me to say that, I am a rails developer.
One thing that rails is not nice about is efficiency.
It’s elegant, its compact, but, the simplest things can have unforseen consequences unless you understand at a fundamental level, whats going on.
The persistence layer of Rails is a class called ActiveRecord (Specifically, most of it is tied up in ActiveRecord::Base)
When you want to do something with every row in a table, or with a subset of rows, you want to do something like this…
ModelClass.find(:all).each do |item| ... do stuff ... end
Take it from me, that would be bad.
What it would do is instantiate every row in the table as an object and build an array of those objects in memory. Thats fine if you have 10, 100, 1000 rows in that table, but what happens when you have 100000 or a million? Well, we would be in for a long wait.