Throw Not Away, That Which Can Be Recycled

I’m in the process of optimizing some old prototype line rendering code. This code isn’t broken, but on every frame, it creates many line geometry objects, renders them once, and throws them away, never to be used again. Then it creates more line objects on the next time–repeat.

Allocating memory all of the time like this is generally a slow, unwanted cycle, and why throw away memory only to use more memory to do roughly the same thing as before?

Here are a few descriptions of why this may be slow:

Imagine that if every time you wrote something on your whiteboard, you ran to the nearest store to buy another whiteboard for the next time you needed to write something. Alternatively, you could just erase or overwrite the notes you no longer needed, saving you the trips.

Another analogy:

You’re playing fetch with your trusted canine companion, but the dog is a figment of your imagination. Whenever you throw another tennis ball into the field it’s long-gone somewhere in the bushes. You run out of tennis balls, so you spend the rest of the afternoon looking for them, and then continue the game. Or you actually have a dog (Springer Spaniel?), and he or she fetches the old balls back for you to use again. In the meantime, you throw one of your remaining tennis balls. No waiting.

Creating something only to throw away doesn’t make sense if you can reuse your rsources.

The idea is to create as many objects (in particular, entities with create/destroy usage patterns) as you think you will need at the beginning of your program–one big allocation. Then your program should request these objects as-needed and set their properties. In this case, the lines have points. You should then mark those objects “free” if you don’t need them anymore. Then the next time you need a new line, the system will just give you an already-existing line from the pool, which you can initialize with different points. This way you avoid allocating at run-time unless you need more entities than you specified at start, in which case you’ll see another large allocation. Either that, or you know exactly the maximum number of entities you’ll ever need. Then you may just want to reserve that number of memory slots to avoid allocation in the middle of your render loop.

In short, recycle memory when you can and when it makes sense to do so. For more information, look into “memory pools,” “memory arenas,” and other memory allocation strategies.