[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [pygame] Frantic memory usage



And once again without the line wrapping...

How I lost Hundreds of MB in Two Weeks

0. Introduction

This article summarizes steps I took (mostly per the advice of the
Pygame mailing list) to shrink the memory footprint of my game, Frantic.
 Frantic is a 2D space shooter in the spirit of Odyssey II title, UFO
(which is in the spirit of Asteroids).  It is written in Python and uses
the Pygame library for graphics, sound, music, and input handling.  I
recently released the first nine levels to the Pygame mailing list as a
demo and was mortified when the initial reviews came back.  

"Takes forever to load!"

"Eats more memory than Quake III!"

"How can such a small game eat so much memory?"

How *could* such a small game eat so much memory?  I thought I was doing
things correctly and my image sizes weren't onerous.  At least
compressed on the hard disk they weren't.  And therein lies my first
problem - I had no idea how much memory my images were consuming at
runtime.  Let's begin there.

1.  Auditing your image memory consumption

Determining how much memory your images are consuming is easy, assuming
you're loading them into pygame.Surface objects:

mem = surface.get_height() * surface.get_width() *
surface.get_bytesize()

That's all there is to it.  You may want to divide the result by 1024 to
get output in kilobytes.  

My explosion animations were 70 frames, 200x200 pixels each.  Since I
specified no pixel depth when calling convert() on the loaded image, it
defaulted to four bytes per pixel (or 32 bits per pixel).  So, doing the
math, you get the following:

70 * 200 * 200 * 4 = 11,200,000 bytes = 10.7 Mb of memory

That's 10.7 Mb for *one* explosion animation which takes up only a
handful of Kb on my hard disk.  To exacerbate the situation, Frantic has
custom explosions for each enemy, each eating 10.7 Mb of memory.  More
on that later, but suffice it to say that this little revelation made me
rethink my explosion design.

2. Caching loaded images

I actually had a caching mechanism implemented in Frantic, but it was
rendered obsolete when I made a change to the way I loaded my images. 
The cache was a simple dict that used the image name as a key and the
loaded surface as the value.  If an already loaded image was requested,
the cached image was returned.

However, about midway through the development of Frantic I condensed
each prerendered animation into a "filmstrip" -- a single image that
holds all of the animation frames.  The image loading function now loads
the filmstrip and returns the parsed out animation frames in an array.  
The oversight on my part was not updating the caching mechanism to cache
the individual frames.  It still cached the filmstrip, but that
filmstrip could be (and was) parsed multiple times if the image was
loaded more than once.   

The solution was easy.  The array of individual frames is now cached and
the filmstrip is released from memory.   If the same image name is
requested, the cached frames are returned.  

3. Load only what you need each level

Another big problem with Frantic was that everything was loaded at
startup time.   When frantic started out as a small, simple game this
method worked quite well.   However, as the number of levels, enemies,
and images grew, it really bogged down processing at startup as well as
memory usage in general.   

The solution was to load only the enemies, powerups, etc. that were
needed on each level.  Correspondingly, they were unloaded when they
were no longer needed.  At startup time, Frantic builds a table of
objects to load and another of objects to unload.  The keys to these
tables are the level numbers on which the loading and unloading is to
occur.   Since no ememy in Frantic hangs around longer than three levels
(the length of a mission), there is an upper bound to the amount of
memory that will be consumed at any time in the game.

4.  Quality vs. conservation

As I mentioned earlier, my explosion animations consisted of 70 frames
of 200x200 images at 4 bytes per pixel.  They looked really nice.  I got
several compliments on them.  

But they were memory eating pigs.  To make matters worse, each enemy had
*two* types of explosion animations - one for when they were destroyed
by the user controlled starship, and another more spectacular version
for chain reactions.  So, each enemy was lugging a piggy 21.4 Mb of
explosion images around.  OINK!

This had to change.   For starters, I eliminated the chain reaction
explosion images and cut memory usage in half.   The effect was barely
noticeable during gameplay.   Next, I specified 16 bits per pixel in my
call to convert() when loading the image, again halving the memory
consumed.   Finally, I massaged the particle systems used to generate
the explosions and trimmed them down to 24 frames each, meaning the
memory consumed by each was now:

24 * 200 * 200 * 2 = 1920000 bytes = 1.88 Mb

And since the chain reaction explosions were eliminated, that's all the
explosion image memory each enemy consumes now, down from 21.4 Mb.  

This accounted for huge memory savings in Frantic.  On the last level of
each mission, there are four enemies and the starship loaded in memory. 
In the old model, each had two hoggy explosions loaded, meaning 214 Mb
of memory was consumed just for explosions!  Now the same five objects
consume just 9.4 Mb for explosions.  And you know what?  They really
don't look that much different.

5. Simple animations can be performed at runtime

There are a couple instances in Frantic where messages are displayed to
the user in the middle of the screen.  Each message graphic is 400x100
pixels, and for effect they are faded in and out.  This fading animation
was prerendered like all other animations.  

By now you can probably see where this is going.  I can't recall how
many frames were required in the various message animations, but it was
far more than is necessary:  one.  By using the
pygame.Surface.set_alpha() function, each message now consists of one
frame instead of dozens.  The alpha value is increased and decreased
accordingly to provide the fade effect, and memory consumed by these
trivial animations was reduced by orders of magnitude.

6. Summary

In summary, if you want to keep your Pygame applications fit and trim,
follow these guidelines:

- Keep track of how much memory you're consuming

- Cache all loaded images

- Load images just before they're needed, unload them just after they're
not

- Consider tradeoffs in image quality and memory consumption

- Perform simple animations at runtime



On Wed, 19 Jul 2006 14:56:07 -0400, "David Mikesell"
<dave_mikesell@xxxxxxxxxxx> said:
> Here's a first draft.   Comments and suggestions welcome.
> 
> How I Lost Hundreds of MB in Two Weeks
> 
> 0. Introduction
> 
> This article summarizes steps I took (mostly per the advice of the
> Pygame
> mailing list) to shrink the memory footprint of my game, Frantic. 
> Frantic is
> a 2D space shooter in the spirit of Odyssey II title, UFO (which is in
> the 
> spirit of Asteroids).  It is written in Python and uses the Pygame
> library for 
> graphics, sound, music, and input handling.  I recently released the
> first nine
> levels to the Pygame mailing list as a demo and was mortified when the
> initial
> reviews came back.  
> 
> "Takes forever to load!"
> 
> "Eats more memory than Quake III!"
> 
> "How can such a small game eat so much memory?"
> 
> How *could* such a small game eat so much memory?  I thought I was doing
> things
> correctly and my image sizes weren't onerous.  At least compressed on
> the hard
> disk they weren't.  And therein lies my first problem - I had no idea
> how much
> memory my images were consuming at runtime.  Let's begin there.
> 
> 1.  Auditing your image memory consumption
> 
> Determining how much memory your images are consuming is easy, assuming
> you're
> loading them into pygame.Surface objects:
> 
> mem = surface.get_height() * surface.get_width() *
> surface.get_bytesize()
> 
> That's all there is to it.  You may want to divide the result by 1024 to
> get 
> output in kilobytes.  
> 
> My explosion animations were 70 frames, 200x200 pixels each.  Since I
> specified
> no pixel depth when calling convert() on the loaded image, it defaulted
> to
> four bytes per pixel (or 32 bits per pixel).  So, doing the math, you
> get the
> following:
> 
> 70 * 200 * 200 * 4 = 11,200,000 bytes = 10.7 Mb of memory
> 
> That's 10.7 Mb for *one* explosion animation which takes up only a
> handful of 
> Kb on my hard disk.  To exacerbate the situation, Frantic has custom
> explosions
> for each enemy, each eating 10.7 Mb of memory.  More on that later, but
> suffice
> it to say that this little revelation made me rethink my explosion
> design.
> 
> 2. Caching loaded images
> 
> I actually had a caching mechanism implemented in Frantic, but it was
> rendered
> obsolete when I made a change to the way I loaded my images.  The cache
> was a
> simple dict that used the image name as a key and the loaded surface as
> the 
> value.  If an already loaded image was requested, the cached image was
> returned.
> 
> However, about midway through the development of Frantic I condensed
> each 
> prerendered animation into a "filmstrip" -- a single image that holds
> all of 
> the animation frames.  The image loading function now loads the
> filmstrip and
> returns the parsed out animation frames in an array.   The oversight on
> my part
> was not updating the caching mechanism to cache the individual frames. 
> It
> still cached the filmstrip, but that filmstrip could be (and was) parsed 
> multiple times if the image was loaded more than once.   
> 
> The solution was easy.  The array of individual frames is now cached and
> the
> filmstrip is released from memory.   If the same image name is
> requested, the
> cached frames are returned.  
> 
> 3. Load only what you need each level
> 
> Another big problem with Frantic was that everything was loaded at
> startup
> time.   When frantic started out as a small, simple game this method
> worked 
> quite well.   However, as the number of levels, enemies, and images
> grew, it 
> really bogged down processing at startup as well as memory usage in
> general.   
> 
> The solution was to load only the enemies, powerups, etc. that were
> needed on 
> each level.  Correspondingly, they were unloaded when they were no
> longer 
> needed.  At startup time, Frantic builds a table of objects to load and
> another 
> of objects to unload.  The keys to these tables are the level numbers on
> which
> the loading and unloading is to occur.   Since no ememy in Frantic hangs
> around
> longer than three levels (the length of a mission), there is an upper
> bound to
> the amount of memory that will be consumed at any time in the game.
> 
> 4.  Quality vs. conservation
> 
> As I mentioned earlier, my explosion animations consisted of 70 frames
> of
> 200x200 images at 4 bytes per pixel.  They looked really nice.  I got
> several
> compliments on them.  
> 
> But they were memory eating pigs.  To make matters worse, each enemy had
> *two* 
> types of explosion animations - one for when they were destroyed by the
> user
> controlled starship, and another more spectacular version for chain
> reactions.
> So, each enemy was lugging a piggy 21.4 Mb of explosion images around. 
> OINK!
> 
> This had to change.   For starters, I eliminated the chain reaction
> explosion
> images and cut memory usage in half.   The effect was barely noticeable
> during
> gameplay.   Next, I specified 16 bits per pixel in my call to convert()
> when
> loading the image, again halving the memory consumed.   Finally, I
> massaged the
> particle systems used to generate the explosions and trimmed them down
> to 24
> frames each, meaning the memory consumed by each was now:
> 
> 24 * 200 * 200 * 2 = 1920000 bytes = 1.88 Mb
> 
> And since the chain reaction explosions were eliminated, that's all the 
> explosion image memory each enemy consumes now, down from 21.4 Mb.  
> 
> This accounted for huge memory savings in Frantic.  On the last level of
> each 
> mission, there are four enemies and the starship loaded in memory.  In
> the old 
> model, each had two hoggy explosions loaded, meaning 214 Mb of memory
> was 
> consumed just for explosions!  Now the same five objects consume just
> 9.4 Mb 
> for explosions.  And you know what?  They really don't look that much
> different.
> 
> 5. Simple animations can be performed at runtime
> 
> There are a couple instances in Frantic where messages are displayed to
> the
> user in the middle of the screen.  Each message graphic is 400x100
> pixels, and
> for effect they are faded in and out.  This fading animation was
> prerendered
> like all other animations.  
> 
> By now you can probably see where this is going.  I can't recall how
> many
> frames were required in the various message animations, but it was far
> more 
> than is necessary:  one.  By using the pygame.Surface.set_alpha()
> function, 
> each message now consists of one frame instead of dozens.  The alpha
> value is
> increased and decreased accordingly to provide the fade effect, and
> memory
> consumed by these trivial animations was reduced by orders of magnitude.
> 
> 6. Summary
> 
> In summary, if you want to keep your Pygame applications fit and trim,
> follow
> these guidelines:
> 
> - Keep track of how much memory you're consuming
> 
> - Cache all loaded images
> 
> - Load images just before they're needed, unload them just after they're
> not
> 
> - Consider tradeoffs in image quality and memory consumption
> 
> - Perform simple animations at runtime
> 
> 
> On Wed, 19 Jul 2006 12:56:44 +1000, "René Dudfield" <renesd@xxxxxxxxx>
> said:
> > It could go in the wiki somewhere.  Or we could upload some html.
> > 
> > On 7/19/06, Brian Fisher <brian@xxxxxxxxxxxxxxxxxxx> wrote:
> > > On 7/18/06, R. Alan Monroe <amonroe@xxxxxxxxxxxxxxx> wrote:
> > > > >> weeks ago.  RAM footprint ranges from 56 - 64 Mb depending on the number
> > > >
> > > > > Got that down to 43 - 52 Mb now.   Amazing how much I was wasting...
> > > >
> > > > The saga of how you fixed it all might make a good article for
> > > > gamedev.net or the like.
> > > >
> > > I would imagine it would have the most relevance to a pygame user so
> > > I'd be interested to find it on some page pygame-y... Is there some
> > > place on the pygame site where such a tale would be good to place?
> > >