Re: [pygame] Frantic memory usage

["David Mikesell" <dave_mikesell@xxxxxxxxxxx>]

The explosions are definitely a problem.   Roughly 4.6 Mb of memory for
each, and considering there are two types (regular, ricochet hit) for
each enemy and one for the starship, as much as 41.4 Mb can be hogged on
a level just by explosion images (because there are never more than four
enemies per level, including the boss UFO).

I'm going to refactor these, perhaps shrinking the frame count down from
60, and definitely reusing graphics and altering at runtime to give a
different effect for each enemy (e.g. rotating, coloring, etc.).   This
will make constant the memory used by explosions, even if enemy counts
go up per level.

On Tue, 11 Jul 2006 09:47:23 -0700, "Brian Fisher"
<brian@xxxxxxxxxxxxxxxxxxx> said:
> you have a lot of fairly big images in animations... frames are all
> going to be loaded in uncompressed, so they will likely be bigger in
> memory than on disk. You might want to take your image cache for your
> split images and give it some function that can return the sum of the
> expected memory size of all the currently loaded surfaces (width x
> height x bytes per pixel), and have that print out when you want to
> see it - It will help you break down how much of your memory
> allocation is due to wasted/duplicated/leaked memory due to code, and
> how much memory is due to having lots of big images
> 
> On 7/11/06, David Mikesell <dave_mikesell@xxxxxxxxxxx> wrote:
> > Rats - looked at the wrong process table entry.  Had two python
> > processes going - mine still hogging.  Back to the drawing board.
> >
> > On Tue, 11 Jul 2006 12:05:07 -0400, "David Mikesell"
> > <dave_mikesell@xxxxxxxxxxx> said:
> > > Down to 5Mb of memory!   I refactored what Brian suggested with the
> > > images, as well as some other messy code.   More testing to do, but I'll
> > > post the new version later tonight or tomorrow.
> > >
> > > Thanks, Brian!
> > >
> > > On Tue, 11 Jul 2006 06:51:53 -0400, "David Mikesell"
> > > <dave_mikesell@xxxxxxxxxxx> said:
> > > > Awesome catch, Brian.  I thought I was removing the artifacts managers
> > > > from all containers when they were through, but apparently there's a bug
> > > > in that code.   Even so, I still should be caching/removing the parsed
> > > > filmstrips.  I'll rework the image loading/unloading today and hopefully
> > > > release a better version tonight.
> > > >
> > > > Thanks very much,
> > > >
> > > > Dave
> > > >
> > > > p.s. I told you it was something braindead
> > > >
> > > > On Tue, 11 Jul 2006 01:49:50 -0700, "Brian Fisher"
> > > > <brian@xxxxxxxxxxxxxxxxxxx> said:
> > > > > One memory-related thing I noticed right away is that each time I
> > > > > start a new game, another 40MB gets allocated (so it's leaking mem).
> > > > > It looks like each time you load a level you create new "collision
> > > > > artifacts" managers, but the old managers get kept around. Each of
> > > > > those managers finds the filmstrip it needs and splits it up into
> > > > > surfaces. You don't load more than one copy of the underlying .png's
> > > > > due to your cache, but each manager happily creates it's own surfaces
> > > > > from the film strip (so you end up with lots of copies of the
> > > > > animations). You may want to think about caching the split filmstrips,
> > > > > instead of caching the .pngs, and not storing the loaded pngs at all
> > > > > (what do you need them for after you've split them anyways?)
> > > > >
> > > > > It looks like the explosion animations in particular take like 10MB a
> > > > > piece...  which is no suprise at 14000x200 for the filmstrip
> > > > > (14000x200x4bytes per pixel ~ 10MB)
> > > > >
> > > > > In case you are curious, I was able to find where the 40MB for level
> > > > > loading is being allocated by binary search - basically run the game a
> > > > > few times, inserting a bunch of prints and time.sleep(2) calls in
> > > > > order to give me a chance to look at memory usage in the task manager
> > > > > after large blocks of code, before new stuff ran. It showed that the
> > > > > mem was allocated in load_level_objects in game_scene.py:
> > > > >           self.collisions = levels.get_collision_table()
> > > > >           for o in game['collision_artifacts']:
> > > > >                   if o['load'] == self.level:
> > > > >                           manager_name = o['name']
> > > > >                           is_collidable = o['collidable']
> > > > >                           log.write_debug_msg( "Loading %s" % manager_name )
> > > > >                           manager_class = import_by_name( "%s.%s" % ( manager_name, manager_name ) )
> > > > >                           manager = manager_class()
> > > > >                           self.collision_artifact_managers[manager_name] = manager
> > > > >                           self.game_object_managers.append( manager )
> > > > >                           if is_collidable:
> > > > >                                   self.collidable_collision_artifact_managers.append( manager )
> > > > >
> > > > > what happens there, is that even though the old manager is not in the
> > > > > collision_artifact_managers dict anymore, it's never removed from the
> > > > > game_object_managers or collidable_collision_artifact_managers lists,
> > > > > which is why they keep their memory around. (as a test, I made sure to
> > > > > delete the old managers from those lists if they were already in the
> > > > > dict, and then loading the level again didn't leak memory anymore)
> > > > >                           if (manager_name in self.collision_artifact_managers):
> > > > >                                   old_manager = self.collision_artifact_managers[manager_name]
> > > > >                                   self.game_object_managers.remove(old_manager)
> > > > >                                   if is_collidable:
> > > > >                                           self.collidable_collision_artifact_managers.remove(old_manager)
> > > > >                                   print "skipping", manager_name
> > > > >
> > > > >
> > > > > On 7/10/06, David Mikesell <dave_mikesell@xxxxxxxxxxx> wrote:
> > > > > > I found some image filmstrips I was not unloading after they're cut up
> > > > > > into image arrays, and it freed up a decent amount of memory, but my
> > > > > > game is still a hog.  I totalled the amount of disk space for all images
> > > > > > used by the starship (different weapons, explosions, thruster, etc.),
> > > > > > and it's 600Kb.   However, when I comment out the line that loads the
> > > > > > starship_manager, the memory usage drops by a whopping 17Mb.   It's even
> > > > > > worse for the first enemy, which is loaded when the first level begins.
> > > > > > 355Kb of images, but the memory usage skyrockets another 40Mb when it
> > > > > > loads on the first level.
> > > > > >
> > > > > > I'm doing something braindead somewhere, just haven't tracked it down
> > > > > > yet.   The code is spread across dozens of source files, but if anyone's
> > > > > > interested it's at http://medievalelks.com/code.zip.  I honestly don't
> > > > > > expect anyone to take the time to look at such a volume, but I've been
> > > > > > requested several times to post it so there it is.   I didn't include
> > > > > > any data files, just code.
> > > > > >
> > > > > > Anyway, I'm making some progress.   Oh, and I scaled down the game to
> > > > > > 640x480, which also helps the performance.   Removed some
> > > > > > redundant-sounding music loops, too.   Download size is now a svelte
> > > > > > 13Mb :-).
> > > > > >
> >