[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [pygame] Cheap COW clones for surfaces



I'm not sure that adding a hidden expensive operation that will make it
work, but slow, instead of simply raising an error or doing what the
user requested and letting them see the weird results is a good idea.
Writing a performant game then becomes akin to navigating a minefield,
where you have to do everything just right, blindly, otherwise it will
still work (so you can't see where the mistakes are), but slow.

On Mon, 18 Jun 2018 12:36:45 +0200
Thomas Kluyver <takowl@xxxxxxxxx> wrote:

> Even for responsible adults, it's valuable to build abstractions that
> give you the results you expect when you do the obvious thing. The
> 'responsible adults' principle means we can say "you can modify this
> private attribute if you know what you're doing". It doesn't mean
> that you can expect everyone to read the docs to find out how to use
> it right.
> 
> On 18 June 2018 at 12:17, Daniel Pope <mauve@xxxxxxxxxxxxxx> wrote:
> 
> > Pygame Zero programmers are not necessarily responsible or adults.
> > Overall that idea would break several of our documented principles:
> >
> > http://pygame-zero.readthedocs.io/en/stable/principles.html
> >
> >
> > On Mon, 18 Jun 2018, 11:09 Radomir Dopieralski,
> > <pygame@xxxxxxxxxxxx> wrote:
> >  
> >> You can't prevent it, but you can explain in the documentation
> >> that it is discouraged and what happens when they do it. Python
> >> programmers are responsible adults after all, no?
> >>
> >> Otherwise you start writing Java in Python...
> >>
> >> On Mon, 18 Jun 2018 10:24:25 +0100
> >> Daniel Pope <mauve@xxxxxxxxxxxxxx> wrote:
> >>  
> >> > The drawing thread should handle all blits to the screen as well
> >> > as flip. But in a framework we cannot prevent users creating
> >> > surfaces and mutating them in the logic thread.
> >> >
> >> > On Mon, 18 Jun 2018, 09:55 Radomir Dopieralski,
> >> > <pygame@xxxxxxxxxxxx> wrote:
> >> >  
> >> > > Them again, if you assume that the drawing thread is doing all
> >> > > the blits, then the situation is reversed — it's the only
> >> > > thread that needs write access to the surfaces, and everything
> >> > > is good again. In your scenario, we still get the correct
> >> > > result, because all the blits are executed by the drawing
> >> > > thread in the same order in which they were scheduled —
> >> > > including the blits that modify your temporary surface with
> >> > > the digits. As long as all the modifications happen either on
> >> > > one side or the other, we are good.
> >> > >
> >> > > On Mon, 18 Jun 2018 01:09:16 +0200
> >> > > Radomir Dopieralski <pygame@xxxxxxxxxxxx> wrote:
> >> > >  
> >> > > > Oh, I think I misunderstood what the drawing thread was
> >> > > > supposed to do. I thought it would only handle the "flip"
> >> > > > and/or "update" calls, that is, sending the pixel data of
> >> > > > the screen surface to the graphic cards, which is the
> >> > > > slowest part of any pygame program. But from what you write,
> >> > > > I understand it's supposed to handle all blits as well?
> >> > > >
> >> > > > On Sun, 17 Jun 2018 17:33:54 -0500
> >> > > > Luke Paireepinart <rabidpoobear@xxxxxxxxx> wrote:
> >> > > >  
> >> > > > > Radomir, there is another side effect - if the surfaces
> >> > > > > are not copied, then whatever content was in surf a at the
> >> > > > > moment that it was supposed to be drawn now no longer
> >> > > > > exists. For example, say the logic thread is font
> >> > > > > rendering the numbers 1 thru 9, and then blitting these to
> >> > > > > positions that are not overlapping each other, but it is
> >> > > > > reusing a surface for the font render call. In the
> >> > > > > synchronous example, 1, 2, 3, etc would be drawn. Make
> >> > > > > that asynchronous and you could end up with 9,9,9,9,9....
> >> > > > >
> >> > > > > If you are multithreading a game, the "main" thread should
> >> > > > > be mutating the game state, not the surfaces directly. The
> >> > > > > render thread should be the view of the model state at the
> >> > > > > snapshot in time at which the render is called. The logic
> >> > > > > thread should have a control that is determining the time
> >> > > > > between updates and applying the appropriate amount of
> >> > > > > ticks to the game state / emulating physics / etc. Then
> >> > > > > whenever the previous draw has completed, the render
> >> > > > > thread would snapshot the state and represent it
> >> > > > > appropriately (either redrawing or comparing to its
> >> > > > > previous state). If there needs to be multiple draw calls
> >> > > > > to represent the state properly since the last state, then
> >> > > > > they can all be done before the display buffer is flipped.
> >> > > > >
> >> > > > > In the 1-9 example, each font would be represented by a
> >> > > > > game object that had a positional element, and you could
> >> > > > > add them to the map asynchronous of the draws since the
> >> > > > > render thread would display them on the next iteration of
> >> > > > > the render loop where they existed.
> >> > > > >
> >> > > > >
> >> > > > > On Sun, Jun 17, 2018, 3:49 PM Radomir Dopieralski
> >> > > > > <pygame@xxxxxxxxxxxx> wrote:
> >> > > > >  
> >> > > > > > Of course strictly speaking there is a difference in
> >> > > > > > behavior, however, from the practical point of view, the
> >> > > > > > difference boils down to the fact that something gets
> >> > > > > > drawn half a frame earlier rather than half a frame
> >> > > > > > later (because the command to draw it was given in
> >> > > > > > between the frames). Unless you have less than 6 frames
> >> > > > > > a second, the difference wouldn't be easy to notice,
> >> > > > > > especially since it would be rather rare too, since you
> >> > > > > > would need to get very specific timings to even have it
> >> > > > > > happen. I wonder if it's worth the effort.
> >> > > > > >
> >> > > > > > On Sun, 17 Jun 2018 21:40:56 +0100
> >> > > > > > Daniel Pope <mauve@xxxxxxxxxxxxxx> wrote:
> >> > > > > >  
> >> > > > > > > The mutatation would be on the logic thread side.
> >> > > > > > > Consider this:
> >> > > > > > >
> >> > > > > > > draw_to_screen(surf_a,  ...)
> >> > > > > > > if cond:
> >> > > > > > >     surf_a.blit(surf_b, ...)
> >> > > > > > > draw_to_screen(surf_a, ...)
> >> > > > > > >
> >> > > > > > > Currently draw_to_screen() is synchronous. I'd like it
> >> > > > > > > to queue the blit instead, to happen in another
> >> > > > > > > thread. If I just did
> >> > > > > > >
> >> > > > > > > def draw_to_screen(surf, ...):
> >> > > > > > >     draw_queue.put((surf, ...))
> >> > > > > > >
> >> > > > > > > then there's a race condition - surf_a may be drawn
> >> > > > > > > twice with the surf_b update.
> >> > > > > > >
> >> > > > > > > If draw_to_screen() is implemented like
> >> > > > > > >
> >> > > > > > > def draw_to_screen(surf, ...):
> >> > > > > > >      draw_queue.put((surf.copy(), ...))
> >> > > > > > >
> >> > > > > > > Then I get no change in behaviour, but I copy on every
> >> > > > > > > blit, on the logic thread, ie 2 copies per frame
> >> > > > > > > regardless of whether cond is True.
> >> > > > > > >
> >> > > > > > > Changing the implementation of copy to create a COW
> >> > > > > > > clone means that the buffer copy actually happens at
> >> > > > > > > this line, if it is hit:
> >> > > > > > >
> >> > > > > > > surf_a.blit(surf_b, ...)
> >> > > > > > >
> >> > > > > > > Which means that there's 1 copy if cond is True and 0
> >> > > > > > > if cond is False.
> >> > > > > > >
> >> > > > > > > On Sun, 17 Jun 2018, 20:59 Radomir Dopieralski,
> >> > > > > > > <pygame@xxxxxxxxxxxx> wrote:
> >> > > > > > >  
> >> > > > > > > > On Sun, 17 Jun 2018 17:48:26 +0200
> >> > > > > > > > Daniel Pope <mauve@xxxxxxxxxxxxxx> wrote:
> >> > > > > > > >  
> >> > > > > > > > > I have been thinking for some time about how to
> >> > > > > > > > > optimise Pygame Zero games on Raspberry Pi. Most
> >> > > > > > > > > Pi models have multiple cores and an obvious
> >> > > > > > > > > improvement is to parallelise. Logic has to be
> >> > > > > > > > > synchronous but I would like to offload painting
> >> > > > > > > > > the screen to a separate thread.
> >> > > > > > > > >
> >> > > > > > > > > The problem I would have is in passing references
> >> > > > > > > > > to mutable objects between threads. Primarily this
> >> > > > > > > > > applies to Surface objects, which are mutable and
> >> > > > > > > > > expensive to copy. If I have a draw thread that
> >> > > > > > > > > maintains a queue of screen blit operations, I
> >> > > > > > > > > want the queue to hold references to surface data
> >> > > > > > > > > that won't change even if I later mutate the
> >> > > > > > > > > surfaces in the logic thread.  
> >> > > > > > > >
> >> > > > > > > > Sorry if I am missing something obvious, but it
> >> > > > > > > > seems to me that the draw thread doesn't need to
> >> > > > > > > > mutate the surfaces? I mean, it only accesses them
> >> > > > > > > > in a read-only fashion to render them. So you don't
> >> > > > > > > > need to pass references to mutable objects — the
> >> > > > > > > > drawing thread can get a read-only reference. Why do
> >> > > > > > > > you need it to have a reference to non-changing
> >> > > > > > > > data? After all, if it changes, you will have to
> >> > > > > > > > re-draw it in the next frame anyways. Unless the
> >> > > > > > > > drawing thread is more than one frame behind (which
> >> > > > > > > > really shouldn't happen), you don't care about the
> >> > > > > > > > data changing.  

-- 
Radomir Dopieralski