Re: SV: SV: SV: SV: Progress Update

[Pierre Phaneuf <pp@ludusdesign.com>]

Bjarke Hammersholt Roune wrote:

> > I see, and this is a good reason. Personally, I'd have named that class
> > something more like StaticHashTable, to reflect its immutability and
> > leave the name free for a "normal" hash table. But maybe I'm nitpicking
> > here.
> 
> Well, then I'd end up with names like ConstStaticHashTableIterator. I
> decided ConstHashTableIterator was long enough... ;)
> 
> Not a bad point. Fixing it would be only a matter of a search-and-replace
> operation, though.

Such a specialized class with such a general name as "HashTable" is not
a very good idea...

> > They *cannot* all hash to the same location, since I there is a load
> > limit. When an entry has to be added to a full bucket, I grow the hash
> > table. Of course, pathogen hash patterns (many keys hashing to similar
> > values) could cause excessive growing of the table, but that life with
> > hash tables.
> 
> That gives you more space, but still, if every entry in your table hashes to
> the value 1, then I don't see what you can do about that, nomatter how large
> your table is: the rest of the positions in the table would just be empty.

Then, you have a problem with your hashing algorithm. Your hash table
would also have a problem, wouldn't it? It would have a huge number of
entries in one of the hash position. Your's would be way faster than
mine because of the binary searchable sorted list, but it wouldn't be
anything like a correctly hashed table.

> > By "worth the effort", I meant in programmer-time. You had to program
> > two different hash tables and each time you add stuff to one, you have
> > to invoke a bunch of methods and juggle conversions around.
> >
> > I have *one* hash table implementation and I do ht->set("thekey",
> > "thevalue") and that's it.
> 
> DynHashTable<...> dynTable
> 
> table.ConvertToDynTable(dynTable);
> dynTable.AddEntry(entry);
> table.CreateFromDynTable(dynTable);
> 
> Well, is this much worse? Again, the requirements to the HashTable is quite
> specific.

Not *much* worse, but the point isn't only programmer-time for users,
but for your time.

You must apply a similar development approach to the rest of
PenguinPlay, right? If doing the same thing takes me 10% of the time it
took you and performs 90% of the way your implementation would (and
would be "fixable" if I invested more time, but I could do when I choose
to), it would still take me considerable time to finish a usable version
of a project of the scope of PenguinPlay.

Do you think that it'll get any easier by taking 10 times as much effort
to get that last 10% right from the start?

> > Opening of the files? Heck, in Quadra, there is about *10* such file
> > opens over all the course of a typical short game, which includes
> > reading the configuration (once), reading the highscore file (once),
> > loading the resources (once) and loading the highscoring demos (when you
> > click on the button).
> >
> > Per frame, we have an average of zero file opening. Okay, so how would
> > you rate the investment in time now for a game such as Quadra?
> 
> My response is that it would make no sense for a game that opens only 10
> files to utilize any kind of advanced IO library, and the Quadra developers
> is therefore clearly not in the group of people we are targetting with this
> product.

I said 10 files opening because I counted the resource file as one
opening. The resource file contains a few hundreds files. I guess I was
a bit misleading with this. Anyway, we don't "open" files all that often
anyway, from time to time during game play, we do (level ups that change
the "theme", for example), that's about it.

The point is that improving the color conversion process for example
yields results that are multiple orders of magnitude greater than the
best resource file system could give us. I wish the PenguinPlay system
was strongly implanted, and I just feel the efforts should be wisely
invested for this to happen.

> > This is better stuff (the CD-ROM caching).
> 
> Quadra still wouldn't care... ;)

The resource file is 5.5 MB. Caching *could* help, if CD-ROMs are
implicated in the thing (CD-ROMs can be *very* slow if they have to spin
up). :-)

> > For example, do you know what Unix does when an interrupt breaks a
> > system call in half and messes the internal state of the system call?
> > NOTHING! It just returns with EINTR (interrupted system call)!
> > Outrageous, isn't it? :-)
> >
> > But the good news is that this is fixable. You just have the libc, which
> > repeats the system calls until they work. :-)
> >
> > Did they "do it properly"? Nope! But it's fixable...
> 
> I'd say it isn't fixable if it takes an external codebase to fix it. That's
> called a work-around. Like when you don't do something specific in a program
> because you know that it'll die if you do, or only do it in a specific way.

This is all documented, and is a design choice made for the simplicity
of the design, which is a valid goal. The fact that this was fixable in
a user-space library rather than making it "properly" in kernel-space
just prompted them to go for it and leave this to user-space, keeping
the system-specific kernel-space clean of such complexity.

This made Unix stronger rather than weaker, as it would seem at first
glance!

Does it really matter to the user doing an fread() that the libc
actually is doing a loop of read() calls? Nope. Don't worry, everything
is fine!

> > Yep, but they did have at least a much better solution and people didn't
> > care. NeXT also had a much better system and look where they are now
> > (OpenStep programming is a dream come true by the way, simply
> > fantastic).
> 
> In some way, they must've failed, it seems... :)

Their problem was cost it seems. They had fantastic machines, operating
system, programming tools and user interfaces (look at Window Maker or
AfterStep today for a taste of it).

> Well, it doesn't matter what OS they choose, PenguinFile will still work.
> And PenguinFile doesn't have any competetion. That's why it was started.
> There's a gap there, and if there is only one lib to fill it...

Doom, Quake, Quadra and numerous others have archive support. What
boggles me about PenguinFile and PenguinPlay is that archive support is
often done as an afterthought and still done nearly as good as it could
be, good enough anyway. What is an *afterthought* in most project is a
major undertaking in PenguinPlay, is that expected?

> Not in all aspects. There are things Windows currenlty does much better than
> any Unix. Like aid for the disabled or an easy GUI (for beginners, not for
> people who are actually serious).

Aid for the disabled and easy GUI can also be done in Unix.

This isn't the point anyway. The point is that technical superiority
isn't the key to success. It is *part* of it, and is sure a very
desirable feature to have, but if you don't get *something* out soon,
there will be a system that will cover this area, and it could be of the
"fundamentally flawed, unfixable" kind. I do not care for "perfection"
right now, I only care for not getting invaded by stupid stuff! Give me
something *fixable* eventually, NOW!

-- 
Pierre Phaneuf
http://ludusdesign.com/