SV: PFile notes

["Bjarke Hammersholt Roune" <asmild@post6.tele.dk>]

> >> >> (2) The user mounts the new PakFile with the ppfMF_write flag.
> >> >> The code for
> >> >> this creates a new PakFile object using the first constructor, which
> >> >> recognizes the Pak as a new one, allowing the adding of files.
> >> >>
> >> >Hmm... Is there a good reason for doing it in this way?
> >>
> >> Yes. That way ppfCopy can be a generic function, just copying
> files from A
> >> to B.
> >>
> >What is the problem that makes it impossible for ppfCopy() to be generic
> >otherwise?
>
> Not impossible, but harder.
> If the target file system is not mounted at copying time but has to be
> treated specially ppfCopy would be much more complex.
>
Why wouldn't it be mounted?

> For me "user" == user of LibPP, i.e. programmer of a game. So I guess we
> agree here.
>
Ahh, ok. When I say user, I usually mean gamer. I usually user the word
client when I mean "programmer who use PFile".

> >A program migth also do something like this:
> >
> >"Datafile currupt - cannot continue"
> >
> >Which is quite undesireable.
>
> As undesirable as a "file not found - cannot continue" and about
> as likely.
>
Well, it worked before wihtout this file (or it couldn't have failed in
writing it), and not finding a file is more common, and therefore something
clients will be more likely to hanlde graciously.

But, well, yes, this is a somewhat small point.

> >migth want to consider having a PakFile::CreateEmptyPak() function, which
> >doens't create split-second-life-time PakFile objects.
>
> Ok, if you mind about the extra few cycles...
>
Ah, well, it was more that that's a somewhat weird way to do it. In any
case, fixing sometihng like that would be very low priority.

> Well, take some Pak with n dirs. The XHeader uses, say, 16 bytes per dir
> (position and length of the dir info for easier memory mapping). Now you
> add a dir - and both the data in the Pak grows and the XHeader grows. If
> it's at the start of the Pak then you lost. If it's at the end you lost
> (ok, you can always make sure it's at the very end and always read the Pak
> from its end backwards until you reach the XHeader ID to determine its
> starting pos). If it's somewhere in the middle you lost.
> In other words - it has to be relocateable.
>
Well, if you want to use it like that, you can put it after the directory
and file header data, and relocate it along with the header when the pak
grows. That also guanrantees that you don't have any unused gaps in the pak,
which might otherwise be created.

> >> What I meant is that the generic Directory class has a PakFile specific
> >> function (SetSerializingPakFile ()).
> >>
> >Well, yes, but that's only when Directory is serializing *from*
> a *pak*. I
> >don't see how that is bad. Directory is reponsible for loading a
> directory
> >hierachy from a pak file. It needs to know what pakfile to tell
> each loaded
>
> No. Directory is responsible for maintaining a collection of files and/or
> subdirectories. A Directory is in no way related to a PakFile. Some of the
> classes derived from it are, others are not.
>
That's a good point. See my other E-mail.

> Ok, right. The main writing code is in Directory and HashTable. Which
> explains that I didn't find it for so long, because it shoudn't be there.
>
See my other E-mail.

> >> (which is BTW outdated according to its comment) AFAIS.
> >>
> >The MISC template parameter candidate which is in HashTable.h is strictly
> >there for documentation porpuses, showing what interface a valid
> candidate
> >for a MISC template parameter must have. I believe it says this.
>
> Yup, but it also says it's outdated. I don't know what you meant with that
> comment though... ;)
>
Well, the example MISC candidate doesn't look like a proper candidate
should, so its outdated for its porpuse: documentation.

> I can't completely follow you, but ok... The system has to be able to
> handle the following scenarios nicely:
>
> (1) Creating a dir from a static source with optimizations (e.g.
> an existing
> PakFile containing the hash keys of the entries, hashtable size etc) and
> eventually adding 1-2 directories later (some FS mounted below etc)
>
> (2) Creating a dir structure file by file (e.g. when creating a Pak or
> reading an ftp dir) with only very rare read accesses to that struct, and
> afterwards optimizing the hashtable for read accesses and eventually
> serialization.
>
That would be handled fine with the very simple to implement solution I
outlined above.

> >> That app needs to keep track about what files will be stored
> anyway, i.e.
> >> it has to have some internal representation of the target hierarchy.
> >>
> >Why would it duplicate this functionality, when PFile would be able to do
> >this for it?
>
> Because it somehow has to display what's stored in the Pak? At least it
> *should* have this feature ... ;)
> Do you think WinZip would be as common if it wouldn't offer the
> possibility
> to see what is stored in the archive? ;)
>
Why wouldn't it just read the PFile structure when finding out what to
display? Why duplicate all of this?

> >What exactly do you mean? Do you mean like duplicating directory
> hierarchy
> >from the native file system?
>
> from anywhere to anywhere.
> Example:
>
> ppfCopy ("/usr", "/mnt/usrpak", ppfGF_recursive);
>
> will copy the directory "/usr" and everything below it,
> recursively, to the
> directory "/mnt/usrpak" and then returns.
>
Yes, but that requires that /usr contains the exact directory and file
structure you want.

How would you do that if the files you wanted to include were scattered all
over?

like half the files in /usr needs to be added, plus alot of files in alot of
other locations.

> >> And it's much more intuitive if the copy operation actually
> happens when
> >> you call, well, the copy command. And that this copy command
> only succeeds
> >> if the copy operation succeeded.
> >>
> >The copied file will be there as soon as it is copied, and it
> will be there
>
> ppfCopy ("some/weird/file", "/mnt/newpak", 0);
> ppFILE *WFile = ppfOpen ("/mnt/newpak/file", "rb");
>
> Sure?
>
Doesn't it do this? Well, that's just a question of making it do that. Quite
simple, I believe.

> >the next time the pack is opened also. The pak is invalid while its being
> >built anyway, so there is absolutely no difference for the client in this
>
> Does it have to be invalid while it is being built? When storing all dir
> info at the end (both in terms of location and time) there's no
> more reason
> for this.
>
Well... it doesn't *have* to be invalid.

You do realise you'll have to move (ie, serialize) the complete directory
and file structure to the end of the pak each and every time you add a file
to the pak? (I sense we aren't talking about the same thing here)

>
> PS: I'll rename ppFILE to ppfFILE for consistency sake (even though it
> sounds ugly ;)
>
Good idea :)