13-09-2002 15:55									#1
	Understanding Tot Sprite File Format Full PM-box? Change here!

It's a pity that Tot spr format is unknown, i usually play Civ without animations (since i removed spr files a lot of time ago to play scenarios), some days ago i reinstalled Tot and started a game: i was delighted looking at animations of tanks or bombers.
Unfortunately it seems that Draco eventually lost interest in Tot (but i still hope to see him surface here again) so it seems we have to do the whole work alone.
A couple of days ago i started peeking the SPR files and attached you'll find a translated (and polished) version of my notes, they're far from complete but i think they're a starting point.

Greetings,
Angelo

Attachment: sprformat.txt
This has been downloaded 25 time(s).

	13-09-2002 17:17									#2
	Full PM-box? Change here!

Hooray for Angelo! If there's anything I ,as a computer illiterate, can do to support you, just let me know.

	13-09-2002 18:08									#3
	Inflate your Upload Space

Ditto for me.
Downloaded your notes but I'm already I'm afraid, so I'm not likely to be too much help.

	13-09-2002 20:04									#4
	Lose 30 kilos (of popups)

Great work... I think I've found a little more thanks to your info . I'm not too familiar with all the correct terms or common "techniques" in file formats, so I just rant a little and guess away:

STATIC.SPR

The header consists of a whole set of DWORDs (or long integers, a sequence of 4 bytes in any case), most of them are offset pointers or so. I'll list them
1. 0 (for whatever reason )
2. 12, pointer to the next "block" at offset 12
3. 1636, a pointer to the end of the "header"
4. 0, offset of the first image (starting after the header, add value (3) to get the "real" offset)
5. - 408. remaining image offsets 405 in total
409. + (3) = end of file

And you didn't miscount the number of occurrences of "4000 0000 4000 0000" it is indeed 406, but near offset 139111 (0x21F67) there are two closely following eachother, one of them is either superfluous (an empty image not entirely removed correctly?) or perhaps just a coincidental occurrence of the sequence as part of the image encoding. In any case, the second of these two (at offset 139127 or 0x21F77) is NOT represented in the offset listing in the header.

UNIT01.SPR

Again lots of DWORDs, but not all offsets this time:

1. 12, pointer to byte 12 or DWORD (4)
2. 1324, pointer to the start of the image offset list in the header
3. 1928, pointer to the end of the header
328 DWORDs I haven't investigated yet
151 DWORDs, being the offsets of the images (works the same as with Static.spr: the first offset is 0 and is relative to the end of the header, so add (3) to get the "real" offset)

Edit: the last offset in the list always points to the end of the file, so for the UNIT01.SPR that would mean there are 150 images.

Last edited by Mercator on 14-09-2002 at 17:56

	17-09-2002 16:04									#5
	Put an end to popups!

Mercator, this is a great news!
Yes, offsets! i tried to use them directly as pointers but i fell in useless places of the file, sure, i had to sum header offset, it's clear now.

Unfortunately, the format in which is coded the image is still unknown, i wasn't able to find a single match with any standard image format.

I peeked in the Tot exe file and found that the real name of sprite file format is SPRT... but this is not very useful after all

Always in the exe there were references to several graphic formats: PCX, IFF ILBM (Used mainly on Amiga!!!), GIF and obviously BMP... this tells us that probably Microprose used a pre-made graphic library but i don't think this is useful too...

Greetings,
Angelo

	19-09-2002 00:09									#6
	Support Apolyton buy from Amazon

I haven't had any luck with the images either. I copied one presumed image to a separate file, and tried to open it as every possible image file PSP recognized... But still no luck. Then again, it might well be that it requires some little extra fixed header information added in front... Maybe DracoOmega (the one who said he was working on an editor) can give us just a little more info.

	23-09-2002 23:27									#7
	Support Apolyton buy from Amazon

This is Draco Omega here (under a different name due to login problems).

I figure everyone must either hate me now or think that I made the whole thing up and never figured it out. The truth is that I started writing the program several times with the best of intentions, but it was taking a while and it ended up getting abandoned. I guess I'll never end up finishing it myself so I'll just give the format out and let you guys handle that end of things. Again I apologize....

I had almost all of the format figured out within the first 3 days of looking at it, but my notes were sketchy to begin with and they probably got thrown out when I moved to university last month. (Also my old computer got beat up my the postal service and is currently inaccesable to me, but if I can manage to get a look at my old source code, some more things will become clear)This will be pretty much off the top of my head and incomplete, although it should probably help a little. And I'll post any more details as I remember them.

Firstly, there is no sort of compression used on the images, nor are they a standard format that I am aware of. They are 16-bit images, with each pixel taking to bytes (obviously) but not quite in the standard fashion. The bits are in standard 5-5-5 R-G-B format, but the unused bit is used here to represent whether that pixel should be painted with the team color (adjusted naturally to the shade of the pixel)

The image is divided into a number of rows of pixels, of which each row has a 2 byte header. I'm not 100% sure if this order is correct, but I think the first byte is double how many pixels this row is offset from the far left of the image. The second is the length in bytes of the row.
They could be in the other order, though.

Each row does not be the same length, but starts at the first non-transparent pixel of that visiable row and ends with the last one. This basically saves the space that would be wasted with the large ammount of transparent space on the left and right of the sprite.

There is also a header to each image that is either 45 or 46 bytes in length (I can't remember which) Unfortunatly, I can't remember the order of the entried at all, but I can at least tell you what some of them are, so other may be able to find them.

There is a value stating the true top, bottom, left, and right bounds of the image (with a maximum of 64x64 as in units.bmp) Also, is the size of the main image data. Some other, less used values relating to the position of the health bar are there as well.

Darn... As I'm writing this, I realize how little use it is and how little I remember. Well, I'll trie to get the source code off my old computer as soon as possible and post a more detailed, accurate, and complete version of the format, but I'm afraid that this will have to do until then, such as it is...... Oh, and if you have any questions about my poorly written description, feel free to ask.

	24-09-2002 03:35									#8
	Support Apolyton, buy Galactic Civilizations

quote: Each row does not be the same length, but starts at the first non-transparent pixel of that visiable row and ends with the last one. This basically saves the space that would be wasted with the large ammount of transparent space on the left and right of the sprite.

Thanks DO/Z! How does the format store the offset of the first non-transparent pixel for each row?

	25-09-2002 20:44									#9
	Support Apolyton, buy GURPS/ Alpha Centauri

quote: [...] each row has a 2 byte header. [...] I think the first byte is double how many pixels this row is offset from the far left of the image. The second is the length in bytes of the row. They could be in the other order, though.

Marvellous! I think that's more than enough information to get us started (although any more is highly appreciated). Thanks Draco/Zarion!

	29-09-2002 14:14									#10
	Suffering from ads?

Any progress? I'm just a cheerleader, I guess, but I'm sure hoping you guys can solve this mystery!

	29-09-2002 20:59									#11
	Support Apolyton, buy Call to Power 2

It's great to hear you again Draco/Zarion !
Thank you very much, you've saved us from searching tonnes of image formats (unfortunately i've already looked hundreds of them ...)

Actually i still have to find some time to work on this, but i think our first goal should be to define a correct format of a single image in the .spr,
We already know where to look and, thanks to Draco, we also know what to look, so it shouldn't be very difficult.

After that... well, we will see ...

Greetings,
Angelo

	03-10-2002 21:38									#12
	Suffering from ads?

I take it no one is bothering with this then???

Why didnt Microprose publish information on it in hte first place to encourage editing & making new units?

So are there no animated custom units for this game i take it, made by fans

	04-10-2002 00:24									#13
	Support Apolyton, pre-order Civilization IV

I think people are still working on it, SMIFFGIG. It may take some time to decode the thing.

I don't really know why Microprose didn't include a sprite editor or at least provide some way to modify the sprites. I think they had Starcraft in mind, where existing graphics are used in scenarios to represent things that look entirely different. But Civ is a very different game. IMHO, had they done so, ToT would have been much more popular. Indeed, the graphics (both animated and not) are it's greatest downfall.

	04-10-2002 00:31									#14
	Increase Your PM Length

quote: think they had Starcraft in mind, where existing graphics are used in scenarios to represent things that look entirely different

Lol i know EXACTLY what you mean (no no thats not a world war 2 spitfire it is a fusion powered space cruiser)

quote: had they done so, ToT would have been much more popular

Without a doubt, had the potential to what civ2 was in modability and fan created scenarios

quote: I think people are still working on it, SMIFFGIG. It may take some time to decode the thing.

Well good luck to them, who ever they are. I look forward to the results

P.S. It would certainly open alot of doors for ToT

	30-10-2002 17:27									#15
	Support Apolyton buy from Amazon

* bump *

I found out some more (attached)... And more importantly, I've made a little utility. So far it can only export all images in the Static.spr files to 24-bit bitmaps though (the info from Zarion was more than enough to figure out how to "decode" the images )...
But at least I'm getting somewhere.

Zarion (or Angelo?)... Do you by any chance have more info on the header sections of the Resource and Units sprite files? That seems to be the only thing I haven't figured out yet, but it's a rather important bit I'd say.

Attachment: spr format.3.txt
This has been downloaded 13 time(s).

	31-10-2002 03:12									#16
	Support Apolyton buy from Amazon

I wouldn't mind getting my grubby hands on that utility.

	05-11-2002 22:25									#17
	Support Apolyton buy from Amazon

quote: I found out some more (attached)... And more importantly, I've made a little utility. So far it can only export all images in the Static.spr files to 24-bit bitmaps though (the info from Zarion was more than enough to figure out how to "decode" the images )...

Sounds like a sprite editor to me!

If the bitmaps can be put back into a SPR.....

	06-11-2002 02:22									#18
	Enter the AD-FREE zone

Not yet...

	16-11-2002 12:58									#19
	Support Apolyton or Terrorists Win

I wonder if the sprite files are in PPM format?

http://astronomy.swin.edu.au/~pbourke/dataformats/ppm/

	18-11-2002 22:28									#20
	Help yourself to an AD-FREE life

Nope

They're in some custom "compressed" 15/16-bit bitmap format. I don't think it's a pre-existing format, because the "compression" is done to reduce all the superfluous transparent "pink" surrounding the images. I don't think many would really need such compression (except game designers).

I have figured out the exact details of the single images in the sprite, I'm not too sure about all the file header details, though...

In any case, I'm working on a sprite editor, which will be able to export sprites to a pair of bitmaps, and import them again. A pair of bitmaps, since one bitmap holds the "normal" image, the other holds the civilization-color mask.

I now have procedures ready that can convert between bitmaps/sprites, but no procedures yet to retrieve the images from the sprite file (I started over since last time, to be able to support the unitXX.spr and resource.spr)...

It all depends on how busy I'm gonna be with other things how long it will take me from here.

	18-11-2002 22:39									#21
	Support Apolyton or Terrorists Win

Can't wait..... !

	21-11-2002 16:24									#22
	Inflate your Upload Space

This is a breakthru that may just refresh interest in ToT,
Also it will be a stunning discovery for we ToT scenario makers!

Good luck, gentlemen!!!

	25-11-2002 18:29									#23
	Avatar Enlargement: We've got the solution

It's probably going to take a while yet... I have to do some serious thinking on what the interface is gonna look like.
Some better news: I found out a bunch more about the animation information in the resource/units sprite files:
- You can specify whether you want to loop an animation or not (or even, say, only loop the latter half of an animation, after an initial first half).
- You can make animations semi-transparent, like the die animation fade-out. Since you can specify the transparency level for each frame, you can do quite some more complicated things with it (say, have an entirely semi-transparent "Ghost" unit).
- Frames can be mirrored, saving you from making unnecessary additional frames.

There are quite a few more technical intricacies, which are especially challenging to program correctly: animations aren't saved in full, one after the other. Instead, the header specifies which animation to use, and for each animation which of the images in the file to use. That means you could reuse both animations and frames indefinitely while only one instance is actually saved in file.

For the static unit images, only 5 directions are stored: North, Northeast, East, Southeast and South. The remaining 3 are created by mirroring on of the above.
For the animated unit images, all 8 directions are stored, but, as stated earlier, you can reuse and mirror individual frames.

I'll post the new file format description soon...

(and the eager creators will have to wait a little longer )

	28-11-2002 02:36									#24
	Full PM-box? Change here!

Here's the text file I promised...
No progress since my last post though (well, my planning on paper has come a little further).

Attachment: spr format.3.txt
This has been downloaded 21 time(s).