+==========================================================================================+ |Keen 1-3 type graphics format: | +==========================================================================================+ This covers the files EGAHEAD, EGALATCH and EGASPRIT, the header, unmasked and masked (sprite) files respectively. Keen uses EGA type graphics, in which an image is stored as 4 (or 5) 'planes' of data. (4 for unmasked graphics, blue, green, red and intensity, 5 for unmasked graphics, which also has a 'masked' plane.) In each plane 8 pixels are stored per byte, (this is why graphic sizes tend to be multiples of 8 pixels wide.) giving in total an approximately 4-bit form of storage. When uncompressed the files are usually 'raw data'; much like a bitmap without a header. The egahead file tells Keen where in the files to read from, and how much. A pixel (Or rather 8 pixels) is composed by adding all the values from the four planes for each pixel. So black is 0000, and dark blue is 1000 (Light blue is 1001) etc. In summary: GRAPHICS -> 4 PLANES -> 8 PIXEL LINES -> 0\1 BITS BGRL: COLOR 0000: Black 0001: Dark grey 0010: Red 0011: Light red 0100: Green 0101: Light green 0110: Brown 0111: Light brown (Yellow) 1000: Dark blue 1001: Light blue 1010: Purple 1011: Light purple (Pink) 1100: Turqoise 1101: Light Turquoise 1110: Grey 1111: White +------------------------------------------------------------------------------------------+ |Keen 1-3 Graphics Header (EGAHEAD): | +------------------------------------------------------------------------------------------+ Used in all Keen 1-3 engine games. (Dangerous Dave has this stored internally.) An important note about Keen is that it can only retrieve data from 'rounded ($X0) addresses; this means that all pointers end in 0 ($00030, $00150...) and that any data not stored starting at a rounded address will be 'clipped' (Though the sprites have a way around this...) The graphic names are not required and do little. Many of the values in Keen are zero and so can be ignored, unless you want to do something fancy with your files. ------------------------------------------------------------------------------- EGAHEAD STRUCTURE: HEADER UNMASKED ENTRIES ... .. . MASKED ENTRIES ... .. . ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- HEADER: 0 4 Latplansiz Size of EGALATCH plane; should be one quarter of the size of the UNCOMPRESSED EGALATCH size. This is the size of the four planes in the EGALATCH file 4 4 Sprplansiz Size of EGASPRIT plane; should be one fifth of the size of the UNCOMPRESSED EGASPRIT size. This is the size of the five planes in the EGASPRIT file 8 4 Bmpdatstart Where in the EGAHEAD the entries for unmasked graphics (Excluding font and tiles.) start. Should always be byte 48 12 4 Sprdatstart Where in the EGAHEAD the entries for masked graphics (Sprites) start; by default this is right after the unmasked graphics. 16 2 Fontnum Number of 8x8 font entries are in the font; since font is written first in EGALATCH, this x 8 in bytes defines the offset into each plane that the unmasked bitmaps start 18 4 Fontloc Offset in plane where font data starts. Should be zero since font is first. 22 2 Unknum Used for the ending screen until this was removed. Number of screen graphics. 24 4 Unkloc Used for screen graphics until removal. Offset in plane where screen data starts. 28 2 Tilenum Number of 16x16 tiles 30 4 Tileloc Offset in EGALATCH plane where tile data starts. = fontnum x 8 34 2 Bmpnum Number of unmasked bitmaps 36 4 Bmploc Offset in plane where unmasked bitmap data starts. 40 2 Spritenum Number of sprite images 42 4 Spriteloc Offset in EGASPRIT plane of start of sprite data. Is, of course, zero. 46 2 Compression Add 2 to this byte if EGALATCH is compressed, add 1 to it if EGAHEAD is compressed. Thus uncompressed graphics have this set at 0 and fully compressed at 3 48 16x Unm Ent Unmasked graphic entries of 16 bytes each. ? 128x Msk Ent Masked graphic entries of 128 bytes each. ------------------------------------------------ UNMASKED BITMAP ENTRIES: ? 2 Size h The width of the graphic divided by 8 +2 2 Size v The height of the graphic in pixels; if this cannot be divided into neat 16 byte pieces, the extra data, usually 8 bytes, is added to the size. +4 4 Loc When added to the graphic offset in the header, gives the location of the start of the graphic data in the plane. For the first graphic this is thus zero. +8 8 Name Name of the graphic, padded with nuls ------------------------------------------------ ------------------------------------------------ MASKED BITMAP ENTRIES: (SPRITES) ? 2 Width The width of the graphic divided by 8 +2 2 Height The height of the graphic in pixels; the same rule applies as for unmasked graphics except now we have: +4 2 Loc offset Usually 8, this is the number of bytes 'extra' that must be added to the location to reach the start of the sprite dats. This appears when a sprite is so small, usually 8x8 pixels, that it doesn't fill a multiple of 16 bytes. This will affect ALL sprites after the abberation until another one occurs to fix the shortfall. +6 2 Location Multiplying this by 16 bytes gives the location of the start of the sprite data in the EGA plane. +8 4 Hitbox ul Location of the upper-left corner of the sprite's hitbox or colision rectangle, in pixels, starting from 0,0. First two bytes are the h location and the next two are the v location. Values are the actual value x 256 +12 4 Hitbox br Same as above, for the bottom left corner. +16 16 Name The sprite name, usually includes a number and is usually only 10 bytes long. +32 3*32 Copies Keen uses these entries for smooth movement; each 32 byte entry is a copy of the inital sprite shifted 2 pixels left. Keen automatically generates these copies, but needs the size bytes (The width is always 8 pixels wider than initial.), everything else can be blank. ------------------------------------------------ ------------------------------------------------------------------------------- +------------------------------------------------------------------------------------------+ |Unmasked Graphics (EGALATCH): | +------------------------------------------------------------------------------------------+ This file stores the 8x8 font, 16x16 tiles and unmasked bitmap images in Keen 1-3. Other Keen engine games tend to have these stored seperately. The default structure is exactly as outlined above, but can be easily changed, the font can be placed after the bitmaps for example, by chaning the EGAHEAD file. In Keen 1 this file can be LZW compressed, though this was suh a bad idea that ID abandonded it in latter games. It is a very annoying feature. Uncompressed files are, of course, raw data. Each tile or font character, being regular, is its own seperate entry and these are assumed to be spaced regularly after a given point. Each unmasked bitmap has to be given its own offset, since they can be any size. ------------------------------------------------------------------------------- DEFAULT FILE STRUCTURE: 0 8x Font 256 8x8 font entries taking up 8 bytes each 2048 32x Tiles x rows of 13 16x16 tiles taking up 32 bytes each ? ? Bitmaps x unmasked bitmaps, taking up variable amounts of space This structure is not fixed, and may in fact be altered dramatically by altering the offsets in the header file ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- COMPRESSED: 0 4 Dec size Size of decompressed file, in bytes 4 2 Max dic Size of the maximum length of dictionary code words, in bits 6 ? Data LZW compressed data ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- UNCOMPRESSED: 0 x Pane 1 Raw data for the BLUE plane x x Pane 2 Raw data for the GREEN plane 2x x Pane 3 Raw data for the RED plane 3x x Pane 4 Raw data for the LIGHT plane ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- COMPRESSION: LZW compression is the most difficult Keen compression to understand. What makes it worse is that, while it COMPRESSES on the byte level, it outputs a longer string of BITS. Compressed files thus consist of strings of code bits representing strings of bytes. To understand this, it is best to explain how the file is compressed first. To start off with. the compression program has a 'dictionary'; a table that lists 256 possible bytes and a few other entries; thus, the byte $04 is entry 4 in this table: 0000 - 00 (character) 0001 - 01 (character) ... 00FE - FE (character) 00FF - FF (character) 0100 - Reserved for errors in the Keen implementation 0101 - Reserved for end of compressed data in the Keen implementation 0102 - (not set) 0103 - (not set) ... To start with, the program gets the first byte of a file and stores it in memory as say, (W). It then outputs the 9-bit code for that byte. (So if the first byte was $00, then that is entry 0, (W) = 0 and 9 bits of 0 are written to compressed file.) Now that it has a W it gets serious. It looks for the next byte (N) and looks to see if (W + N) is in the dictionary. If it IS, then it adds (N) to (W) and moves on. If it ISN"T, then (W + N) is added as a new entry to the dictionary, the code for (W + N) is written to the compressed file and (N) is turned into (W). THEN it moves on. So, initially the program has only a few strings in its dictionary, it adds new ones every few bytes and each code stands for only a few bytes. But as the dictionary gets larger, new entries happen less often, and each new entry stands for more bytes. But WAIT! You yell, the dictionary is getting bigger, but the 9 byte codes will only let you have 512 entries, then what? Then the program starts using 10 bit codes, then 11, then 12, allowing a dictionary of 4096 entries. And then..? Well, then it stops, because Keen is slow. Instead of adding new enries to the dictionary, or even replacing old ones, it simply checks to see if a string of bytes is in the dictionary, then writes the code for the longest string that is. Thus, when the dictionary is full, compression rates drop, resulting in less than 50% compression. To decompress, the program starts with an empty dictionary, and starts reading 9 bit codes and deciphers them, adding new entries and rebuilding the dictionary. If you don't get all that, don't worry, few do! But here is a summary: Compression: 1.) Start with our dictionary with 257 entries in it. Codes are 9-bits long 2.) (W) is empty. (Nothing) 3.) Get a new byte (N) -> if (W + N) is in the dictionary, then (W) = (W + N) -> Then goto step 3.) again if (W + N) is NOT in the dictionary, then add (W + C) as a new dictionary entry -> Then write the code for that entry number to output -> Then (W) becomes (N) Goto step 3.) again 4.) When we run out of stuff to compress, we write the code for (W) to output 5.) Then we write the code for entry 257 (STOP!) to output 6.) And stop. => If we run out of 9-bit codes (512 entries) then we start using 10-bit codes; any 9-bit codes written after this will be 10-bits long. Same for 11 and but NOT 12 bit codes, 12 is as far as we go (Notice bytes 5&6 in the file are $000C, or 12) => Once the dictionary is full (4096 entries), then do THIS: 3.) Get a new byte (N) -> if (W + N) is in the dictionary, then (W) = (W + N) -> Then goto step 3.) again if (W + N) is NOT in the dictionary, then write the code for (W) to output -> Then (W) becomes (N) Decompresson: 1.) Start with our dictionary with 257 entries in it. Codes are 9-bits long 2.) (W) is empty. (Nothing) 3.) Read a code (C) -> If the code has an entry in the dictionary, then (W) = (W) + (C) Goto step 3.) again If the code DOESN'T have an entry in the dictionary, then we make a new entry of (W + C) Then write (W + C) as output Then goto 3.) again 4.) If we get the code for entry 257, write the code for (W) as output 5.) And stop. => If we run out of 9-bit codes (512 entries) then we start using 10-bit codes; any 9-bit codes written after this will be 10-bits long. Same for 11 and but NOT 12 bit codes, 12 is as far as we go. => Once the dictionary is full (4096 entries), then do THIS: 3.) Read a code (C) -> If the code has an entry in the dictionary, then (W) = (W) + (C) Goto step 3.) again If the code DOESN'T have an entry in the dictionary, then write (W) as output Then (W) = (C) Goto step 3 again ------------------------------------------------------------------------------- +------------------------------------------------------------------------------------------+ |Sprite Graphics (EGASPRIT): | +------------------------------------------------------------------------------------------+ The EGASPRIT file is exactly the same as the EGALATCH, both in structure and compression; however it consists of FIVE planes and the sprite entries are 'irregular' (Of variable size.) ------------------------------------------------------------------------------- COMPRESSED: 0 4 Dec size Size of decompressed file, in bytes 4 2 Max dic Size of the maximum length of dictionary code words, in bits 6 ? Data LZW compressed data ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- UNCOMPRESSED: 0 x Pane 1 Raw data for the BLUE plane x x Pane 2 Raw data for the GREEN plane 2x x Pane 3 Raw data for the RED plane 3x x Pane 4 Raw data for the LIGHT plane 4x x Pane 5 Raw data for the MASK plane ------------------------------------------------------------------------------- +------------------------------------------------------------------------------------------+ |Keen 1-3 Screens (FINALE.CKX et al) | +------------------------------------------------------------------------------------------+ These files are always compressed, there is not way to stop this. ------------------------------------------------------------------------------- COMPRESSED FILE: 0 4 Uncmp size The uncompressed size of the file in bytes. For Keen this is always $00 $80 $00 $00, or 32'768 bytes. 4 x RLE data Image data, RLE compressed. ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- UNCOMPRESSED: Consists of graphics data in EGA format. That is, b,g,r,l (Blue, green, red, light) planes, as in the EGALATCH. Each plane is 8000 bytes in size, holding 7'808 bytes of image data and 192 blank 'padding bytes' This is read from byte 0 as the EGALATCH graphics, with the image being 320x200 in size. ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- COMPRESSION: This is simple RLE, where data is stored in one of two ways, either as a string of 'literals' to be copied directly, or as a 'repeatable' byte to be repeated n times. The data starts with a signal byte saying what to do. If the byte has a value of 128 or over, then the NEXT (value - 127) bytes are copied to file. If it is below 128, then the NEXT byte is copied (value + 3) times. Compression ends when there is no more file to read. 1. Check if we've reached the end of the file, if yes, stop. Otherwise... 2. Read a byte 3. If the byte value >= 128, then read and output the next (value - 127) bytes 4. If the byte value < 128, then read the next byte and output it (value + 3) times. -------------------------------------------------------------------------------