| author | ecalot
<ecalot> 2004-07-09 00:56:47 UTC |
| committer | ecalot
<ecalot> 2004-07-09 00:56:47 UTC |
| parent | 049824e0568560044f5ac2b09a6587914184b273 |
| FP/doc/FormatSpecifications | +122 | -42 |
| FP/doc/FormatSpecifications.tex | +122 | -42 |
diff --git a/FP/doc/FormatSpecifications b/FP/doc/FormatSpecifications index a75a0a3..886ddca 100644 --- a/FP/doc/FormatSpecifications +++ b/FP/doc/FormatSpecifications @@ -1,38 +1,40 @@ Table of Contents - +~~~~~ ~~ ~~~~~~~~ 1. Preamble .......................................................... 31 -2. Introduction ...................................................... 35 +2. Introduction ...................................................... 36 3. Primitives ........................................................ 54 -3.1. DAT reading and writing primitives .............................. 58 -3.2. DAT reading primitives .......................................... 70 -3.3. DAT writing primitives .......................................... 77 +3.1. DAT reading and writing primitives .............................. 61 +3.2. DAT reading primitives .......................................... 73 +3.3. DAT writing primitives .......................................... 80 4. File Specifications ............................................... 85 -4.1. General file specs, index and checksums ......................... 87 -4.2. Images ......................................................... 152 -4.2.1 Headers ....................................................... 155 -4.2.2 Algorithms .................................................... 178 -4.2.2.1 Run length encoding (RLE) ................................... 194 -4.2.2.2 Custom LZ ................................................... 205 -4.3. Palettes ....................................................... 208 -4.4. Levels ......................................................... 215 -4.4.1 Unknown blocks ................................................ 243 -4.4.2 Room mapping .................................................. 260 -4.4.3 Room linking .................................................. 366 -4.4.4 Guard handling ................................................ 382 -4.4.5 Starting Position ............................................. 409 -4.4.6 Door events ................................................... 423 -4.5. Digital Waves .................................................. 467 -4.6. Midi music ..................................................... 477 -4.7. Internal PC Speaker ............................................ 480 -4.8. Binary files ................................................... 485 -5. Credits .......................................................... 491 -6. License .......................................................... 503 +4.1. General file specs, index and checksums ......................... 91 +4.2. Images ......................................................... 156 +4.2.1 Headers ....................................................... 159 +4.2.2 Algorithms .................................................... 182 +4.2.2.1 Run length encoding (RLE) ................................... 198 +4.2.2.2 LZ variant (LZG) ............................................ 209 +4.3. Palettes ....................................................... 286 +4.4. Levels ......................................................... 293 +4.4.1 Unknown blocks ................................................ 321 +4.4.2 Room mapping .................................................. 338 +4.4.3 Room linking .................................................. 444 +4.4.4 Guard handling ................................................ 460 +4.4.5 Starting Position ............................................. 487 +4.4.6 Door events ................................................... 501 +4.5. Digital Waves .................................................. 545 +4.6. Midi music ..................................................... 555 +4.7. Internal PC Speaker ............................................ 558 +4.8. Binary files ................................................... 563 +5. Credits .......................................................... 569 +6. License .......................................................... 582 1. Preamble + ~~~~~~~~ This file was written thanks to the reverse engineering made by several people, see the credits section. 2. Introduction + ~~~~~~~~~~~~ There are two versions of the DAT file format: DAT v1.0 used in POP 1.x and DAT v2.0 used in POP 2. In this document we will specify DAT v1.0. @@ -52,6 +54,7 @@ Table of Contents is another resource and must be shared by a group of images. 3. Primitives + ~~~~~~~~~~ This section shows how the PR dat handling primitives works, this library is useful to access resources without having to worry about the format. @@ -83,6 +86,7 @@ Table of Contents char* backupExtension); 4. File Specifications + ~~~~ ~~~~~~~~~~~~~~ 4.1. General file specs, index and checksums All DAT files has an index, this index has a number of items count and @@ -179,8 +183,8 @@ Table of Contents RAW_LR means that the data wasn't compressed, it is used for small images the format is saved from left to right (LR) serializing a line to the next integer byte if necessary. In case the image was 16 colors - two pixels per byte (4bpp) will be used, in case the image was 2 - colors, 8 pixels per byte (1bpp) will be used. + two pixels per byte (4bpp) will be used, in case the image was 2 + colors, 8 pixels per byte (1bpp) will be used. RLE_LR has a Run length encoding (RLE) algorithm, after uncompressed the image can be read as a RAW_LR. RLE_UD is the same as RLE_LR except that after uncompressed the image must @@ -188,7 +192,7 @@ Table of Contents LZG_LR has any kind of variant of the LZ77 algorithm (the sliding windows algorithm), here we named it LZG in honor of Lance Groody, the original coder. - After uncompressed it may be handled as RAW_LR + After uncompressed it may be handled as RAW_LR. LZG_UD Uses LZG compression but is drawn from top to bottom as RLE_UD 4.2.2.1 Run length encoding (RLE) @@ -202,8 +206,82 @@ Table of Contents If you reach a control byte but the image size is passed, then you have completed the image. -4.2.2.2 Custom LZ - Use the source, Luke. +4.2.2.2 LZ variant (LZG) + This is a simplified algorithm explaination: + + There is a slide window initialized with zeros. + The first byte is a maskbyte. + For each of the 8 bits in the maskbyte the next actions must be performed: + If the bit is 1 print the next unread byte to the slide window + If the bit is a zero read the next two bytes as control bytes with the + following format: + - 10 bits for the slide position (S). Add 66 to this number. + - 6 bits for the repetition number (R). Add 3 to this number. + Then print the next R bytes starting with the S'th byte of the slide + window. + After all the maskbyte is read and processed, the next byte is another + maskbyte. Use the same procedure to finish uncompressing the file. + + This version of the algorithm is limited to 1024 bytes due to slide window + size. In case you want to know the full algorithm and see how it works for + bigger images, use the source, Luke. + + This is the uncompression function source: + (note that this is part of PR that is under the GPL license) + + /* A big number */ + #define MAX_MOD_SIZE_IN_LZG 32001 + + /* modulus to be used in the 10 bits of the algorithm */ + #define MAX_MXD_SIZE_IN_LZG 0x400 + + /* LZG expansion algorithm sub function */ + unsigned char popBit(unsigned char *byte) { + unsigned char bit=(unsigned char)((*byte)&1); + (*byte)>>=1; + return bit; + } + + /* Expands LZ Groody algorithm. This is the core of PR */ + int expandLzg(const unsigned char* array, int arraySize, + unsigned char* image, int imageSize) { + char k; + int location,h,cursor=0,pos=0; + unsigned char maskbyte,rep; + + /* img is a unsigned char[] sized 0x400 */ + /* clean output garbage */ + for(location=0;location<MAX_MOD_SIZE_IN_LZG;img[location]=0,location++); + + /* main loop */ + while (cursor<imageSize) { + maskbyte=array[pos++]; + for (k=8;k&&(cursor<imageSize);k--) { + if (popBit(&maskbyte)) { + img[cursor++]=array[pos++]; + } else { + /* + * 10 bits for the slide position (S). Add 66 to this number. + * 6 bits for the repetition number (R). Add 3 to this number. + */ + location=66+(((rep=array[pos])&3)<<8)+(unsigned char)array[pos+1]; + pos+=2; + rep=(unsigned char)((rep>>2)+3); + + /* Here is the difference between big and small images */ + while (rep--) { + h=cursor/MAX_MXD_SIZE_IN_LZG- + ((location%MAX_MXD_SIZE_IN_LZG)>(cursor%MAX_MXD_SIZE_IN_LZG)); + img[cursor++]=img[ + ((h<0)?0:h)*MAX_MXD_SIZE_IN_LZG+ + (location++)%MAX_MXD_SIZE_IN_LZG + ]; + } + } + } + } + return ((pos==arraySize)&(cursor==imageSize)); /* 0 is ERROR, 1 is SUCCESS */ + } 4.3. Palettes Palettes have 100 bytes allways, after 4 bytes from the beginning the @@ -453,12 +531,12 @@ Table of Contents Let's define: Screen as S and it is a number from 1 to 24 (5 bits) S = s1 s2 s3 s4 s5 - where sn is the bit n of the binary representation of S - Location as L and is a number from 0 to 29 (5 bits) + where sn is the bit n of the binary representation of S + Location as L and is a number from 0 to 29 (5 bits) L = l1 l2 l3 l4 l5 - where ln is the bit n of the binary representation of L + where ln is the bit n of the binary representation of L This number is according to the location format specifications. - Trigger-next as T and is a 0 or 1 (1 bit) + Trigger-next as T and is a 0 or 1 (1 bit) T = t1 Byte I has the form: t1 s4 s5 l1 l2 l3 l4 l5 @@ -489,23 +567,25 @@ Just raw sound format. 5. Credits + ~~~~~~~ This document: - Writing Enrique Calot + Writing . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot Reverse Engineering: - Indexes Enrique Calot - Levels Enrique Calot - Images Tammo Jan Dijkema - RLE Compression Tammo Jan Dijkema - LZG Compression Anke Balderer - Sounds Christian Lundheim + Indexes . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot + Levels . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot + Images . . . . . . . . . . . . . . . . . . . . . . . Tammo Jan Dijkema + RLE Compression . . . . . . . . . . . . . . . . . . . Tammo Jan Dijkema + LZG Compression . . . . . . . . . . . . . . . . . . . . . Anke Balderer + Sounds . . . . . . . . . . . . . . . . . . . . . . . Christian Lundheim 6. License + ~~~~~~~ Copyright (c) 2004 Princed Project Team Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled - "GNU Free Documentation License". + "GNU Free Documentation License". diff --git a/FP/doc/FormatSpecifications.tex b/FP/doc/FormatSpecifications.tex index a75a0a3..886ddca 100644 --- a/FP/doc/FormatSpecifications.tex +++ b/FP/doc/FormatSpecifications.tex @@ -1,38 +1,40 @@ Table of Contents - +~~~~~ ~~ ~~~~~~~~ 1. Preamble .......................................................... 31 -2. Introduction ...................................................... 35 +2. Introduction ...................................................... 36 3. Primitives ........................................................ 54 -3.1. DAT reading and writing primitives .............................. 58 -3.2. DAT reading primitives .......................................... 70 -3.3. DAT writing primitives .......................................... 77 +3.1. DAT reading and writing primitives .............................. 61 +3.2. DAT reading primitives .......................................... 73 +3.3. DAT writing primitives .......................................... 80 4. File Specifications ............................................... 85 -4.1. General file specs, index and checksums ......................... 87 -4.2. Images ......................................................... 152 -4.2.1 Headers ....................................................... 155 -4.2.2 Algorithms .................................................... 178 -4.2.2.1 Run length encoding (RLE) ................................... 194 -4.2.2.2 Custom LZ ................................................... 205 -4.3. Palettes ....................................................... 208 -4.4. Levels ......................................................... 215 -4.4.1 Unknown blocks ................................................ 243 -4.4.2 Room mapping .................................................. 260 -4.4.3 Room linking .................................................. 366 -4.4.4 Guard handling ................................................ 382 -4.4.5 Starting Position ............................................. 409 -4.4.6 Door events ................................................... 423 -4.5. Digital Waves .................................................. 467 -4.6. Midi music ..................................................... 477 -4.7. Internal PC Speaker ............................................ 480 -4.8. Binary files ................................................... 485 -5. Credits .......................................................... 491 -6. License .......................................................... 503 +4.1. General file specs, index and checksums ......................... 91 +4.2. Images ......................................................... 156 +4.2.1 Headers ....................................................... 159 +4.2.2 Algorithms .................................................... 182 +4.2.2.1 Run length encoding (RLE) ................................... 198 +4.2.2.2 LZ variant (LZG) ............................................ 209 +4.3. Palettes ....................................................... 286 +4.4. Levels ......................................................... 293 +4.4.1 Unknown blocks ................................................ 321 +4.4.2 Room mapping .................................................. 338 +4.4.3 Room linking .................................................. 444 +4.4.4 Guard handling ................................................ 460 +4.4.5 Starting Position ............................................. 487 +4.4.6 Door events ................................................... 501 +4.5. Digital Waves .................................................. 545 +4.6. Midi music ..................................................... 555 +4.7. Internal PC Speaker ............................................ 558 +4.8. Binary files ................................................... 563 +5. Credits .......................................................... 569 +6. License .......................................................... 582 1. Preamble + ~~~~~~~~ This file was written thanks to the reverse engineering made by several people, see the credits section. 2. Introduction + ~~~~~~~~~~~~ There are two versions of the DAT file format: DAT v1.0 used in POP 1.x and DAT v2.0 used in POP 2. In this document we will specify DAT v1.0. @@ -52,6 +54,7 @@ Table of Contents is another resource and must be shared by a group of images. 3. Primitives + ~~~~~~~~~~ This section shows how the PR dat handling primitives works, this library is useful to access resources without having to worry about the format. @@ -83,6 +86,7 @@ Table of Contents char* backupExtension); 4. File Specifications + ~~~~ ~~~~~~~~~~~~~~ 4.1. General file specs, index and checksums All DAT files has an index, this index has a number of items count and @@ -179,8 +183,8 @@ Table of Contents RAW_LR means that the data wasn't compressed, it is used for small images the format is saved from left to right (LR) serializing a line to the next integer byte if necessary. In case the image was 16 colors - two pixels per byte (4bpp) will be used, in case the image was 2 - colors, 8 pixels per byte (1bpp) will be used. + two pixels per byte (4bpp) will be used, in case the image was 2 + colors, 8 pixels per byte (1bpp) will be used. RLE_LR has a Run length encoding (RLE) algorithm, after uncompressed the image can be read as a RAW_LR. RLE_UD is the same as RLE_LR except that after uncompressed the image must @@ -188,7 +192,7 @@ Table of Contents LZG_LR has any kind of variant of the LZ77 algorithm (the sliding windows algorithm), here we named it LZG in honor of Lance Groody, the original coder. - After uncompressed it may be handled as RAW_LR + After uncompressed it may be handled as RAW_LR. LZG_UD Uses LZG compression but is drawn from top to bottom as RLE_UD 4.2.2.1 Run length encoding (RLE) @@ -202,8 +206,82 @@ Table of Contents If you reach a control byte but the image size is passed, then you have completed the image. -4.2.2.2 Custom LZ - Use the source, Luke. +4.2.2.2 LZ variant (LZG) + This is a simplified algorithm explaination: + + There is a slide window initialized with zeros. + The first byte is a maskbyte. + For each of the 8 bits in the maskbyte the next actions must be performed: + If the bit is 1 print the next unread byte to the slide window + If the bit is a zero read the next two bytes as control bytes with the + following format: + - 10 bits for the slide position (S). Add 66 to this number. + - 6 bits for the repetition number (R). Add 3 to this number. + Then print the next R bytes starting with the S'th byte of the slide + window. + After all the maskbyte is read and processed, the next byte is another + maskbyte. Use the same procedure to finish uncompressing the file. + + This version of the algorithm is limited to 1024 bytes due to slide window + size. In case you want to know the full algorithm and see how it works for + bigger images, use the source, Luke. + + This is the uncompression function source: + (note that this is part of PR that is under the GPL license) + + /* A big number */ + #define MAX_MOD_SIZE_IN_LZG 32001 + + /* modulus to be used in the 10 bits of the algorithm */ + #define MAX_MXD_SIZE_IN_LZG 0x400 + + /* LZG expansion algorithm sub function */ + unsigned char popBit(unsigned char *byte) { + unsigned char bit=(unsigned char)((*byte)&1); + (*byte)>>=1; + return bit; + } + + /* Expands LZ Groody algorithm. This is the core of PR */ + int expandLzg(const unsigned char* array, int arraySize, + unsigned char* image, int imageSize) { + char k; + int location,h,cursor=0,pos=0; + unsigned char maskbyte,rep; + + /* img is a unsigned char[] sized 0x400 */ + /* clean output garbage */ + for(location=0;location<MAX_MOD_SIZE_IN_LZG;img[location]=0,location++); + + /* main loop */ + while (cursor<imageSize) { + maskbyte=array[pos++]; + for (k=8;k&&(cursor<imageSize);k--) { + if (popBit(&maskbyte)) { + img[cursor++]=array[pos++]; + } else { + /* + * 10 bits for the slide position (S). Add 66 to this number. + * 6 bits for the repetition number (R). Add 3 to this number. + */ + location=66+(((rep=array[pos])&3)<<8)+(unsigned char)array[pos+1]; + pos+=2; + rep=(unsigned char)((rep>>2)+3); + + /* Here is the difference between big and small images */ + while (rep--) { + h=cursor/MAX_MXD_SIZE_IN_LZG- + ((location%MAX_MXD_SIZE_IN_LZG)>(cursor%MAX_MXD_SIZE_IN_LZG)); + img[cursor++]=img[ + ((h<0)?0:h)*MAX_MXD_SIZE_IN_LZG+ + (location++)%MAX_MXD_SIZE_IN_LZG + ]; + } + } + } + } + return ((pos==arraySize)&(cursor==imageSize)); /* 0 is ERROR, 1 is SUCCESS */ + } 4.3. Palettes Palettes have 100 bytes allways, after 4 bytes from the beginning the @@ -453,12 +531,12 @@ Table of Contents Let's define: Screen as S and it is a number from 1 to 24 (5 bits) S = s1 s2 s3 s4 s5 - where sn is the bit n of the binary representation of S - Location as L and is a number from 0 to 29 (5 bits) + where sn is the bit n of the binary representation of S + Location as L and is a number from 0 to 29 (5 bits) L = l1 l2 l3 l4 l5 - where ln is the bit n of the binary representation of L + where ln is the bit n of the binary representation of L This number is according to the location format specifications. - Trigger-next as T and is a 0 or 1 (1 bit) + Trigger-next as T and is a 0 or 1 (1 bit) T = t1 Byte I has the form: t1 s4 s5 l1 l2 l3 l4 l5 @@ -489,23 +567,25 @@ Just raw sound format. 5. Credits + ~~~~~~~ This document: - Writing Enrique Calot + Writing . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot Reverse Engineering: - Indexes Enrique Calot - Levels Enrique Calot - Images Tammo Jan Dijkema - RLE Compression Tammo Jan Dijkema - LZG Compression Anke Balderer - Sounds Christian Lundheim + Indexes . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot + Levels . . . . . . . . . . . . . . . . . . . . . . . . . Enrique Calot + Images . . . . . . . . . . . . . . . . . . . . . . . Tammo Jan Dijkema + RLE Compression . . . . . . . . . . . . . . . . . . . Tammo Jan Dijkema + LZG Compression . . . . . . . . . . . . . . . . . . . . . Anke Balderer + Sounds . . . . . . . . . . . . . . . . . . . . . . . Christian Lundheim 6. License + ~~~~~~~ Copyright (c) 2004 Princed Project Team Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled - "GNU Free Documentation License". + "GNU Free Documentation License".