BLTE

From wowdev
Jump to navigation Jump to search

Any files stored inside TACT and CASC data files are BLTE (likely short for blocktable encoding) encoded, which means before reading anything in the file, first you have to decode it.

It consists of these chunks in the following order:

  • Header
  • ChunkInfo (only if Header.headerSize > 0)
  • Data


To read a BLTE encoded file:

  1. Read the Header chunk
  2. Read the ChunkInfo chunk if Header.headerSize > 0
  3. Read each of the Data chunks and combine them to create the complete file

Note: If there is no ChunkInfo struct, there is just one Data chunk.


  • Header
Offset (Hex) Type Name Description
0x00 char[4] FileSignature "BLTE"
0x04 uint32_t [BE] headerSize Size of the BLTE header (BLTE header = Header + ChunkInfo).


  • ChunkInfoEntry
Offset (Hex) Type Name Description
0x00 uint32_t [BE] compressedSize Compressed size of the chunk (the compression mode byte is included).
0x04 uint32_t [BE] decompressedSize Decompressed size of the chunk.
0x08 char[16] checksum The checksum of the compressed chunk (the compression mode byte is included).


  • ChunkInfo
Offset (Hex) Type Name Description
0x00 uint8_t [BE] flags Flags of some sort.
0x02 uint24_t [BE] chunkCount The number of chunks.
0x04 ChunkInfoEntry[chunkCount] chunks The chunk info for the chunks in the file.


if either flags != 0xF or chunkCount == 0, the file is deemed badly formatted.

  • Data
Offset (Hex) Type Name Description
0x00 char encodingMode Available values: N, Z, F, E
0x01 char[ChunkInfo.compressedSize - 1] data The encoded data.


Example implementation as Binary Template can be found here: BLTE-Template


Encoding modes:

  • N: Plain data.
  • Z: Zlib encoded data.
  • 4: lz4hc encoded data.

The encoded data is preceded by the two header bytes as specified by the zlib RFC [1] (Section 2.2). 78 DA most of the time.

Reading bits left-to-right:

var compressionInfo = reader.ReadBits(4);
var compressionMethod = reader.ReadBits(4);
var flevel = reader.ReadBits(2);
var fdict = reader.ReadBit();
var fcheck = reader.ReadBits(5);

In .NET you can basically skip these bytes and use DeflateStream. Make sure to wrap around the chunk so it internally does not try to consume bytes from the following chunk.

  • F: Recursively encoded BLTE data.
  • E: encrypted: one of salsa20, arc4, rc4.
struct 
{
  unsigned char key_name_length;              // 0x8
  unsigned char key_name[key_name_length];
  unsigned char IV_length;                    // 0x4
  unsigned char IV[IV_length];
  char type; // 'S': salsa20, 'A': arc4
} E_chunk;

key_name is resolved by client to the actual key. keys are distributed via keyrings and some keys are hardcoded.

Encoding Specification (ESpec)

ESpecs are string-based representations of the encoding of BLTE-encoded data files that serve as recipes for the patcher to produce a binary-identical encoded output file (as patching operates on the unencoded data). The information they contain is redundant with the information in the BLTE header of the file itself, but due to ESpec shorthand notations multiple ESpecs can encode the same output for the same input (e.g. "b:256*2=z", "b:{256=z,256=z}", etc.), and the same ESpec can result in different output block configurations for inputs of different sizes (e.g. 500 bytes of input results in 2 blocks and 600 bytes 3 blocks with "b:256*=z"). They are used most extensively in Encoding files.

An example parser can be found at this gist.

The strings are not whitespace-tolerant. They use the following EBNF grammar (concatenations are omitted), where e-spec is a top level string:

e-spec          =  ( 'n' )
                |  ( 'z' [ ':' ( zip-level | '{' zip-level ',' zip-bits '}' ) ] )
                |  ( 'e' ':' '{' encryption-key ',' encryption-iv ',' e-spec '}' )
                |  ( 'b' ':' ( final-subchunk | '{' ( [{block-subchunk ','}] final-subchunk ) '}' ) )
                ;

block-subchunk  =  block-size-spec '=' e-spec ;
block-size-spec =  block-size [ '*' block-count ] ; (* block-count (1 if unspecified) blocks of block-size bytes *)
block-size      =  number [ block-unit ] ;
block-unit      =  'K'                          (* count * 2^10 *)
                |  'M'                          (* count * 2^20 *)
                ;
block-count     =  number ;
final-subchunk  =  final-size-spec '=' e-spec ;
final-size-spec =  block-size-spec
                |  block-size '*'               (* greedy spec of block-size blocks (last block <= block-size) *)
                |  '*'                          (* greedy block *)
                ;

zip-level       = number
                ;
zip-bits        = number | ( 'm' 'p' 'q' )
                ;

encryption-key  =  ? eight byte upper-hex encoded key name ?
                ;
encryption-iv   =  ? four byte hex-string IV value ?
                ;
  • where a greedy final-size-spec consumes all remaining bytes in the parent block or file
    • "all remaining bytes" may be 0, producing no blocks
  • where zip-level defaults to 9 and zip-bits to 15 if not given
    • zip-bits with value 'mpq' means that zip-bits is calculated using based on chunk size like in MPQ:
      • if size <= 0x200, then zip-bits is 9
      • if size <= 0x400, then zip-bits is 10
      • if size <= 0x800, then zip-bits is 11
      • if size <= 0x1000, then zip-bits is 12
      • if size <= 0x2000, then zip-bits is 13
      • if size <= 0x4000, then zip-bits is 14
      • otherwise zip-bits is 15
      • Technically, for block sizes <= 0x100, zip-bits should be 8, but this seems to not be the case in BLTE (likely Zlib doesn't support a window size of 8 bits).

Examples

b:{164=z,16K*565=z,1656=z,140164=z}
  • blocks on top level
    • 164 bytes of zip with level=9, bits=15
    • 565 blocks of 16Kb zip chunks with level=9, bits=15
    • 1656 bytes of zip with level=9, bits=15
    • 140164 of zip with level=9, bits=15
b:{1768=z,66443=n}
  • blocks on top level
    • 1768 bytes of zip with level=9, bits=15
    • 66443 bytes of raw data
b:{256K*=e:{237DA26C65073F42,06FC152E,z}}
  • blocks on top level
    • unspecified count of 256 kb chunks encrypted with key 237DA26C65073F42 and IV 06FC152E
      • containing zip data with level=9, bits=15
      • last chunk <= 256 kb
z
  • zipped data on top level with level=9, bits=15
b:{22=n,31943=z,211232=n,27037696=n,138656=n,17747968=n,*=z}
  • blocks on top level
    • 22 bytes of raw data
    • 31943 bytes of zip with level=9, bits=15
    • 211232 bytes of raw data
    • 27037696 bytes of raw data
    • 138656 bytes of raw data
    • 17747968 bytes of raw data
    • an unspecified amount of zipped data with level=9, bits=15
b:{16K*=z:{6,mpq}}
  • blocks on top level
    • unspecified count of 16 kb chunks of zipped data with level=6, bits=14 (see MPQ behavior above)
      • last chunk <= 16 kb