Difference between revisions of "BLS"

From wowdev
Jump to navigation Jump to search
Line 20: Line 20:
 
  ''/*0x04*/'' uint32_t version; // version, 0x10003 in WotLK, 0x10004 in MoP/WoD
 
  ''/*0x04*/'' uint32_t version; // version, 0x10003 in WotLK, 0x10004 in MoP/WoD
 
  ''/*0x08*/'' uint32_t permutationCount; // (from old definition)
 
  ''/*0x08*/'' uint32_t permutationCount; // (from old definition)
  ''/*0x0c*/'' uint32_t unknown;
+
  ''/*0x0c*/'' uint32_t nShaders; // the expected number of shaders in this file (this is the number
  ''/*0x10*/'' uint32_t unknown2;
+
// of distinct shaders that will be found if you inspect the file after decompression)
  ''/*0x14*/'' uint32_t unknown3;
+
  ''/*0x10*/'' uint32_t ofsCompressedChunks; // offset to array of offsets to compressed chunks
  ''/*0x18*/'' uint32_t ofsCompressedData; // offset to the start of the compressed (zlib) data bytes
+
  ''/*0x14*/'' uint32_t nCompressedChunks; // number of compressed chunks
 +
  ''/*0x18*/'' uint32_t ofsCompressedData; // offset to the start of the compressed (zlib) data  
 +
// bytes (all offsets from the array above are offset by this much)
  
This header is present at 0x00 in the BLS file, there is a significant amount of unknown data between the end of this header and the start of the compressed data section that is unexplored. In order to continue reading the data from a BLS file the data at ofsCompressedData must be decompressed with zlib inflate. The data at ofsCompressedData should begin ''78 9C'', the zlib magic header (for default compression).
+
This header is present at 0x00 in the BLS file, there is a significant amount of unknown data between the end of this header and the start of the compressed data section that is unexplored. In order to continue reading the data from a BLS file the data at ofsCompressedData must be decompressed with zlib inflate. There are multiple chunks of separately compressed zlib data (all sections start with ''78 9C'', the zlib magic header for default compression). To decompress the file, take the ofsCompressedData offset, then for each chunk read the offset from ofsCompressedChunks:
  
Once decompressed, the old WotLK structure is present inside the decompressed data.
+
(unsigned char *)ptr + header->ofsCompressedData + ofsCompressedChunks[index]
 +
 
 +
Each decompressed chunk should be appended to the last. Some of the shader text spans multiple chunks. The offsets just help with finding the start of the zlib header for each chunk. Inside the decompressed data, the old WotLK block header is present, along with a shader preamble (WoD+).
  
 
===Shader Block===
 
===Shader Block===
Line 47: Line 51:
 
The length field dictates where this block ends, in order to iterate over all blocks in BLS file you would start at the first block, read ''len'' and then skip to the next block using that. In MoP, the shader text was present immediately after the BLSBlock structure. However, in WoD there is a significant amount of new data between the end of this structure and the beginning of the shader text. Experimentally, there appears to be a header in front of this data that holds the offset to the text itself.
 
The length field dictates where this block ends, in order to iterate over all blocks in BLS file you would start at the first block, read ''len'' and then skip to the next block using that. In MoP, the shader text was present immediately after the BLSBlock structure. However, in WoD there is a significant amount of new data between the end of this structure and the beginning of the shader text. Experimentally, there appears to be a header in front of this data that holds the offset to the text itself.
  
  struct BLSBlock2 {
+
  struct BLSPreamble {
  ''/*0x00*/'' uint32_t header;
+
  ''/*0x00*/'' char magic[4]; // in GLSL shaders this is "GLS3"
  ''/*0x04*/'' uint32_t unknown[4];
+
  ''/*0x04*/'' uint32_t unknown[4]; // these look like flag fields, I'd guess they identify the capabilities of a given shader variant
  ''/*0x14*/'' uint32_t offset;
+
  ''/*0x14*/'' uint32_t offset; // the offset from the block start to the shader text
 
  }
 
  }
  

Revision as of 04:27, 3 May 2015

BLS is the container format that stores the GPU shaders used to render the world. In WoD, there are now four different shader types under the Shaders\* directory.

  • Vertex
    • (versions: arbvp1, vp40, glvs_150, ps_2_0, ps_3_0, ps_4_0, ps_5_0)
  • Fragment
    • (versions: arbfp1, fp40, glfs_150, ps_2_0, ps_3_0, ps_4_0, ps_5_0)
  • Geometry
    • (versions: glgs_150, gs_4_0, gs_5_0)
  • Hull/Domain (equivalent to Tessellation in OpenGL, except WoW currently only has these shaders for DX)
    • (versions: ds_5_0/hs_50)

In MoP+, the BLS format changed significantly (the version increased from 1.3 to 1.4), a new header has been introduced and all of the shader text inside the BLS files is now contained inside compressed chunks. Very little appears to be known about the structure of the format itself, and the meaning of the various fields. Below is enough information to be able to extract the compressed chunks and read out shader text from the various shader components. Additionally, the Shaders\Effects folder contains WFX files that allow WoW to map shader types from models onto the various shader components (although, there are occasions in the M2 source code where shaders are picked in code).

BLS Format

The data that used to be present in this page seemed to be quite out of data and reflected the BLS format as of WotLK (or earlier). These updated structures can help write code to read in BLS files as of MoP/WoD but contain very little by way of analysis on most of the fields themselves.

Header

struct BLSHeader {
/*0x00*/	char magic[4];			// FourCC-style magic header, "GXVS", "GXPS", etc. Character order reversed in-file.
/*0x04*/	uint32_t version;		// version, 0x10003 in WotLK, 0x10004 in MoP/WoD
/*0x08*/	uint32_t permutationCount;	// (from old definition)
/*0x0c*/	uint32_t nShaders;		// the expected number of shaders in this file (this is the number
						// of distinct shaders that will be found if you inspect the file after decompression)
/*0x10*/	uint32_t ofsCompressedChunks;	// offset to array of offsets to compressed chunks
/*0x14*/	uint32_t nCompressedChunks;	// number of compressed chunks
/*0x18*/	uint32_t ofsCompressedData;	// offset to the start of the compressed (zlib) data 
						// bytes (all offsets from the array above are offset by this much)

This header is present at 0x00 in the BLS file, there is a significant amount of unknown data between the end of this header and the start of the compressed data section that is unexplored. In order to continue reading the data from a BLS file the data at ofsCompressedData must be decompressed with zlib inflate. There are multiple chunks of separately compressed zlib data (all sections start with 78 9C, the zlib magic header for default compression). To decompress the file, take the ofsCompressedData offset, then for each chunk read the offset from ofsCompressedChunks:

(unsigned char *)ptr + header->ofsCompressedData + ofsCompressedChunks[index]

Each decompressed chunk should be appended to the last. Some of the shader text spans multiple chunks. The offsets just help with finding the start of the zlib header for each chunk. Inside the decompressed data, the old WotLK block header is present, along with a shader preamble (WoD+).

Shader Block

The decompressed BLS data is formatted in the same format that old WotLK shaders used, with the exception of WoD shaders that have a new (important) structure that is needed to find the start of the shader text. It no longer appears to be true that the top-level permutationCount field represents the number of shaders present in the inner BLS data.

Note: all the instances of BLSBlock are padded to the next nearest 4-byte alignment.

struct BLSBlock {
/*0x00*/	uint32_t flags;
/*0x04*/	uint32_t flags2;
/*0x08*/	uint32_t unknown;
/*0x0c*/	uint32_t unknown2;
/*0x10*/	uint32_t unknown3;
/*0x14*/	uint32_t unknown4;
/*0x18*/	uint32_t unknown5;
/*0x1c*/	uint32_t len;
}

The length field dictates where this block ends, in order to iterate over all blocks in BLS file you would start at the first block, read len and then skip to the next block using that. In MoP, the shader text was present immediately after the BLSBlock structure. However, in WoD there is a significant amount of new data between the end of this structure and the beginning of the shader text. Experimentally, there appears to be a header in front of this data that holds the offset to the text itself.

struct BLSPreamble {
/*0x00*/	char magic[4];		// in GLSL shaders this is "GLS3"
/*0x04*/	uint32_t unknown[4];	// these look like flag fields, I'd guess they identify the capabilities of a given shader variant
/*0x14*/	uint32_t offset;	// the offset from the block start to the shader text
}

The shader text starts at offset of (BLSBlock + BLSBlock2.offset). The shader text is then present in the format used by the particular graphics API used in that subdirectory. (ie. glvs/glfs == OpenGL GLSL, arbvp1/arbfp1 == OpenGL ARB assembly, vs_*/ps_* == DirectX, etc).

Note: in 6.1.2 the OpenGL GLSL (glvs/glfs/etc) shaders appear to have been recompiled with a newer version of the HLSL cross-compiler and they are exceptionally informative. In particular, there are named uniform (constant) blocks that indicate the meaning of the variables.