BLS: Difference between revisions

From wowdev
Jump to navigation Jump to search
No edit summary
No edit summary
Line 12: Line 12:
In MoP+, the BLS format changed significantly (the version increased from 1.3 to 1.4), a new header has been introduced and all of the shader text inside the BLS files is now contained inside compressed chunks. Very little appears to be known about the structure of the format itself, and the meaning of the various fields. Below is enough information to be able to extract the compressed chunks and read out shader text from the various shader components. Additionally, the Shaders\Effects folder contains [[WFX]] files that allow WoW to map shader types from models onto the various shader components (although, there are occasions in the M2 source code where shaders are picked in code).
In MoP+, the BLS format changed significantly (the version increased from 1.3 to 1.4), a new header has been introduced and all of the shader text inside the BLS files is now contained inside compressed chunks. Very little appears to be known about the structure of the format itself, and the meaning of the various fields. Below is enough information to be able to extract the compressed chunks and read out shader text from the various shader components. Additionally, the Shaders\Effects folder contains [[WFX]] files that allow WoW to map shader types from models onto the various shader components (although, there are occasions in the M2 source code where shaders are picked in code).


==Header==
==BLS Format==
*Main header (0xC bytes)
The data that used to be present in this page seemed to be quite out of data and reflected the BLS format as of WotLK (or earlier). These updated structures can help write code to read in BLS files as of MoP/WoD but contain very little by way of analysis on most of the fields themselves.
This header is in all files - pixel and vertex shaders in all profiles.
 
===Header===
  struct BLSHeader {
  struct BLSHeader {
  ''/*0x00*/'' char[4] magix; // in reverse character order: "SVXG" in case of a vertex shader, "SPXG" in case of a fragment shader
  ''/*0x00*/'' char magic[4]; // FourCC-style magic header, "GXVS", "GXPS", etc. Character order reversed in-file.
  ''/*0x04*/'' uint32 version; // Always 0x10003 - version 1.3 of format
  ''/*0x04*/'' uint32_t version; // version, 0x10003 in WotLK, 0x10004 in MoP/WoD
  ''/*0x08*/'' uint32 permutationCount;
  ''/*0x08*/'' uint32_t permutationCount; // (from old definition)
  ''/*0x0C*/''
''/*0x0c*/'' uint32_t unknown;
  };
  ''/*0x10*/'' uint32_t unknown2;
  ''/*0x14*/'' uint32_t unknown3;
''/*0x18*/'' uint32_t ofsCompressedData; // offset to the start of the compressed (zlib) data bytes
 
This header is present at 0x00 in the BLS file, there is a significant amount of unknown data between the end of this header and the start of the compressed data section that is unexplored. In order to continue reading the data from a BLS file the data at ofsCompressedData must be decompressed with zlib inflate. The data at ofsCompressedData should begin ''78 9C'', the zlib magic header (for default compression).
 
Once decompressed, the old WotLK structure is present inside the decompressed data.
 
===Shader Block===
The decompressed BLS data is formatted in the same format that old WotLK shaders used, with the exception of WoD shaders that have a new (important) structure that is needed to find the start of the shader text. It no longer appears to be true that the top-level ''permutationCount'' field represents the number of shaders present in the inner BLS data.
 
'''Note:''' all the instances of ''BLSBlock'' are padded to the next nearest 4-byte alignment.


==Blocks==
There are permutationCount blocks of the following structure. They are padded to 0x*0, 0x*4, 0x*8 and 0x*C.
  struct BLSBlock {
  struct BLSBlock {
  ''/*0x00*/'' DWORD flags0; // seen: 0x3FE80 in pixel shaders; 0x1A0F in vertex shaders. there may be more ..
  ''/*0x00*/'' uint32_t flags;
  ''/*0x04*/'' DWORD flags4; // seen: 0x200 in pixel shaders; 0x3FEC1 in vertex shaders (there may be more ..)
''/*0x04*/'' uint32_t flags2;
  ''/*0x08*/'' DWORD unk8; // Never seen anything in here.
  ''/*0x08*/'' uint32_t unknown;
  ''/*0x0C*/'' uint32 size; // Tells you how large the block actually is.
''/*0x0c*/'' uint32_t unknown2;
  ''/*0x10*/'' char data[size]; // In whatever format defined.
  ''/*0x10*/'' uint32_t unknown3;
  ''/*----*/''
''/*0x14*/'' uint32_t unknown4;
  };
  ''/*0x18*/'' uint32_t unknown5;
''/*0x1c*/'' uint32_t len;
}
 
The length field dictates where this block ends, in order to iterate over all blocks in BLS file you would start at the first block, read ''len'' and then skip to the next block using that. In MoP, the shader text was present immediately after the BLSBlock structure. However, in WoD there is a significant amount of new data between the end of this structure and the beginning of the shader text. Experimentally, there appears to be a header in front of this data that holds the offset to the text itself.
 
struct BLSBlock2 {
  ''/*0x00*/'' uint32_t header;
''/*0x04*/'' uint32_t unknown[4];
  ''/*0x14*/'' uint32_t offset;
  }
 
The shader text starts at offset of (BLSBlock + BLSBlock2.offset). The shader text is then present in the format used by the particular graphics API used in that subdirectory. (ie. glvs/glfs == OpenGL GLSL, arbvp1/arbfp1 == OpenGL ARB assembly, vs_*/ps_* == DirectX, etc).


'''Note:''' in 6.1.2 the OpenGL GLSL (glvs/glfs/etc) shaders appear to have been recompiled with a newer version of the HLSL cross-compiler and they are ''exceptionally'' informative. In particular, there are named uniform (constant) blocks that indicate the meaning of the variables.


[[Category:Format]]
[[Category:Format]]

Revision as of 21:33, 2 May 2015

BLS is the container format that stores the GPU shaders used to render the world. In WoD, there are now four different shader types under the Shaders\* directory.

  • Vertex
    • (versions: arbvp1, vp40, glvs_150, ps_2_0, ps_3_0, ps_4_0, ps_5_0)
  • Fragment
    • (versions: arbfp1, fp40, glfs_150, ps_2_0, ps_3_0, ps_4_0, ps_5_0)
  • Geometry
    • (versions: glgs_150, gs_4_0, gs_5_0)
  • Hull/Domain (equivalent to Tessellation in OpenGL, except WoW currently only has these shaders for DX)
    • (versions: ds_5_0/hs_50)

In MoP+, the BLS format changed significantly (the version increased from 1.3 to 1.4), a new header has been introduced and all of the shader text inside the BLS files is now contained inside compressed chunks. Very little appears to be known about the structure of the format itself, and the meaning of the various fields. Below is enough information to be able to extract the compressed chunks and read out shader text from the various shader components. Additionally, the Shaders\Effects folder contains WFX files that allow WoW to map shader types from models onto the various shader components (although, there are occasions in the M2 source code where shaders are picked in code).

BLS Format

The data that used to be present in this page seemed to be quite out of data and reflected the BLS format as of WotLK (or earlier). These updated structures can help write code to read in BLS files as of MoP/WoD but contain very little by way of analysis on most of the fields themselves.

Header

struct BLSHeader {
/*0x00*/	char magic[4];			// FourCC-style magic header, "GXVS", "GXPS", etc. Character order reversed in-file.
/*0x04*/	uint32_t version;		// version, 0x10003 in WotLK, 0x10004 in MoP/WoD
/*0x08*/	uint32_t permutationCount;	// (from old definition)
/*0x0c*/	uint32_t unknown;
/*0x10*/	uint32_t unknown2;
/*0x14*/	uint32_t unknown3;
/*0x18*/	uint32_t ofsCompressedData;	// offset to the start of the compressed (zlib) data bytes

This header is present at 0x00 in the BLS file, there is a significant amount of unknown data between the end of this header and the start of the compressed data section that is unexplored. In order to continue reading the data from a BLS file the data at ofsCompressedData must be decompressed with zlib inflate. The data at ofsCompressedData should begin 78 9C, the zlib magic header (for default compression).

Once decompressed, the old WotLK structure is present inside the decompressed data.

Shader Block

The decompressed BLS data is formatted in the same format that old WotLK shaders used, with the exception of WoD shaders that have a new (important) structure that is needed to find the start of the shader text. It no longer appears to be true that the top-level permutationCount field represents the number of shaders present in the inner BLS data.

Note: all the instances of BLSBlock are padded to the next nearest 4-byte alignment.

struct BLSBlock {
/*0x00*/	uint32_t flags;
/*0x04*/	uint32_t flags2;
/*0x08*/	uint32_t unknown;
/*0x0c*/	uint32_t unknown2;
/*0x10*/	uint32_t unknown3;
/*0x14*/	uint32_t unknown4;
/*0x18*/	uint32_t unknown5;
/*0x1c*/	uint32_t len;
}

The length field dictates where this block ends, in order to iterate over all blocks in BLS file you would start at the first block, read len and then skip to the next block using that. In MoP, the shader text was present immediately after the BLSBlock structure. However, in WoD there is a significant amount of new data between the end of this structure and the beginning of the shader text. Experimentally, there appears to be a header in front of this data that holds the offset to the text itself.

struct BLSBlock2 {
/*0x00*/	uint32_t header;
/*0x04*/	uint32_t unknown[4];
/*0x14*/	uint32_t offset;
}

The shader text starts at offset of (BLSBlock + BLSBlock2.offset). The shader text is then present in the format used by the particular graphics API used in that subdirectory. (ie. glvs/glfs == OpenGL GLSL, arbvp1/arbfp1 == OpenGL ARB assembly, vs_*/ps_* == DirectX, etc).

Note: in 6.1.2 the OpenGL GLSL (glvs/glfs/etc) shaders appear to have been recompiled with a newer version of the HLSL cross-compiler and they are exceptionally informative. In particular, there are named uniform (constant) blocks that indicate the meaning of the variables.