M2/Loading

Shader Substitution

WotLK Notes

In Wrath of the Lich King, the value on disk for M2Batch->shader is not the value used at runtime to select the appropriate shader effect.

Rather, the game runs through a series of substitution code paths, after which, the value is used to select the right effect from the BLS files.

Typically, the value on disk for M2Batch->shader is 0, with one large exception: if flag 0x08 is set in the M2, the value in M2Batch->shader is actually an index offset inside the M2TextureCombinerCombos table.

Bit Layout of M2Batch->shader

Just before the effect is loaded from BLS, the layout of M2Batch->shader is as follows:

Bits	Notes
0, 1, 2	Texture 2 Combiner Mode
3	Texture 2 Env Mapped
4, 5, 6	Texture 1 Combiner Mode
7	Texture 1 Env Mapped
8, 9, 10, 11, 12, 13	Unknown
14	Set for T2 coords??
15	Skips simple substitution??

CM2Shared::SubstituteSimpleShaders

The following is reversed from 3.3.5a (12340):

void __fastcall CM2Shared::SubstituteSimpleShaders(CM2Shared *this) {
  uint32_t batchCount = this->skinProfile->batches.count;

  if (batchCount == 0) {
    return;
  }

  uint32_t batchIndex;

  for (batchIndex = 0; batchIndex < batchCount; ++batchIndex) {
    M2Batch batch = this->skinProfile->batches.data[batchIndex];

    if (batch->shader & 0x8000) {
      continue;
    }

    M2Material material = &this->data->materials.data[batch->materialIndex];

    // Flag 0x08: use_combiner_combos
    if (this->data->flags & 0x08) {
      uint32_t textureCombinerComboIndex = batch->shader;
      uint32_t textureCoordComboIndex = batch->textureCoordComboIndex;

      batch->shader = 0;

      uint16_t shader[2];

      uint32_t textureCount = batch->textureCount;
      uint32_t textureIndex;

      for (textureIndex = 0; textureIndex < textureCount; ++textureIndex) {
        bool isFirstTexture = textureIndex == 0;
        bool isLastTexture = textureIndex == textureCount - 1;

        uint16_t textureCombiner = 0;

        // If this is the first texture and the batch material's blending mode is opaque,
        // override the combiner mode to opaque; otherwise, use the combiner mode from the
        // combiner combos
        if (isFirstTexture && material->blendMode == 0) {
          textureCombiner = 0;
        } else {
          textureCombiner = this->data->textureCombinerCombos.data[textureCombinerComboIndex + textureIndex];
        }

        shader[textureIndex] = textureCombiner;

        uint16_t textureCoord = this->data->textureCoordCombos.data[textureCoordComboIndex + textureIndex];

        // If the texture coord is env, set env bit for texture
        if (textureCoord > 2u) {
           shader[textureIndex] |= 8u;
        }

        // If this is the last texture and the texture coord is T2, enable bit 15
        if (isLastTexture && textureCoord == 1) {
          batch->shader |= 0x4000u;
        }
      }

      batch->shader |= shader[0] << 4 | shader[1];
    } else {
      uint16_t shader = 0;

      uint16_t textureCombiner = 0;

      // If the material blend mode is opaque, force the combiner to opaque; otherwise,
      // default combiner to mod
      if (material->blendMode == 0) {
        textureCombiner = 0;
      } else {
        textureCombiner = 1;
      }

      shader = textureCombiner;

      uint16_t textureCoord = this->data->textureCoordCombos.data[textureCoordComboIndex];

      // If the texture coord is env, set env bit for texture
      if (textureCoord > 2u) {
        shader |= 8u;
      }

      // If the texture coord is T2, enable bit 15
      if (textureCoord == 1) {
        batch->shader |= 0x4000;
      }

      batch->shader |= shader << 4;
    }
  }
}

CM2Shared::SubstituteSpecializedShaders

TODO