M2/Loading

From wowdev
Jump to navigation Jump to search

Shader Substitution

The following information is relevant for M2s in the Wrath of the Lich King and Cataclyms eras. See the notes for each expansion below.

It appears the shader substitution logic was completely removed in Mists of Pandaria, and no longer exists in the client.

Cata Notes

In Cataclysm, the shader substitution logic only runs for M2s with a version <= 264.

It seems Blizzard moved the shader substitution logic to their tooling at some point during Cataclysm, but kept the substitution paths around in the event an older M2 was loaded by the client.

WotLK Notes

In Wrath of the Lich King, the value on disk for M2Batch->shader is not the value used at runtime to select the appropriate shader effect.

Rather, the game runs through a series of substitution code paths, after which, the value is used to select the right effect from the BLS files.

These substitution functions are called from CM2Shared::InitializeSkinProfile, and are always run.

Typically, the value on disk for M2Batch->shader is 0, with one large exception: if flag 0x08 is set in the M2, the value in M2Batch->shader is actually an index offset inside the M2TextureCombinerCombos table.

Bit Layout of M2Batch->shader After Substitution

After shader substitution, the layout of M2Batch->shader is as follows:

Bits Notes
0, 1, 2 Texture 2 Op
3 Texture 2 Env Map
4, 5, 6 Texture 1 Op
7 Texture 1 Env Map
8, 9, 10, 11, 12, 13 Unused??
14 Set for T2 coords??
15 Specialized Effect

Note that if specialized effect is set, bits 0 - 14 simply hold a single number, which is used to look up the specialized effect in CM2Shared::GetEffect.

CM2Shared::SubstituteSimpleShaders

The following is reversed from 3.3.5a (12340).

void __fastcall CM2Shared::SubstituteSimpleShaders(CM2Shared *this) {
  uint32_t batchCount = this->skinProfile->batches.count;

  if (batchCount == 0) {
    return;
  }

  uint32_t batchIndex;

  for (batchIndex = 0; batchIndex < batchCount; ++batchIndex) {
    M2Batch batch = this->skinProfile->batches.data[batchIndex];

    if (batch->shader & 0x8000) {
      continue;
    }

    M2Material material = &this->data->materials.data[batch->materialIndex];

    // Flag 0x08: use_combiner_combos
    if (this->data->flags & 0x08) {
      uint32_t textureCombinerComboIndex = batch->shader;
      uint32_t textureCoordComboIndex = batch->textureCoordComboIndex;

      batch->shader = 0;

      uint16_t shader[2];

      uint32_t textureCount = batch->textureCount;
      uint32_t textureIndex;

      for (textureIndex = 0; textureIndex < textureCount; ++textureIndex) {
        bool isFirstTexture = textureIndex == 0;
        bool isLastTexture = textureIndex == textureCount - 1;

        uint16_t textureCombiner;

        // If this is the first texture and the batch material's blending mode is opaque,
        // override the combiner mode to opaque; otherwise, use the combiner mode from the
        // combiner combos
        if (isFirstTexture && material->blendMode == M2BLEND_OPAQUE) {
          textureCombiner = M2COMBINER_OPAQUE;
        } else {
          textureCombiner = this->data->textureCombinerCombos.data[textureCombinerComboIndex + textureIndex];
        }

        shader[textureIndex] = textureCombiner;

        uint16_t textureCoord = this->data->textureCoordCombos.data[textureCoordComboIndex + textureIndex];

        // If the texture coord is env, set env bit for texture
        if (textureCoord > 2u) {
           shader[textureIndex] |= M2COMBINER_ENVMAP;
        }

        // If this is the last texture and the texture coord is T2, enable bit 15
        if (isLastTexture && textureCoord == 1) {
          batch->shader |= 0x4000u;
        }
      }

      batch->shader |= shader[0] << 4 | shader[1];
    } else {
      uint16_t shader = 0;

      uint16_t textureCombiner;

      // If the material blend mode is opaque, force the combiner to opaque; otherwise,
      // default combiner to mod
      if (material->blendMode == M2BLEND_OPAQUE) {
        textureCombiner = M2COMBINER_OPAQUE;
      } else {
        textureCombiner = M2COMBINER_MOD;
      }

      shader = textureCombiner;

      uint16_t textureCoord = this->data->textureCoordCombos.data[textureCoordComboIndex];

      // If the texture coord is env, set env bit for texture
      if (textureCoord > 2u) {
        shader |= M2COMBINER_ENVMAP;
      }

      // If the texture coord is T2, enable bit 15
      if (textureCoord == 1) {
        batch->shader |= 0x4000;
      }

      batch->shader |= shader << 4;
    }
  }
}

CM2Shared::SubstituteSpecializedShaders

This function consolidates more sophisticated batch layers into single batches with specialized shaders.

Batch layering is controlled by M2Batch->materialLayer.

The three (todo: 4?) possible outcomes of this particular substitution function are:

  • Combiners_Opaque_Mod2xNA_Alpha
  • Combiners_Opaque_AddAlpha
  • Combiners_Opaque_AddAlpha_Alpha

When two batches are consolidated, only the first batch layer remains active.

Combiners_Opaque_Mod2xNA_Alpha

Combiners_Opaque_Mod2xNA_Alpha is substituted when the following requirements are met:

Layer Requirement
1 + 2 Batch materials share the same setting for M2Material flag 0x01 (lighting)
1 + 2 Batches share the same texture
1 + 2 Batches share the same texture weight
1 Batch material blend mode is M2BLEND_OPAQUE
1 Batch texture count is 2
1 Batch combiner 2 is M2COMBINER_MOD2X or M2COMBINER_MOD2X_NA
1 Batch texture coord is T1_Env
2 Batch material blend mode is M2BLEND_ALPHA_KEY or M2BLEND_ALPHA
2 Batch texture count is 1

Combiners_Opaque_AddAlpha

TODO

Combiners_Opaque_AddAlpha_Alpha

TODO

Shader Selection

When the game loads an M2 skin, CShaderEffects are created for all M2Batches based on the value in M2Batch->shader.

See below for the various functions involved.

CM2Shared::GetEffect

The following is reversed from 3.3.5a (12340).

CShaderEffect* __thiscall CM2Shared::GetEffect(CM2Shared *this, M2Batch *batch) {
  CShaderEffect* effect;

  // Simple effect
  if (!(batch->shader & 0x8000)) {
    effect = CM2Shared::CreateSimpleEffect(this, batch->textureCount, batch->shader, batch->textureCoordComboIndex);

    // Fallback
    // 0000000000010001
    // 1 texture: Combiners_Mod
    // 2 texture: Combiners_Mod_Mod
    if (!effect) {
      effect = CM2Shared::CreateSimpleEffect(this, batch->textureCount, 0x11u, batch->textureCoordComboIndex);
    }

    return effect;
  }

  // Advanced effect
  char* vsName = 0;
  char* psName = 0;
  uint32_t colorOp = 0;
  uint32_t alphaOp = 0;

  switch (batch->shader & 0x7FFF) {
    case 0u:
      return NULL;
    case 1u:
      vsName = "Diffuse_T1_Env";
      psName = "Combiners_Opaque_Mod2xNA_Alpha";
      colorOp = 0;
      alphaOp = 3;
      break;
    case 2u:
      vsName = "Diffuse_T1_Env";
      psName = "Combiners_Opaque_AddAlpha";
      colorOp = 0;
      alphaOp = 3;
      break;
    case 3u:
      vsName = "Diffuse_T1_Env";
      psName = "Combiners_Opaque_AddAlpha_Alpha";
      colorOp = 0;
      alphaOp = 3;
      break;
    default:
      break;
  }

  char* effectName;

  // Create effect name for hashing
  strcpy(&effectName, vsName);
  strcat(&effectName, psName);

  effect = CShaderEffectManager::GetEffect(&effectName);

  if (!effect) {
    effect = CShaderEffectManager::CreateEffect(&effectName);

    CShaderEffect::InitEffect(effect, vsName, psName);

    CShaderEffect::InitFixedFuncPass(effect, &colorOp, &alphaOp, 1);

    // TODO verify
    if (!effect) {
      effect = CM2Shared::CreateSimpleEffect(this, batch->textureCount, 0x11u, batch->textureCoordComboIndex);
    }
  }

  return effect;
}

CM2Shared::CreateSimpleEffect

The following is reversed from 3.3.5a (12340).

CShaderEffect *__thiscall CM2Shared::CreateSimpleEffect(CM2Shared *this, uint32_t textureCount, uint16_t shader, uint16_t textureCoordComboIndex) {
  uint32_t combiner[2];
  uint32_t env[2];

  combiner[0] = (shader >> 4) & M2COMBINER_OP_MASK;  // T1 Combiner
  combiner[1] = (shader >> 0) & M2COMBINER_OP_MASK;  // T2 Combiner
  envmap[0]   = (shader >> 4) & M2COMBINER_ENVMAP;   // T1 Env Mapped
  envmap[1]   = (shader >> 0) & M2COMBINER_ENVMAP;   // T2 Env Mapped

  char* vsName;
  char* psName;

  // 1 texture
  if (textureCount == 1) {
    if (envmap[0]) {
      vsName = "Diffuse_Env";
    } else {
      if (this->data->textureCoordCombos.data[textureCoordIndex] === 0) {
        vsName = "Diffuse_T1";
      } else {
        vsName = "Diffuse_T2";
      }
    }

    switch (combiner[0]) {
      case 0u:
        psName = "Combiners_Opaque";
        break;
      case 1u:
        psName = "Combiners_Mod";
        break;
      case 2u:
        psName = "Combiners_Decal";
        break;
      case 3u:
        psName = "Combiners_Add";
        break;
      case 4u:
        psName = "Combiners_Mod2x";
        break;
      case 5u:
        psName = "Combiners_Fade";
        break;
      default:
        psName = "Combiners_Mod";
        break;
    }
  // 2 textures
  } else {
    if (envmap[0] && envmap[1]) {
      vsName = "Diffuse_Env_Env";
    } else if (envmap[0]) {
      vsName = "Diffuse_Env_T2";
    } else if (envmap[1]) {
      vsName = "Diffuse_T1_Env";
    } else {
      vsName = "Diffuse_T1_T2";
    }

    switch (combiner[0]) {
      case 0u:
        switch (combiner[1]) {
          case 0u:
            psName = "Combiners_Opaque_Opaque";
            break;
          case 3u:
            psName = "Combiners_Opaque_Add";
            break;
          case 4u:
            psName = "Combiners_Opaque_Mod2x";
            break;
          case 6u:
            psName = "Combiners_Opaque_Mod2xNA";
            break;
          case 7u:
            psName = "Combiners_Opaque_AddNA";
            break;
          default:
            psName = "Combiners_Opaque_Mod";
            break;
        }
      case 1u:
        switch (combiner[1]) {
          case 0u:
            psName = "Combiners_Mod_Opaque";
            break;
          case 1u:
            psName = "Combiners_Mod_Mod";
            break;
          case 3u:
            psName = "Combiners_Mod_Add";
            break;
          case 4u:
            psName = "Combiners_Mod_Mod2x";
            break;
          case 6u:
            psName = "Combiners_Mod_Mod2xNA";
            break;
          case 7u:
            psName = "Combiners_Mod_AddNA";
            break;
          default:
            psName = "Combiners_Mod_Mod";
            break;
        }
      case 3u:
        switch (combiner[1]) {
          case 1u:
            psName = "Combiners_Add_Mod";
            break;
          default:
            return NULL;
            break;
        }
      case 4u:
        switch (combiner[1]) {
          case 4u:
            psName = "Combiners_Mod2x_Mod2x";
            break;
          default:
            return NULL;
            break;
        }
    }
  }

  char* effectName;

  // Create effect name for hashing
  strcpy(&effectName, vsName);
  strcat(&effectName, psName);

  CShaderEffect* effect;

  effect = CShaderEffectManager::GetEffect(&effectName);

  if (!effect) {
    effect = CShaderEffectManager::CreateEffect(&effectName);

    CShaderEffect::InitEffect(effect, vsName, psName);

    // M2GetCombinerOps (inlined)
    for (int textureIndex = 0; textureIndex < textureCount; textureIndex++) {
      colorOps[textureIndex] = colorOpTable[combiner[textureIndex]];
      alphaOps[textureIndex] = alphaOpTable[combiner[textureIndex]];
    }

    CShaderEffect::InitFixedFuncPass(effect, colorOps, alphaOps, textureCount);
  }

  return effect;
}