https://wowdev.wiki/api.php?action=feedcontributions&user=Simca&feedformat=atomwowdev - User contributions [en]2024-03-29T01:16:41ZUser contributionsMediaWiki 1.39.7https://wowdev.wiki/index.php?title=CreatureCache.wdb&diff=33685CreatureCache.wdb2021-05-01T00:05:29Z<p>Simca: Corrected final int field in Classic structure.</p>
<hr />
<div>The CreatureCache.wdb holds basic of the information for creatures you have seen in game. <br />
*Signature: WMOB<br />
<br />
==Structure==<br />
===1.13.2.31882===<br />
{|<br />
|-<br />
! Column !! Field !! Type !! Notes<br />
|-<br />
| 1 || ID || int ||<br />
|-<br />
| 2 || Length || int || if length > 0 the subsequent fields will be defined<br />
|-<br />
| 3 || TitleLength || uint:11 bits ||<br />
|-<br />
| 4 || TitleAltLength || uint:11 bits ||<br />
|-<br />
| 5 || CursorNameLength || uint:6 bits ||<br />
|-<br />
| 6 || CLS_Bit_1 || uint:1 bit || unknown purpose, usually 1<br />
|-<br />
| 7 || Leader || uint:1 bit ||<br />
|-<br />
| 8 || Name0Length || uint:11 bits ||<br />
|-<br />
| 9 || NameAlt0Length || uint:11 bits ||<br />
|-<br />
| 10 || Name1Length || uint:11 bits ||<br />
|-<br />
| 11 || NameAlt1Length || uint:11 bits ||<br />
|-<br />
| 12 || Name2Length || uint:11 bits ||<br />
|-<br />
| 13 || NameAlt2Length || uint:11 bits ||<br />
|-<br />
| 14 || Name3Length || uint:11 bits ||<br />
|-<br />
| 15 || NameAlt3Length || uint:11 bits ||<br />
|-<br />
| 16 || Name0 || char[Name0Length] ||<br />
|-<br />
| 17 || NameAlt0 || char[NameAlt0Length] ||<br />
|-<br />
| 18 || Name1 || char[Name1Length] ||<br />
|-<br />
| 19 || NameAlt1 || char[NameAlt1Length] ||<br />
|-<br />
| 20 || Name2 || char[Name2Length] ||<br />
|-<br />
| 21 || NameAlt2 || char[NameAlt2Length] ||<br />
|-<br />
| 22 || Name3 || char[Name3Length] ||<br />
|-<br />
| 23 || NameAlt3 || char[NameAlt3Length] ||<br />
|-<br />
| 24 || Flags || int[2] ||<br />
|-<br />
| 25 || CreatureType || int ||<br />
|-<br />
| 26 || CreatureFamily || int ||<br />
|-<br />
| 27 || Classification || int ||<br />
|-<br />
| 28 || ProxyCreatureID || int[2] ||<br />
|-<br />
| 29 || CLS_Int_1 || int ||<br />
|-<br />
| 30 || NumCreatureDisplays || int ||<br />
|-<br />
| 31 || UNK_BFA_Multiplier || float ||<br />
|-<br />
| || ''CreatureDisplay'' || struct[NumCreatureDisplays] ||<br />
|-<br />
| 32 || CreatureDisplay.DisplayInfoID || int || <br />
|-<br />
| 33 || CreatureDisplay.Scale || float ||<br />
|-<br />
| 34 || CreatureDisplay.Probability || float ||<br />
|-<br />
| 35 || HPMultiplier || float ||<br />
|-<br />
| 36 || EnergyMultiplier || float ||<br />
|-<br />
| 37 || NumQuestItems || uint ||<br />
|-<br />
| 38 || CreatureMovementInfoID || int ||<br />
|-<br />
| 39 || RequiredExpansion || int ||<br />
|-<br />
| 40 || TrackingQuestID || int ||<br />
|-<br />
| 41 || VignetteID || int ||<br />
|-<br />
| 42 || CreatureClassMask || int || Some type of 'creature class type' expressed as a bitfield (2^ID); 1 = Warrior, 2 = Rogue, 8 = Caster or something like that<br />
|-<br />
| 43 || Title || char[TitleLength] ||<br />
|-<br />
| 44 || TitleAlt || char[TitleAltLength] ||<br />
|-<br />
| 45 || CursorName || char[CursorNameLength] || defined if CursorNameLength != 1<br />
|-<br />
| 46 || QuestItemID || int[NumQuestItems] ||<br />
|}<br />
<br />
===2.3.0===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=1|build=1.13.2.31882}} client version<br />
'''Column Field Type Notes''' <br />
1 creatureID Integer <br />
2 entryLength Integer WDB Files <br />
3 name String <br />
4 name String <br />
5 name String <br />
6 name String <br />
7 description String Shown as <Description> under their name in game <br />
8 cursor String Cursor shown when hovering <br />
9 typeID Integer [[CreatureType.dbc]]<br />
10 familyID Integer [[CreatureFamily.dbc]] <br />
11 flagID Integer Creature Flag Types <br />
12 Unknown <br />
13 spellDataID [[CreatureSpellData.dbc]] <br />
14 DisplayID Integer [[CreatureDisplayInfo.dbc]] <br />
15 DisplayID2 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
16 DisplayID3 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
17 DisplayID4 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
18 Unknown Floats Floats? <br />
19 Unknown Floats Floats? <br />
20 RacialLeader uint8 2-byte. Bosses seem to have this set at 256 while everything else is either 0 or 1.<br />
<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type<br />
1 Name String[4]<br />
2 Title String<br />
3 Flags Integer<br />
4 CreatureType Integer<br />
5 CreatureFamily Integer<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=CreatureCache.wdb&diff=33684CreatureCache.wdb2021-05-01T00:02:50Z<p>Simca: Added a couple of field names.</p>
<hr />
<div>The CreatureCache.wdb holds basic of the information for creatures you have seen in game. <br />
*Signature: WMOB<br />
<br />
==Structure==<br />
===1.13.2.31882===<br />
{|<br />
|-<br />
! Column !! Field !! Type !! Notes<br />
|-<br />
| 1 || ID || int ||<br />
|-<br />
| 2 || Length || int || if length > 0 the subsequent fields will be defined<br />
|-<br />
| 3 || TitleLength || uint:11 bits ||<br />
|-<br />
| 4 || TitleAltLength || uint:11 bits ||<br />
|-<br />
| 5 || CursorNameLength || uint:6 bits ||<br />
|-<br />
| 6 || CLS_Bit_1 || uint:1 bit || unknown purpose, usually 1<br />
|-<br />
| 7 || Leader || uint:1 bit ||<br />
|-<br />
| 8 || Name0Length || uint:11 bits ||<br />
|-<br />
| 9 || NameAlt0Length || uint:11 bits ||<br />
|-<br />
| 10 || Name1Length || uint:11 bits ||<br />
|-<br />
| 11 || NameAlt1Length || uint:11 bits ||<br />
|-<br />
| 12 || Name2Length || uint:11 bits ||<br />
|-<br />
| 13 || NameAlt2Length || uint:11 bits ||<br />
|-<br />
| 14 || Name3Length || uint:11 bits ||<br />
|-<br />
| 15 || NameAlt3Length || uint:11 bits ||<br />
|-<br />
| 16 || Name0 || char[Name0Length] ||<br />
|-<br />
| 17 || NameAlt0 || char[NameAlt0Length] ||<br />
|-<br />
| 18 || Name1 || char[Name1Length] ||<br />
|-<br />
| 19 || NameAlt1 || char[NameAlt1Length] ||<br />
|-<br />
| 20 || Name2 || char[Name2Length] ||<br />
|-<br />
| 21 || NameAlt2 || char[NameAlt2Length] ||<br />
|-<br />
| 22 || Name3 || char[Name3Length] ||<br />
|-<br />
| 23 || NameAlt3 || char[NameAlt3Length] ||<br />
|-<br />
| 24 || Flags || int[2] ||<br />
|-<br />
| 25 || CreatureType || int ||<br />
|-<br />
| 26 || CreatureFamily || int ||<br />
|-<br />
| 27 || Classification || int ||<br />
|-<br />
| 28 || ProxyCreatureID || int[2] ||<br />
|-<br />
| 29 || CLS_Int_1 || int ||<br />
|-<br />
| 30 || NumCreatureDisplays || int ||<br />
|-<br />
| 31 || UNK_BFA_Multiplier || float ||<br />
|-<br />
| || ''CreatureDisplay'' || struct[NumCreatureDisplays] ||<br />
|-<br />
| 32 || CreatureDisplay.DisplayInfoID || int || <br />
|-<br />
| 33 || CreatureDisplay.Scale || float ||<br />
|-<br />
| 34 || CreatureDisplay.Probability || float ||<br />
|-<br />
| 35 || HPMultiplier || float ||<br />
|-<br />
| 36 || EnergyMultiplier || float ||<br />
|-<br />
| 37 || NumQuestItems || uint ||<br />
|-<br />
| 38 || CreatureMovementInfoID || int ||<br />
|-<br />
| 39 || RequiredExpansion || int ||<br />
|-<br />
| 40 || TrackingQuestID || int ||<br />
|-<br />
| 41 || VignetteID || int ||<br />
|-<br />
| 42 || ExpansionID || int || Should always be 1 since it's Classic<br />
|-<br />
| 43 || Title || char[TitleLength] ||<br />
|-<br />
| 44 || TitleAlt || char[TitleAltLength] ||<br />
|-<br />
| 45 || CursorName || char[CursorNameLength] || defined if CursorNameLength != 1<br />
|-<br />
| 46 || QuestItemID || int[NumQuestItems] ||<br />
|}<br />
<br />
===2.3.0===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=1|build=1.13.2.31882}} client version<br />
'''Column Field Type Notes''' <br />
1 creatureID Integer <br />
2 entryLength Integer WDB Files <br />
3 name String <br />
4 name String <br />
5 name String <br />
6 name String <br />
7 description String Shown as <Description> under their name in game <br />
8 cursor String Cursor shown when hovering <br />
9 typeID Integer [[CreatureType.dbc]]<br />
10 familyID Integer [[CreatureFamily.dbc]] <br />
11 flagID Integer Creature Flag Types <br />
12 Unknown <br />
13 spellDataID [[CreatureSpellData.dbc]] <br />
14 DisplayID Integer [[CreatureDisplayInfo.dbc]] <br />
15 DisplayID2 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
16 DisplayID3 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
17 DisplayID4 Integer [[CreatureDisplayInfo.dbc]] - Added in 2.2 <br />
18 Unknown Floats Floats? <br />
19 Unknown Floats Floats? <br />
20 RacialLeader uint8 2-byte. Bosses seem to have this set at 256 while everything else is either 0 or 1.<br />
<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type<br />
1 Name String[4]<br />
2 Title String<br />
3 Flags Integer<br />
4 CreatureType Integer<br />
5 CreatureFamily Integer<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=M2/.skel&diff=30811M2/.skel2020-09-15T19:30:30Z<p>Simca: /* SKL1 */ Named first field.</p>
<hr />
<div>These files replace some blocks from the [[M2]] MD20 data. The chunks doing that have a fixed header followed by raw data, as with MD20. The headers have offsets into the respective chunk. This means that there are no fixed sizes for most chunks, and not all bytes of the chunk are actually used but include alignment and padding as well.<br />
<br />
=SKL1=<br />
struct {<br />
uint32_t flags; // Assumed flags; always 0x100 as of {{Template:Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2.25079}}.<br />
M2Array<char> name;<br />
uint8_t _0x0c[4]; // Is this already part of the data? Always 0 as of {{Template:Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2.25079}}.<br />
} skeleton_l_header;<br />
uint8_t skeleton_l_raw_data[];<br />
<br />
=SKA1=<br />
struct {<br />
M2Array<M2Attachment> attachments;<br />
M2Array<uint16_t> attachment_lookup_table;<br />
} skeleton_attachment_header;<br />
uint8_t skeleton_attachment_raw_data[];<br />
<br />
=SKB1=<br />
struct {<br />
M2Array<M2CompBone> bones;<br />
M2Array<uint16_t> key_bone_lookup;<br />
} skeleton_bone_header;<br />
uint8_t skeleton_bone_raw_data[];<br />
<br />
=SKS1=<br />
struct {<br />
M2Array<M2Loop> global_loops;<br />
M2Array<M2Sequence> sequences;<br />
M2Array<uint16_t> sequence_lookups;<br />
uint8_t _0x18[8]; // Is this already part of the data? Always 0 as of {{Template:Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2.25079}}.<br />
} skeleton_sequence_header;<br />
uint8_t skeleton_sequence_raw_data[];<br />
<br />
=SKPD=<br />
struct {<br />
uint8_t _0x00[8]; // Always 0 as of {{Template:Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2.25079}}.<br />
uint32_t parent_skel_file_id;<br />
uint8_t _0x0c[4]; // Always 0 as of {{Template:Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2.25079}}.<br />
} skeleton_parent_data;<br />
<br />
The parent skeleton file data id is used for de-duplication. E.g. lightforgeddraeneimale references draeneimale_hd. lightforgeddraeneimale does not have any [[M2/.anim]] files, but uses those of draeneimale_hd via parent-link (does not even have an AFID chunk). Note that it appears that the child [[M2/.skel]]s still have <tt>SK*1</tt> chunks, so apparently only <tt>*FID</tt> are shared?!<br />
<br />
=AFID=<br />
Same structure and semantics as in [[M2#AFID|M2.AFID]].<br />
<br />
=BFID=<br />
Same structure and semantics as in [[M2#BFID|M2.BFID]].</div>Simcahttps://wowdev.wiki/index.php?title=QuestCache.wdb&diff=29848QuestCache.wdb2020-04-27T23:51:25Z<p>Simca: /* Version 9.0.1.33978 */</p>
<hr />
<div>The [[QuestCache.wdb]] holds most of the information for quest you have seen in game. <br />
*Signature: WQST <br />
<br />
==Structure==<br />
<br />
===Version 9.0.1.33978===<br />
Blizzard removed all three fields related to level (QuestLevel, QuestMaxScalingLevel, and QuestMinLevel) in 9.0.1. Presumably, this is replaced with a lookup to ContentTuning in order to acquire that information. However, I don't know where that lookup happens (maybe in a separate packet that isn't cached?). Regardless, quest levels aren't a thing we have easy access to anymore. Additionally, Blizzard added two new fields to the end of the structure (B30993_Int_1 and B31984_Int_1). The purpose of both is unknown. They were added in the build for which they were named after.<br />
<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int Quest_UNK_27075; // Unknown, but seems to frequently mirror SuggestedGroupNum. Theory: Maximum party size number for LFG Tool to create a group<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3;<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
uint B30993_Int_1; // Unknown - only set on Warfront-related quests; has values of 12, 113, 114, and 115<br />
uint B31984_Int_1; // Unknown - always 0 so far<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 8.0.1.27075===<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int QuestLevel; // Recommended level to complete the quest<br />
int Quest_UNK_27075; // Unknown. Almost always 0 but sometimes 3 or 5.<br />
int QuestMaxScalingLevel; // Maximum level that the quest reward and difficulty will scale to<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestMinLevel; // Required level to pick up the quest<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3;<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 3.3.5?===<br />
'''Column Field Type Notes''' <br />
1 questID Integer <br />
2 entryLength Integer WDB Files <br />
3 DuplicatedQuestID Integer This is a duplicate of the quest ID <br />
4 QuestType Integer 2 or 0, 0 appears to be when it's a quick click-through quest like repeatable quests (but not exclusively) <br />
5 QuestLevel Integer The level of the quest. This is not the level at which the quest is atainable at. <br />
6 areaID/sortID Integer AreaTable.dbc if negative the value points to: [[QuestSort.dbc]] <br />
7 infoID Integer [[QuestInfo.dbc]] <br />
8 SuggestedPlayers Integer <br />
9 FactionID Integer [[Faction.dbc]] <br />
10 FactionAmount Integer e.g. 3000 <br />
11 Unknown Integer Always 0? <br />
12 Unknown Integer Always 0? <br />
13 nextQuestID Integer The quest that follows this quest <br />
14 coins Integer Value is in Copper - Coins rewarded on completion <br />
15 SubExp70 Integer Value is in Copper - Coins rewarded on lvl 70 instead of experience (guess) <br />
16 RewardSpellID Integer [[Spell.dbc]]: Spell or ability that is added to players spellbook upon completion. This is probably an argument passed into EffectOnPlayer somehow, as this is sometimes a duplicated value as the EffectOnPlayer field, and it's not a spell that the player learns. (Such as the Razorhide quest turnin). <br />
17 EffectOnPlayer Integer [[Spell.dbc]]: Spell/effect cast on player when completing. <br />
18 startingItemID Integer [[ItemCache.wdb]] The item that you are given when you start the quest, such as a package to deliver. <br />
19 QuestFlags BitMask<br />
20 givenItem1 Integer [[ItemCache.wdb]] <br />
21 givenItem1Amount Integer <br />
22 givenItem2 Integer [[ItemCache.wdb]]<br />
23 givenItem2Amount Integer <br />
24 givenItem3 Integer [[ItemCache.wdb]]<br />
25 givenItem3Amount Integer <br />
26 givenItem4 Integer [[ItemCache.wdb]] <br />
27 givenItem4Amount Integer <br />
28 choiceItem1 Integer [[ItemCache.wdb]] <br />
29 choiceItem1Amount Integer <br />
30 choiceItem2 Integer [[ItemCache.wdb]] <br />
31 choiceItem2Amount Integer <br />
32 choiceItem3 Integer [[ItemCache.wdb]] <br />
33 choiceItem3Amount Integer <br />
34 choiceItem4 Integer [[ItemCache.wdb]] <br />
35 choiceItem4Amount Integer <br />
36 choiceItem5 Integer [[ItemCache.wdb]] <br />
37 choiceItem5Amount Integer <br />
38 choiceItem6 Integer [[ItemCache.wdb]] <br />
39 choiceItem6Amount Integer <br />
40 Unknown Integer <br />
41 Unknown Integer <br />
42 Unknown Integer <br />
43 Unknown Integer <br />
44 name String <br />
45 description String <br />
46 details String <br />
47 subdescription String <br />
48 killCreature1 Integer [[CreatureCache.wdb]] <br />
49 killCreature1Amount Integer <br />
50 collectItem1 Integer [[ItemCache.wdb]] <br />
51 collectItem1Amount Integer <br />
52 killCreature2 Integer [[CreatureCache.wdb]] <br />
53 killCreature2Amount Integer <br />
54 collectItem2 Integer [[ItemCache.wdb]] <br />
55 collectItem2Amount Integer <br />
56 killCreature3 Integer [[CreatureCache.wdb]]<br />
57 killCreature3Amount Integer <br />
58 collectItem3 Integer [[ItemCache.wdb]] <br />
59 collectItem3Amount Integer <br />
60 killCreature4 Integer [[CreatureCache.wdb]] <br />
61 killCreature4Amount Integer <br />
62 collectItem4 Integer [[ItemCache.wdb]] <br />
63 collectItem4Amount Integer <br />
64 Objective1 String If not killCreature1 or collectItem1 this is the objective if set <br />
65 Objective2 String If not killCreature2 or collectItem2 this is the objective if set <br />
66 Objective3 String If not killCreature3 or collectItem3 this is the objective if set <br />
67 Objective4 String If not killCreature4 or collectItem4 this is the objective if set <br />
<br />
===Version 0.5.3.3368===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type''' <br />
1 QuestId Integer<br />
2 QuestType Integer<br />
3 QuestLevel Integer<br />
4 QuestSortID Integer<br />
5 QuestInfoID Integer<br />
6 RewardNextQuest Integer<br />
7 RewardMoney Integer<br />
8 StartItem Integer<br />
9 RewardItems Integer[4]<br />
10 RewardAmount Integer[4]<br />
11 RewardChoiceItems Integer[6]<br />
12 RewardChoiceAmount Integer[6]<br />
13 POIContinent Integer<br />
14 POIx Float<br />
15 POIy Float<br />
16 POIPriority Integer<br />
17 LogTitle Char[128]<br />
18 LogDescription Char[1024]<br />
19 QuestDescription Char[1024]<br />
20 AreaDescription Char[128]<br />
21 MonsterToKill Integer[4]<br />
22 MonsterToKillQuantity Integer[4]<br />
23 ItemToGet Integer[4]<br />
24 ItemToGetQuantity Integer[4]<br />
25 GetDescription Char[4][64]<br />
<br />
==Quest Flags==<br />
1 (1) Deliver <br />
2 (2) Kill <br />
3 (4) Speak To <br />
4 (8) Repeatable? Q:Shareable?<br />
5 (16) Exploration <br />
6 (32) Timed Quest <br />
7 (64) Raid Quest Tagged as "(Raid)" (to verify)<br />
8 (128) Reputation <br />
9 (256) Unknown <br />
10 (512) Unknown <br />
11 (1024) Unknown <br />
12 (2048) Unknown <br />
13 (4096) Daily This bit is set if it's a Daily Quest<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=QuestCache.wdb&diff=29847QuestCache.wdb2020-04-27T23:48:26Z<p>Simca: /* Structure */ Added 9.0.1.33978 QuestCache structure.</p>
<hr />
<div>The [[QuestCache.wdb]] holds most of the information for quest you have seen in game. <br />
*Signature: WQST <br />
<br />
==Structure==<br />
<br />
===Version 9.0.1.33978===<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int Quest_UNK_27075; // Unknown, but seems to frequently mirror SuggestedGroupNum. Theory: Maximum party size number for LFG Tool to create a group<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3;<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
uint B30993_Int_1; // Unknown - only set on Warfront-related quests; has values of 12, 113, 114, and 115<br />
uint B31984_Int_1; // Unknown - always 0 so far<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 8.0.1.27075===<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int QuestLevel; // Recommended level to complete the quest<br />
int Quest_UNK_27075; // Unknown. Almost always 0 but sometimes 3 or 5.<br />
int QuestMaxScalingLevel; // Maximum level that the quest reward and difficulty will scale to<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestMinLevel; // Required level to pick up the quest<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3;<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 3.3.5?===<br />
'''Column Field Type Notes''' <br />
1 questID Integer <br />
2 entryLength Integer WDB Files <br />
3 DuplicatedQuestID Integer This is a duplicate of the quest ID <br />
4 QuestType Integer 2 or 0, 0 appears to be when it's a quick click-through quest like repeatable quests (but not exclusively) <br />
5 QuestLevel Integer The level of the quest. This is not the level at which the quest is atainable at. <br />
6 areaID/sortID Integer AreaTable.dbc if negative the value points to: [[QuestSort.dbc]] <br />
7 infoID Integer [[QuestInfo.dbc]] <br />
8 SuggestedPlayers Integer <br />
9 FactionID Integer [[Faction.dbc]] <br />
10 FactionAmount Integer e.g. 3000 <br />
11 Unknown Integer Always 0? <br />
12 Unknown Integer Always 0? <br />
13 nextQuestID Integer The quest that follows this quest <br />
14 coins Integer Value is in Copper - Coins rewarded on completion <br />
15 SubExp70 Integer Value is in Copper - Coins rewarded on lvl 70 instead of experience (guess) <br />
16 RewardSpellID Integer [[Spell.dbc]]: Spell or ability that is added to players spellbook upon completion. This is probably an argument passed into EffectOnPlayer somehow, as this is sometimes a duplicated value as the EffectOnPlayer field, and it's not a spell that the player learns. (Such as the Razorhide quest turnin). <br />
17 EffectOnPlayer Integer [[Spell.dbc]]: Spell/effect cast on player when completing. <br />
18 startingItemID Integer [[ItemCache.wdb]] The item that you are given when you start the quest, such as a package to deliver. <br />
19 QuestFlags BitMask<br />
20 givenItem1 Integer [[ItemCache.wdb]] <br />
21 givenItem1Amount Integer <br />
22 givenItem2 Integer [[ItemCache.wdb]]<br />
23 givenItem2Amount Integer <br />
24 givenItem3 Integer [[ItemCache.wdb]]<br />
25 givenItem3Amount Integer <br />
26 givenItem4 Integer [[ItemCache.wdb]] <br />
27 givenItem4Amount Integer <br />
28 choiceItem1 Integer [[ItemCache.wdb]] <br />
29 choiceItem1Amount Integer <br />
30 choiceItem2 Integer [[ItemCache.wdb]] <br />
31 choiceItem2Amount Integer <br />
32 choiceItem3 Integer [[ItemCache.wdb]] <br />
33 choiceItem3Amount Integer <br />
34 choiceItem4 Integer [[ItemCache.wdb]] <br />
35 choiceItem4Amount Integer <br />
36 choiceItem5 Integer [[ItemCache.wdb]] <br />
37 choiceItem5Amount Integer <br />
38 choiceItem6 Integer [[ItemCache.wdb]] <br />
39 choiceItem6Amount Integer <br />
40 Unknown Integer <br />
41 Unknown Integer <br />
42 Unknown Integer <br />
43 Unknown Integer <br />
44 name String <br />
45 description String <br />
46 details String <br />
47 subdescription String <br />
48 killCreature1 Integer [[CreatureCache.wdb]] <br />
49 killCreature1Amount Integer <br />
50 collectItem1 Integer [[ItemCache.wdb]] <br />
51 collectItem1Amount Integer <br />
52 killCreature2 Integer [[CreatureCache.wdb]] <br />
53 killCreature2Amount Integer <br />
54 collectItem2 Integer [[ItemCache.wdb]] <br />
55 collectItem2Amount Integer <br />
56 killCreature3 Integer [[CreatureCache.wdb]]<br />
57 killCreature3Amount Integer <br />
58 collectItem3 Integer [[ItemCache.wdb]] <br />
59 collectItem3Amount Integer <br />
60 killCreature4 Integer [[CreatureCache.wdb]] <br />
61 killCreature4Amount Integer <br />
62 collectItem4 Integer [[ItemCache.wdb]] <br />
63 collectItem4Amount Integer <br />
64 Objective1 String If not killCreature1 or collectItem1 this is the objective if set <br />
65 Objective2 String If not killCreature2 or collectItem2 this is the objective if set <br />
66 Objective3 String If not killCreature3 or collectItem3 this is the objective if set <br />
67 Objective4 String If not killCreature4 or collectItem4 this is the objective if set <br />
<br />
===Version 0.5.3.3368===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type''' <br />
1 QuestId Integer<br />
2 QuestType Integer<br />
3 QuestLevel Integer<br />
4 QuestSortID Integer<br />
5 QuestInfoID Integer<br />
6 RewardNextQuest Integer<br />
7 RewardMoney Integer<br />
8 StartItem Integer<br />
9 RewardItems Integer[4]<br />
10 RewardAmount Integer[4]<br />
11 RewardChoiceItems Integer[6]<br />
12 RewardChoiceAmount Integer[6]<br />
13 POIContinent Integer<br />
14 POIx Float<br />
15 POIy Float<br />
16 POIPriority Integer<br />
17 LogTitle Char[128]<br />
18 LogDescription Char[1024]<br />
19 QuestDescription Char[1024]<br />
20 AreaDescription Char[128]<br />
21 MonsterToKill Integer[4]<br />
22 MonsterToKillQuantity Integer[4]<br />
23 ItemToGet Integer[4]<br />
24 ItemToGetQuantity Integer[4]<br />
25 GetDescription Char[4][64]<br />
<br />
==Quest Flags==<br />
1 (1) Deliver <br />
2 (2) Kill <br />
3 (4) Speak To <br />
4 (8) Repeatable? Q:Shareable?<br />
5 (16) Exploration <br />
6 (32) Timed Quest <br />
7 (64) Raid Quest Tagged as "(Raid)" (to verify)<br />
8 (128) Reputation <br />
9 (256) Unknown <br />
10 (512) Unknown <br />
11 (1024) Unknown <br />
12 (2048) Unknown <br />
13 (4096) Daily This bit is set if it's a Daily Quest<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=QuestCache.wdb&diff=29846QuestCache.wdb2020-04-27T23:42:44Z<p>Simca: /* Structure */ After future data has been added, that field is 100% Flags3.</p>
<hr />
<div>The [[QuestCache.wdb]] holds most of the information for quest you have seen in game. <br />
*Signature: WQST <br />
<br />
==Structure==<br />
<br />
===Version 8.0.1.27075===<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int QuestLevel; // Recommended level to complete the quest<br />
int Quest_UNK_27075; // Unknown. Almost always 0 but sometimes 3 or 5.<br />
int QuestMaxScalingLevel; // Maximum level that the quest reward and difficulty will scale to<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestMinLevel; // Required level to pick up the quest<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3;<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 3.3.5?===<br />
'''Column Field Type Notes''' <br />
1 questID Integer <br />
2 entryLength Integer WDB Files <br />
3 DuplicatedQuestID Integer This is a duplicate of the quest ID <br />
4 QuestType Integer 2 or 0, 0 appears to be when it's a quick click-through quest like repeatable quests (but not exclusively) <br />
5 QuestLevel Integer The level of the quest. This is not the level at which the quest is atainable at. <br />
6 areaID/sortID Integer AreaTable.dbc if negative the value points to: [[QuestSort.dbc]] <br />
7 infoID Integer [[QuestInfo.dbc]] <br />
8 SuggestedPlayers Integer <br />
9 FactionID Integer [[Faction.dbc]] <br />
10 FactionAmount Integer e.g. 3000 <br />
11 Unknown Integer Always 0? <br />
12 Unknown Integer Always 0? <br />
13 nextQuestID Integer The quest that follows this quest <br />
14 coins Integer Value is in Copper - Coins rewarded on completion <br />
15 SubExp70 Integer Value is in Copper - Coins rewarded on lvl 70 instead of experience (guess) <br />
16 RewardSpellID Integer [[Spell.dbc]]: Spell or ability that is added to players spellbook upon completion. This is probably an argument passed into EffectOnPlayer somehow, as this is sometimes a duplicated value as the EffectOnPlayer field, and it's not a spell that the player learns. (Such as the Razorhide quest turnin). <br />
17 EffectOnPlayer Integer [[Spell.dbc]]: Spell/effect cast on player when completing. <br />
18 startingItemID Integer [[ItemCache.wdb]] The item that you are given when you start the quest, such as a package to deliver. <br />
19 QuestFlags BitMask<br />
20 givenItem1 Integer [[ItemCache.wdb]] <br />
21 givenItem1Amount Integer <br />
22 givenItem2 Integer [[ItemCache.wdb]]<br />
23 givenItem2Amount Integer <br />
24 givenItem3 Integer [[ItemCache.wdb]]<br />
25 givenItem3Amount Integer <br />
26 givenItem4 Integer [[ItemCache.wdb]] <br />
27 givenItem4Amount Integer <br />
28 choiceItem1 Integer [[ItemCache.wdb]] <br />
29 choiceItem1Amount Integer <br />
30 choiceItem2 Integer [[ItemCache.wdb]] <br />
31 choiceItem2Amount Integer <br />
32 choiceItem3 Integer [[ItemCache.wdb]] <br />
33 choiceItem3Amount Integer <br />
34 choiceItem4 Integer [[ItemCache.wdb]] <br />
35 choiceItem4Amount Integer <br />
36 choiceItem5 Integer [[ItemCache.wdb]] <br />
37 choiceItem5Amount Integer <br />
38 choiceItem6 Integer [[ItemCache.wdb]] <br />
39 choiceItem6Amount Integer <br />
40 Unknown Integer <br />
41 Unknown Integer <br />
42 Unknown Integer <br />
43 Unknown Integer <br />
44 name String <br />
45 description String <br />
46 details String <br />
47 subdescription String <br />
48 killCreature1 Integer [[CreatureCache.wdb]] <br />
49 killCreature1Amount Integer <br />
50 collectItem1 Integer [[ItemCache.wdb]] <br />
51 collectItem1Amount Integer <br />
52 killCreature2 Integer [[CreatureCache.wdb]] <br />
53 killCreature2Amount Integer <br />
54 collectItem2 Integer [[ItemCache.wdb]] <br />
55 collectItem2Amount Integer <br />
56 killCreature3 Integer [[CreatureCache.wdb]]<br />
57 killCreature3Amount Integer <br />
58 collectItem3 Integer [[ItemCache.wdb]] <br />
59 collectItem3Amount Integer <br />
60 killCreature4 Integer [[CreatureCache.wdb]] <br />
61 killCreature4Amount Integer <br />
62 collectItem4 Integer [[ItemCache.wdb]] <br />
63 collectItem4Amount Integer <br />
64 Objective1 String If not killCreature1 or collectItem1 this is the objective if set <br />
65 Objective2 String If not killCreature2 or collectItem2 this is the objective if set <br />
66 Objective3 String If not killCreature3 or collectItem3 this is the objective if set <br />
67 Objective4 String If not killCreature4 or collectItem4 this is the objective if set <br />
<br />
===Version 0.5.3.3368===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type''' <br />
1 QuestId Integer<br />
2 QuestType Integer<br />
3 QuestLevel Integer<br />
4 QuestSortID Integer<br />
5 QuestInfoID Integer<br />
6 RewardNextQuest Integer<br />
7 RewardMoney Integer<br />
8 StartItem Integer<br />
9 RewardItems Integer[4]<br />
10 RewardAmount Integer[4]<br />
11 RewardChoiceItems Integer[6]<br />
12 RewardChoiceAmount Integer[6]<br />
13 POIContinent Integer<br />
14 POIx Float<br />
15 POIy Float<br />
16 POIPriority Integer<br />
17 LogTitle Char[128]<br />
18 LogDescription Char[1024]<br />
19 QuestDescription Char[1024]<br />
20 AreaDescription Char[128]<br />
21 MonsterToKill Integer[4]<br />
22 MonsterToKillQuantity Integer[4]<br />
23 ItemToGet Integer[4]<br />
24 ItemToGetQuantity Integer[4]<br />
25 GetDescription Char[4][64]<br />
<br />
==Quest Flags==<br />
1 (1) Deliver <br />
2 (2) Kill <br />
3 (4) Speak To <br />
4 (8) Repeatable? Q:Shareable?<br />
5 (16) Exploration <br />
6 (32) Timed Quest <br />
7 (64) Raid Quest Tagged as "(Raid)" (to verify)<br />
8 (128) Reputation <br />
9 (256) Unknown <br />
10 (512) Unknown <br />
11 (1024) Unknown <br />
12 (2048) Unknown <br />
13 (4096) Daily This bit is set if it's a Daily Quest<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=WDB&diff=28772WDB2020-01-15T19:01:33Z<p>Simca: Reconciled differences in repeated tables to result in a single, better table.</p>
<hr />
<div>The WDB files are found inside the WDB folder. The client caches data it receives from the server. This is mainly done to reduce network traffic.<br />
<br />
Not all WDB caches are saved to disk. This is defined in the client by the <tt>DBCache</tt>'s constructor which contains a persistent parameter.<br />
<br />
== Header ==<br />
<br />
*WDB files < 1.6: Header length is 16 bytes<br />
*WDB files >=1.6: Header length is 20 bytes (Verified till 1.9.4)<br />
*WDB files >=3.0.8-9506: Header length is 24 bytes<br />
<br />
'''Offset Type Description'''<br />
0x00 char[4] Identifier - depends on the WDB file, will be explained later (reversed!)<br />
0x04 uint32 Client Version - Version of the client (lo -> hi encoding)<br />
0x08 char[4] Client Locale - The Locale of the client (reversed!) {{Template:Sandbox/VersionRange|min_expansionlevel=1|min_build=1.6.0}}<br />
0x0C uint32 Record Size - Something to do with row length, there is no consistent way of retrieving it as of yet. In Alpha this is row length with strings treated as 4 bytes.<br />
0x10 uint32 Record Version - A manually updated versioning field - except for WoWCache.wdb which is read from <code>WardenCachedModule::Version</code><br />
0x14 uint32 Cache Version - A packet based versioning field set via SMSG_CLIENTCACHE_VERSION {{Template:Sandbox/VersionRange|min_expansionlevel=3|min_build=3.0.8}}<br />
<br />
== WDB files ==<br />
<br />
'''File Signature''' <br />
[[ArenaTeamCache.wdb]] WATM {{Template:Sandbox/VersionRange|min_expansionlevel=3|max_expansionlevel=4}}<br />
[[BattlePetNameCache.wdb]] WBPN {{Template:Unverified|{{Template:Sandbox/VersionRange|min_expansionlevel=5|max_expansionlevel=6}} Not seen > 6.2.4}}<br />
[[CreatureCache.wdb]] WMOB <br />
[[DanceCache.wdb]] WDAN {{Template:Sandbox/VersionRange|min_expansionlevel=3|max_expansionlevel=4}}<br />
[[GameObjectCache.wdb]] WGOB <br />
[[GuildStatsCache.wdb]] WGLD<br />
[[ItemCache.wdb]] WIDB {{Template:Sandbox/VersionRange|max_expansionlevel=3}}<br />
[[ItemNameCache.wdb]] WNDB {{Template:Sandbox/VersionRange|max_expansionlevel=3}}<br />
[[ItemTextCache.wdb]] WITX <br />
[[NameCache.wdb]] WNAM<br />
[[NPCCache.wdb]] WNPC <br />
[[PageTextCache.wdb]] WPTX <br />
[[PetitionCache.wdb]] WPTN<br />
[[PetNameCache.wdb]] WPNM<br />
[[QuestCache.wdb]] WQST <br />
[[RealmCache.wdb]] WRLM {{Template:Unverified|{{Template:Sandbox/VersionRange|min_expansionlevel=5|max_expansionlevel=6|max_build=6.0.1.18179}} Not seen ≥ 6.2.3}}<br />
[[WOWCache.wdb]] WRDN<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=ADB&diff=28581ADB2019-12-29T01:40:37Z<p>Simca: /* Version 7 */ Changed 'sha256' to 'verification_hash'.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
The structures described here are used in [[ADB|ADB files]], which are a cache of dynamically streamed database entries, typically used for hotfixes and content that Blizzard wants to hide from dataminers until the last minute.<br />
<br />
For more information, please check out the page on [[DB2|DB2 files]]. This article assumes you already have a basic understanding of how they function.<br />
<br />
=Table content structures=<br />
This page describes the structure of ADB files. For a list of existing DB2 files (and therefore ADB files) and their contents, see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=WCH3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WCH4 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[DB2#WDB2]] for how to adapt this structure for its .DB2 counterpart.<br />
struct adb_header<br />
{<br />
uint32_t magic; // 'WCH3'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero for ADBs<br />
};<br />
<br />
template<typename record_type><br />
struct adb_file<br />
{<br />
adb_header header;<br />
// static_assert (header.record_size == sizeof (record_type));<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WCH4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings and an optional block after the string block that contains non-inline IDs.<br />
<br />
It is worth noting that min_id and max_id fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present.<br />
<br />
Note that, in most cases, the non-inline IDs structure still exists, even if it is made completely redundant by the existence of the offset map. After all, why do work once when you could do it twice?<br />
<br />
==Structure==<br />
See [[DB2#WDB3]] for how to adapt this structure for its .DB2 counterpart.<br />
template<typename record_type><br />
struct wch4_file<br />
{<br />
adb_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WCH5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.22345|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and was phased out in favor of WCH6 in Legion (Patch 7.0.3 Build 21414). As such, support for WCH5 is likely important for almost nobody.<br />
<br />
While not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
It's worth noting that WCH5 is missing two header fields compared to its .DB2 counterpart. Since the deduplication technology we call 'copy_table' is not employed in any of the .ADB formats, copy_table_size variable was always zero anyway. As for flags, Blizzard always loads the .DB2 file before loading the .ADB file, so the flags field being in both files is just redundant (for them); any .ADB file will be read using the flags field from the .DB2 file of the same name. This is inconvenient for us, as most (if not all) of us load .ADB files standalone at the moment, but there are workarounds (such as having a configuration file to remember the last used flags value for each named .DB2 file).<br />
<br />
After Legion build 21737, an additional header field was added (right before 'build') called 'layout_hash'. This field's appearance is similar to the change to the [[DB2#WDB5|WDB5]] header made in the same build, except in that case, the 'build' field was removed when the 'layout_hash' field was added. For the post-21737 WCH5 header, both fields are present.<br />
<br />
==Structure==<br />
See [[DB2#WDB4]] or [[DB2#WDB5]] for how to adapt this structure for its .DB2 counterpart.<br />
struct wch5_adb_pre_21737_header<br />
{<br />
uint32_t magic; // 'WCH5'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
struct wch5_adb_post_21737_header<br />
{<br />
uint32_t magic; // 'WCH5' for .adb (cache)<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4 but the entire array only counts as '1' field for WCH5<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // from build 21737 onward, this field is present<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
template<typename record_type><br />
struct wch5_file<br />
{<br />
wch5_adb_post_21737_header header;<br />
struct offset_map_entry // Optional - can be detected by pulling 'flags' from relevant DB2<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count]; // Optional - can be detected by pulling 'flags' from relevant DB2<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WCH6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22345|max_expansionlevel=7|max_build=7.0.3.22451|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22345) and was phased out in favor of WCH7 in Legion (Patch 7.0.3 Build 22451).<br />
<br />
Literally nothing changed in WCH6. The introduction of this incremented signature was probably a temporary countermeasure to combat a problem Blizzard was having with 'build' not being updated successfully. This would leave ADBs around basically forever (since they are only wiped when current build is greater than the one in the ADBs being loaded), which could cause conflicts if you downloaded an updated client with new DB2s.<br />
<br />
=WCH7=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22451|max_expansionlevel=7|max_build=7.0.3.22484|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22451) and was phased out in favor of WCH8 in Legion in the next build (Patch 7.0.3 Build 22484).<br />
<br />
A new header field has been added between 'record_count' and 'field_count'. This field designates the size of a new table which is located between the string table and the non-inline IDs table. All records are padded to 'record_size'.<br />
<br />
=WCH8=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22484|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22484) and is still in use today. It was used alongside [[DB2#WDB5|WDB5]].<br />
<br />
The only difference to WCH7 is the padding of files with an offset map. These are padded as per the WCH5 specification again; presumably as this was unnecessarily changed in WCH7.<br />
<br />
=DBCache.bin=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7}}<br />
[[DB2#WDB5|WDB5]] and [[ADB#WCH8|WCH8]] were retired simultaneously in Patch 7.2.0 Build 23436. All files in the 'ADB' folder were consolidated into a single new file, DBCache.bin. It contains server-provided new and changed records for all client-side database tables. It even sometimes contains server-side database changes (entries with an unknown 'table_hash'). It is theorized that the file is a sort of 'packet replayer' for the WoW executable, used for updating client-side DB2s to latest hotfixes on boot up, before confirming with the server about the newest changes.<br />
<br />
The file uses the same record structures as the WoW binary requires but with none of the mechanics found in newer [[DB2]] formats. There is no offset map, field compression of any kind, and no deduplication. Strings are inline, and IDs are always provided before the record, regardless of whether or not there is an ID value in the record data additionally.<br />
<br />
There have previously been two versions of this page which different naming:<br />
* The <tt>index</tt> is not exactly an index but a push ID. Multiple records will have the same "index" when pushed at the same time.<br />
* <tt>version</tt> was referred to as <tt>region_ish</tt> in both header and entry, which may be utter bogus though. It listed 1 = us, 3 = eu, 4 = ptr, which does indeed seem to be more version than region. This would mean though that entries should also refer to it as <tt>version</tt>.<br />
* <tt>sha256</tt> was marked for {{Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2}} rather than {{Sandbox/PrettyVersion|expansionlevel=7|build=7.2.0.23436}}.<br />
<br />
==Version 1==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.2.0.23780|max_exclusive=1}}<br />
<br />
struct dbcache_file<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry entry;<br />
}<br />
};<br />
<br />
==Version 2 through Version 4==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23780|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
<br />
Version 2 added a new field directly after 'magic' in 'dbcache_entry'. At first, it was thought that the new field was just an expansion of the signature, as the field looked like 'XFTH2' for a long time. However, it appears to be something different.<br />
<br />
There were no structure changes in Version 3 or Version 4.<br />
<br />
struct dbcache_file_v2<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 5 and Version 6==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436}}<br />
<br />
Version 5 added a new field directly after 'build_id' in 'dbcache_file_header'. It is a verification hash, calculated using the current file plus a specific random number embedded in the current WoW executable. This number changes for every build. This was likely added in order to prevent players from being able to load arbitrary gamedata easily using custom DBCache.bin files. (Note that this is still entirely possible, just much more tedious.)<br />
<br />
There were no structure changes in Version 6.<br />
<br />
struct dbcache_file_v5<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t verification_hash[32];<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 7==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8}}<br />
<br />
Version 7 removed the 'region_ish' field and moved the 'data_size' two fields back towards the end of the entry structure.<br />
<br />
struct dbcache_file_v7<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t verification_hash[32];<br />
};<br />
<br />
struct dbcache_entry_v7<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint32_t data_size; // Moved from before 'table_hash' to after 'record_id' in Version 7<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v7 entry;<br />
}<br />
};<br />
<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=ADB&diff=28580ADB2019-12-29T01:40:20Z<p>Simca: /* Version 5 and Version 6 */ Changed 'sha256' to 'verification_hash'.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
The structures described here are used in [[ADB|ADB files]], which are a cache of dynamically streamed database entries, typically used for hotfixes and content that Blizzard wants to hide from dataminers until the last minute.<br />
<br />
For more information, please check out the page on [[DB2|DB2 files]]. This article assumes you already have a basic understanding of how they function.<br />
<br />
=Table content structures=<br />
This page describes the structure of ADB files. For a list of existing DB2 files (and therefore ADB files) and their contents, see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=WCH3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WCH4 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[DB2#WDB2]] for how to adapt this structure for its .DB2 counterpart.<br />
struct adb_header<br />
{<br />
uint32_t magic; // 'WCH3'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero for ADBs<br />
};<br />
<br />
template<typename record_type><br />
struct adb_file<br />
{<br />
adb_header header;<br />
// static_assert (header.record_size == sizeof (record_type));<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WCH4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings and an optional block after the string block that contains non-inline IDs.<br />
<br />
It is worth noting that min_id and max_id fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present.<br />
<br />
Note that, in most cases, the non-inline IDs structure still exists, even if it is made completely redundant by the existence of the offset map. After all, why do work once when you could do it twice?<br />
<br />
==Structure==<br />
See [[DB2#WDB3]] for how to adapt this structure for its .DB2 counterpart.<br />
template<typename record_type><br />
struct wch4_file<br />
{<br />
adb_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WCH5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.22345|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and was phased out in favor of WCH6 in Legion (Patch 7.0.3 Build 21414). As such, support for WCH5 is likely important for almost nobody.<br />
<br />
While not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
It's worth noting that WCH5 is missing two header fields compared to its .DB2 counterpart. Since the deduplication technology we call 'copy_table' is not employed in any of the .ADB formats, copy_table_size variable was always zero anyway. As for flags, Blizzard always loads the .DB2 file before loading the .ADB file, so the flags field being in both files is just redundant (for them); any .ADB file will be read using the flags field from the .DB2 file of the same name. This is inconvenient for us, as most (if not all) of us load .ADB files standalone at the moment, but there are workarounds (such as having a configuration file to remember the last used flags value for each named .DB2 file).<br />
<br />
After Legion build 21737, an additional header field was added (right before 'build') called 'layout_hash'. This field's appearance is similar to the change to the [[DB2#WDB5|WDB5]] header made in the same build, except in that case, the 'build' field was removed when the 'layout_hash' field was added. For the post-21737 WCH5 header, both fields are present.<br />
<br />
==Structure==<br />
See [[DB2#WDB4]] or [[DB2#WDB5]] for how to adapt this structure for its .DB2 counterpart.<br />
struct wch5_adb_pre_21737_header<br />
{<br />
uint32_t magic; // 'WCH5'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
struct wch5_adb_post_21737_header<br />
{<br />
uint32_t magic; // 'WCH5' for .adb (cache)<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4 but the entire array only counts as '1' field for WCH5<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // from build 21737 onward, this field is present<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
template<typename record_type><br />
struct wch5_file<br />
{<br />
wch5_adb_post_21737_header header;<br />
struct offset_map_entry // Optional - can be detected by pulling 'flags' from relevant DB2<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count]; // Optional - can be detected by pulling 'flags' from relevant DB2<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WCH6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22345|max_expansionlevel=7|max_build=7.0.3.22451|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22345) and was phased out in favor of WCH7 in Legion (Patch 7.0.3 Build 22451).<br />
<br />
Literally nothing changed in WCH6. The introduction of this incremented signature was probably a temporary countermeasure to combat a problem Blizzard was having with 'build' not being updated successfully. This would leave ADBs around basically forever (since they are only wiped when current build is greater than the one in the ADBs being loaded), which could cause conflicts if you downloaded an updated client with new DB2s.<br />
<br />
=WCH7=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22451|max_expansionlevel=7|max_build=7.0.3.22484|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22451) and was phased out in favor of WCH8 in Legion in the next build (Patch 7.0.3 Build 22484).<br />
<br />
A new header field has been added between 'record_count' and 'field_count'. This field designates the size of a new table which is located between the string table and the non-inline IDs table. All records are padded to 'record_size'.<br />
<br />
=WCH8=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22484|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22484) and is still in use today. It was used alongside [[DB2#WDB5|WDB5]].<br />
<br />
The only difference to WCH7 is the padding of files with an offset map. These are padded as per the WCH5 specification again; presumably as this was unnecessarily changed in WCH7.<br />
<br />
=DBCache.bin=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7}}<br />
[[DB2#WDB5|WDB5]] and [[ADB#WCH8|WCH8]] were retired simultaneously in Patch 7.2.0 Build 23436. All files in the 'ADB' folder were consolidated into a single new file, DBCache.bin. It contains server-provided new and changed records for all client-side database tables. It even sometimes contains server-side database changes (entries with an unknown 'table_hash'). It is theorized that the file is a sort of 'packet replayer' for the WoW executable, used for updating client-side DB2s to latest hotfixes on boot up, before confirming with the server about the newest changes.<br />
<br />
The file uses the same record structures as the WoW binary requires but with none of the mechanics found in newer [[DB2]] formats. There is no offset map, field compression of any kind, and no deduplication. Strings are inline, and IDs are always provided before the record, regardless of whether or not there is an ID value in the record data additionally.<br />
<br />
There have previously been two versions of this page which different naming:<br />
* The <tt>index</tt> is not exactly an index but a push ID. Multiple records will have the same "index" when pushed at the same time.<br />
* <tt>version</tt> was referred to as <tt>region_ish</tt> in both header and entry, which may be utter bogus though. It listed 1 = us, 3 = eu, 4 = ptr, which does indeed seem to be more version than region. This would mean though that entries should also refer to it as <tt>version</tt>.<br />
* <tt>sha256</tt> was marked for {{Sandbox/PrettyVersion|expansionlevel=7|build=7.3.2}} rather than {{Sandbox/PrettyVersion|expansionlevel=7|build=7.2.0.23436}}.<br />
<br />
==Version 1==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.2.0.23780|max_exclusive=1}}<br />
<br />
struct dbcache_file<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry entry;<br />
}<br />
};<br />
<br />
==Version 2 through Version 4==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23780|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
<br />
Version 2 added a new field directly after 'magic' in 'dbcache_entry'. At first, it was thought that the new field was just an expansion of the signature, as the field looked like 'XFTH2' for a long time. However, it appears to be something different.<br />
<br />
There were no structure changes in Version 3 or Version 4.<br />
<br />
struct dbcache_file_v2<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 5 and Version 6==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436}}<br />
<br />
Version 5 added a new field directly after 'build_id' in 'dbcache_file_header'. It is a verification hash, calculated using the current file plus a specific random number embedded in the current WoW executable. This number changes for every build. This was likely added in order to prevent players from being able to load arbitrary gamedata easily using custom DBCache.bin files. (Note that this is still entirely possible, just much more tedious.)<br />
<br />
There were no structure changes in Version 6.<br />
<br />
struct dbcache_file_v5<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t verification_hash[32];<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 7==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8}}<br />
<br />
Version 7 removed the 'region_ish' field and moved the 'data_size' two fields back towards the end of the entry structure.<br />
<br />
struct dbcache_file_v7<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t sha256[32];<br />
};<br />
<br />
struct dbcache_entry_v7<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint32_t data_size; // Moved from before 'table_hash' to after 'record_id' in Version 7<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v7 entry;<br />
}<br />
};<br />
<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=28577DB22019-12-28T17:33:24Z<p>Simca: Fixed signs on WDC2+ string offset changes</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for pallet_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - combined size of record data after this section - combined size of string blocks before this section<br />
<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Additionally, it calculates the combined size of the string block data before this section and adds that value into the result. Lastly, it calculates the combined size of all record data blocks after this section and subtracts that value from the result. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the parts where the 'combined size of string block data before this section' and 'combined size of all record data blocks after this section' need to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before. When calculating WDC2+ string offsets, you need to subtract out the record data size of any sections that occur after the current section and subtract out the size of any string data sections that occur before the current section. This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[section_headers.offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[section_headers.offset_map_id_count];<br />
};<br />
section data_sections[header.section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=TACT&diff=28531TACT2019-12-25T15:54:16Z<p>Simca: /* World of Warcraft */ Added 275's TactKey.</p>
<hr />
<div>TACT ('''T'''rusted '''A'''pplication '''C'''ontent '''T'''ransfer) is the name of the content transfer part of [[NGDP]]. It is often used to install games that use [[CASC]] filesystems, but also supports non-CASC based products.<br />
<br />
=Product Information=<br />
Upon initial release, TACT used regular HTTP to retrieve information for its products. In early 2018, Blizzard started working on a new system that has superseded this. More information about the new system can be found on the [[Ribbit]] page. As of March/April 2019 Ribbit is now the primary system for retrieving version information for most products. HTTP version information has already diverged in some places and should probably only be used as a fallback.<br />
<br />
==HTTP URLs==<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="width: 200px" | URL<br />
! style="width: 550px" | Description<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/cdns || a table of CDN domains available with game data per region<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/versions || current version, buildconfig, cdnconfig, productconfig and optionally keyring per region<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/bgdl || similar to versions, but tailored for use by the Battle.net App background downloader<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blobs || contains InstallBlobMD5 and GameBlobMD5<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blob/game || a blob file that regulates game functionality for the Battle.net App<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blob/install || a blob file that regulates installer functionality for the game in the Battle.net App<br />
|}<br />
<br />
==Products==<br />
The following products are known to have existed at one point. When this page refers to a product it refers to the TACT Product. Agent UIDs are specific to [[Agent]].<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Non-Games<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent UID<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| agent || agent || Battle.net Agent || style="color: green;" | Active<br />
|-<br />
| agent_test || || Probably Agent test || <br />
|-<br />
| bna || battle.net || Battle.net App || style="color: green;" | Active<br />
|-<br />
| bts || || Bootstrapper || style="color: orange;" | Partial (only versions as of now, cdnpath=tpr/bnt004)<br />
|-<br />
| catalogs || || Catalog || style="color: green;" | Active<br />
|-<br />
| clnt || || Client || style="color: red;" | Deprecated<br />
|-<br />
| demo || || || style="color: orange;" | Partial<br />
|-<br />
| test || || || style="color: red;" | Deprecated<br />
|}<br />
<br />
Most games have multiple products. Tables for these are collapsed by default to not affect page length. Click Expand to see all known products for a game. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Diablo III <br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| d3 || diablo3''(_locale)'' || Diablo 3 Retail || style="color: green;" | Active<br />
|-<br />
| d3b || || Diablo 3 Beta (2013) || style="color: orange;" | Partial<br />
|-<br />
| d3cn || d3cn || Diablo 3 China || style="color: green;" | Active<br />
|-<br />
| d3cnt || || Diablo 3 China Test (?) || style="color: orange;" | Unused (everything empty)<br />
|-<br />
| d3t || diablo3_ptr''(_locale)'' || Diablo 3 Test || style="color: green;" | Active<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Heroes of the Storm<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| bnt || || Heroes of the Storm Alpha || style="color: red;" | Deprecated<br />
|-<br />
| hero || heroes || Heroes of the Storm Retail || style="color: green;" | Active<br />
|-<br />
| heroc || heroes_tournament || Heroes of the Storm Tournament || style="color: green;" | Active<br />
|-<br />
| herot || heroes_ptr || Heroes of the Storm Test || style="color: green;" | Active<br />
|-<br />
| storm || || Heroes of the Storm || style="color: red;" | Deprecated<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Hearthstone<br />
|-<br />
| hsb || hs_beta || Hearthstone Retail || style="color: green;" | Active<br />
|-<br />
| hsc || hs_tournament || Hearthstone Chournament || style="color: green;" | Active<br />
|-<br />
| hst || || Hearthstone Test || style="color: orange;" | Partial<br />
|-<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Overwatch (aka Prometheus)<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| pro || prometheus || Overwatch Retail || style="color: green;" | Active<br />
|-<br />
| proc || prometheus_tournament || Overwatch Tournament US || style="color: green;" | Active<br />
|-<br />
| proc_cn || prometheus_tournament_cn || Overwatch Tournament China || style="color: green;" | Active<br />
|-<br />
| proc_eu || prometheus_tournament_eu || Overwatch Tournament Europe || style="color: green;" | Active<br />
|-<br />
| proc_kr || prometheus_tournament_kr || Overwatch Tournament Korea || style="color: green;" | Active<br />
|-<br />
| proc2 || prometheus_tournament2 || Overwatch Professional 2 || style="color: green;" | Active<br />
|-<br />
| proc2_cn || prometheus_tournament2_cn || Overwatch Professional 2 China || style="color: green;" | Active<br />
|-<br />
| proc2_eu || prometheus_tournament2_eu || Overwatch Professional 2 Europe || style="color: green;" | Active<br />
|-<br />
| proc2_kr || prometheus_tournament2_kr || Overwatch Professional 2 Korea || style="color: green;" | Active<br />
|-<br />
| proc3 || prometheus_tournament3 || Overwatch Tournament (Dev) || <br />
|-<br />
| procr || prometheus_tournament_viewer || Overwatch League Stage 3 || <br />
|-<br />
| procr2 || prometheus_tournament_viewer_2 || Overwatch League Stage 2 || <br />
|-<br />
| prodev || prometheus_dev || Overwatch Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| proe || || || Not on public CDNs<br />
|-<br />
| prot || prometheus_test || Overwatch Test || style="color: green;" | Active<br />
|-<br />
| prov || prometheus_vendor || Overwatch Vendor || style="color: blue;" | Active (encrypted)<br />
|-<br />
| proms || prometheus_viewer || Overwatch World Cup Viewer || style="color: orange;"| Partial<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|StarCraft 1<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
<br />
! width="200" | Status<br />
|-<br />
| s1 || s1 || StarCraft 1 || style="color: green;" | Active<br />
|-<br />
| s1a || || Starcraft 1 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| s1t || s1_ptr || StarCraft 1 Test || style="color: green;" | Active<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|StarCraft II<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| s2 || s2''(_locale)'' || StarCraft II Retail || style="color: green;" | Active<br />
|-<br />
| s2b || || StarCraft II Beta || style="color: red;" | Deprecated<br />
|-<br />
| s2t || s2_ptr''(_locale)'' || StarCraft II Test || style="color: red;" | Deprecated<br />
|-<br />
| sc2 || || StarCraft II || style="color: red;" | Deprecated<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Warcraft III<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| w3 || w3 || Warcraft III || style="color: green;"| Active<br />
|-<br />
| w3t || w3_ptr || Warcraft III Public Test || style="color: green;"| Active<br />
|-<br />
| war3 || || Warcraft III (old) || style="color: orange;"| Partial<br />
|-<br />
| w3b || w3_beta || Warcraft III: Reforged Beta ||<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|World of Warcraft<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| wow || wow''(_locale)'' || World of Warcraft Retail || style="color: green; | Active<br />
|-<br />
| wow_beta || wow_beta || World of Warcraft Alpha/Beta || style="color: green; | Active<br />
|-<br />
| wow_classic || wow_test, later wow_classic || World of Warcraft Classic || style="color: green; | Active<br />
|-<br />
| wow_classic_beta || wow_classic_beta || World of Warcraft Classic Beta || style="color: green; | Active<br />
|- <br />
| wowdev || || World of Warcraft Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowdemo || || World of Warcraft (Classic) Demo || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowe1 || wow_event1 || World of Warcraft Event 1 || style="color: green;" | Active<br />
|-<br />
| wowe2 || wow_event2 || World of Warcraft Event 2 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowe3 || wow_event3''(_locale)'' || World of Warcraft Event 3 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowt || wow_ptr''(_locale)'' || World of Warcraft Test || style="color: green;" | Active<br />
|-<br />
| wowv || wow_vendor || World of Warcraft Vendor || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowv2 || wow_vendor2 || World of Warcraft Vendor 2 (Classic) || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowz || wow_submission || World of Warcraft Submission (previously Vendor) || style="color: green; | Active<br />
|}<br />
<br />
===Non-Blizzard===<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Call of Duty<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| viper || viper || Call of Duty Black Ops 4 || style="color: green;" | Active<br />
|-<br />
| viperdev || viper_alpha || Call of Duty Black Ops 4 - Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| viperv1 || viper_vendor || Call of Duty Black Ops 4 Vendor || <br />
|-<br />
| odin || odin_placeholder → odin || Call of Duty: Modern Warfare || <br />
|-<br />
| odindev || odin_dev || Call of Duty: Modern Warfare Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| odinv1 || odin_vendor1 || Call of Duty: Modern Warfare Vendor 4 ||<br />
|-<br />
| odinv2 || odin_vendor2 || Call of Duty: Modern Warfare Vendor 4 ||<br />
|-<br />
| odinv3 || odin_vendor3 || Call of Duty: Modern Warfare Vendor 4 ||<br />
|-<br />
| odinv4 || odin_vendor4 || Call of Duty: Modern Warfare Vendor 4 ||<br />
|-<br />
| odina || odin_alpha || another cod mw dev channel ||<br />
|-<br />
| odinb || odin_beta || public (paid) beta ||<br />
|-<br />
| odine || odin_event || tournament/event<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="4"|Destiny 2<br />
|-<br />
! width="90" | TACT Product<br />
! width="90" | Agent Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| dst2 || destiny_2 || Destiny 2 || style="color: green;" | Active<br />
|-<br />
| dst2a || destiny2_alpha || Destiny 2 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| dst2dev || destiny2_takehome || Destiny 2 "takehome" development || style="color: orange;" | Partial (encrypted?)<br />
|-<br />
| dst2e1 || destiny2_event || Destiny 2 Event || Active; probably 1..9<br />
|-<br />
| dst2igr || || Destiny 2 Internet Game Room || style="color: green;" | Active<br />
|-<br />
| dst2t || destiny2_ptr || Destiny 2 Public Test || style="color: green;" | Active<br />
|}<br />
<br />
=Glossary=<br />
This page uses many terms. Here's an overview of stuff you need to know to be able to understand most of the explanations below. <br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="width: 100px"| Term<br />
! style="width: 550px" | Description<br />
|-<br />
|Filename || The file's real filename. Note that one file in TACT can have many names referring to it - essentially, one encoding key can map to many different name hashes.<br />
|-<br />
|Name Hash/Lookup || The filename after being hashed with the [[#hashpath|Jenkins Hash]].<br />
|-<br />
|Content Hash/CKey || The MD5 of the entire file in its uncompressed state; the purest representation of the data.<br />
|-<br />
|Encoding Hash/EKey || MD5 hash of the potentially encoded file. For unencoded files, the content hash/CKey. For chunkless [[BLTE]] files lacking a chunk table, this hash covers the entire encoded file. For chunked [[BLTE]] files, this hash covers only the [[BLTE]] headers including chunk table, as the chunk table contains hashes of the content of each chunk. A given file can be encoded in many ways, and a single content hash may potentially have multiple encoding keys.<br />
|-<br />
| CDN Key || The key used to lookup a file on the CDN. Same as encoding hash/EKey.<br />
|}<br />
<br />
=File types=<br />
Files are stored on CDNs in the following format: <tt>http://(cdnsHost)/(cdnsPath)/(pathType)/(FirstTwoHexOfHash)/(SecondTwoHexOfHash)/(FullHash)</tt><br />
<br />
Current list of CDNs for a specific product can be found in the <tt>http://us.patch.battle.net:1119/(product)/cdns</tt> file or via [[Ribbit]] in the <tt>v1/products/(product/)cdns</tt> endpoint. Blizzard regularly shuffles around CDNs and sometimes even adds/removes CDNs, so be sure to parse <tt>cdns</tt> to stay up to date. The CDN path found in the <tt>cdns</tt> file very rarely changes, the only known occurence is Overwatch changing from <tt>tpr/pro</tt> to <tt>tpr/ovw</tt>.<br />
<br />
Known path types are:<br />
* config - contains the three types of config files: Build configs, CDN configs, and Patch configs<br />
* data - contains archives, indexes, and unarchived standalone files (typically binaries, mp3s, and movies and files mentioned in buildconfig like root, install and download)<br />
* patch - contains patch manifests, files, archives, indexes<br />
<br />
Blizzard regularly cleans old builds from the CDN so any example files mentioned in this article might be unavailable at the time of reading (feel free to update them to something more recent if this is the case).<br />
<br />
Product configs referred to in the versions file are generally saved in <tt>tpr/configs</tt> for all games, but to be sure this path should be found by parsing the cdns file for the product in question.<br />
==Config Files==<br />
<br />
===Build Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/22/38/2238ab9c57b672457a2fa6fe2107b388</tt><br />
<br />
Some of the files listed in this file are explained later on in this article.<br />
<br />
What follows is a table for all known variables that have been seen in a buildconfig.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Root|root]] || Content hash of the '''decoded''' root file, look this up in encoding to get the '''encoded''' hash.<br />
|-<br />
| [[#Install|install]] || First key is the content hash of the '''decoded''' install file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash/CDN key.<br />
|-<br />
| install-size || Install size(s) corresponding to the install hash(es). Absent in older WoW builds.<br />
|-<br />
| [[#Download|download]] || First key is the content hash of the '''decoded''' download file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash.<br />
|-<br />
| download-size || Download size(s) corresponding to the download hash(es). Absent in older WoW builds.<br />
|-<br />
| [[#Download Size|size]] || CKey and EKey of the download size file, respectively. Introduced in WoW build 27547.<br />
|-<br />
| size-size || Download size sizes corresponding to the download size keys.<br />
|-<br />
| build-partial-priority || Content hash:block size pairs for priority non-archived files, ordered by priority. The block size appears to be the amount to request in each partial HTTP request. These files are [[BLTE]]-encoded and usually but not always split into blocks of fixed uncompressed size where the partial priority block size is a power-of-two multiple of the file's uncompressed block size. First seen in build 24116. Optional.<br />
|-<br />
| partial-priority || Content hash of a partial download file containing priority files to download first; note that the sizes in the download file entries are download block sizes, '''not''' file sizes. First seen in build 22231, replaced by build-partial-priority in build 24116. Optional.<br />
|-<br />
| partial-priority-size || Unknown: always 0 if present. Present if partial-priority is present.<br />
|-<br />
| [[#Encoding|encoding]] || First key is the content hash of the '''decoded''' encoding file. Second key is the '''encoded''' hash. If either none or 1 is found, the game (at least Wow) switches to plain-data (?) . Seen in build 20173<br />
|-<br />
| encoding-size || Encoding sizes corresponding to the encoding hashes.<br />
|-<br />
| [[#Patch|patch]] || The manifest of patchable data files. Does not include the metadata files e.g. encoding, whose patches are specified in the patch config file. Optional.<br />
|-<br />
| patch-size || Size of the patch manifest, if any. Optional.<br />
|-<br />
| [[#Patch_Config|patch-config]] || Content hash of non-encoded patch config (see Patch Config)<br />
|-<br />
| build-attributes || Optional. Seen in agent.<br />
|-<br />
| build-branch || Optional. Presumably the SCM branch built.<br />
|-<br />
| build-comments || Optional.<br />
|-<br />
| build-creator || Optional. Presumably the user who submitted the build.<br />
|-<br />
| build-fixed-hash || Optional. Seen in S2.<br />
|-<br />
| build-replay-hash || Optional. Seen in S2.<br />
|-<br />
| build-name || Optional? Name of the build<br />
|-<br />
| build-playbuild-installer || Optional? Type of installer for the Battle.net app to use<br />
|-<br />
| build-product || Optional? Product name<br />
|-<br />
| build-t1-manifest-version || Optional.<br />
|-<br />
| build-uid || Optional? Program code (see Products)<br />
|}<br />
<br />
===CDN Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/42/33/423364147752a596911aa1de2ff1f6a4</tt><br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Archives|archives]] || CDN keys of all archives (and by appending .index to the hash their indexes)<br />
|-<br />
| [[#Archive-Group_Index_.28.index.29|archive-group]] || CDN key of the the combined index file (see Archive-Group Index)<br />
|-<br />
| file-index || CDN key of .index file for unarchived files. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| file-index-size || Size of unarchived file .index.<br />
|-<br />
| [[#patch-archives|patch-archives]] || CDN keys of patch archives (needs research)<br />
|-<br />
| patch-archive-group || CDN key of probably the combined patch index file (needs research)<br />
|-<br />
| [[#Build_Config|builds]] || List of build configs this config supports (optional)<br />
|-<br />
| patch-file-index || CDN key of .index file for unarchived patches. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| patch-file-index-size || Size of unarchived patch .index.<br />
|}<br />
<br />
===Patch Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/20/f0/20f0593dd1b9fdaeaa7a808f83d48f1d</tt><br />
<br />
This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 in build 19027 (October 10th, 2014).<br />
The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material.<br />
The structure and purpose of all of the fields of this file requires further research.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| patch-entry || Repeats 3 times with patch entries for install, download and encoding file. Builds using the partial-priority (not build-partial-priority) mechanism may additionally have a partial-priority entry.<br />
|-<br />
| patch || Patch manifest file<br />
|-<br />
| patch-size || Size of patch file (optional?)<br />
|}<br />
<br />
====patch-entry====<br />
<br />
The format of these strings is:<br />
<br />
<pre>patch-entry = <type> <content hash> <content size> <BLTE-encoding key> <BLTE-encoded size> <encoding string></pre><br />
<br />
followed by sets of<br />
<br />
<pre><old BLTE-encoding key> <old content size> <patch hash> <patch size></pre><br />
<br />
This serves to pair old files to the patch needed to bring them up to date, as well as providing the information required to detect if the file is already up to date.<br />
<br />
==Data Files==<br />
<br />
Example index: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef.index</tt><br />
<br />
Example archive: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef</tt><br />
<br />
==Patch Files==<br />
[[#Patch]]<br />
<br />
==Encoding table==<br />
The encoding file maps content hashes <tt>C-Key</tt>s to encoded-file hashes <tt>E-Key</tt>s. In addition, there is information on how the files are [[BLTE]]-encoded by <tt>E-Spec</tt>s. <br />
<br />
Blocks in this file are, in this order<br />
<br />
* header<br />
* encoding specification data <tt>ESpec</tt><br />
* content key → encoding key table <tt>CEKeyPageTable</tt><br />
* encoding key → encoding spec table <tt>EKeySpecPageTable</tt><br />
* encoding specification data for the encoding file itself<br />
<br />
An incomplete/outdated 010 Editor template can be found at [https://gist.github.com/heksesang/fdda3e4f8a5ed53b71ed this gist] which can be used to understand page handling.<br />
<br />
====Header====<br />
Header is a constant 0x16 bytes giving size information for the other blocks, mostly.<br />
<br />
struct {<br />
/*0x00*/ char signature[2]; // "EN"<br />
enum {<br />
encoding_version_1 = 1, // {{Sandbox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125}}<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ckey;<br />
/*0x04*/ uint8_BE_t hash_size_ekey;<br />
/*0x05*/ uint16_BE_t CEKeyPageTable_page_size_kb; // in kilo bytes. e.g. 4 in here → 4096 byte pages (default)<br />
/*0x07*/ uint16_BE_t EKeySpecPageTable_page_size_kb; // ^<br />
/*0x09*/ uint32_BE_t CEKeyPageTable_page_count;<br />
/*0x0D*/ uint32_BE_t EKeySpecPageTable_page_count;<br />
/*0x11*/ uint8_BE_t _unknown_x11; // 0 -- sometimes assumed to be part of ESpec_block_size, but actually asserted to be zero by agent<br />
/*0x12*/ uint32_BE_t ESpec_block_size;<br />
/*0x16*/<br />
} header;<br />
<br />
====ESpec====<br />
Encoding specification strings are just a blob of zero terminated strings referenced by <tt>EKeySpecPageTable</tt> with the accumulated size of <tt>header.ESpec_block_size</tt> (including zero terminators).<br />
<br />
The definition of the format for these strings is described on the [[BLTE#Encoding_Specification_.28ESpec.29|BLTE]] page.<br />
<br />
====Page Tables====<br />
The format of the two page tables is the same: <br />
<br />
* an index for fast key → page access, followed by<br />
* the actual pages with specific content<br />
<br />
In both cases, the entries in the lists have the same count, and semi-dynamic size, depending on <tt>header.hash_size_ckey</tt> and <tt>header.hash_size_ekey</tt>. Note that the page checksum size is fixed to MD5's 16 bytes. The position comments below assume the standard key size of 16 bytes.<br />
<br />
struct page_index_t {<br />
/*0x00*/ char first_Xkey[header.hash_size_Xkey]; // where X is c for CEKeyPageTable and e for EKeySpecPageTable<br />
/*0x10*/ char page_md5[0x10];<br />
/*0x20 usually*/<br />
};<br />
<br />
The pages themselves are filled as much possible with actual entry structs. They are padded to the end for alignment of pages. Pages don't actually have to be full.<br />
<br />
=====CEKeyPageTable=====<br />
This table maps one <tt>ckey</tt> to one or more <tt>ekey</tt>s. This means that there can be multiple representations, e.g. an encrypted and an unencrypted version. This isn't usually the case and on reading any of them can be picked, since they represent the same file content. It can be used to pick an already downloaded archive, or unencrypted one, or handle deleted archives.<br />
<br />
struct ckey_ekey_entry_t {<br />
/*0x00*/ uint8_BE_t keyCount;<br />
/*0x01*/ uint40_BE_t file_size; // of the non-encoded version of the file<br />
/*0x06*/ char ckey[header.hash_size_ckey]; // this ckey is represented by…<br />
/*0x16*/ char ekey[header.hash_size_ekey][keyCount]; // …these ekeys<br />
/*0x26 usually*/<br />
} page_entries[];<br />
<br />
=====EKeySpecPageTable=====<br />
This table maps one <tt>ekey</tt> to the corresponding <tt>espec</tt> describing how the encoding happened.<br />
<br />
struct ekey_espec_entry_t {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint32_BE_t espec_index; // not an offset but an index, assuming zero-terminated espec strings<br />
/*0x14*/ uint40_BE_t file_size; // of the encoded version of the file<br />
/*0x19 usually*/<br />
} page_entries[];<br />
<br />
==Install manifest==<br />
File signature: "IN"<br />
<br />
The install file lists files installed on disk. Since the install file is shared by architectures and OSs, there are also tags to select a subset of files. When using multiple tags, a binary combination of the bitfields of files to be installed can be created. <br />
<br />
====Header Structure====<br />
The file begins with a 10 byte header describing the number of tags and files listed. '''Structure names were invented by the author of this page.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[2] || FileSignature || "IN"<br />
|-<br />
| 0x02 || uint8_t || Version? || 1<br />
|-<br />
| 0x03 || uint8_t || hash_size || size of hashes used for files (usually md5 -> 16)<br />
|-<br />
| 0x04 || uint16_BE_t || num_tags || number of tags in header of file<br />
|- <br />
| 0x06 || uint32_BE_t || num_entries || The number of entries in the body of the file<br />
|}<br />
<br />
====Tags Structure====<br />
After the header, an array with information about available tags follows. Each tag has a bitfield listing the files installed when the given tag is chosen. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || name || <br />
|- <br />
| uint16_BE_t || type || A number shared amongst specific flags. Actual meaning is [[#Product_Specific|specific to products]].<br />
|- <br />
| char[divru (header.entries, CHAR_BIT)] || files || A bitfield that lists which files are installed when the specified tag is installed.<br />
|}<br />
<br />
====Files Structure====<br />
The remainder of the file is populated by a list of files with their content hash, each a variable size (due to the strings). Structure names were invented by the author of this page.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || FileName || The name of the file.<br />
|- <br />
| char[header.hash_size] || hash || The hash of the uncompressed file. Usually MD5.<br />
|- <br />
| uint32_BE_t || Size || The size of the file.<br />
|}<br />
<br />
====C-like structure====<br />
char I; char N;<br />
uint8_BE_t _unk3;<br />
uint8_BE_t hash_size;<br />
uint16_BE_t num_tags;<br />
uint32_BE_t num_files;<br />
<br />
struct {<br />
string name;<br />
uint16_BE_t type;<br />
char flags[divru (num_files, CHAR_BIT)];<br />
} tags[num_tags];<br />
<br />
struct {<br />
string name;<br />
char hash[hash_size];<br />
uint32_BE_t size;<br />
} files[num_files];<br />
<br />
==Download manifest==<br />
The download file lists all files stored in the data archives.<br />
The client uses this to download files ahead of time, without it, the client will download on demand which can lead to issues. <br />
The download priority is set inside the entries with 0 being the highest and 2 the lowest however, if the game is running, missing assets in the player's vicinity take precedence.<br />
<br />
Just like the install file, the download file is shared across all architectures and locales so utilizes the same bitfield-tag system to assess what subset of files are needed.<br />
<br />
NOTE: partial-priority download files do not contain the actual file sizes but redefine the FileSize field to ChunkSize.<br />
<br />
This file has this structure:<br />
<br />
* Header<br />
* Entries[Header.EntryCount]<br />
* Tags[Header.TagsCount]<br />
<br />
====Download Header====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "DL")<br />
|-<br />
| char || Version || 1 < 7.3.0, 2<br />
|-<br />
| char || ChecksumSize || Always 0x10<br />
|-<br />
| char || unk || ??? Always 1<br />
|-<br />
| int [BE] || EntryCount || The amount of file entries in this file<br />
|-<br />
| short [BE] || TagCount || The amount of tag entries in this file<br />
|}<br />
<br />
====Download Entry====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="120" | Version Added<br />
! width="810" | Description<br />
|-<br />
| char || Unk || 2 || ??? Appears to be a boolean. Currently only set to 1 on 4 specific records<br />
|-<br />
| char[16] || Hash || 1 || This hash is found in every node of the encoding file. (Reverse lookup) MD5<br />
|-<br />
| uint40_t [BE] || FileSize || 1 || The compressed size of the file<br />
|-<br />
| char || DownloadPriority || 1 || 0 = highest, 2 = lowest; -1 for the Install since {{Unverified|{{Sandbox/PrettyVersion|expansionlevel=8}}}}<br />
|-<br />
| char[4] || Unk || 1 || ???<br />
|}<br />
<br />
====Download Tag====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| string || Name || A C-String indicating this tag's Name.<br />
|-<br />
| short [BE] || Type || Hash type<br />
|-<br />
| char[N] || Bits || an array of size N = Header.EntryCount / 8 + (Header.EntryCount % 8 > 0 ? 1 : 0); that is basically a massive bit mess. Use Schroeppel's 8 bits reverse function on it to have bits.<br />
|}<br />
<br />
====code-ish====<br />
struct {<br />
/*0x00*/ char signature[2]; // "DL"<br />
enum {<br />
download_version_1 = 1,<br />
download_version_2 = 2, // {{Unverified|{{Sandbox/VersionRange|min_expansionlevel=7|min_build=7.3.0.???}}}}<br />
download_version_3 = 3,<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ekey;<br />
/*0x04*/ uint8_BE_t has_checksum_in_entry;<br />
/*0x05*/ uint32_BE_t entry_count;<br />
/*0x09*/ uint16_BE_t tag_count;<br />
/*0x0b*/<br />
<br />
#if version ≥ download_version_2<br />
/*0x0b*/ uint8_BE_t {{Unverified|number_of_flag_bytes}}; // defaults to 0, up to {{Unverified|4}}<br />
/*0x0c*/<br />
<br />
#if version >= download_version_3<br />
/*0x0c*/ uint8_BE_t {{Unverified|base_priority}}; // defaults to 0<br />
/*0x0d*/ char _unknown_0d[3]; // As of 1.15.6.2-test4, this is explicitly 0. It is ignored on reading.<br />
/*0x10*/<br />
#endif<br />
#endif<br />
} header;<br />
<br />
struct {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint40_BE_t file_size;<br />
/*0x15*/ uint8_BE_t priority; // {{Unverified|header.base_priority is subtracted on parse}}<br />
/*0x16*/<br />
<br />
#if header.has_checksum_in_entry<br />
/*0x16*/ uint32_BE_t checksum;<br />
/*0x1a*/<br />
#endif<br />
<br />
#if header.version ≥ download_version_2<br />
enum {<br />
download_flag_plugin = 1, // "plugin"<br />
download_flag_plugin_data = 2, // "plugin-data"<br />
}; <br />
uint8_BE_t flags[header.number_of_flag_bytes]; // {{Unverified|defaults to 0 if no flag bytes present}}<br />
#endif<br />
<br />
} entries[header.entry_count];<br />
<br />
struct {<br />
char const name[]; // this string is zero terminated, no fixed size<br />
// thus for readability we start offset at 0 here.<br />
/*0+00*/ uint16_BE_t type; // game specific. usually architecture, category, locale, os, region or alike.<br />
/*0+02*/ char mask[divru (header.entry_count._, CHAR_BIT)]; // if bit is set, entries[bit] is part of this tag<br />
/*0x??*/<br />
} tag[header.tag_count];<br />
<br />
==Download Size ==<br />
Build 27547 introduced the Download Size file of unknown purpose. The file is a stripped-down Download file with partial EKeys and files sorted by encoded files size. The purpose of this file is not clear.<br />
<br />
struct Header<br />
{<br />
char signature[2]; // "DS"<br />
uint8_t version; // only known: 1 (as of agent 6700)<br />
uint8_t ekeySize; // 9, up to 23 possible, but agent 6700 hardcodes 9, so, uh.<br />
uint32_BE_t numFiles;<br />
uint16_BE_t numTags;<br />
uint40_BE_t totalSize; // Size of all files combined<br />
};<br />
<br />
struct TagEntry<br />
{<br />
char name[]; // Null-terminated<br />
uint16_BE_t type;<br />
char fileMask[(hdr.numFiles + 7) / 8];<br />
}<br />
<br />
struct FileEntry<br />
{<br />
char ekey[hdr.ekeySize];<br />
uint32_BE_t esize;<br />
};<br />
<br />
SizeHeader hdr;<br />
TagEntry tags[hdr.numTags];<br />
FileEntries files[hdr.numFiles]; // Sorted descending by esize<br />
<br />
==Patch==<br />
<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "PA")<br />
|-<br />
| char || version || 1 or 2<br />
|-<br />
| char || file_key_size || <= 0x10<br />
|-<br />
| char || size_b || <= 0x10<br />
|-<br />
| char || patch_key_size || <= 0x10<br />
|-<br />
| char || block_size_bits || 2 <= block_size_bits <= 24. block size == 2^block_size_bits.<br />
|-<br />
| short || block_count ||<br />
|-<br />
| char || flags ||<br />
|-<br />
| char[16] || EncodingCkey || ckey for encoding file<br />
|-<br />
| char[16] || EncodingEkey || ekey for encoding file<br />
|-<br />
| int || DecodedSize || Decoded encoding file size in bytes<br />
|-<br />
| int || EncodedSize || Encoded encoding file size in bytes<br />
|-<br />
| char || EspecLength || Length of the following string<br />
|-<br />
| char[EspecLength] || EncodingEspec || espec of encoding file<br />
|-<br />
| char[] || ??? || byte array containing blocks entries, blocks, and optional patch tail<br />
|}<br />
<br />
header+entries needs to be less than 0x10000 bytes (at least in wow-18179). md5sum is only checked for header+entries, file might be larger thus.<br />
<br />
struct PatchManifest_Header<br />
{<br />
uint16_BE_t magic; // 'PA'<br />
uint8_t version; // 1 or 2<br />
uint8_t file_key_size; // <= 0x10<br />
uint8_t size_b; // <= 0x10<br />
uint8_t patch_key_size; // <= 0x10<br />
uint8_t block_size_bits; // 12 <= block_size_bits <= 24. max block size == 2^block_size_bits<br />
uint16_BE_t block_count; // (file_key_size + 20) * entry_count + sizeof (PatchManifest_Header) < 0x10000<br />
uint8_t flags; // &1: plain-data mode manifest, &2: extended header <br />
<br />
#if encoding_information_apparently_added_after_18179<br />
uint8_t encoding_ckey[16];<br />
uint8_t encoding_ekey[16]; // probably since PA2<br />
uint32_BE_t decoded_size;<br />
uint32_BE_t encoded_size;<br />
uint8_t encoding_espec_length;<br />
char encoding_format[encoding_espec_length];<br />
#endif<br />
} header;<br />
<br />
struct PatchManifest_Block<br />
{<br />
uint8_t last_file_ckey[header.file_key_size];<br />
uint8_t md5_of_block[16];<br />
uint32_BE_t block_offset; // in this file<br />
} blocks[header.block_count]; // sorted ascending by key<br />
<br />
// at positions given in PatchManifest_Block<br />
struct block<br />
{<br />
struct<br />
{<br />
uint8_t num_patches; // <= 0x10.<br />
uint8_t target_file_ckey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
struct<br />
{<br />
uint8_t source_file_ekey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
uint8_t patch_ekey[header.patch_key_size];<br />
uint32_BE_t patch_size;<br />
uint8_t unk; // some sort of patch index number. first entry seems to always be 1<br />
} patches[num_patches];<br />
} files[]; // count unspecified: read until the next file num_patches would be 0 <br />
// OR block would exceed max block size<br />
};<br />
<br />
// some files have a block of data after the last block of the patch manifest (which may be shorter than max block size). this block appears to be a patch of encoding, but the format is not understood.<br />
<br />
=Product Specific=<br />
In this section, the game/usage specific parts of CASC are describe. While CASC is a generic format, a lot of stuff is hardcoded, like hash sizes. Other parts are actually left up to the implementation, like root files or download tags.<br />
<br />
==World of Warcraft==<br />
===Root===<br />
As of 8.2 there was changes to this format. The client currently parses backwards compatible. There are three changes:<br />
* The file is now chunked with the old content being in a MFST (ManiFeST) chuck.<br />
* blocks can be without name hashes and thus ID-only. <br />
* the name and content hashes are no longer interleaved but two separate arrays<br />
** In memory, the manifest is kept in a FDID → entry map, so when reverse engineering the parser one may think that fnv1a is relevant, but that’s merely the hashmap insert. <br />
** To parse that variable format the client uses to pointers to chash/name and a stride (of 16 for key only and 24 for with name) rather than one for records. It also determines the name hash/key hash base pointers depending on format, allowing them to have one parse able to read all format versions.<br />
<br />
enum locale_flags : uint32_t {<br />
enUS = 0x2,<br />
koKR = 0x4,<br />
frFR = 0x10,<br />
deDE = 0x20,<br />
zhCN = 0x40,<br />
esES = 0x80,<br />
zhTW = 0x100,<br />
enGB = 0x200,<br />
enCN = 0x400,<br />
enTW = 0x800,<br />
esMX = 0x1000,<br />
ruRU = 0x2000,<br />
ptBR = 0x4000,<br />
itIT = 0x8000,<br />
ptPT = 0x10000,<br />
};<br />
enum content_flags : uint32_t {<br />
LoadOnWindows = 0x8, // macOS clients do not read block if flags & 0x108 != 0<br />
LoadOnMacOS = 0x10, // windows clients do not read block if flags & 0x110 != 0<br />
LowViolence = 0x80,<br />
{{Unverified|DoNotLoad}} = 0x100, // neither macOS nor windows clients read blocks with this flag set. {{Unverified|LoadOnMysteryPlatform}}?<br />
{{Unverified|UpdatePlugin}} = 0x800, // only ever set for UpdatePlugin.dll and UpdatePlugin.dylib<br />
Encrypted = 0x8000000,<br />
NoNameHash = 0x10000000,<br />
{{Unverified|UncommonResolution}} = 0x20000000, // denotes non-1280px wide cinematics<br />
Bundle = 0x40000000,<br />
NoCompression = 0x80000000,<br />
};<br />
struct MD5Hash {<br />
char bytes[0x10];<br />
};<br />
<br />
if (format_version == 8.2) {<br />
uint32_t magic; // 'MFST'<br />
uint32_t total_file_count; // sum of all blocks' num_records<br />
uint32_t named_file_count; // sum of those blocks' num_records that have NoNameHash not set<br />
}<br />
<br />
// bool allow_non_named_files = total_file_count ≠named_file_count || format_version < 8.2;<br />
// bool use_old_record_format = magic ≠'MFST';<br />
<br />
struct CASBlock {<br />
uint32_t num_records;<br />
content_flags flags;<br />
locale_flags locale;<br />
int32_t fileDataIDDeltas[num_records]; // each block starts with 0, +1 is implicit per entry, so consecutive ids will have delta=0<br />
<br />
if (use_old_record_format) {<br />
struct CASRecord {<br />
MD5Hash content_key; // MD5 hash of the file's raw data<br />
uint64_t name_hash; // Jenkins96 (lookup3) hash of the file's path <br />
} records[num_records];<br />
}<br />
else {<br />
MD5Hash content_key[num_records]; // MD5 hash of the file's raw data<br />
if (!(allow_non_named_files && flags & NoNameHash)) {<br />
uint64_t name_hash[num_records]; // Jenkins96 (lookup3) hash of the file's path <br />
}<br />
}<br />
<br />
int32 file_data_id (size_t index) const<br />
{<br />
return index == 0<br />
? fileDataIDDeltas[index]<br />
: file_data_id (index - 1) + 1 + fileDataIDDeltas[index];<br />
}<br />
} blocks[]; // count: fill up until end of file<br />
====hashpath====<br />
This function is used in the root file by WoW and other older MPQ-based games to calculate filename lookups.<br />
<br />
hashpath (string path) → uint32_t<br />
{<br />
string normalized = toupper (path).replace (from: '/', to: '\\')<br />
uint32_t pc = 0, pb = 0;<br />
hashlittle2 (normalized, strlen (normalized), &pc, &pb);<br />
return pc;<br />
}<br />
<br />
===Tags===<br />
Values depend on versions, semantic categories are cross version.<br />
<br />
* '''Platform''': The deployment target, i.e. Windows or OSX<br />
* '''Architecture''': Sub-division of the deployment target, i.e. x86_32 or x86_64<br />
* '''Locale''': The same as in [[Localization]]: Files specific to a single localisation of the game.<br />
* '''Region''': Equivalent to the patch server regions, i.e. us, eu, kr, tw, cn.<br />
* '''Category''': A replacement for the MPQ system to tag low priority downloads: speech, text<br />
* '''Alternate''': A special category for censored content.<br />
<br />
====Version specific values====<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125|max_expansionlevel=6|max_build=6.0.1.18761}}<br />
<br />
Architecture = 1, Locale = 2, Platform = 3<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18764|max_expansionlevel=6|max_build=6.2.2.20426}}<br />
<br />
Architecture = 1, Category = 2, Locale = 3, Platform = 4, Region = 5<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.2.2.20438|max_expansionlevel=7}}<br />
<br />
Platform = 1, Architecture = 2, Locale = 3, Region = 4, Category = 5<br />
<br />
{{SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26604|note={{Unverified|Actually alternate was just missing above but schlumpf doesn't want to verify since when}}}}<br />
<br />
enum {<br />
platform = 1,<br />
architecture = 2,<br />
locale = 3,<br />
region = 4,<br />
category = 5,<br />
alternate = 0x4000,<br />
};<br />
<br />
===Shared storage===<br />
As of 8.1 both the WoW client and Battle.net app have started supporting sharing storage containers ([CASC]). There is one Data folder in the root folder with storage for multiple products.<br />
Products and product-specific data (Cache, Erros, Interface/Addons, Logs, WTF, etc) are stored in subfolders. <br />
<br />
Root folder has a .build.info with cached build information for each product sharing the storage, each product subfolder also has a .flavor.info file with the TACT product name (e.g. wowt or wow_classic_beta, should be the same as in .build.info). Both of these files are written by the Battle.net agent and not by the clients themselves. However, if these two files are available the clients will use cached information in the .build.info file instead of getting it from remote.<br />
<br />
=Armadillo=<br />
If <tt>$program/blob/game</tt> or <tt>productconfig</tt> contains <tt>decryption_key_name</tt>, data on CDN is encrypted. The key is retrieved from the keychain or manually entered (a base32 encoded version that contains key and checksum is used). It is stored in <tt>$key_name.ak</tt>. The key file contains the key followed by a the first four bytes of the md5 hash of the key for verification purposes.<br />
<br />
struct {<br />
/*0x00*/ unsigned char key[0x10];<br />
/*0x10*/ unsigned char md5_of_key[0x4]; // first four bytes only<br />
/*0x14*/<br />
} ak_file;<br />
<br />
Encryption uses Salsa20. The IV is the last 8 bytes (16 characters) of the cdn hash.<br />
<br />
==Known keys==<br />
decryption_key_name used_by key checksum base32<br />
<br />
# Starcraft 1<br />
sc1Dev s1a F6 79 DC 38 E0 C3 65 FB 48 2E 48 A7 48 90 9D 29 19 F5 BB 88 6Z45YOHAYNS7WSBOJCTUREE5FEM7LO4I<br />
<br />
# Overwatch<br />
pro pro (pre-launch) <br />
proc proc3<br />
prolivedev prodev <br />
provendor prov<br />
<br />
# World of Warcraft<br />
wowdemo wowdemo<br />
wowdevalpha wowdev<br />
wowvendor wowv<br />
<br />
# Destiny 2<br />
destiny2_openbeta dst2<br />
destiny_dev dst2a (pre-alpha)<br />
destiny_event dst2e1<br />
destiny_live dst2a<br />
dst2livedev dst2dev<br />
<br />
# Call of Duty: Black Ops 4<br />
viperlivedev viperdev<br />
<br />
# Odin<br />
odinlivedev odindev<br />
<br />
=CDN File Organization=<br />
Data for every CASC-based game exists on the CDN in one of two places at any given time. To reduce file-system and download overhead many files are packed into archives and indexed by archive indices, both of a different format than the CASC installations found on client systems; other "unarchived", "standalone", or "loose" files are stored as separate files on the CDN that must be downloaded independently. There are at least three different reasons why a particular file is found in one or the other:<br />
# Small files typically incur larger filesystem overhead and benefit most from being packed into archives. A rough rule of thumb appears to be that files smaller than 2 MB or so are put in archives. Presumably larger files are not archived because they make it more difficult to minimize the number of archive files, which are limited in size (a 2 MB file limit would limit unused space in an archive to under 0.8%, given enough data to form a full archive to begin with).<br />
# Key files such as encoding, partial-priority, TVFS (Warcraft 3 root), and the download, install, and patch manifests, as well as their respective patches, are typically stored loose for quick access and are rarely ever found in archives.<br />
# As games evolve old files become obsolete and are removed from the CDN. However, the archive system means that archives can only be removed when every file in the archive is no longer needed, potentially wasting large amounts of space on the CDN - the exact opposite of the purpose of bundling files into archives to begin with. Thus as the amount of unused data in an archive grows over time, files still in use may be converted to loose files to allow the archive to be purged from the CDN, even when the files are unusually small to be found independently.<br />
<br />
Previously, no official indices/manifests of loose files existed, and they could only be found by subtracting archived files from file lists in encoding or manifest files. Beginning with Warcraft 3 and subsequently being deployed more widely on 7/24/2018, new fields in the CDN config file link to index files containing "all" loose data or patch files on the current CDNs.<br />
<br />
===Archives===<br />
Archives are extensionless 256 MB files that are usually only stored on the Blizzard CDNs. Their naming follows the standard URL hash format using the '/data/' path type.<br />
<br />
The structure of the archives is presumably just file fragment after file fragment. You will never need to parse it because you can just look up offset + size of your file fragment in the index files and then take the piece directly out of the archive.<br />
<br />
The fragments are all [[BLTE]] encoded.<br />
<br />
The filename is '''NOT''' the hash of the archive content but the hash of the '''index''''s footer.<br />
<br />
===Archive Indexes (.index)===<br />
These '.index' files reveal to the user where the compressed game files are located within the archives. All indexes (except the Archive-Group index, see below) are named after their archive (only difference is these have an extension).<br />
'.index' files are stored on the CDN using the standard hash naming scheme (remember they have an extension though). They are also located in the directory 'INSTALL_DIR/Data/indices/' for a WoW install.<br />
<br />
====Normal Index Entry Structure====<br />
*'''The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null EKey fields. The last chunk is a table-of-contents, listing the LAST EKey in each chunk and checksum of each block. All checksums are the first checksumSize bytes of the MD5 of the respective data. Structure names were invented by the author of this page.'''<br />
<br />
struct index_entry {<br />
char EKey[footer.keySizeInBytes];<br />
uint_BE_t(footer.sizeBytes) blte_encoded_size;<br />
uint_BE_t(footer.offsetBytes) offset_to_blte_encoded_data_in_archive;<br />
};<br />
<br />
struct index_block {<br />
static constexpr const block_size = footer.blockSizeKb << 10;<br />
index_entry entries[block_size / sizeof (index_entry)];<br />
char padding[block_size - sizeof (entries)];<br />
} blocks[];<br />
<br />
struct {<br />
struct {<br />
char last_EKey[footer.keySizeInBytes]; // last EKey of a block<br />
} entries[num_blocks];<br />
<br />
struct {<br />
char lower_part_of_md5_of_block[footer.checksumSize]; <br />
} blocks_hash[num_blocks];<br />
} toc;<br />
<br />
struct { // note: size comments assume 0x10 checksum size<br />
/*0x00*/ char toc_hash[checksumSize]; // client tries to read with 0x10, then backs off in size when smaller<br />
/*0x10*/ char version?; // always 1<br />
/*0x11*/ char _11; // 0<br />
/*0x12*/ char _12; // 0<br />
/*0x13*/ char blockSizeKb?; // Normally 4. Left-shifted by 10. Believed to be block size in KB.<br />
/*0x14*/ char offsetBytes; // Normally 4 for archive indices, 5 for patch group indices, 6 for group indices, and 0 for loose file indices<br />
/*0x15*/ char sizeBytes; // Normally 4<br />
/*0x16*/ char keySizeInBytes; // Normally 16<br />
/*0x17*/ char checksumSize; // Normally 8, <= 0x10<br />
/*0x18*/ uint32_t numElements; // BigEndian in _old_ versions (e.g. 18179)<br />
/*0x1C*/ char footerChecksum[checksumSize];<br />
/*0x??*/<br />
} footer;<br />
<br />
* footerChecksum is calculated over the footer beginning with version ''when footerChecksum is zeroed''<br />
* The archive/index name is the MD5 of the footer beginning with toc_hash<br />
<br />
===archive-group===<br />
<tt>archive-group</tt> is actually a very special <tt>.index</tt> file that is typically much larger than other indices. There is '''no''' archive attached, it is only the index. Also, the index is '''not''' found on the CDN! It is assembled by combining all index files given in the CDN config, on client side. The hash is just for verification purposes. Client side a file is assembled using the normal index file format, for caching.<br />
<br />
It has a single difference in format to normal indices: While other indices have their <tt>offsetBytes</tt> long <tt>offset</tt> field point into the archive, for archive groups, the field also has an <tt>archiveIndex</tt>:<br />
<br />
struct {<br />
uint16_BE_t archiveIndex; // Index of the archive in the CDN config's archive list<br />
uint32_BE_t offsetBytes; // The offset within the specified archive<br />
};<br />
<br />
It does not only having the offset, but also an index into a file. Semantically that's still an offset, so no further field for size is used.<br />
<br />
It is suggested you do not just parse indices by <tt>.index</tt> filename locally but take the config files into account. An easy heuristic is that if <tt>offsetBytes</tt> is not 4, it is a special index, either loose files or a group.<br />
<br />
===patch-archives===<br />
Patch archives are the /patch/ equivalent to the regular (data) archives on the CDN. Like archives, these are binary blobs of fragments indexed by an accompanying .index file with the same name. Again, the index is a <tt>hash, size, offset</tt> tuple, but the hash is the content hash rather than an encoding hash.<br />
<br />
Most files in patch archives are [[Patching Files|ZBSDIFF1]] blobs, though in principle any file that might be in the /patch/ namespace may be found in patch archives and must be handled accordingly.<br />
<br />
===patch-archive-group===<br />
See [[#archive-group|archive-group]]. There is no known difference other than the combined data being <tt>patch-archives</tt>.<br />
<br />
=TACT keys=<br />
TACT keys are used to encrypt data in files. In most cases, TACT keys are streamed to clients when content (such as cinematics in WoW or skins in Overwatch) is unlocked.<br />
<br />
== Battle.net app==<br />
key_name key type seen in<br />
2C547F26A2613E01 37C50C102D4C9E3A5AC069F072B1417D salsa20 Battle.net App Alpha 1.5.0<br />
<br />
== Overwatch ==<br />
Overwatch has keys included in the client/keyring, but they can also be streamed from the server.<br />
<div class="mw-collapsible mw-collapsed toccolours "><br />
The key table is collapsed by default to reduce page length. Click expand on the right to expand. <br />
<div class="mw-collapsible-content" id="mw-customcollapsible-myDivision"><br />
key_name key type seen in method used for<br />
FB680CB6A8BF81F3 62D90EFA7F36D71C398AE2F1FE37BDB9 salsa20 0.8.0.24919 keyring<br />
402CD9D8D6BFED98 AEB0EADEA47612FE6C041A03958DF241 salsa20 0.8.0.24919 keyring<br />
DBD3371554F60306 34E397ACE6DD30EEFDC98A2AB093CD3C salsa20 0.8.0.24919 keyring<br />
11A9203C9881710A 2E2CB8C397C2F24ED0B5E452F18DC267 salsa20 0.8.0.24919 keyring<br />
A19C4F859F6EFA54 0196CB6F5ECBAD7CB5283891B9712B4B salsa20 0.8.0.24919 keyring<br />
87AEBBC9C4E6B601 685E86C6063DFDA6C9E85298076B3D42 salsa20 0.8.0.24919 keyring<br />
DEE3A0521EFF6F03 AD740CE3FFFF9231468126985708E1B9 salsa20 0.8.0.24919 keyring<br />
8C9106108AA84F07 53D859DDA2635A38DC32E72B11B32F29 salsa20 0.8.0.24919 keyring<br />
49166D358A34D815 667868CD94EA0135B9B16C93B1124ABA salsa20 0.8.0.24919 keyring<br />
1463A87356778D14 69BD2A78D05C503E93994959B30E5AEC salsa20 ≤ 1.0.3.0 keyring<br />
5E152DE44DFBEE01 E45A1793B37EE31A8EB85CEE0EEE1B68 salsa20 ≤ 1.0.3.0 keyring<br />
9B1F39EE592CA415 54A99F081CAD0D08F7E336F4368E894C salsa20 ≤ 1.0.3.0 keyring<br />
24C8B75890AD5917 31100C00FDE0CE18BBB33F3AC15B309F salsa20 ≤ 1.0.3.0 keyring<br />
EA658B75FDD4890F DEC7A4E721F425D133039895C36036F8 salsa20 ≤ 1.0.3.0 keyring<br />
026FDCDF8C5C7105 8F41809DA55366AD416D3C337459EEE3 salsa20 keyring<br />
CAE3FAC925F20402 98B78E8774BF275093CB1B5FC714511B salsa20 keyring<br />
061581CA8496C80C DA2EF5052DB917380B8AA6EF7A5F8E6A salsa20 keyring<br />
BE2CB0FAD3698123 902A1285836CE6DA5895020DD603B065 salsa20 keyring<br />
57A5A33B226B8E0A FDFC35C99B9DB11A326260CA246ACB41 salsa20 1.1.0.0.30200 keyring Ana<br />
42B9AB1AF5015920 C68778823C964C6F247ACC0F4A2584F8 salsa20 1.2.0.1.30684 keyring Summer Games<br />
4F0FE18E9FA1AC1A 89381C748F6531BBFCD97753D06CC3CD salsa20 1.2.0.1.30684 keyring<br />
7758B2CF1E4E3E1B 3DE60D37C664723595F27C5CDBF08BFA salsa20 1.2.0.1.30684 keyring<br />
E5317801B3561125 7DD051199F8401F95E4C03C884DCEA33 salsa20 1.4.0.2.32143 keyring Halloween Terror<br />
16B866D7BA3A8036 1395E882BF25B481F61A4D621141DA6E salsa20 1.4.1.0.31804 keyring Bastion Blizzcon 2016 skin<br />
11131FFDA0D18D30 C32AD1B82528E0A456897B3CE1C2D27E salsa20 1.5.0.1.32795 keyring Sombra<br />
CAC6B95B2724144A 73E4BEA145DF2B89B65AEF02F83FA260 salsa20 1.5.0.1.32795 keyring Ecopoint: Antarctica<br />
B7DBC693758A5C36 BC3A92BFE302518D91CC30790671BF10 salsa20 1.5.0.1.32795 keyring Genji Oni Skin (HotS Nexus Challenge)<br />
90CA73B2CDE3164B 5CBFF11F22720BACC2AE6AAD8FE53317 salsa20 1.6.1.0.33236 keyring Oasis map<br />
6DD3212FB942714A E02C1643602EC16C3AE2A4D254A08FD9 salsa20 1.6.1.0.33236 keyring<br />
11DDB470ABCBA130 66198766B1C4AF7589EFD13AD4DD667A salsa20 1.6.1.0.33236 keyring Winter Wonderland<br />
5BEF27EEE95E0B4B 36BCD2B551FF1C84AA3A3994CCEB033E salsa20 keyring<br />
9359B46E49D2DA42 173D65E7FCAE298A9363BD6AA189F200 salsa20 keyring Diablo's 20th anniversary<br />
1A46302EF8896F34 8029AD5451D4BC18E9D0F5AC449DC055 salsa20 1.7.0.2.34156 keyring Year of the Rooster<br />
693529F7D40A064C CE54873C62DAA48EFF27FCC032BD07E3 salsa20 1.8.0.0.34470 keyring CTF Maps<br />
388B85AEEDCB685D D926E659D04A096B24C19151076D379A salsa20 1.8.0.0.34470 keyring Numbani Update (Doomfist teaser)<br />
E218F69AAC6C104D F43D12C94A9A528497971F1CBE41AD4D salsa20 1.9.0.0.34986 keyring Orisa<br />
F432F0425363F250 BA69F2B33C2768F5F29BFE78A5A1FAD5 salsa20 1.10.0.0.35455 keyring Uprising<br />
061D52F86830B35D D779F9C6CC9A4BE103A4E90A7338F793 salsa20 1.10.0.0.35455 keyring D.Va Officer Skin (HotS Nexus Challenge 2)<br />
1275C84CF113EF65 CF58B6933EAF98AF53E76F8426CC7E6C salsa20 keyring<br />
D9C7C7AC0F14C868 3AFDF68E3A5D63BABA1E6821883F067D salsa20 keyring<br />
BD4E42661A432951 6DE8E28C8511644D5595FC45E5351472 salsa20 1.11.0.0.36376 keyring Anniversary event<br />
C43CB14355249451 0EA2B44F96A269A386856D049A3DEC86 salsa20 1.12.0.0.37104 keyring Horizon Lunar Colony<br />
E6D914F8E4744953 C8477C289DCE66D9136507A33AA33301 salsa20 1.13.0.0.37646 keyring Doomfist<br />
5694C503F8C80178 7F4CF1C1FBBAD92B184336D677EBF937 salsa20 1.13.0.0.37646 keyring Doomfist<br />
21DBFD65F3E54269 AB580C3837CAF8A461F243A566B2AE4D salsa20 1.13.0.0.37646 keyring Summer Games 2017<br />
27ABA5F88DD8D078 ???????????????????????????????? salsa20 1.13.0.0.37646 ??????? ??????<br />
21E1F90E71D33C71 328742339162B32676C803C2255931A6 salsa20 1.14.1.0.39083 keyring Deathmatch<br />
D9CB055BCDD40B6E 49FB4477A4A0825327E9A73682BECD0C salsa20 1.15.0.0.????? keyring Junkertown<br />
8175CE3C694C6659 E3F3FA7726C70D26AE130D969DDDF399 salsa20 1.16.0.0.40011 keyring Halloween 2017<br />
B8DE51690075435A C07E9260BB711217E7DE6FED911F4296 salsa20 1.16.0.0.????? keyring Winston Blizzcon 2017 skin<br />
F6CF23955B5D437D AEBA227328A5B0AA9F51DAE3F6A7DFE4 salsa20 1.17.0.2.41350 keyring Moira<br />
0E4D9426F2891F5C 9FF064C38BE52CCDF73748180F628205 salsa20 1.18.1.2.42076 keyring Winter Wonderland 2017<br />
9240BA6A2A0CF684 DF2E37D78B43108FA6242068B70D1F65 salsa20 1.19.1.3.42563 keyring Overwatch League<br />
82297FBAB7F5EB80 B534C20965852FB15AECAC17E381B417 salsa20 1.19.1.3.42563 keyring Jan 2017 Lootbox Update<br />
9ADF00AA1A174A69 9A4AC899261A2F1C6969F39397C358E7 salsa20 1.19.1.3.42563 keyring Blizzard World<br />
CFA05AA76B49F881 526DDDEF19BF373C25B629A334CD7237 salsa20 1.19.1.3.42563 keyring WoW's Battle For Azeroth Preorder<br />
493455579DA0B18E C0BABF72AD2C05DFC14017D1ADBF5977 salsa20 1.19.3.1.43036 keyring Inaugral Season Spray/Icon ??<br />
6362C5AD65DAE686 62F603D5390F763ED51773F0164FEDB5 salsa20 1.19.3.1.43036 keyring White/Gray OWL Skins ??<br />
8162E5313A9C135D F407834D9521587C5012B0A59D7E064B salsa20 1.20.0.2.43435 keyring Lunar New Year 2018 (Dog) / Ayutthaya / Comp CTF<br />
68EAE8FDC008C381 ???????????????????????????????? salsa20 1.20.0.2.43435 n/a ??<br />
F412C6327C4BF091 6FAFC648CBF1C2115B769593C170E732 salsa20 1.20.0.2.43435 keyring SC2 20th Anniversary (Kerrigan Skin)<br />
3B3ED0874091B174 5D09C2688B1D9F1A4DB64602C1661D24 salsa20 1.21.?.?.????? keyring Brigitte<br />
37FD04E05D2A6292 F06455E56CD144914295F2EF153D23BA salsa20 1.21.?.?.????? keyring Brigitte Cosmetics<br />
C0DDC77552BE5794 F2BB7F35990E2900CBD877B4D3A7139C salsa20 1.22.?.?.????? keyring OWL away skins<br />
68D8EB839DC15D75 29AFFECA5299C4140A12A66F954EF1E3 salsa20 1.22.?.?.????? keyring Archives 2018 (Retribution)<br />
209F33BBAC9D1295 BD535438D0CDEE0E9567E0EF671C809F salsa20 1.23.?.?.????? ??????? Rialto<br />
A55F8C6F20454D94 42D76285412461B0C75AB75FB52F596E salsa20 1.23.?.?.????? ??????? Mercy BCRF Items<br />
3EEEDB8E7C29A09B 837AC79305E4BFBA3B2226F1B98200BF salsa20 1.24.?.?.????? ??????? Anniversary 2018 Map/Items<br />
22C1AF6758F8449E 28AA1BD2B9A1E3633989B1BBF64AEAC4 salsa20 1.26.?.?.????? ??????? Emily / Comp Season 11 / Comp 3v3 Items<br />
11B2E01B9331799E 9A1C99303D6A58978E29C98873327A6A salsa20 1.26.?.?.????? ??????? Wrecking Ball<br />
8498337C740329B3 EC73D663E3FC72416B3E467915708B4F salsa20 1.26.?.?.????? ??????? Wrecking Ball Cosmetics<br />
E6E8BCABE3CC96C1 57727E52600665EAC02BD87F0692898F salsa20 1.26.?.?.????? ??????? OWL Grand Finals Unlocks / Lucio Emote<br />
5D4AC0DC6F3113BA D1C9F1FF69585D1398EBA0463481F5CF salsa20 1.27.?.?.????? ??????? Summer Games 2018<br />
27F9D85973DCD5AF C5FE1015BCE0B7848F022868AFF654D1 salsa20 1.27.?.?.????? ??????? Summer Games 2018<br />
67AAD845CC0F03BD 6CD8AD3F37F54ABEC7630294D49041BF salsa20 1.27.?.?.????? ??????? D.Va Nano Cola Challenge<br />
CA13F0C79042A1A0 FCD89CE8812E6346076FC82DD7A92487 salsa20 1.28.?.?.????? ??????? Busan<br />
C4D84093A32684BD 38E182423EEDD8E3F57AC1D407B470D0 salsa20 1.28.?.?.????? ??????? Blizzcon 2018 Sombra Demon Hunter Skin<br />
0BFE5A2B3C606BA1 D6419B8E42820B0B24E08DA444A06822 salsa20 1.29.?.?.????? ??????? Halloween Terror 2018<br />
402CD9D8D6BFED98 AEB0EADEA47612FE6C041A03958DF241 salsa20 1.29.?.?.????? keyring ??<br />
F1CBDF48147D26C6 4B6944695157D43D33E40B5692445ADB salsa20 1.30.?.?.????? ??????? 1.30 + World Cup Viewer<br />
01A48CFAE25F85CD 60D77A58062D03DC693C01954DD18021 salsa20 1.30.?.?.????? ??????? 1.30 + World Cup Viewer<br />
FEBBF66DAEF6C9BE 4A220AE3A6808ED2697410C94C1CE970 salsa20 1.30.0.1.????? ??????? Ashe<br />
2AD4834DAB3986AB EA6971F78ABEBD2AF883D46A5486F39F salsa20 1.31.?.?.????? ??????? Winter 2018<br />
D89B24D62F00A04E C3A4D010AAA7287A3EACCD45ED471345 salsa20 1.31.?.?.????? ??????? Bastet Challenge<br />
32DDC40236DAEA7B 2DBDE4FB9FDA776AA294870854FE9B02 salsa20 1.31.?.?.????? ??????? Lunar New Year 2019 (Pig)<br />
F481EFC2302EE415 37A7F2B87D0B8B700456C6A92C71D6DD salsa20 1.33.?.?.????? ??????? Paris Map<br />
D1AC8C1903524D9A D781D0AA35E5C106BCA7CF01DEBD1494 salsa20 1.34.0.1.55918 stream Baptiste<br />
71EEBE93590AA903 0CCD10D4553EEC7E97FD36A9E8ADF0FF salsa20 1.34.0.1.55918 stream Baptiste Cosmetics<br />
</div></div><br />
<br />
== World of Warcraft ==<br />
WoW's [[DB/TactKey|TactKey.db2]] and [[DB/TactKeyLookup|TactKeyLookup.db2]] contain keys and key_names (called lookups there) respectively. Either can be streamed from server.<br />
These files cannot be bruteforced by requesting hotfix data (Blizzard has guards in place).<br />
<br />
The id field in the table below corresponds to the id field in the db2's.<br />
<br />
key_name key type id seen in used for<br />
FA505078126ACB3E BDC51862ABED79B2DE48C8E7E66C6200 salsa20 15 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
FF813F7D062AC0BC AA0B5C77F088CCC2D39049BD267F066D salsa20 25 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
D1E9B5EDF9283668 8E4A2579894E38B4AB9058BA5C7328EE salsa20 39 WOW-20740patch7.0.1_Beta Enchanted Torch pet<br />
B76729641141CB34 9849D1AA7B1FD09819C5C66283A326EC salsa20 40 WOW-20740patch7.0.1_Beta Enchanted Pen pet<br />
FFB9469FF16E6BF8 D514BD1909A9E5DC8703F4B8BB1DFD9A salsa20 41 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
23C5B5DF837A226C 1406E2D873B6FC99217A180881DA8D62 salsa20 42 WOW-20740patch7.0.1_Beta Enchanted Cauldron pet<br />
3AE403EF40AC3037 ???????????????????????????????? salsa20 51 WOW-21249patch7.0.3_Beta not used between 7.0 and 7.3<br />
E2854509C471C554 433265F0CDEB2F4E65C0EE7008714D9E salsa20 52 WOW-21249patch7.0.3_Beta Warcraft movie items<br />
8EE2CB82178C995A DA6AFC989ED6CAD279885992C037A8EE salsa20 55 WOW-21531patch7.0.3_Beta BlizzCon 2016 Murlocs<br />
5813810F4EC9B005 01BE8B43142DD99A9E690FAD288B6082 salsa20 56 WOW-21531patch7.0.3_Beta Fel Kitten<br />
7F9E217166ED43EA 05FC927B9F4F5B05568142912A052B0F salsa20 57 WOW-21531patch7.0.3_Beta Legion music <br />
C4A8D364D23793F7 D1AC20FD14957FABC27196E9F6E7024A salsa20 58 WOW-21691patch7.0.3_Beta Demon Hunter #1 cinematic (legion_dh1)<br />
40A234AEBCF2C6E5 C6C5F6C7F735D7D94C87267FA4994D45 salsa20 59 WOW-21691patch7.0.3_Beta Demon Hunter #2 cinematic (legion_dh2)<br />
9CF7DFCFCBCE4AE5 72A97A24A998E3A5500F3871F37628C0 salsa20 60 WOW-21691patch7.0.3_Beta Val'sharah #1 cinematic (legion_val_yd)<br />
4E4BDECAB8485B4F 3832D7C42AAC9268F00BE7B6B48EC9AF salsa20 61 WOW-21691patch7.0.3_Beta Val'sharah #2 cinematic (legion_val_yx)<br />
94A50AC54EFF70E4 C2501A72654B96F86350C5A927962F7A salsa20 62 WOW-21691patch7.0.3_Beta Sylvanas warchief cinematic (legion_org_vs)<br />
BA973B0E01DE1C2C D83BBCB46CC438B17A48E76C4F5654A3 salsa20 63 WOW-21691patch7.0.3_Beta Stormheim Sylvanas vs Greymane cinematic (legion_sth)<br />
494A6F8E8E108BEF F0FDE1D29B274F6E7DBDB7FF815FE910 salsa20 64 WOW-21691patch7.0.3_Beta Harbingers Gul'dan video (legion_hrb_g)<br />
918D6DD0C3849002 857090D926BB28AEDA4BF028CACC4BA3 salsa20 65 WOW-21691patch7.0.3_Beta Harbingers Khadgar video (legion_hrb_k)<br />
0B5F6957915ADDCA 4DD0DC82B101C80ABAC0A4D57E67F859 salsa20 66 WOW-21691patch7.0.3_Beta Harbingers Illidan video (legion_hrb_i)<br />
794F25C6CD8AB62B 76583BDACD5257A3F73D1598A2CA2D99 salsa20 67 WOW-21846patch7.0.3_Beta Suramar cinematic (legion_su_i)<br />
A9633A54C1673D21 1F8D467F5D6D411F8A548B6329A5087E salsa20 68 WOW-21846patch7.0.3_Beta legion_su_r cinematic<br />
5E5D896B3E163DEA 8ACE8DB169E2F98AC36AD52C088E77C1 salsa20 69 WOW-21846patch7.0.3_Beta Broken Shore intro cinematic (legion_bs_i)<br />
0EBE36B5010DFD7F 9A89CC7E3ACB29CF14C60BC13B1E4616 salsa20 70 WOW-21846patch7.0.3_Beta Alliance Broken Shore cinematic (legion_bs_a)<br />
01E828CFFA450C0F 972B6E74420EC519E6F9D97D594AA37C salsa20 71 WOW-21846patch7.0.3_Beta Horde Broken Shore cinematic (legion_bs_h)<br />
4A7BD170FE18E6AE AB55AE1BF0C7C519AFF028C15610A45B salsa20 72 WOW-21846patch7.0.3_Beta Khadgar & Light's Heart cinematic (legion_iq_lv)<br />
69549CB975E87C4F 7B6FA382E1FAD1465C851E3F4734A1B3 salsa20 73 WOW-21846patch7.0.3_Beta legion_iq_id cinematic<br />
460C92C372B2A166 946D5659F2FAF327C0B7EC828B748ADB salsa20 74 WOW-21952patch7.0.3_Beta Stormheim Alliance cinematic (legion_g_a_sth)<br />
8165D801CCA11962 CD0C0FFAAD9363EC14DD25ECDD2A5B62 salsa20 75 WOW-21952patch7.0.3_Beta Stormheim Horde cinematic (legion_g_h_sth)<br />
A3F1C999090ADAC9 B72FEF4A01488A88FF02280AA07A92BB salsa20 81 WOW-22578patch7.1.0_PTR Firecat Mount<br />
18AFDF5191923610 ???????????????????????????????? salsa20 82 WOW-22578patch7.1.0_PTR not used between 7.1 and 7.3<br />
3C258426058FBD93 ???????????????????????????????? salsa20 91 WOW-23436patch7.2.0_PTR not used between 7.2 and 7.3<br />
094E9A0474876B98 E533BB6D65727A5832680D620B0BC10B salsa20 92 WOW-23910patch7.2.5_PTR shadowstalkerpanthermount, shadowstalkerpantherpet<br />
3DB25CB86A40335E 02990B12260C1E9FDD73FE47CBAB7024 salsa20 93 WOW-23789patch7.2.0_PTR legion_72_ots<br />
0DCD81945F4B4686 1B789B87FB3C9238D528997BFAB44186 salsa20 94 WOW-23789patch7.2.0_PTR legion_72_tst<br />
486A2A3A2803BE89 32679EA7B0F99EBF4FA170E847EA439A salsa20 95 WOW-23789patch7.2.0_PTR legion_72_ars<br />
71F69446AD848E06 E79AEB88B1509F628F38208201741C30 salsa20 97 WOW-24473patch7.3.0_PTR BlizzCon 2017 Mounts (AllianceShipMount and HordeZeppelinMount)<br />
211FCD1265A928E9 A736FBF58D587B3972CE154A86AE4540 salsa20 98 WOW-24473patch7.3.0_PTR Shadow fox pet (store) <br />
0ADC9E327E42E98C 017B3472C1DEE304FA0B2FF8E53FF7D6 salsa20 99 WOW-23910patch7.2.5_PTR legion_72_tsf<br />
BAE9F621B60174F1 38C3FB39B4971760B4B982FE9F095014 salsa20 100 WOW-24727patch7.3.0_PTR Rejection of the Gift cinematic (legion_73_agi)<br />
34DE1EEADC97115E 2E3A53D59A491E5CD173F337F7CD8C61 salsa20 101 WOW-24727patch7.3.0_PTR Resurrection of Alleria Windrunner cinematic (legion_73_avt)<br />
E07E107F1390A3DF 290D27B0E871F8C5B14A14E514D0F0D9 salsa20 102 WOW-25079patch7.3.2_PTR Tottle battle pet, Raptor mount, Horse mount (104 files)<br />
32690BF74DE12530 A2556210AE5422E6D61EDAAF122CB637 salsa20 103 WOW-24781patch7.3.0_PTR legion_73_pan<br />
BF3734B1DCB04696 48946123050B00A7EFB1C029EE6CC438 salsa20 104 WOW-25079patch7.3.2_PTR legion_73_afn<br />
74F4F78002A5A1BE C14EEC8D5AEEF93FA811D450B4E46E91 salsa20 105 WOW-25079patch7.3.2_PTR SilithusPhase01 map<br />
423F07656CA27D23 ???????????????????????????????? salsa20 107 WOW-25600patch7.3.5_PTR bltestmap<br />
0691678F83E8A75D ???????????????????????????????? salsa20 108 WOW-25600patch7.3.5_PTR filedataid 1782602-1782603<br />
324498590F550556 ???????????????????????????????? salsa20 109 WOW-25600patch7.3.5_PTR filedataid 1782615-1782619<br />
C02C78F40BEF5998 ???????????????????????????????? salsa20 110 WOW-25600patch7.3.5_PTR test/testtexture.blp (fdid 1782613)<br />
47011412CCAAB541 ???????????????????????????????? salsa20 111 WOW-25600patch7.3.5_PTR unused in 25600<br />
23B6F5764CE2DDD6 ???????????????????????????????? salsa20 112 WOW-25600patch7.3.5_PTR unused in 25600<br />
8E00C6F405873583 ???????????????????????????????? salsa20 113 WOW-25600patch7.3.5_PTR filedataid 1783470-1783472, tileset/test/bltesttileset*.blp<br />
78482170E4CFD4A6 768540C20A5B153583AD7F53130C58FE salsa20 114 WOW-25600patch7.3.5_PTR Magni Bronzebeard VO<br />
B1EB52A64BFAF7BF 458133AA43949A141632C4F8596DE2B0 salsa20 115 WOW-25600patch7.3.5_PTR dogmount, 50 files<br />
FC6F20EE98D208F6 57790E48D35500E70DF812594F507BE7 salsa20 117 WOW-25632patch7.3.5_PTR bfa shop stuff<br />
402CFABF2020D9B7 67197BCD9D0EF0C4085378FAA69A3264 salsa20 118 WOW-25678patch7.3.5_PTR bfa ad texture<br />
6FA0420E902B4FBE 27B750184E5329C4E4455CBD3E1FD5AB salsa20 119 WOW-25744patch7.3.5_PTR Legion epilogue cinematics<br />
1076074F2B350A2D 88BF0CD0D5BA159AE7CB916AFBE13865 salsa20 121 WOW-26287patch8.0.1_Beta skiff<br />
816F00C1322CDF52 6F832299A7578957EE86B7F9F15B0188 salsa20 122 WOW-26287patch8.0.1_Beta snowkid<br />
DDD295C82E60DB3C 3429CC5927D1629765974FD9AFAB7580 salsa20 123 WOW-26287patch8.0.1_Beta redbird<br />
83E96F07F259F799 91F7D0E7A02CDE0DE0BD367FABCB8A6E salsa20 124 WOW-26522patch8.0.1_Beta BlizzCon 2018 (Alliance and Horde banners and cloaks)<br />
49FBFE8A717F03D5 C7437770CF153A3135FA6DC5E4C85E65 salsa20 225 WOW-27826patch8.1.0_PTR Meatwagon mount (Warcraft 3: Reforged)<br />
C1E5D7408A7D4484 A7D88E52749FA5459D644523F8359651 salsa20 226 WOW-26871patch8.0.1_Beta Sylvanas Warbringer cinematic<br />
E46276EB9E1A9854 CCCA36E302F9459B1D60526A31BE77C8 salsa20 227 WOW-26871patch8.0.1_Beta ltc_a, ltc_h and ltt cinematics<br />
D245B671DD78648C 19DCB4D45A658B54351DB7DDC81DE79E salsa20 228 WOW-26871patch8.0.1_Beta stz, zia, kta, jnm & ja cinematics<br />
4C596E12D36DDFC3 B8731926389499CBD4ADBF5006CA0391 salsa20 229 WOW-26871patch8.0.1_Beta bar cinematic<br />
0C9ABD5081C06411 25A77CD800197EE6A32DD63F04E115FA salsa20 230 WOW-26871patch8.0.1_Beta zcf cinematic<br />
3C6243057F3D9B24 58AE3E064210E3EDF9C1259CDE914C5D salsa20 231 WOW-26871patch8.0.1_Beta ktf cinematic<br />
7827FBE24427E27D 34A432042073CD0B51627068D2E0BD3E salsa20 232 WOW-26871patch8.0.1_Beta rot cinematic<br />
FAF9237E1186CF66 AE787840041E9B4198F479714DAD562C salsa20 233 WOW-28048patch8.1.0_PTR DB2 partial encryption test battle pet<br />
5DD92EE32BBF9ABD ???????????????????????????????? salsa20 234 WOW-27004patch8.0.1_Subm filedataid 2238294, interface/icons/ui_shop_bcv.blp<br />
0B68A7AF5F85F7EE 27AA011082F5E8BBBD71D1BA04F6ABA4 salsa20 236 WOW-28151patch8.1.0_PTR flying pig mount<br />
01531713C83FCC39 ???????????????????????????????? salsa20 237 WOW-28151patch8.1.0_PTR fdid 2460009 & 2460732, a cape texture and icon<br />
76E4F6739A35E8D7 05CF276722E7165C5A4F6595256A0BFB salsa20 238 WOW-28294patch8.1.0_PTR Sylverian Dreamer mount<br />
66033F28DC01923C 9F9519861490C5A9FFD4D82A6D0067DB salsa20 239 WOW-28294patch8.1.0_PTR Vulpine Familiar mount<br />
FCF34A9B05AE7E6A E7C2C8F77E30AC240F39EC23971296E5 salsa20 240 WOW-28151patch8.1.0_PTR Alliance fireworks<br />
E2F6BD41298A2AB9 C5DC1BB43B8CF3F085D6986826B928EC salsa20 241 WOW-28151patch8.1.0_PTR Horde fireworks<br />
14C4257E557B49A1 064A9709F42D50CB5F8B94BC1ACFDD5D salsa20 242 WOW-28440patch8.1.0_PTR dor cinematic<br />
1254E65319C6EEFF 79D2B3D1CCB015474E7158813864B8E6 salsa20 243 WOW-28440patch8.1.0_PTR akt cinematic<br />
C8753773ADF1174C 1E0E37D42EE5CE5E8067F0394B0905F2 salsa20 244 WOW-28938patch8.1.5_PTR Obsidian Worldbreaker mount & Lil' Nefarian pet<br />
2170BCAA9FA96E22 6DDA6D48D72DC8005DB9DC15368D35BC salsa20 245 WOW-28938patch8.1.5_PTR baby alpaca pet<br />
75485627AA225F4D ???????????????????????????????? salsa20 246 WOW-28938patch8.1.5_PTR fdid 2741546 (a creature that uses baby raptor pet sounds), 2741548, 2741549<br />
08717B15BF3C7955 4B06BF9D17663CEB3312EA3C69FBC5DD salsa20 248 WOW-29220patch8.1.5_PTR inv_encrypted20.blp (fdid 2823166)<br />
D19DCF7ACA8D96D6 ???????????????????????????????? salsa20 249 WOW-30080patch8.2.0_PTR starts at fdid 2843110, 10 files<br />
9FD609902B4B2E07 ABE0C5F9C123E6E24E7BEA43C2BF00AC salsa20 250 WOW-29418patch8.1.5_PTR Derek Proudmoore cinematic (dpr, 5 files)<br />
CB26B441FAE4C8CD ???????????????????????????????? salsa20 251 WOW-30080patch8.2.0_PTR fdid 2888623, 2892270, 2892271, 2892272, 2892274, 2892275<br />
A98C7594F55C02F0 EEDB77473B721DED6204A976C9A661E7 salsa20 252 WOW-30080patch8.2.0_PTR BlizzCon 2019 - Murloc pets<br />
259EE68CD9E76DBA 465D784F1019661CCF417FE466801283 salsa20 253 WOW-30080patch8.2.0_PTR Alabaster mounts (30 files)<br />
6A026290FBDB3754 3D2D620850A6765DD591224F605B949A salsa20 255 WOW-30080patch8.2.0_PTR BlizzCon 2019 - Wendigo transmog set<br />
CF72FD04608D36ED A0A889976D02FA8D00F7AF0017AD721F salsa20 257 WOW-30262patch8.2.0_PTR Azshara Warbringer cinematic (5 files)<br />
17F07C2E3A45DB3D 6D3886BDB91E715AE7182D9F3A08F2C9 salsa20 258 WOW-30262patch8.2.0_PTR Solesa Naksu Nazjatar phase (34 files)<br />
DFAB5841B87802B5 ???????????????????????????????? salsa20 259 WOW-31337patch8.2.5_PTR starts at fdid 3016206, 8 files, 3016206 is a creature that uses felbat sounds<br />
C050FA06BB0538F6 C552F5D0B72231502D2547314E6015F7 salsa20 260 WOW-30495patch8.2.0_PTR Crossroads cinematic (5 files)<br />
AB5CDD3FC321831F E1384F5B06EBBCD333695AA6FFC68318 salsa20 261 WOW-30495patch8.2.0_PTR Azshara kill cinematic (5 files)<br />
A7B7D1F12395040E 36AD3B31273F1EBCEE8520AAA74B12F2 salsa20 262 WOW-30495patch8.2.0_PTR Nazjatar intro cinematics (9 files)<br />
83A2AB72DD8AE992 023CFF062B19A529B9F14F9B7AAAC5BB salsa20 263 WOW-31337patch8.2.5_PTR 8.2.5 War Campaign scenario/models<br />
BEAF567CC45362F0 8BD3ED792405D9EE742BF6AFA944578A salsa20 264 WOW-31337patch8.2.5_PTR 8.2.5 War Campaign quests/vo<br />
7BB3A77FD8D14783 4C94E3609CFE0A82000A0BD46069AC6F salsa20 265 WOW-31337patch8.2.5_PTR 8.2.5 War Campaign epilogue quests<br />
8F4098E2470FE0C8 AA718D1F1A23078D49AD0C606A72F3D5 salsa20 266 WOW-31337patch8.2.5_PTR 8.2.5 War Campaign epilogue in-game cinematic<br />
6AC5C837A2027A6B B0B7CE091763D15E7F69A8E2342CDD7C salsa20 267 WOW-31337patch8.2.5_PTR Shadowlands CE rewards<br />
302AAD8B1F441D95 24B86438CF02538649E5BA672FD5993A salsa20 271 WOW-31337patch8.2.5_PTR RaF mounts & armor<br />
5C909F00088734B9 ???????????????????????????????? salsa20 272 WOW-31337patch8.2.5_PTR 7 files, 3054070 is a creature that uses yak sounds, likely a mount<br />
F785977C76DE9C77 ???????????????????????????????? salsa20 273 WOW-31337patch8.2.5_PTR starts at fdid 3071600, 313 files, Winter Veil?<br />
1CDAF3931871BEC3 66B4D34A3AF30E5EB7F414F6C30AAF4F salsa20 275 WOW-31337patch8.2.5_PTR Winter Veil 2019 toy/achievement (28 files)<br />
814E1AB43F3F9345 B65E2A63A116AA251FA5D7B0BAABF778 salsa20 276 WOW-31599patch8.2.5_PTR The Negotiation cinematic (5 files)<br />
1FBE97A317FFBEFA BD71F78D43117C68724BB6E0D9577E08 salsa20 277 WOW-31599patch8.2.5_PTR Reckoning cinematic (5 files)<br />
30581F81528FB27C ???????????????????????????????? salsa20 278 WOW-32044patch8.3.0_PTR Contains creature that uses monkey sounds<br />
4287F49A5BB366DA ???????????????????????????????? salsa20 279 WOW-31599patch8.2.5_PTR Unused in 8.2.5<br />
D134F430A45C1CF2 543DA784D4BD2428CFB5EBFEBA762A90 salsa20 280 WOW-32044patch8.3.0_PTR Encrypted map w/ Icecrown assets, Darion Mograine VO<br />
01C82EE0725EDA3A ???????????????????????????????? salsa20 281 WOW-31812patch8.2.5_PTR Unused in 8.2.5<br />
04C0C50B5BE0CC78 ???????????????????????????????? salsa20 282 WOW-31812patch8.2.5_PTR Unused in 8.2.5<br />
A26FD104489B3DE5 ???????????????????????????????? salsa20 283 WOW-31812patch8.2.5_PTR Unused in 8.2.5<br />
EA6C3B8F210A077F ???????????????????????????????? salsa20 284 WOW-32044patch8.3.0_PTR 18 files, 3159888 is a creature that uses gronn sounds, likely a mount<br />
4A738212694AD0B6 ???????????????????????????????? salsa20 285 WOW-32044patch8.3.0_PTR unused in 32044<br />
2A430C60DDCC75FF ???????????????????????????????? salsa20 286 WOW-32044patch8.3.0_PTR 254 files, appears to be items<br />
<br />
== Warcraft III: Reforged==<br />
key_name key type id seen in used for<br />
6E4296823E7D561E C0BFA2943AC3E92286E4443EE3560D65 salsa20 ?? 1.32.0.13369 Base content (Beta Week 0)<br />
E04D60E31DDEBF63 263DB5C402DA8D4D686309CB2E3254D0 salsa20 ?? 1.32.0.13445 Base content (Beta Week 1)</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=28375DB22019-12-11T21:22:33Z<p>Simca: Clarified WDC2+ string changes further - the calculation varies by section index.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for pallet_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - combined size of record data after this section + combined size of string blocks before this section<br />
<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Additionally, it calculates the combined size of the string block data before this section and adds that value into the result. Lastly, it calculates the combined size of all record data blocks after this section and subtracts that value from the result. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'combined size of string block data before this section' needs to be subtracted out of and the part where the 'combined size of all record data blocks after this section' needs to be added into the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before. When calculating WDC2+ string offsets, you need to subtract out the record data size of any sections that occur after the current section and add in the size of any string data sections that occur before the current section. This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[section_headers.offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[section_headers.offset_map_id_count];<br />
};<br />
section data_sections[header.section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=28360DB22019-12-10T11:39:50Z<p>Simca: /* WDC3 */ Corrected problem with Post-WDC2 string reading. Thanks Kruithne for the heads-up and Barncastle for the fix.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for pallet_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - combined size of string blocks outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - (StringTableSize - SectionHeaders[si].StringTableSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the string block data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of string block data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (string table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[section_headers.offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[section_headers.offset_map_id_count];<br />
};<br />
section data_sections[header.section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=28359DB22019-12-10T11:38:44Z<p>Simca: /* WDC2 */ Corrected problem with Post-WDC2 string reading. Thanks Kruithne for the heads-up and Barncastle for the fix.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for pallet_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - combined size of string blocks outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - (StringTableSize - SectionHeaders[si].StringTableSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the string block data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of string block data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[section_headers.offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[section_headers.offset_map_id_count];<br />
};<br />
section data_sections[header.section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=M2&diff=26855M22019-06-01T08:42:25Z<p>Simca: /* M2ParticleOld */ Fixed comment alignment of my last edit.</p>
<hr />
<div>[[M2]] files (also called [[MDX]]) contain model objects. Each [[M2]] file describes the vertices, faces, materials, texture names, animations and properties of one model. [[M2]] files don't have a chunked format like most other WoW formats (except in Legion). Since it is chunked in Legion, all offsets are relative to beginning of the MD21 chunk's data rather than the beginning of the file. <br />
<br />
Models are used for doodads (decoration objects), players, monsters and really everything in the game except for [[ADT|Terrain]] and [[WMO]]s.<br />
<br />
==Header==<br />
The header has mostly the layout of number-offset pairs, containing the number of a particular record in the file, and the offset. These appear at fixed places in the header. Record sizes are not specified in the file.<br />
<br />
struct <br />
{<br />
// note: Offsets are for {{Template:Sandbox/VersionRange|min_expansionlevel=3}}!<br />
/*0x000*/ uint32_t magic; // "MD20". [[M2#Legion|Legion]] uses a chunked file format starting with MD21.<br />
/*0x004*/ uint32_t [[#Versions|version]];<br />
/*0x008*/ {{Template:Type/M2Array|char}} name; // should be globally unique, used to reload by name in internal clients<br />
<br />
/*0x010*/ struct<br />
{<br />
uint32_t flag_tilt_x : 1;<br />
uint32_t flag_tilt_y : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
uint32_t flag_use_texture_combiner_combos : 1; // add textureCombinerCombos array to end of data<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
uint32_t flag_load_phys_data : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=6}}<br />
uint32_t flag_unk_0x80 : 1; // with this flag unset, demon hunter tattoos stop glowing<br />
// since {{Template:Unverified|{{Template:Sandbox/PrettyVersion|expansionlevel=4|build=4.0.1.12911}}}} {{Template:Unverified|every}} model now has this flag<br />
uint32_t flag_camera_related : 1; // TODO: verify version<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=7}} // TODO: verify version, these are just added based on where I first saw them -- schlumpf.<br />
uint32_t flag_new_particle_record : 1; // In CATA: new version of ParticleEmitters. By default, length of M2ParticleOld is 476. <br />
// But if 0x200 is set or if version is bigger than 271, length of M2ParticleOld is 492.<br />
uint32_t flag_unk_0x400 : 1;<br />
uint32_t flag_texture_transforms_use_bone_sequences : 1; // {{Template:Sandbox/VersionRange|min_expansionlevel=6}} 0x800 -- When set, texture transforms are animated using the sequence being played on the bone found by index in tex_unit_lookup_table[textureTransformIndex], instead of using the sequence being played on the model's first bone. Example model: 6DU_HellfireRaid_FelSiege03_Creature<br />
uint32_t flag_unk_0x1000 : 1;<br />
uint32_t flag_unk_0x2000 : 1; // seen in various legion models<br />
uint32_t flag_unk_0x4000 : 1;<br />
uint32_t flag_unk_0x8000 : 1; // seen in UI_MainMenu_Legion<br />
uint32_t flag_unk_0x10000 : 1;<br />
uint32_t flag_unk_0x20000 : 1;<br />
uint32_t flag_unk_0x40000 : 1;<br />
uint32_t flag_unk_0x80000 : 1;<br />
uint32_t flag_unk_0x100000 : 1;<br />
uint32_t flag_unk_0x200000 : 1; // apparently: use 24500 upgraded model format: chunked .anim files, change in the exporter reordering sequence+bone blocks before name<br />
#endif<br />
#endif<br />
#endif<br />
#endif<br />
} global_flags;<br />
<br />
/*0x014*/ {{Template:Type/M2Array|M2Loop|section=Global_sequences}} global_loops; // Timestamps used in [[#Global_Sequences_2|global looping animations]].<br />
/*0x01C*/ {{Template:Type/M2Array|M2Sequence|section=Animation_sequences}} sequences; // Information about the animations in the model.<br />
/*0x024*/ {{Template:Type/M2Array|uint16_t|section=Animation_Lookup}} sequenceIdxHashById; // Mapping of [[#Animation_sequences|sequence IDs]] to the entries in the [[#Animation_sequences|Animation sequences]] block.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?|section=Playable_Animation_Lookup}} playable_animation_lookup;<br />
#endif<br />
/*0x02C*/ {{Template:Type/M2Array|M2CompBone|section=Bones}} bones; // MAX_BONES = 0x100 => Creature\SlimeGiant\GiantSlime.M2 has 312 bones ({{Template:Sandbox/PrettyVersion|expansionlevel=3}})<br />
/*0x034*/ {{Template:Type/M2Array|uint16_t|section=Key-Bone_Lookup}} boneIndicesById; //Lookup table for key skeletal bones. (alt. name: key_bone_lookup)<br />
/*0x03C*/ {{Template:Type/M2Array|M2Vertex|section=Vertices}} vertices;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|M2SkinProfile|section=Views_.28LOD.29}} skin_profiles;<br />
#else<br />
/*0x044*/ uint32_t num_skin_profiles; // [[#Views_.28LOD.29|Views (LOD)]] are now in [[M2/.skin|.skins]].<br />
#endif<br />
<br />
/*0x048*/ {{Template:Type/M2Array|M2Color|section=Colors}} colors; // Color and alpha animations definitions.<br />
/*0x050*/ {{Template:Type/M2Array|M2Texture|section=Textures}} textures;<br />
/*0x058*/ {{Template:Type/M2Array|M2TextureWeight|section=Transparency}} texture_weights; // Transparency of textures.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?}} unknown;<br />
#endif<br />
/*0x060*/ {{Template:Type/M2Array|M2TextureTransform|section=Texture_Transforms}} texture_transforms;<br />
/*0x068*/ {{Template:Type/M2Array|uint16_t|section=Replacable_texture_lookup}} textureIndicesById; // (alt. name: replacable_texture_lookup)<br />
/*0x070*/ {{Template:Type/M2Array|M2Material|section=Render_flags}} materials; // Blending modes / render flags.<br />
/*0x078*/ {{Template:Type/M2Array|uint16_t|section=Bone_Lookup_Table}} boneCombos; // (alt. name: bone_lookup_table)<br />
/*0x080*/ {{Template:Type/M2Array|uint16_t|section=Texture_lookup_table}} textureCombos; // (alt. name: texture_lookup_table)<br />
/*0x088*/ {{Template:Type/M2Array|uint16_t|section=Texture_unit_lookup_table}} textureTransformBoneMap; // (alt. name: tex_unit_lookup_table)<br />
/*0x090*/ {{Template:Type/M2Array|uint16_t|section=Transparency_lookup_table}} textureWeightCombos; // (alt. name: transparency_lookup_table)<br />
/*0x098*/ {{Template:Type/M2Array|uint16_t|section=Texture_Transforms_lookup_table}} textureTransformCombos; // (alt. name: texture_transforms_lookup_table)<br />
<br />
/*0x0A0*/ {{Template:Type|CAaBox}} bounding_box; // min/max( [1].z, 2.0277779f ) - 0.16f seems to be the maximum camera height<br />
/*0x0B8*/ float bounding_sphere_radius; // detail doodad draw dist = clamp (bounding_sphere_radius * detailDoodadDensityFade * detailDoodadDist, …)<br />
/*0x0BC*/ {{Template:Type|CAaBox}} collision_box;<br />
/*0x0D4*/ float collision_sphere_radius;<br />
<br />
/*0x0D8*/ {{Template:Type/M2Array|uint16_t|section=Triangles}} collisionIndices; // (alt. name: collision_triangles)<br />
/*0x0E0*/ {{Template:Type/M2Array|C3Vector|section=Vertices_2}} collisionPositions; // (alt. name: collision_vertices)<br />
/*0x0E8*/ {{Template:Type/M2Array|C3Vector|section=Normals}} collisionFaceNormals; // (alt. name: collision_normals) <br />
/*0x0F0*/ {{Template:Type/M2Array|M2Attachment|section=Attachments}} attachments; // position of equipped weapons or effects<br />
/*0x0F8*/ {{Template:Type/M2Array|uint16_t|section=Attachment_Lookup}} attachmentIndicesById; // (alt. name: attachment_lookup_table)<br />
/*0x100*/ {{Template:Type/M2Array|M2Event|section=Events}} events; // Used for playing sounds when dying and a lot else.<br />
/*0x108*/ {{Template:Type/M2Array|M2Light|section=Lights}} lights; // Lights are mainly used in loginscreens but in wands and some doodads too.<br />
/*0x110*/ {{Template:Type/M2Array|M2Camera|section=Cameras}} cameras; // The cameras are present in most models for having a model in the character tab. <br />
/*0x118*/ {{Template:Type/M2Array|uint16_t|section=Camera_lookup_table}} cameraIndicesById; // (alt. name: camera_lookup_table)<br />
/*0x120*/ {{Template:Type/M2Array|M2Ribbon|section=Ribbon_emitters}} ribbon_emitters; // Things swirling around. See the CoT-entrance for light-trails.<br />
/*0x128*/ {{Template:Type/M2Array|M2Particle|section=Particle_emitters}} particle_emitters;<br />
<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
if (flag_use_texture_combiner_combos)<br />
{<br />
/*0x130*/ {{Template:Type/M2Array|uint16_t|section=Blend_mode_overrides}} textureCombinerCombos; // When set, textures blending is overriden by the associated array.<br />
}<br />
#endif<br />
} header;<br />
<br />
==Types==<br />
struct M2Bounds {<br />
CAaBox extent;<br />
float radius;<br />
};<br />
template<typename T><br />
struct M2Array {<br />
uint32_t size;<br />
uint32_t offset; // pointer to T, relative to begin of m2 data block (i.e. MD21 chunk content or begin of file)<br />
};<br />
struct M2TrackBase {<br />
uint16_t trackType;<br />
uint16_t loopIndex;<br />
M2Array<M2SequenceTimes> sequenceTimes;<br />
};<br />
template<typename T> <br />
struct M2PartTrack {<br />
M2Array<fixed16> times;<br />
M2Array<T> values;<br />
};<br />
template<typename T> <br />
struct M2SplineKey {<br />
T value;<br />
T inTan;<br />
T outTan;<br />
};<br />
struct M2Range {<br />
uint32_t minimum;<br />
uint32_t maximum;<br />
};<br />
<br />
==Versions==<br />
Files get handled differently depending on this! Ranges are inclusive.<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Version !! width="190" | Extension<br />
|-<br />
| 272-274 || Legion<br />
|-<br />
| 272 || Warlords of Draenor<br />
|-<br />
| 272 || Mists of Pandaria<br />
|-<br />
| 265-272 || Cataclysm<br />
|-<br />
| 264 || Wrath of the Lich King<br />
|-<br />
| 260-263 || The Burning Crusade<br />
|-<br />
| 256-257 || Classic<br />
|-<br />
| 256 || Pre-Release<br />
|}<br />
<br />
==Chunks==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
<br />
From Legion and up, the file might be [[Chunk|chunked]] instead. If this is the case, the magic will be anything but 'MD20' and the m2 data will be in the 'MD21' chunk. If the first magic is 'MD20', it will be loaded just fine like it did previously. Note that the chunks can be in any order with MD21 often being first. <br />
<br />
'''NOTE''': Unlike all other chunked formats in WoW, chunk names in M2 are '''NOT''' reversed. Example: AFID == AFID in file.<br />
<br />
===MD21===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
The MD21 chunk contains the not yet chunked data, just like in the old format. The content literally is an pre legion file, starting with the MD20 magic. This also implies that all offsets inside this chunk are relative to the ''chunk'', not the ''file''.<br />
M2Data pre_legion_style_data;<br />
<br />
===PFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}.phys</tt>}}<br />
uint32_t phys_file_id;<br />
<br />
===SFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${view}.skin</tt> and <tt>${basename}_lod${lodband}.skin</tt>}}<br />
uint32_t skinFileDataIDs[header.nViews];<br />
uint32_t lod_skinFileDataIDs[lodBands /* 2? */];<br />
Some model files, for example 'Creature\NightborneFemaleCitizen\NightborneFemaleCitizen.m2' have 4 skin files and 2 lod files but only 20 bytes are in chunk. In chunk there are 4 skins and 1 lod present.<br />
<br />
Lod skins are selected based on distance to entity/doodad and chosen based on GetCVar("entityLodDist")/X and GetCVar("doodadLodDist")/X where X - distance. Lods are ignored when "M2UseLOD" CVar is set to 0.<br />
<br />
===AFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${anim_id}-${sub_anim_id}.anim</tt>}}<br />
struct<br />
{<br />
uint16_t anim_id;<br />
uint16_t sub_anim_id;<br />
uint32_t file_id; // might be 0 for "none" (so this is probably not sparse, even if it could be)<br />
} anim_file_ids[];<br />
<br />
===BFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}_${i}.bone</tt>}}<br />
uint32_t boneFileDataIDs[];<br />
<br />
===TXAC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=It is unknown what this replaced. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
char unk[2]; // likely used in CM2SceneRender::SetupTextureTransforms and uploaded to the shader directly. 0 otherwise.<br />
} texture_ac[m2data.header.materials.count + m2data.header.particles.count];<br />
<br />
===EXPT===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
} extended_particle[m2data.header.particles.count];<br />
<br />
Probably outdated chunk after introduction of EXP2 chunk. If EXP2 doesnt exist, client tries to reconstruct it with data from EXPT chunk.<br />
<br />
===EXP2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown}}}}<br />
<br />
<source><br />
struct EXP2<br />
{<br />
M2Array<Exp2Record> content;<br />
};<br />
<br />
struct Exp2Record<br />
{<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
M2PartTrack<fixed16> unk3;<br />
};<br />
</source><br />
<br />
<br />
The length of this M2Array is the same as length of particle_emitters<br />
<br />
===PABC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to replace {{Unverified|[[#Animation_Lookup]]}} from old file. {{Unverified|Exact build unknown}}}}<br />
M2Array<uint16t> m_replacementParentSequenceLookups; // only seen in quillboarbrute*.m2. Contains AnimationIds<br />
<br />
Replaces parentSequencesLookups. But unlike header.sequence_lookups of parent model, this is straight array and not a map. If index with target animation is not found in here, parentSequencesLookups are used instead.<br />
<br />
This chunk called BlacklistAnimData in client.<br />
<br />
Client doesnt seem to use found index and thus whole array is used only to check if the target animation is present.<br />
<br />
===PADC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Moves texture weights from old file to a chunk. {{Unverified|Exact build unknown}}}}<br />
Defines replacement for header.texture_weights (WHY?)<br />
<br />
<source lang="cpp"><br />
struct PADC {<br />
M2Array<M2TextureWeight> texture_weights;<br />
} <br />
</source><br />
<br />
===PSBC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
<br />
Defines ParentSequenceBounds<br />
M2Array<M2Bounds> parentSequenceBounds;<br />
<br />
===PEDC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
M2Array<M2TrackBase> parentEventData;<br />
<br />
===SKID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=This used to be filename based, using <tt>${basename}.skel</tt>. {{Unverified|Exact build unknown}}}}<br />
uint32_t SKeletonfileID; // links to [[M2/.skel]]<br />
<br />
===TXID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629|note=Replaces the filename for {{#Textures}} with hardcoded type.}}<br />
Replaces in-file texture filenames. <br />
struct {<br />
uint32_t fileDataID;<br />
} textureID[]<br />
<br />
===LDV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629}}<br />
<br />
Defines LodData<br />
<br />
<source lang="cpp"><br />
struct LodData<br />
{<br />
uint16 unk0; <br />
uint16 lodCount; //maxLod = lodCount-1; <br />
float unk2_f;<br />
uint8_t particleBoneLod[4]; //lod serves as indes into this array<br />
_DWORD unk4;<br />
};<br />
</source><br />
<br />
Somehow defines _lod%0d.skin files. On pandarenfemale.m2, lodCount == 4. SFID has 7 files first 4 are ordinary .skin files and last 3 are _lod%0d.skin files. Enumeration for _lod%0d.skin files for that model starts from 1, and last file in SFID is pandarenfemale_lod03.skin So technically maxLod indeed represents maximum Lod<br />
<br />
unk2_f is used in formula, but it's purpose is unknown<br />
fmaxf(fminf(740.0 / unk2_f, 5.0), 0.5);<br />
<br />
LodData.particleBoneLod works this way: Each model has current lod which is [0..3]. Next:<br />
<source lang="cpp"><br />
if ( lod < 1 )<br />
result = 0;<br />
<br />
if ( LodData)<br />
result = (0x10000 << LodData->particleBoneLod[lod]);<br />
else<br />
result = (0x10000 << (lod- 1));<br />
<br />
...<br />
//For each ParticleEmitter and related M2Particle record<br />
if ( result & M2CompBone[M2Particle->old.boneIndex].flags ) {<br />
//Do not animate this emitter<br />
}<br />
</source><br />
<br />
<br />
[[User:Deamon|Deamon]] ([[User talk:Deamon|talk]])<br />
<br />
===RPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>recursion_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} recursive_particle_models[particle count];<br />
<br />
===GPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>geometry_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} geometry_particle_models[particle count];<br />
<br />
===WFV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV1 {<br />
// unknown<br />
};<br />
<br />
===WFV2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV2 {<br />
// unknown<br />
};<br />
<br />
===PGD1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=1|min_build=1.13.2.30172}}<br />
struct PGD1Entry {<br />
char _0x00[2];<br />
};<br />
M2Array<PGD1Entry> p_g_d_v1; // count appears to be equivalent to particle count<br />
<br />
=Skeleton and animation=<br />
==Global sequences==<br />
A list of timestamps that act as upper limits for global sequence ranges.<br />
struct M2Loop<br />
{<br />
uint32_t timestamp;<br />
} loops[];<br />
<br />
==Standard animation block==<br />
* {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} uses a single-timeline approach, chaining all animations into one long piece and separating them via begin and end given in animation data. {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, each animation has an own timeline.<br />
* Animation blocks contain a list of lists of timestamps and a list of lists of values, where the first list is by animation and the second one by timestamp-entry.<br />
*'''Many values that change with time are specified using blocks like the following.'''<br />
<br />
template<typename T><br />
struct M2Array<br />
{<br />
/*0x00*/ uint32_t number;<br />
/*0x04*/ uint32_t offset_elements;<br />
/*0x08*/<br />
};<br />
<br />
struct M2TrackBase<br />
{<br />
/*0x00*/ uint16_t interpolation_type;<br />
/*0x02*/ uint16_t global_sequence;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}}<br />
M2Array<pair<uint32_t>> interpolation_ranges; // no longer required {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, as implicit by minimum and maximum timestamp per sequence.<br />
M2Array<uint32_t> timestamps;<br />
#else<br />
/*0x04*/ M2Array<M2Array<uint32_t>> timestamps;<br />
#endif<br />
/*0x0C*/<br />
};<br />
<br />
template<typename T><br />
struct M2Track : M2TrackBase<br />
{<br />
/*0x00*/ // base <br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} <br />
M2Array<T> values;<br />
#else<br />
/*0x0C*/ M2Array<M2Array<T>> values;<br />
#endif<br />
/*0x14*/<br />
};<br />
<br />
* Thus, as example, with<br />
<br />
{{Template:Type|M2CompBone|link=M2#Bones}} b;<br />
<br />
one may get the number of animations having translation information with<br />
<br />
b.translation.timestamps.number<br />
<br />
and the number of timestamps in the first animation using<br />
<br />
b.translation.timestamps.elements[0].number<br />
<br />
and the first timestamp value of the first animation via<br />
<br />
b.translation.timestamps.elements[0].elements[0]<br />
<br />
The actual translation vector for animation 0 at timestamp 0 is at<br />
<br />
b.translation.values.elements[0].elements[0]<br />
<br />
* Some timestamps/values.elements entries may have number/elements = 0, if for that animation id no animation is given.<br />
* [[#.anim_files|.anim]] files are just a blob of data which may as well be in the main model file, that is pointed to by the first array_ref layer.<br />
* [model file name][animation id]-[animation sub-id][[#.anim_files|.anim]]<br />
* it seems like it is possible to detect if animation data is stored in-m2 or externally via<br />
** All animations which have flags & 0x20 are stored internally. <br />
** Animations which do not have flags & 0x20 are not stored internally. <br />
** Animations which do not have flags & 0x20 AND do not have flags & 0x40 are in [[#.anim_files|.anim]] files<br />
** Animations which do not have flags & 0x20 AND DO have flags 0x40 are stored... somewhere. I have no clue.<br />
<br />
===Global Sequences===<br />
If a block has a sequence >= 0, the block has a completely separate max timestamp. This is the value in the model's ofsGlobalSequences table; index into that table with this sequence value and use that as the block's max timestamp. Blocks that use these global sequences also only have one track, so at the same time as clipping the current timestamp to the max time above, interpolated value should always be taken from track 0 in the block. <br />
<br />
A global sequence is completely unrelated to animations. It just always loops. This way, the sequence is not interrupted when an animation is launched.<br />
<br />
This appears to be frequently used by models that don't have more conventional animations (login screen animations, items/weapons with animated effects, etc).<br />
<br />
-- Rour, additionally, these sequences can be longer or shorter than whatever animation is running for a given model, so I recommend taking a global scene timestamp and then clipping that value into the given max timestamp range. Otherwise animations will appear to reset when the regular animation loops, which is not good.<br />
<br />
===Interpolation===<br />
* If the interpolation type is 0, then values will change instantly at the timestamp, with no interpolation whatsoever.<br />
* If the interpolation type is 1, then the block linearly interpolates between keyframe values (lerp for vectors/colours, nlerp for quaternions).<br />
* If the interpolation type is 2, then cubic bezier spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the first control point is the first spline key's <code>value</code>, the second control point is the first spline key's <code>tanOut</code>, the third control point is the second spline key's <code>tanIn</code>, and the fourth control point is the second spline key's <code>value</code>.<br />
* If the interpolation type is 3, then cubic hermite spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the starting point is the first spline key's <code>value</code>, the starting tangent is the first spline key's <code>tanOut</code>, the ending tangent is the second spline key's <code>tanIn</code>, and the ending point is the second spline key's <code>value</code>.<br />
<br />
'''NOTE:''' There is confusion about type <code>2</code> and <code>3</code> being <code>hermite/bezier</code> or <code>bezier/hermite</code>. Alpha says that <code>2 = hermite</code>, WoD says that <code>2 = bezier</code>. This was changed when the format went from MDL to M2. --[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 01:53, 4 September 2017 (CEST), --[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) 19:15, 14 November 2018 (GMT)<br />
<br />
In WotLK 2 appears to be bezier, which pretty much confirms it being bezier for wotlk+. TBC and classic need further checking though. -- [[User:Skarn|Skarn]] ([[User talk:Skarn|talk]]) 01:38, 16 June 2018 (CEST)<br />
<br />
==.anim files==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=3}}<br />
<br />
Low priority sequences (e.g. emotes, one shot animations) are in extra files to allow for lazy loading. These files are raw data of timestamps and values for animation blocks. Blizzard's exporter prefers to align blocks to a 16 byte boundary, but that's not required.<br />
<br />
'''The client loads .anim files if (M2Sequence.flags & 0x130 ) == 0.''' The .anim file to use is <code>"%s%04d-%02d.anim" % (model_filename_without_extension, anim.id, anim.sub_anim_id)</code>.<br />
<br />
===Legion 24500===<br />
In Legion, these files are optionally chunked now. They are chunked either<br />
* if M2 header's 0x200000 flag is set and thus the new mid expansion format change is used<br />
* if the M2 has a .skel file<br />
<br />
For new format M2s, .anim is pretty much unchanged except that there is the AFM2 chunk header. The AFSA and AFSB chunks do not appear in that case. If it is a .skel file based model, the chunks are present and animation data is split into bone and attach data. The AFM2 chunk then contains the animation data for ????, the AFSA chunk that for attachments and the AFSB chunk that for bones. See .skel files for that.<br />
<br />
====AFM2====<br />
The same content as an old anim file would have. In fact, files that were just converted to the new format are bit identical except for the chunk header.<br />
====AFSA====<br />
skeleton data for attachments<br />
====AFSB====<br />
skeleton data for bones<br />
<br />
==Animation sequences==<br />
List of animations present in the model. <br />
struct M2Sequence<br />
{<br />
uint16_t id; // Animation id in [[AnimationData.dbc]]<br />
uint16_t variationIndex; // Sub-animation id: Which number in a row of animations this one is.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
uint32_t start_timestamp;<br />
uint32_t end_timestamp;<br />
#else<br />
uint32_t duration; // The length of this animation sequence in milliseconds.<br />
#endif<br />
float movespeed; // This is the speed the character moves with in this animation.<br />
uint32_t flags; // See below.<br />
int16_t frequency; // This is used to determine how often the animation is played. For all animations of the same type, this adds up to 0x7FFF (32767).<br />
uint16_t _padding;<br />
M2Range replay; // May both be 0 to not repeat. Client will pick a random number of repetitions within bounds if given.<br />
#if version < ??? < 6.0.1<br />
uint32_t blendTime;<br />
#else<br />
uint16_t blendTimeIn; // The client blends (lerp) animation states between animations where the end and start values differ. This specifies how long that blending takes. Values: 0, 50, 100, 150, 200, 250, 300, 350, 500.<br />
uint16_t blendTimeOut; // The client blends between this sequence and the next sequence for blendTimeOut milliseconds.<br />
#endif<br />
// For both blendTimeIn and blendTimeOut, the client plays both sequences simultaneously while interpolating between their animation transforms.<br />
{{Template:Type|M2Bounds}} bounds;<br />
int16_t variationNext; // id of the following animation of this AnimationID, points to an Index or is -1 if none.<br />
uint16_t aliasNext; // id in the list of animations. Used to find actual animation if this sequence is an alias (flags & 0x40)<br />
} sequences[];<br />
<br />
--[[User:Koward|Koward]] ([[User talk:Koward|talk]]) 09:50, 18 December 2015 (UTC) In M2 v274 (Legion), it looks like blend_time has been divided in two uint16_t, and for standard animations the old blend_time is duplicated in both fields (ex : uint32 150 becomes two uint16 150). Maybe start and end blend_time values ? See Creature/GennGreymane/GennGreymane.m2 .<br />
===Flags===<br />
''One thing I saw in the source is that "-1 animationblocks" in bones wont get parsed if 0x20 is not set.''<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="500" | Description<br />
|-<br />
| 0x01 || Sets 0x80 when loaded. (M2Init)<br />
|-<br />
| 0x02 ||<br />
|-<br />
| 0x04 ||<br />
|-<br />
| 0x08 ||<br />
|-<br />
| 0x10 || apparently set during runtime in CM2Shared::LoadLowPrioritySequence for all entries of a loaded sequence (including aliases)<br />
|-<br />
| 0x20 || primary bone sequence -- If set, the animation data is in the .m2 file. If not set, the animation data is in an .anim file. {{Template:Unverified|Was named 'looped animation' by schlumpf years ago, without source.}}<br />
|-<br />
| 0x40 || has next / is alias (To find the animation data, the client skips these by following aliasNext until an animation without 0x40 is found.)<br />
|-<br />
| 0x80 || Blended animation (if either side of a transition has 0x80, lerp between end->start states, unless end==start by comparing bone values)<br />
|-<br />
| 0x100 || sequence stored in model ?<br />
|-<br />
| 0x200 ||<br />
|-<br />
| 0x400 ||<br />
|-<br />
| 0x800 || seen in Legion 24500 models<br />
|}<br />
<br />
-- Rour, some animations rely on blending to look right. The MoP mage CM shoulders only animate half of their movement and rely on lerping back to the start position to look correct.<br />
<br />
=== Animation Lookup===<br />
Hash table for Animations in [[AnimationData.dbc]].<br />
<br />
struct<br />
{<br />
uint16_t animation_index; // Index at [[#Animation_sequences|ofsAnimations]] which represents the animation in [[AnimationData.dbc]]. -1 if none.<br />
} animation_lookups[];<br />
<br />
The hash used is <tt>anim_id % num_buckets</tt>. If a bucket is used, a stride of <tt>n^2</tt> is added with <tt>n = 1, 2, …</tt> until the entry is matching:<br />
<br />
<source lang="c++"><br />
M2Sequence* find_entry (uint32_t anim_id)<br />
{<br />
size_t i (anim_id % animation_lookups.count);<br />
<br />
for (size_t stride (1); true; ++stride)<br />
{<br />
if (animation_lookups[i] == -1)<br />
{<br />
return nullptr;<br />
}<br />
if (animation_sequences[animation_lookups[i]].id == anim_id)<br />
{<br />
return &animation_sequences[i];<br />
}<br />
<br />
i = (i + stride * stride) % animation_lookups.count;<br />
// so original_i + 1, original_i + 1 + 4, original_i + 1 + 4 + 9, …<br />
}<br />
<br />
[[unreachable]];<br />
}<br />
</source><br />
<br />
The entry referenced is the first in the `nextAlias` chain of a given animation id. Thus, <tt>num_buckets < num_animations</tt>, even if a model would have all animations multiple times.<br />
<br />
=== Playable Animation Lookup ===<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=2|note=Partially inlined into M2Sequences}}<br />
<br />
Lookup table for Playable Animation in [[AnimationData.dbc]]<br />
<br />
'''Offset Type Description'''<br />
0x00 int16 Fallback Animation ID in [[AnimationData.dbc]]<br />
0x02 int16 Flags (0, 1, 3 seen)<br />
<br />
Models don't present all animations sequences. This table maps from global animation list to available animation for the current model. The engine uses it to know which animation clip to play. That's why there are a lot of zeros ("Stand") for creatures.<br />
<br />
Flags are used to modify how the clip should be played:<br />
'''Value Meaning'''<br />
0 Play normal<br />
1 Play backwards?<br />
3 Freeze<br />
<br />
For instance, the "HumanMale/HumanMale.m2" model has just one "Loot" animation sequence. "LootHold" and "LootUp" are obtained with flags 3 and 1.<br />
<br />
== Bones ==<br />
struct M2CompBone // probably M2Bone {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
{<br />
int32_t key_bone_id; // Back-reference to [[#Key-Bone_Lookup|the key bone lookup table]]. -1 if this is no key bone.<br />
enum<br />
{<br />
spherical_billboard = 0x8,<br />
cylindrical_billboard_lock_x = 0x10,<br />
cylindrical_billboard_lock_y = 0x20,<br />
cylindrical_billboard_lock_z = 0x40,<br />
transformed = 0x200,<br />
kinematic_bone = 0x400, // MoP+: allow [[PHYS|physics]] to influence this bone<br />
helmet_anim_scaled = 0x1000, // set blend_modificator to [[HelmetAnimScaling.dbc|helmetAnimScalingRec]].m_amount for this bone<br />
};<br />
uint32_t flags; <br />
int16_t parent_bone; // Parent bone ID or -1 if there is none.<br />
uint16_t [[M2/.skin#Submeshes|submesh_id]]; // Mesh part ID OR uDistToParent?<br />
union { // only {{Template:Sandbox/VersionRange|min_expansionlevel=2}}?<br />
struct {<br />
uint16_t uDistToFurthDesc;<br />
uint16_t uZRatioOfChain;<br />
} CompressData; // {{Template:Unverified|No model has ever had this part of the union used.}}<br />
uint32_t boneNameCRC; // these are for debugging only. their bone names match those in key bone lookup.<br />
};<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation;<br />
#else<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|M2CompQuat|link=Quaternion_values_and_2.x}}> rotation; // compressed values, default is (32767,32767,32767,65535) == (0,0,0,1) == identity<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scale;<br />
{{Template:Type|C3Vector}} pivot; // The pivot point of that bone.<br />
} bones[];<br />
The bone indices in the vertex definitions seem to index into this data.<br />
<br />
===Billboards===<br />
The billboarding bits are used for various things:<br />
* Light halos around lamps must always face the viewer<br />
* The cannonball stack model (in the Deadmines or Booty Bay), where each cannonball is a crude hemisphere, they always face the viewer to create the illusion of actual cannonballs.<br />
<br />
Spherical and cylindrical billboard bits are mutually exclusive. Only one of them can be used for the bone.<br />
<br />
===Bone Lookup Table===<br />
Lookup table for bones referenced from [[M2/.skin#Submeshes|M2SkinSection]]. <br />
struct <br />
{<br />
uint16_t bone;<br />
} bone_lookup[];<br />
<br />
===Key-Bone Lookup===<br />
Its a lookup table for key skeletal bones like hands, arms, legs, etc. nKeyBoneLookup is 27 for the most models. At static models it is mostly 1.<br />
struct <br />
{<br />
uint16_t bone; // -1 if none<br />
} key_bone_lookup[];<br />
<br />
Official list:<br />
*00 "ArmL"<br />
*01 "ArmR"<br />
*02 "ShoulderL"<br />
*03 "ShoulderR"<br />
*04 "SpineLow"<br />
*05 "Waist"<br />
*06 "Head"<br />
*07 "Jaw"<br />
*08 "IndexFingerR"<br />
*09 "MiddleFingerR"<br />
*10 "PinkyFingerR"<br />
*11 "RingFingerR"<br />
*12 "ThumbR"<br />
*13 "IndexFingerL"<br />
*14 "MiddleFingerL"<br />
*15 "PinkyFingerL"<br />
*16 "RingFingerL"<br />
*17 "ThumbL"<br />
*18 "$BTH"<br />
*19 "$CSR"<br />
*20 "$CSL"<br />
*21 "_Breath"<br />
*22 "_Name"<br />
*23 "_NameMount"<br />
*24 "$CHD"<br />
*25 "$CCH"<br />
*26 "Root"<br />
*27 "Wheel1" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*28 "Wheel2" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*29 "Wheel3" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*30 "Wheel4" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*31 "Wheel5" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*32 "Wheel6" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*33 "Wheel7" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*34 "Wheel8" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
<br />
=Geometry and rendering=<br />
==Vertices==<br />
struct M2Vertex<br />
{<br />
{{Template:Type|C3Vector}} pos;<br />
uint8 bone_weights[4];<br />
uint8 bone_indices[4];<br />
{{Template:Type|C3Vector}} normal;<br />
{{Template:Type|C2Vector}} tex_coords[2]; // two textures, depending on shader used<br />
};<br />
<br />
Models, too, use a Z-up coordinate systems, so in order to convert to Y-up, the X, Y, Z values become (X, -Z, Y).<br />
<br />
-- Rour, the WoW vertex shaders all follow the same pattern, "Diffuse_XX_YY" (or sometimes XX, YY and Env). The particular vertex shader that is used chooses which set of texture coordinates to use. So Diffuse_T1 sends T1 texcoords to the fragment shader. Where Diffuse_T1_T2 sends both (for textures 0 and 1) but Diffuse_T1_T1 sends the same coords for both textures. Etc.<br />
<br />
==Views (LOD)==<br />
Skin profiles describe LOD views onto the model. They use all those lookup tables to be able to reference only parts of the lists while not being dynamically sized within the profile data.<br />
<br />
{{Template:Sandbox/VersionRange|max_expansionlevel=2}} they were stored in the [[M2]] itself, {{Template:Sandbox/VersionRange|min_expansionlevel=3}} they have been moved to [[M2/.skin|.skin]] files. The offsets are relative to the file the skin profile header is defined in. There is one [[M2/.skin|.skin]] file per profile, each with a separate header, while in the inlined version, all headers are sequential. See the [[M2/.skin|.skin]] file page for formats of both versions.<br />
<br />
==Render flags==<br />
struct M2Material<br />
{<br />
uint16_t flags;<br />
uint16_t blending_mode; // apparently a bitfield<br />
} materials[];<br />
*'''Flags:'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="1000" | Meaning<br />
|-<br />
| 0x01 || Unlit<br />
|-<br />
| 0x02 || Unfogged<br />
|-<br />
| 0x04 || Two-sided (no backface culling if set)<br />
|-<br />
| 0x08 || depthTest<br />
|-<br />
| 0x10 || depthWrite<br />
|-<br />
| 0x40 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x80 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x400 || ??? (seen in WoD)<br />
|-<br />
| 0x800 || prevent alpha for custom elements. if set, use (fully) opaque or transparent. (litSphere, shadowMonk) (MoP+)<br />
|}<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! source !! dest !! notes<br />
|-<br />
| 0 || GL_ONE || GL_ZERO || blending disabled (GxBlendStateDesc: 0)<br />
|-<br />
| 1 || GL_ONE || GL_ZERO || Mod (1)<br />
|-<br />
| 2 || GL_SRC_ALPHA || GL_ONE_MINUS_SRC_ALPHA || Decal (2)<br />
|-<br />
| 3 || GL_ONE || GL_ONE || Add (10)<br />
|-<br />
| 4 || GL_SRC_ALPHA || GL_ONE || Mod2x (3)<br />
|-<br />
| 5 || GL_DST_COLOR || GL_ZERO || Fade (4)<br />
|-<br />
| 6 || GL_DST_COLOR || GL_SRC_COLOR || Deeprun Tram glass (5)<br />
|-<br />
| 7 || GL_ONE || GL_ONE_MINUS_SRC_ALPHA || WoD+ (13)<br />
|}<br />
<br />
(blend equation is always GL_FUNC_ADD. Values are retrieved via GxBlendStateDesc's lower 5 bits. no separate blend func for alpha.)<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="100" | Mapped to !! width="1500" | Meaning<br />
|-<br />
| 0 || 0 || Combiners_Opaque (Blend disabled)<br />
|-<br />
| 1 || 1 || Combiners_Mod (Blend enabled, Src = ONE, Dest = ZERO, SrcAlpha = ONE, DestAlpha = ZERO)<br />
|-<br />
| 2 || 1 || Combiners_Decal (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 3 || 1 || Combiners_Add (Blend enabled, Src = SRC_COLOR, Dest = DEST_COLOR, SrcAlpha = SRC_ALPHA, DestAlpha = DEST_ALPHA )<br />
|-<br />
| 4 || 1 || Combiners_Mod2x (Blend enabled, Src = SRC_ALPHA, Dest = ONE, SrcAlpha = SRC_ALPHA, DestAlpha = ONE )<br />
|-<br />
| 5 || 4 || Combiners_Fade (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 6 || 4 || Used in the Deeprun Tram subway glass, supposedly (Blend enabled, Src = DEST_COLOR, Dest = SRC_COLOR, SrcAlpha = DEST_ALPHA, DestAlpha = SRC_ALPHA )<br />
|-<br />
| 7 ||? || New in WoD, needs research! ''Example model: World\Expansion05\Doodads\Shadowmoon\Doodads\6FX_Fire_Grassline_Doodad_blue_LARGE.m2''<br />
|}<br />
''*Blend values are taken from D3D11 debugging of the client<br />
===Blend mode overrides===<br />
If this block is present (globalflags&8) and the "shading" flags of a textureunit wont be &0x8000, blending modes wont get mapped to the values above but to the ones in this block.<br />
<br />
Instead of Mapping[renderflags->blendingmode] it will be UnknownBlock[textureunit->Shading].<br />
<br />
As shading is not &0x8000 and (in their code) needs to be above 0, this may only touch Diffuse_T1.<br />
<br />
According to wod, if the M2 Header has flag 0x08, instead of reading blend mode from M2 [[#Render_flags|RenderFlags]], blendMode is read from the raw blend maps referenced in header.<br />
<br />
var flags = renderFlags[texUnit.renderFlags];<br />
var blendMode = flags >> 16;<br />
if ((header.GlobalModelFlags & 0x08) != 0 && texUnit.shader_id < mBlendMap.Length)<br />
blendMode = mBlendMap[texUnit.shader_id];<br />
<br />
==Texture unit lookup table==<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=4|max_exclusive=1|note=Still present but unused in Cataclysm}}<br />
struct <br />
{<br />
uint16_t unit; // -1, 0, or 1. see below<br />
} texture_units[];<br />
<br />
For models that use multitexturing, this maps given texture unit numbers into actual texture unit numbers (0 or 1).<br />
<br />
Values of -1 seem to mean environment mapping.<br />
<br />
One model is of special interest, Creature/KelThuzad/KelThuzad.m2, which is the only one that has an nTexUnits of 3, and has three texture units specified for some of its submeshes. Sure enough, two of those map to 0 and 1, and one maps to -1.<br />
<br />
More confusion thanks to my favorite "weird" model, World/Generic/Gnome/Passive Doodads/GnomeMachine/GnomeSubwayGlass.m2, which is the translucent, environment mapped glass tunnel in the Deeprun Tram. It only has a single value in this block, -1, which is used for the single texture layer in both render operations in the model. This and the magic with rendering flags/blend modes make up the neat transparent-reflective glass effect, but confuse me even more about how envmapping and such is handled. (and where it seems to get the bluish color from - is it in the model (no color blocks in this particular model), the wmo, a solid background color, or simply the result of the blending used?)<br />
<br />
As a side note, on my (dated) system WoW does every texture unit in a single pass.<br />
<br />
==Colors and transparency==<br />
===Colors===<br />
struct M2Color<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> color; // vertex colors in rgb order<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alpha; // 0 - transparent, 0x7FFF - opaque. Normaly NonInterp<br />
} colors[];<br />
<br />
This block is the M2 equivalent to the GEOA chunk in MDX files, it represents the vertex color and visibility animations for meshes. Referenced from the Texture Unit blocks in the [[M2/.skin|*.skin]]. If a texunit belonging to a submesh has a value of -1 then the submesh doesnot use this block. Contains a separate timeline for transparency values. If no animation is used, the given value is constant.<br />
<br />
===Transparency===<br />
struct M2TextureWeight<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> weight;<br />
} textureWeights[];<br />
<br />
Specifies global transparency values in addition to the values given in the Color block. I assume these are multiplied together eventually.<br />
====Transparency lookup table====<br />
struct <br />
{<br />
uint16_t transparency;<br />
} transparency_lookup[];<br />
Contains indices into the Transparency block. Used by the texture unit definitions in the LOD block.<br />
<br />
==Textures==<br />
*'''Textures are defined globally in a list''', additionally, a lookup table is given, referenced during rendering, to select textures.<br />
struct M2Texture<br />
{<br />
uint32_t type; // see below<br />
uint32_t flags; // see below<br />
M2Array<char> filename; // for non-hardcoded textures (type != 0), this still points to a zero-sized string<br />
} textures[];<br />
<br />
====Texture Types====<br />
Texture type is 0 for regular textures, nonzero for skinned textures (filename not referenced in the M2 file!) For instance, in the NightElfFemale model, her eye glow is a type 0 texture and has a file name, the other 3 textures have types of 1, 2 and 6. The texture filenames for these come from client database files:<br />
*DBFilesClient\[[CharSections.dbc]]<br />
*DBFilesClient\[[CreatureDisplayInfo.dbc]]<br />
*DBFilesClient\[[ItemDisplayInfo.dbc]]<br />
*(possibly more)<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="800" | Meaning<br />
|-<br />
| 0 || - NONE - -- Texture given in filename<br />
|-<br />
| 1 || TEX_COMPONENT_SKIN -- Skin -- Body + clothes<br />
|-<br />
| 2 || TEX_COMPONENT_OBJECT_SKIN -- Object Skin -- Item, Capes ("Item\ObjectComponents\Cape\*.blp")<br />
|-<br />
| 3 || TEX_COMPONENT_WEAPON_BLADE -- Weapon Blade -- Used on several models but not used in the client as far as I see. Armor Reflect?<br />
|-<br />
| 4 || TEX_COMPONENT_WEAPON_HANDLE -- Weapon Handle<br />
|-<br />
| 5 || TEX_COMPONENT_ENVIRONMENT -- (OBSOLETE) Environment (Please remove from source art)<br />
|-<br />
| 6 || TEX_COMPONENT_CHAR_HAIR -- Character Hair<br />
|-<br />
| 7 || TEX_COMPONENT_CHAR_FACIAL_HAIR -- (OBSOLETE) Character Facial Hair (Please remove from source art)<br />
|-<br />
| 8 || TEX_COMPONENT_SKIN_EXTRA -- Skin Extra<br />
|-<br />
| 9 || TEX_COMPONENT_UI_SKIN -- UI Skin -- Used on inventory art M2s (1): inventoryartgeometry.m2 and inventoryartgeometryold.m2<br />
|-<br />
| 10 || TEX_COMPONENT_TAUREN_MANE -- (OBSOLETE) Tauren Mane (Please remove from source art) -- Only used in quillboarpinata.m2. I can't even find something referencing that file. Oo Is it used?<br />
|-<br />
| 11 || TEX_COMPONENT_MONSTER_1 -- Monster Skin 1 -- Skin for creatures or gameobjects #1<br />
|-<br />
| 12 || TEX_COMPONENT_MONSTER_2 -- Monster Skin 2 -- Skin for creatures or gameobjects #2<br />
|-<br />
| 13 || TEX_COMPONENT_MONSTER_3 -- Monster Skin 3 -- Skin for creatures or gameobjects #3<br />
|-<br />
| 14 || TEX_COMPONENT_ITEM_ICON -- Item Icon -- Used on inventory art M2s (2): ui-button.m2 and forcedbackpackitem.m2 (CSimpleModel_ReplaceIconTexture("texture"))<br />
|-<br />
| 15 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Background Color<br />
|-<br />
| 16 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem Color<br />
|-<br />
| 17 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Border Color<br />
|-<br />
| 18 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem<br />
|}<br />
<br />
====Flags====<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Meaning<br />
|-<br />
| 1 || Texture wrap X<br />
|-<br />
| 2 || Texture wrap Y<br />
|}<br />
<br />
====Texture lookup table====<br />
struct <br />
{<br />
uint16_t texture;<br />
} texture_lookup[];<br />
<br />
====Replacable texture lookup====<br />
struct <br />
{<br />
uint16_t replacement;<br />
} texture_replacements[];<br />
<br />
A reverse lookup table for 'replaced' textures, mapping replacable ids to texture indices or -1. Only goes up to the maximum id used in the model.<br />
<br />
Its strange, that HARDCODED is in the list, as a model can have more than one of course. Its just the last one written to the file.<br />
<br />
=Effects=<br />
==Texture Transforms==<br />
*'''This block contains definitions for texture animations,''' for example, flowing water or lava in some models. The keyframe values are used in the texture transform matrix.<br />
<br />
struct M2TextureTransform<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation; // rotation center is texture center (0.5, 0.5)<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scaling;<br />
} textureTransforms[];<br />
<br />
Seems like UV rotation in some models are made against (0.5, 0.5) point instead of (0, 0). At least it's the case for world\goober\g_scourgerunecirclecrystal.m2 <br />
<br />
So to get the proper UV rotation it would be necessary apply rotation this way:<br />
<br />
* Translate UV anim matrix to point (0.5, 0.5)<br />
* Apply rotation mat from quaternion<br />
* UV anim matrix to point (-0.5, -0.5)<br />
<br />
====Texture Transforms lookup table ====<br />
struct <br />
{<br />
uint16_t anim_texture_id; // -1 for static<br />
} anim_texture_lookup[];<br />
<br />
== Ribbon emitters ==<br />
struct M2Ribbon<br />
{<br />
uint32_t ribbonId; // Always (as I have seen): -1.<br />
uint32_t boneIndex; // A bone to attach to.<br />
{{Template:Type|C3Vector}} position; // And a position, relative to that bone.<br />
M2Array<uint16_t> textureIndices; // into [[#Textures|textures]]<br />
M2Array<uint16_t> materialIndices; // into [[#Render_flags|materials]]<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> colorTrack;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alphaTrack; // And an alpha value in a short, where: 0 - transparent, 0x7FFF - opaque.<br />
[[#Standard_animation_block|M2Track]]<float> heightAboveTrack;<br />
[[#Standard_animation_block|M2Track]]<float> heightBelowTrack; // do not set to same!<br />
float edgesPerSecond; // this defines how smooth the ribbon is. A low value may produce a lot of edges.<br />
float edgeLifetime; // the length aka Lifespan. in seconds<br />
float gravity; // use arcsin(val) to get the emission angle in degree<br />
uint16_t textureRows; // tiles in texture<br />
uint16_t textureCols;<br />
[[#Standard_animation_block|M2Track]]<uint16_t> texSlotTrack;<br />
[[#Standard_animation_block|M2Track]]<uchar> visibilityTrack;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}} // TODO: verify version<br />
int16_t priorityPlane;<br />
uint16_t padding;<br />
#endif<br />
} ribbons[];<br />
Some models that contain ribbon emitters and are viewable in the game world are: Wisps in BFD, Al'ar the Phoenix in Tempest Keep and any other phoenix models and the energy trails in the COT (not the actual instance, but the entrance cave in Tanaris Desert). Other models with ribbon emitters are spells and effects.<br />
<br />
''Parameters from the MDL format that are probably in here somewhere: emission rate, rows, cols ...?''<br />
<br />
== Particle emitters ==<br />
'''This is partly wrong as hell!''' Do not rely on this block, at all. It might even be wrong for WotLK.<br />
===M2ParticleOld===<br />
struct M2ParticleOld {<br />
uint32 particleId; // Always (as I have seen): -1.<br />
uint32 flags; // See Below<br />
{{Template:Type|C3Vector}} Position; // The position. Relative to the following bone.<br />
uint16 bone; // The [[#Bones|bone]] its attached to.<br />
union<br />
{<br />
uint16 texture; // And the [[#Texture_definitions|textures]] that are used. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
struct // For multi-textured particles actually three ids<br />
{<br />
uint16_t texture_0 : 5;<br />
uint16_t texture_1 : 5;<br />
uint16_t texture_2 : 5;<br />
uint16_t : 1;<br />
};<br />
#endif<br />
};<br />
M2Array<char> geometry_model_filename; // if given, this emitter spawns models<br />
M2Array<char> recursion_model_filename; // if given, this emitter is an '''alias''' for the (maximum 4) emitters of the given model<br />
<br />
#if >= 263 (late Burning Crusade)<br />
uint8 blendingType; // A blending type for the particle. See Below<br />
uint8 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
uint16 particleColorIndex; // This one is used for [[ParticleColor.dbc]]. See below.<br />
#else<br />
uint16 blendingType; // A blending type for the particle. See Below<br />
uint16 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
#endif<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
{{Template:Type|fixed_point}}<uint8_t, 2, 5> multiTextureParamX[2];<br />
#else<br />
uint8 particleType; // Found below.<br />
uint8 headorTail; // 0 - Head, 1 - Tail, 2 - Both <br />
#endif<br />
uint16 textureTileRotation; // Rotation for the texture tile. (Values: -1,0,1) -- priorityPlane<br />
uint16 textureDimensions_rows; // for tiled textures<br />
uint16 textureDimensions_columns;<br />
[[#Standard_animation_block|M2Track]]<float> emissionSpeed; // Base velocity at which particles are emitted.<br />
[[#Standard_animation_block|M2Track]]<float> speedVariation; // Random variation in particle emission speed. (range: 0 to 1)<br />
[[#Standard_animation_block|M2Track]]<float> verticalRange; // longitude; Drifting away vertically. (range: 0 to pi) For plane generators, this is the maximum polar angle of the initial velocity; <br />
// 0 makes the velocity straight up (+z). For sphere generators, this is the maximum elevation of the initial position; <br />
// 0 makes the initial position entirely in the x-y plane (z=0).<br />
[[#Standard_animation_block|M2Track]]<float> horizontalRange; // latitude; They can do it horizontally too! (range: 0 to 2*pi) For plane generators, this is the maximum azimuth angle of the initial velocity; <br />
// 0 makes the velocity have no sideways (y-axis) component. <br />
// For sphere generators, this is the maximum azimuth angle of the initial position.<br />
[[#Standard_animation_block|M2Track]]<float> gravity; // Not necessarily a float; [[#Compressed Particle Gravity|see below]].<br />
[[#Standard_animation_block|M2Track]]<float> lifespan;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float lifespanVary; // An individual particle's lifespan is added to by <code>lifespanVary * random(-1, 1)</code><br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionRate; <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float emissionRateVary; // This adds to the base emissionRate value the same way as lifespanVary. The random value is different every update.<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaLength; // For plane generators, this is the width of the plane in the x-axis.<br />
// For sphere generators, this is the minimum radius.<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaWidth; // For plane generators, this is the width of the plane in the y-axis.<br />
// For sphere generators, this is the maximum radius.<br />
[[#Standard_animation_block|M2Track]]<float> zSource; // When greater than 0, the initial velocity of the particle is <code>(particle.position - C3Vector(0, 0, zSource)).Normalize()</code><br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C3Vector}}> colorTrack; // Most likely they all have 3 timestamps for {start, middle, end}.<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|fixed16}}> alphaTrack;<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C2Vector}}> scaleTrack;<br />
{{Template:Type|C2Vector}} scaleVary; // A percentage amount to randomly vary the scale of each particle<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> headCellTrack; // Some kind of intensity values seen: 0,16,17,32 (if set to different it will have high intensity)<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> tailCellTrack;<br />
#else<br />
float midPoint; // middleTime; Middle point in lifespan (0 to 1).<br />
{{Type|CImVector}}[3] colorValues; // start, middle, end<br />
float[3] scaleValues;<br />
uint16[3] lifespanUVAnim;<br />
uint16[3] decayUVAnim;<br />
int16[2] tailUVAnim; // start, end<br />
int16[2] tailDecayUVAnim;<br />
#endif<br />
float tailLength;<br />
float twinkleSpeed; // twinkleFPS; has something to do with the spread<br />
float twinklePercent; // same mechanic as MDL twinkleOnOff but non-binary in 0.11.0<br />
{{Template:Type|CRange}} twinkleScale; // min, max<br />
float burstMultiplier; // ivelScale; requires (flags & 0x40)<br />
float drag; // For a non-zero values, instead of travelling linearly the particles seem to slow down sooner. Speed is multiplied by exp( -drag * t ).<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float baseSpin; // Initial rotation of the particle quad<br />
float baseSpinVary;<br />
float spin; // Rotation of the particle quad per second<br />
float spinVary;<br />
#else<br />
float spin; // 0.0 for none, 1.0 to rotate the particle 360 degrees throughout its lifetime.<br />
#endif<br />
<br />
[[#M2Box|M2Box]] tumble;<br />
{{Template:Type|C3Vector}} windVector;<br />
float windTime;<br />
<br />
float followSpeed1;<br />
float followScale1;<br />
float followSpeed2;<br />
float followScale2;<br />
M2Array<C3Vector> splinePoints; // Set only for spline praticle emitter. Contains array of points for spline<br />
[[#Standard_animation_block|M2Track]]<uchar> enabledIn; // (boolean) Appears to be used sparely now, probably there's a flag that links particles to animation sets where they are enabled.<br />
} particles[];<br />
<br />
Spin can be a float value greater or less one. Results look better if use it as a "phase shift": particle_rotate = randfloat(-sys->rotation * pi, sys->rotation * pi); --Igor<br />
<br />
===M2Particle (Cata+)===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
* Cata+ has multi texture support<br />
using fp_6_9 = {{Template:Type|fixed_point}}<uint16_t, 6, 9>;<br />
struct vector_2fp_6_9 { fp_6_9 x; fp_6_9 y; };<br />
struct M2Particle<br />
{<br />
M2ParticleOld old;<br />
vector_2fp_6_9 multiTextureParam0[2];<br />
vector_2fp_6_9 multiTextureParam1[2];<br />
} particles[];<br />
<br />
In addition to these two parameters, <code>ParticleType</code> and <code>HeadOrTail</code> got reused (as in replaced at their current position) as <code>multiTextureParamX[2]</code> where all arrays are one entry per additional texture.<br />
<br />
I don't know if the previous meaning of the two parameters still exists, got moved, or was just never used to begin with. ParticleType appears to be implicit by having <code>flags & 0x10100000</code> (→ multi texture), a model (→ model) or neither (→ default).--[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 23:47, 29 October 2015 (UTC)<br />
<br />
===Particle Flags===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="600" | Description<br />
|-<br />
| 0x1 || Particles are affected by lighting; <br />
|-<br />
| 0x2 || <br />
|-<br />
| 0x4 || <br />
|-<br />
| 0x8 || Particles travel "up" in world space, rather than model.<br />
|-<br />
| 0x10 || Do not Trail<br />
|-<br />
| 0x20 || Unlightning<br />
|-<br />
| 0x40 || Use Burst Multiplier<br />
|-<br />
| 0x80 || Particles in Model Space<br />
|-<br />
| 0x100 || <br />
|-<br />
| 0x200 || spawn position randomized in some way?<br />
|-<br />
| 0x400 || STYLE: Pinned Particles, their quad enlarges from their creation position to where they expand.<br />
|-<br />
| 0x800 || <br />
|-<br />
| 0x1000 || XYQuad Particles. They align to XY axis facing Z axis direction.<br />
|-<br />
| 0x2000 || clamp to ground; call CParticleEmitter2::ProjectParticle<br />
|-<br />
| 0x4000 || <br />
|-<br />
| 0x8000 || <br />
|-<br />
| 0x10000 || ChooseRandomTexture<br />
|-<br />
| 0x20000 || STYLE: "Outward" particles, most emitters have this and their particles move away from the origin, when they don't the particles start at origin+(speed*life) and move towards the origin.<br />
|-<br />
| 0x40000 || STYLE: unknown. In a large proportion of particles this seems to be simply the opposite of the above flag, but in some (e.g. voidgod.m2 or wingedlionmount.m2) both flags are true.<br />
|-<br />
| 0x80000 || If set, ScaleVary affects x and y independently; if not set, ScaleVary.x affects x and y uniformly, and ScaleVary.y is not used.<br />
|-<br />
| 0x200000 || Random FlipBookStart<br />
|-<br />
| 0x400000 || Ignores Distance (or 0x4000000?!, CMapObjDef::SetDoodadEmittersIgnoresDistance has this one)<br />
|-<br />
| 0x800000 || gravity values are compressed vectors instead of z-axis values (see Compressed Particle Gravity below)<br />
|-<br />
| 0x1000000 || bone generator = bone, not joint<br />
|-<br />
| 0x4000000 || do not throttle emission rate based on distance<br />
|-<br />
| 0x10000000 || Particle uses multi-texturing (could be one of the other WoD-specific flags), see multi-textured section.<br />
|}<br />
<br />
--[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) A comparison of 0.10.0's MDX files to 0.11.0's and 1.12.1's M2 files indicates that the [[MDX#Flags|MDX PRE2 flags]] (≥0x8000) were probably unchanged during the switch from MDL to M2, albeit separated into their own enum.<br />
<br />
===ParticleColorIndex===<br />
This is used in conjunction with [[ParticleColor.dbc]] to alter the default colour of particles. If the particle colour is not meant to be changed then its ParticleColorIndex will have a value of zero. If the particle colour may be changed then the value will be 11, 12 or 13, indicating whether the first, second or third Start, Mid and End colours are to be used, respectively. The row of [[ParticleColor.dbc]] to be used is determined its ID value, which should correspond to the ParticleColorID value supplied by [[CreatureDisplayInfo.dbc]] or [[ItemDisplayInfo.dbc]] for creatures or items.<br />
===Particle types===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || "normal" particle<br />
|-<br />
| 1 || large quad from the particle's origin to its position (used in Moonwell water effects)<br />
|-<br />
| 2 || seems to be the same as 0 (found some in the Deeprun Tram blinky-lights-sign thing)<br />
|}<br />
''ParticleType is always 0 and, maybe, now (Flags & 0x40000) != 0 means "particles from origin to position". --Igor''<br />
''Checked and verified --BlinkHawk''<br />
===Particle Blendings===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || glDisable(GL_BLEND); glDisable(GL_ALPHA_TEST);<br />
|-<br />
| 1 || glBlendFunc(GL_SRC_COLOR, GL_ONE);<br />
|-<br />
| 2 || glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);<br />
|-<br />
| 3 || glDisable(GL_BLEND); glEnable(GL_ALPHA_TEST);<br />
|-<br />
| 4 || glBlendFunc(GL_SRC_ALPHA, GL_ONE);<br />
|}<br />
from Modelviewer source<br />
-- Rour, some WoD particle effects are using blend mode 0x7 here.<br />
===The Fake-AnimationBlock===<br />
*Its pretty much like the real one but without the "header".<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Offset !! width="90" | Type !! width="120" | Name !! width="500" | Description<br />
|-<br />
| 0x000 || uint32 || nTimestamps || The number of timestamps.<br />
|-<br />
| 0x004 || uint32 || ofsTimestamps || And the offset to them. The timestamps are shorts! (?)<br />
|-<br />
| 0x008 || uint32 || nKeys || The same number again. This time its the number of Keys / Values.<br />
|-<br />
| 0x00C || uint32 || ofsKeys || And their offset.<br />
|}<br />
But they're unable to change between different animations, so they directly point to the data.<br />
<br />
===Compressed Particle Gravity===<br />
Key values in the gravity track are decompressed at load time from a 4-byte value to a C3Vector.<br />
<br />
<nowiki><br />
struct CompressedParticleGravity {<br />
int8_t x, y;<br />
int16_t z;<br />
};<br />
for (/* each 4-byte value in the particle gravity track */) {<br />
float *pValue;<br />
C3Vector *pDst;<br />
if (particle.flags & 0x800000) {<br />
// interpret the 4 bytes at pValue as CompressedParticleGravity:<br />
CompressedParticleGravity v = new (pValue) CompressedParticleGravity();<br />
C3Vector dir = C3Vector(v.x, v.y, 0) * (1.0f / 128.0f);<br />
float z = sqrtf(1.0f - dir.Dot(dir));<br />
float mag = v.z * 0.04238648f;<br />
if (mag < 0) {<br />
z = -z;<br />
mag = -mag;<br />
}<br />
dir.z = z;<br />
dir *= mag;<br />
*pDst = dir;<br />
} else {<br />
*pDst = C3Vector(0, 0, -(*pValue));<br />
}<br />
}</nowiki><br />
<br />
<br />
===M2Box===<br />
struct M2Box {<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMin;<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMax;<br />
}<br />
<br />
=Miscellaneous=<br />
==Name==<br />
char name[];<br />
<br />
Informative name used for debugging purposes. Not used in retail clients.<br />
<br />
==Bounding volumes==<br />
These blocks give a simplified bounding volume for the model. Characters and creatures have just a simple box.<br />
===Vertices===<br />
This block defines the possible points used for the model. They are referenced in the triangles block later.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} position;<br />
} bounding_vertices[];<br />
<br />
===Triangles===<br />
The number in the header tells us how many uint16s there are, not how many triangles. To use this better, you should group three of them into an array. The nBoundingTriangles/3 indices will tell you which vertices are used for the triangle then. <br />
<br />
struct <br />
{<br />
uint16_t index; // three entries pointing to vertices per triangle<br />
} bounding_triangles[];<br />
<br />
The number nBoundingTriangles once again contains the number of indices used, so divide by 3 to get the number of triangles.<br />
<br />
===Normals===<br />
This one defines a normal per triangle. The vectors are normalized, but Blizzard seems to have some problems getting a simple vector normalized leading in several 0,0,0.999999999 ones. Whatever.<br />
<br />
As each vertex has a corresponding normal vector, it should be true that nBoundingNormals = nBoundingTriangles / 3.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} normal;<br />
} bounding_normals[];<br />
<br />
== Lights ==<br />
struct M2Light<br />
{<br />
/*0x00*/ uint16_t type; // Types are listed below.<br />
/*0x02*/ int16_t bone; // -1 if not attached to a bone<br />
/*0x04*/ {{Template:Type|C3Vector}} position; // relative to bone, if given<br />
/*0x10*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> ambient_color;<br />
/*0x24*/ [[#Standard_animation_block|M2Track]]<float> ambient_intensity; // defaults to 1.0<br />
/*0x38*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> diffuse_color;<br />
/*0x4C*/ [[#Standard_animation_block|M2Track]]<float> diffuse_intensity; // defaults to 1.0<br />
/*0x60*/ [[#Standard_animation_block|M2Track]]<float> attenuation_start;<br />
/*0x74*/ [[#Standard_animation_block|M2Track]]<float> attenuation_end;<br />
/*0x88*/ [[#Standard_animation_block|M2Track]]<uint8_t> visibility; // enabled?<br />
/*0x9C*/<br />
} lights[];<br />
<br />
Two light types:<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Description<br />
|-<br />
| 0 || Directional<br />
|-<br />
| 1 || Point light<br />
|}<br />
<br />
Directional light type is not used (at least in 3.3.5) outside login screen, and doesn't seem to be taken into account in world.<br />
<br />
==Cameras==<br />
These blocks are present in the "flyby" camera models which completely lack geometry and the main menu backdrop models which are supposed to have a fixed camera. Additionally, characters and monsters also have this block. The reason that non-mainmenu and non-geometry M2s have cameras was is you can see the unit's portrait and the character info tab.<br />
<br />
struct M2Camera<br />
{<br />
uint32_t type; // 0: portrait, 1: characterinfo; -1: else (flyby etc.); referenced backwards in the lookup table.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=4|max_exclusive=1}}<br />
float fov; // Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
float far_clip;<br />
float near_clip;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> positions; // How the camera's position moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> target_position; // How the target moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} target_position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> roll; // The camera can have some roll-effect. Its 0 to 2*Pi. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> FoV; //Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
} cameras[];<br />
<br />
===Camera field of view===<br />
<br />
The <tt>fov</tt> included in <tt>M2Camera</tt> is a diagonal field of view (in radians). The client converts it to a vertical field of view at runtime using the following formula:<br />
<br />
<syntaxhighlight lang="cpp"><br />
float vfov = dfov / sqrt(1.0 + pow(aspect, 2.0));<br />
</syntaxhighlight><br />
<br />
The aspect ratio used is determined by the rect being presented on (eg. the game window).<br />
<br />
Note that this formula makes an assumption that the screen being projected to is planar (ie a monitor).<br />
<br />
===Camera lookup table===<br />
This block lists the different cameras existing in the model. The index in the array is also the type. CameraLookupTable[1] is always the character tab camera.<br />
<br />
"-1" type cameras are not referenced.<br />
<br />
If nCameras >= 1, then nCameraLookup will be >= 1 regardless of whether any camera will be actually referenced in it. See interface/glues/models/ui_mainmenu_warlords/ui_mainmenu_warlords.m2. A valid block thus may be -1s only. This appears to be an exporter-quirk rather than a requirement.<br />
<br />
struct<br />
{<br />
uint16_t camera;<br />
} camera_lookup[];<br />
<br />
== Attachments ==<br />
This block specifies a bunch of locations on the body - hands, shoulders, head, back, knees etc. It is used to put items on a character. This seems very likely as this block also contains positions for sheathed weapons, a shield, etc.<br />
struct M2Attachment<br />
{<br />
uint32_t id; // Referenced in the [[#Attachment_Lookup|lookup-block]] below.<br />
uint16_t bone; // attachment base<br />
uint16_t unknown; // see BogBeast.m2 in vanilla for a model having values here<br />
{{Template:Type|C3Vector}} position; // relative to bone; Often this value is the same as bone's pivot point <br />
[[#Standard_animation_block|M2Track]]<uchar> animate_attached; // whether or not the attached model is animated when this model is. only a bool is used. default is true.<br />
} attachments[];<br />
<br />
Meaning depends on type of model. The following are for creatures/characters mainly:<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="50" | ID !! width="250" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
|-<br />
| 0 || Shield / MountMain / ItemVisual0<br />
| 12 || Back<br />
| 24 || Special2<br />
| 36 || Bullet (version: somewhen after alpha)<br />
| 48 || RightFoot<br />
|-<br />
| 1 || HandRight / ItemVisual1<br />
| 13 || ShoulderFlapRight<br />
| 25 || Special3<br />
| 37 || SpellHandOmni (version: somewhen after alpha)<br />
| 49 || ShieldNoGlove<br />
|-<br />
| 2 || HandLeft / ItemVisual2<br />
| 14 || ShoulderFlapLeft<br />
| 26 || SheathMainHand<br />
| 38 || SpellHandDirected (version: somewhen after alpha)<br />
| 50 || SpineLow<br />
|-<br />
| 3 || ElbowRight / ItemVisual3<br />
| 15 || ChestBloodFront<br />
| 27 || SheathOffHand<br />
| 39 || VehicleSeat1 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 51 || AlteredShoulderR<br />
|-<br />
| 4 || ElbowLeft / ItemVisual4<br />
| 16 || ChestBloodBack<br />
| 28 || SheathShield<br />
| 40 || VehicleSeat2 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 52 || AlteredShoulderL<br />
|-<br />
| 5 || ShoulderRight<br />
| 17 || Breath<br />
| 29 || PlayerNameMounted<br />
| 41 || VehicleSeat3 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 53 || BeltBuckle {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
|-<br />
| 6 || ShoulderLeft<br />
| 18 || PlayerName<br />
| 30 || LargeWeaponLeft<br />
| 42 || VehicleSeat4 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 54 || SheathCrossbow<br />
|-<br />
| 7 || KneeRight<br />
| 19 || Base<br />
| 31 || LargeWeaponRight<br />
| 43 || VehicleSeat5 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 55 || HeadTop {{Template:Sandbox/VersionRange|min_expansionlevel=7}}<br />
|-<br />
| 8 || KneeLeft<br />
| 20 || Head<br />
| 32 || HipWeaponLeft<br />
| 44 || VehicleSeat6 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 9 || HipRight<br />
| 21 || SpellLeftHand<br />
| 33 || HipWeaponRight<br />
| 45 || VehicleSeat7 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 10 || HipLeft<br />
| 22 || SpellRightHand<br />
| 34 || Chest<br />
| 46 || VehicleSeat8 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 11 || Helm<br />
| 23 || Special1<br />
| 35 || HandArrow<br />
| 47 || LeftFoot<br />
| || <br />
|}<br />
For weapons, usually 5 of these points are present, which correspond to the 5 columns in [[ItemVisuals.dbc]], which in turn has 5 models from [[ItemVisualEffects.dbc]]. This is for the weapon glowy effects and such. The effect ID is the last column in [[ItemDisplayInfo.dbc]]. They take the ids 0 to 4. Mounts take the id 0 for their rider. Breath (17) is used by CGCamera::FinishLoadingTarget() aswell as some other one. The name above the head of a Unit (CGUnit_C::GetNamePosition) looks for PlayerNameMounted (29), then PlayerName (18).<br />
===Attachment Lookup===<br />
The index of the array defines, which type that attachment is of. Its the same as the list above. The lookups and the id of the animations point in a circle.<br />
struct <br />
{<br />
uint16_t attachment;<br />
} attachment_lookup[];<br />
<br />
==Events==<br />
These events are used for timing sounds for example. You can find the $DTH (death) event on nearly every model. It will play the death sound for the unit.<br />
<br />
The events you can use depend on the way, the model is used. Dynamic objects can shake the camera, doodads shouldn't. Units can do a lot more than other objects.<br />
<br />
Somehow there are some entries, that don't use the $... names but identifiers like "DEST" (destination), "POIN" (point) or "WHEE" (wheel). How they are used? Idk.<br />
<br />
struct M2Event<br />
{<br />
uint32_t identifier; // mostly a 3 character name prefixed with '$'.<br />
uint32_t data; // This data is passed when the event is fired. <br />
uint32_t bone; // Somewhere it has to be attached.<br />
{{Template:Type|C3Vector}} position; // Relative to that bone of course, animated. Pivot without animating.<br />
[[#Standard_animation_block|M2TrackBase]] enabled; // This is a timestamp-only animation block. It is built up the same as a normal [[#Standard_animation_block|AnimationBlocks]], but is missing values, as every timestamp is an implicit "fire now".<br />
} events[];<br />
===Possible Events===<br />
There are a lot more of them. I did not list all up to now.<br />
{| style="background:#FCFCFC || color:black"<br />
! width="50" | ID !! width="40" | data !! width="60" | what !! width="60" | Type !! width="200" | seen to be fired on !! width="600" | Description<br />
|-<br />
| $AH[0-3] || — || PlaySoundKit (customAttack[x]) || || || soundEffect ID is defined by CreatureSoundDataRec::m_customAttack[x]<br />
|-<br />
| $BMD || — || BowMissleDestination || RangedWeapon || ||<br />
|-<br />
| $AIM || || || Vehicles || CGUnit_C::ComputeMissileTrajectory || Position used as MissileFirePos.<br />
|-<br />
| $ALT || || anim_swap_event / DisplayTransition || Unit || || CUnitDisplayTransition_C::UpdateState(1) or CUnitDisplayTransition_C::HandleAnimSwapEvent<br />
|-<br />
| $BL[0-3] || — || FootstepAnimEventHit (left) || Unit || || Backwards<br />
|-<br />
| $BR[0-3] || — || FootstepAnimEventHit (right) || Unit || || Backwards<br />
|-<br />
| $BRT || — || PlaySoundKit (birth) || || || soundEffect ID is defined by CreatureSoundDatarec::m_birthSoundID<br />
|-<br />
| $BTH || — || Breath || Unit || All situations, where nothing happens or breathing. || Adds Special Unit Effect based on unit state (under water, in-snow, …)<br />
|-<br />
| $BWP || — || PlayRangedItemPull (Bow Pull) || Unit || LoadRifle, LoadBow || <br />
|-<br />
| $BWR || — || BowRelease || Unit || AttackRifle, AttackBow, AttackThrown || <br />
|-<br />
| $CAH || — || || Unit || Attack*, *Unarmed, ShieldBash, Special* || attack hold? CGUnit_C::HandleCombatAnimEvent<br />
|-<br />
| $CCH || — || || Unit || mostly not fired, AttackThrown || CEffect::DrawFishingString needs this on the model for getting the string attachments.<br />
|-<br />
| $CFM || — || || Unit || CGCamera::UpdateMountHeightOrOffset || CGCamera::UpdateMountHeightOrOffset: Only z is used. Non-animated. Not used if $CMA<br />
|-<br />
| $CHD || || || Unit || not fired || probably does not exist?!<br />
|-<br />
| $CMA || — || || Unit || || CGCamera::UpdateMountHeightOrOffset: Position for camera<br />
|-<br />
| $CPP || || PlayCombatActionAnimKit || || || parry, anims, depending on some state, also some callback which might do more<br />
|-<br />
| $CSD || soundEntryId || PlayEmoteSound || Unit || Emote* || <br />
|-<br />
| $CSL || — || release_missiles_on_next_update if has_pending_missiles (left) || Unit || AttackRifle, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSR || — || release_missiles_on_next_update if has_pending_missiles (right) || Unit || AttackBow, AttackRifle, AttackThrown, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSS || — || PlayWeaponSwooshSound || || sound played depends on CGUnit_C::GetWeaponSwingType<br />
|-<br />
| $CST || — || release_missiles_on_next_update if has_pending_missiles || Unit || Attack*, *Unarmed, ShieldBash, Special*, SpellCast, Parry*, EmoteEat, EmoteRoar, Kick, ... || $CSL/R/T are also used in CGUnit_C::ComputeDefaultMissileFirePos.<br />
|-<br />
| $CVS || || ||  ? || || Data: SoundEntriesAdvanced.dbc, Sound — Not present in 6.0.1.18179<br />
|-<br />
| $DSE || — || DestroyEmitter || MapObj || || <br />
|-<br />
| $DSL || soundEntryId || DoodadSoundLoop (low priority) | GO || || <br />
|-<br />
| $DSO || soundEntryId || DoodadSoundOneShot || GO || || <br />
|-<br />
| $DTH || — || DeathThud + LootEffect || Unit || Death, Drown, Knockdown || """I'm dead now!"", UnitCombat_C, this plays death sounds and more." Note that this is NOT triggering [[CreatureSoundData.dbc|CreatureSoundDataRec]]::m_soundDeathID, but that is just always triggered as soon as the death animation plays.<br />
|-<br />
| $EAC || — || object package state enter 3, exit 2, 4, 5 || || || <br />
|-<br />
| $EDC || — || object package state enter 5, exit 3, 4, 2 || || || <br />
|-<br />
| $EMV || — || object package state enter 4, exit 3, 2, 5 || || || <br />
|-<br />
| $ESD || — || PlayEmoteStateSound || Unit || || soundEffect ID is implicit by currently played emote<br />
|-<br />
| $EWT || — || object package state enter 2, exit 3, 4, 5 || || || <br />
|-<br />
| $FD[1-9] || — || PlayFidgetSound || || || CreatureSoundDataRec::m_soundFidget (only has 5 entries, so don’t use 6-9)<br />
|-<br />
| $FDX || — || PlayUnitSound (stand) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundStandID<br />
|-<br />
| $FL[0-3] || — || FootstepAnimEventHit (left) || || || Forward<br />
|-<br />
| $FR[0-3] || — || FootstepAnimEventHit (right) || || || Forward<br />
|-<br />
| $FSD || — || HandleFootfallAnimEvent || Unit || Walk, Run (multiple times), ... || Plays some sound. Footstep? Also seen at several emotes etc. where feet are moved. CGUnit_C::HandleFootfallAnimEvent<br />
|-<br />
| $GC[0-3] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x + 6] ({Custom0, Custom1, Custom2, Custom3})<br />
|-<br />
| $GO[0-5] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x] ({Stand, Open, Loop, Close, Destroy, Opened})<br />
|-<br />
| $HIT || — || PlayWoundAnimKit || Unit || Attack*, *Unarmed, ShieldBash, Special* || soundEntryId depends on SpellVisualKit<br />
|-<br />
| $KVS || || || Â ? || || MapLoad.cpp -- not found in 6.0.1.18179<br />
|-<br />
| $RL[0-3] || — || FootstepAnimEventHit (left) || || || Running<br />
|-<br />
| $RR[0-3] || — || FootstepAnimEventHit (right) || || || Running<br />
|-<br />
| $SCD || — || PlaySoundKit (spellCastDirectedSound) || || || soundEffect ID is defined by CreatureSoundDataRec::m_spellCastDirectedSoundID<br />
|-<br />
| $SHK || spellEffectCameraShakesID || AddShake || GO || || <br />
|-<br />
| $SHL || — || ExchangeSheathedWeapon (left) || || Sheath, HipSheath || <br />
|-<br />
| $SHR || — || ExchangeSheathedWeapon (right) || || Sheath, HipSheath || <br />
|-<br />
| $SL[0-3] || — || FootstepAnimEventHit (left) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $SMD || — || PlaySoundKit (submerged) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergedSoundID<br />
|-<br />
| $SMG || — || PlaySoundKit (submerge) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergeSoundID<br />
|-<br />
| $SND || soundEntryId || PlaySoundKit (custom) || GO || || <br />
|-<br />
| $SR[0-3] || — || FootstepAnimEventHit (right) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $STx || || || Mounts || MountTransitionObject::UpdateCharacterData || Not seen in 6.0.1.18179 -- x is {E and B} , sequence time is taken of both, pivot of $STB. (Also, attachment info for attachment 0)<br />
|-<br />
| $TRD || — || HandleSpellEventSound || Unit || EmoteWork*, UseStanding* || soundEffect ID is implicit by SpellRec<br />
|-<br />
| $VG[0-8] || — || HandleBoneAnimGrabEvent || || || <br />
|-<br />
| $VT[0-8] || — || HandleBoneAnimThrowEvent || || || <br />
|-<br />
| $WGG || — || PlayUnitSound (wingGlide) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingGlideID<br />
|-<br />
| $WL[0-3] || — || FootstepAnimEventHit (left) || || || <br />
|-<br />
| $WNG || — || PlayUnitSound (wingFlap) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingFlapID<br />
|-<br />
| $WR[0-3] || — || FootstepAnimEventHit (right) || || || <br />
|-<br />
| $WTB || — || || Weapons || || Weapon Trail Bottom position, also used for Bow String<br />
|-<br />
| $WTT || — || || Weapons || || Weapon Trail Top position<br />
|-<br />
| $WWG || || || Â ? || || Calls some function in the Object VMT. -- Not seen in 6.0.1.18179<br />
|-<br />
| DEST || || || Â ? || || exploding ballista, that one has a really fucked up block. Oo<br />
|-<br />
| POIN || || || Unit || not fired || Data:Â ?, seen on multiple models. Basilisk for example. (6801)<br />
|-<br />
| WHEE || || || Â ? || || Data: 601+, Used on wheels at vehicles.<br />
|-<br />
| BOTT || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|-<br />
| TOP || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|}<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=M2&diff=26854M22019-06-01T08:36:53Z<p>Simca: /* Particle Flags */ From furl's notes.</p>
<hr />
<div>[[M2]] files (also called [[MDX]]) contain model objects. Each [[M2]] file describes the vertices, faces, materials, texture names, animations and properties of one model. [[M2]] files don't have a chunked format like most other WoW formats (except in Legion). Since it is chunked in Legion, all offsets are relative to beginning of the MD21 chunk's data rather than the beginning of the file. <br />
<br />
Models are used for doodads (decoration objects), players, monsters and really everything in the game except for [[ADT|Terrain]] and [[WMO]]s.<br />
<br />
==Header==<br />
The header has mostly the layout of number-offset pairs, containing the number of a particular record in the file, and the offset. These appear at fixed places in the header. Record sizes are not specified in the file.<br />
<br />
struct <br />
{<br />
// note: Offsets are for {{Template:Sandbox/VersionRange|min_expansionlevel=3}}!<br />
/*0x000*/ uint32_t magic; // "MD20". [[M2#Legion|Legion]] uses a chunked file format starting with MD21.<br />
/*0x004*/ uint32_t [[#Versions|version]];<br />
/*0x008*/ {{Template:Type/M2Array|char}} name; // should be globally unique, used to reload by name in internal clients<br />
<br />
/*0x010*/ struct<br />
{<br />
uint32_t flag_tilt_x : 1;<br />
uint32_t flag_tilt_y : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
uint32_t flag_use_texture_combiner_combos : 1; // add textureCombinerCombos array to end of data<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
uint32_t flag_load_phys_data : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=6}}<br />
uint32_t flag_unk_0x80 : 1; // with this flag unset, demon hunter tattoos stop glowing<br />
// since {{Template:Unverified|{{Template:Sandbox/PrettyVersion|expansionlevel=4|build=4.0.1.12911}}}} {{Template:Unverified|every}} model now has this flag<br />
uint32_t flag_camera_related : 1; // TODO: verify version<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=7}} // TODO: verify version, these are just added based on where I first saw them -- schlumpf.<br />
uint32_t flag_new_particle_record : 1; // In CATA: new version of ParticleEmitters. By default, length of M2ParticleOld is 476. <br />
// But if 0x200 is set or if version is bigger than 271, length of M2ParticleOld is 492.<br />
uint32_t flag_unk_0x400 : 1;<br />
uint32_t flag_texture_transforms_use_bone_sequences : 1; // {{Template:Sandbox/VersionRange|min_expansionlevel=6}} 0x800 -- When set, texture transforms are animated using the sequence being played on the bone found by index in tex_unit_lookup_table[textureTransformIndex], instead of using the sequence being played on the model's first bone. Example model: 6DU_HellfireRaid_FelSiege03_Creature<br />
uint32_t flag_unk_0x1000 : 1;<br />
uint32_t flag_unk_0x2000 : 1; // seen in various legion models<br />
uint32_t flag_unk_0x4000 : 1;<br />
uint32_t flag_unk_0x8000 : 1; // seen in UI_MainMenu_Legion<br />
uint32_t flag_unk_0x10000 : 1;<br />
uint32_t flag_unk_0x20000 : 1;<br />
uint32_t flag_unk_0x40000 : 1;<br />
uint32_t flag_unk_0x80000 : 1;<br />
uint32_t flag_unk_0x100000 : 1;<br />
uint32_t flag_unk_0x200000 : 1; // apparently: use 24500 upgraded model format: chunked .anim files, change in the exporter reordering sequence+bone blocks before name<br />
#endif<br />
#endif<br />
#endif<br />
#endif<br />
} global_flags;<br />
<br />
/*0x014*/ {{Template:Type/M2Array|M2Loop|section=Global_sequences}} global_loops; // Timestamps used in [[#Global_Sequences_2|global looping animations]].<br />
/*0x01C*/ {{Template:Type/M2Array|M2Sequence|section=Animation_sequences}} sequences; // Information about the animations in the model.<br />
/*0x024*/ {{Template:Type/M2Array|uint16_t|section=Animation_Lookup}} sequenceIdxHashById; // Mapping of [[#Animation_sequences|sequence IDs]] to the entries in the [[#Animation_sequences|Animation sequences]] block.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?|section=Playable_Animation_Lookup}} playable_animation_lookup;<br />
#endif<br />
/*0x02C*/ {{Template:Type/M2Array|M2CompBone|section=Bones}} bones; // MAX_BONES = 0x100 => Creature\SlimeGiant\GiantSlime.M2 has 312 bones ({{Template:Sandbox/PrettyVersion|expansionlevel=3}})<br />
/*0x034*/ {{Template:Type/M2Array|uint16_t|section=Key-Bone_Lookup}} boneIndicesById; //Lookup table for key skeletal bones. (alt. name: key_bone_lookup)<br />
/*0x03C*/ {{Template:Type/M2Array|M2Vertex|section=Vertices}} vertices;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|M2SkinProfile|section=Views_.28LOD.29}} skin_profiles;<br />
#else<br />
/*0x044*/ uint32_t num_skin_profiles; // [[#Views_.28LOD.29|Views (LOD)]] are now in [[M2/.skin|.skins]].<br />
#endif<br />
<br />
/*0x048*/ {{Template:Type/M2Array|M2Color|section=Colors}} colors; // Color and alpha animations definitions.<br />
/*0x050*/ {{Template:Type/M2Array|M2Texture|section=Textures}} textures;<br />
/*0x058*/ {{Template:Type/M2Array|M2TextureWeight|section=Transparency}} texture_weights; // Transparency of textures.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?}} unknown;<br />
#endif<br />
/*0x060*/ {{Template:Type/M2Array|M2TextureTransform|section=Texture_Transforms}} texture_transforms;<br />
/*0x068*/ {{Template:Type/M2Array|uint16_t|section=Replacable_texture_lookup}} textureIndicesById; // (alt. name: replacable_texture_lookup)<br />
/*0x070*/ {{Template:Type/M2Array|M2Material|section=Render_flags}} materials; // Blending modes / render flags.<br />
/*0x078*/ {{Template:Type/M2Array|uint16_t|section=Bone_Lookup_Table}} boneCombos; // (alt. name: bone_lookup_table)<br />
/*0x080*/ {{Template:Type/M2Array|uint16_t|section=Texture_lookup_table}} textureCombos; // (alt. name: texture_lookup_table)<br />
/*0x088*/ {{Template:Type/M2Array|uint16_t|section=Texture_unit_lookup_table}} textureTransformBoneMap; // (alt. name: tex_unit_lookup_table)<br />
/*0x090*/ {{Template:Type/M2Array|uint16_t|section=Transparency_lookup_table}} textureWeightCombos; // (alt. name: transparency_lookup_table)<br />
/*0x098*/ {{Template:Type/M2Array|uint16_t|section=Texture_Transforms_lookup_table}} textureTransformCombos; // (alt. name: texture_transforms_lookup_table)<br />
<br />
/*0x0A0*/ {{Template:Type|CAaBox}} bounding_box; // min/max( [1].z, 2.0277779f ) - 0.16f seems to be the maximum camera height<br />
/*0x0B8*/ float bounding_sphere_radius; // detail doodad draw dist = clamp (bounding_sphere_radius * detailDoodadDensityFade * detailDoodadDist, …)<br />
/*0x0BC*/ {{Template:Type|CAaBox}} collision_box;<br />
/*0x0D4*/ float collision_sphere_radius;<br />
<br />
/*0x0D8*/ {{Template:Type/M2Array|uint16_t|section=Triangles}} collisionIndices; // (alt. name: collision_triangles)<br />
/*0x0E0*/ {{Template:Type/M2Array|C3Vector|section=Vertices_2}} collisionPositions; // (alt. name: collision_vertices)<br />
/*0x0E8*/ {{Template:Type/M2Array|C3Vector|section=Normals}} collisionFaceNormals; // (alt. name: collision_normals) <br />
/*0x0F0*/ {{Template:Type/M2Array|M2Attachment|section=Attachments}} attachments; // position of equipped weapons or effects<br />
/*0x0F8*/ {{Template:Type/M2Array|uint16_t|section=Attachment_Lookup}} attachmentIndicesById; // (alt. name: attachment_lookup_table)<br />
/*0x100*/ {{Template:Type/M2Array|M2Event|section=Events}} events; // Used for playing sounds when dying and a lot else.<br />
/*0x108*/ {{Template:Type/M2Array|M2Light|section=Lights}} lights; // Lights are mainly used in loginscreens but in wands and some doodads too.<br />
/*0x110*/ {{Template:Type/M2Array|M2Camera|section=Cameras}} cameras; // The cameras are present in most models for having a model in the character tab. <br />
/*0x118*/ {{Template:Type/M2Array|uint16_t|section=Camera_lookup_table}} cameraIndicesById; // (alt. name: camera_lookup_table)<br />
/*0x120*/ {{Template:Type/M2Array|M2Ribbon|section=Ribbon_emitters}} ribbon_emitters; // Things swirling around. See the CoT-entrance for light-trails.<br />
/*0x128*/ {{Template:Type/M2Array|M2Particle|section=Particle_emitters}} particle_emitters;<br />
<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
if (flag_use_texture_combiner_combos)<br />
{<br />
/*0x130*/ {{Template:Type/M2Array|uint16_t|section=Blend_mode_overrides}} textureCombinerCombos; // When set, textures blending is overriden by the associated array.<br />
}<br />
#endif<br />
} header;<br />
<br />
==Types==<br />
struct M2Bounds {<br />
CAaBox extent;<br />
float radius;<br />
};<br />
template<typename T><br />
struct M2Array {<br />
uint32_t size;<br />
uint32_t offset; // pointer to T, relative to begin of m2 data block (i.e. MD21 chunk content or begin of file)<br />
};<br />
struct M2TrackBase {<br />
uint16_t trackType;<br />
uint16_t loopIndex;<br />
M2Array<M2SequenceTimes> sequenceTimes;<br />
};<br />
template<typename T> <br />
struct M2PartTrack {<br />
M2Array<fixed16> times;<br />
M2Array<T> values;<br />
};<br />
template<typename T> <br />
struct M2SplineKey {<br />
T value;<br />
T inTan;<br />
T outTan;<br />
};<br />
struct M2Range {<br />
uint32_t minimum;<br />
uint32_t maximum;<br />
};<br />
<br />
==Versions==<br />
Files get handled differently depending on this! Ranges are inclusive.<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Version !! width="190" | Extension<br />
|-<br />
| 272-274 || Legion<br />
|-<br />
| 272 || Warlords of Draenor<br />
|-<br />
| 272 || Mists of Pandaria<br />
|-<br />
| 265-272 || Cataclysm<br />
|-<br />
| 264 || Wrath of the Lich King<br />
|-<br />
| 260-263 || The Burning Crusade<br />
|-<br />
| 256-257 || Classic<br />
|-<br />
| 256 || Pre-Release<br />
|}<br />
<br />
==Chunks==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
<br />
From Legion and up, the file might be [[Chunk|chunked]] instead. If this is the case, the magic will be anything but 'MD20' and the m2 data will be in the 'MD21' chunk. If the first magic is 'MD20', it will be loaded just fine like it did previously. Note that the chunks can be in any order with MD21 often being first. <br />
<br />
'''NOTE''': Unlike all other chunked formats in WoW, chunk names in M2 are '''NOT''' reversed. Example: AFID == AFID in file.<br />
<br />
===MD21===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
The MD21 chunk contains the not yet chunked data, just like in the old format. The content literally is an pre legion file, starting with the MD20 magic. This also implies that all offsets inside this chunk are relative to the ''chunk'', not the ''file''.<br />
M2Data pre_legion_style_data;<br />
<br />
===PFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}.phys</tt>}}<br />
uint32_t phys_file_id;<br />
<br />
===SFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${view}.skin</tt> and <tt>${basename}_lod${lodband}.skin</tt>}}<br />
uint32_t skinFileDataIDs[header.nViews];<br />
uint32_t lod_skinFileDataIDs[lodBands /* 2? */];<br />
Some model files, for example 'Creature\NightborneFemaleCitizen\NightborneFemaleCitizen.m2' have 4 skin files and 2 lod files but only 20 bytes are in chunk. In chunk there are 4 skins and 1 lod present.<br />
<br />
Lod skins are selected based on distance to entity/doodad and chosen based on GetCVar("entityLodDist")/X and GetCVar("doodadLodDist")/X where X - distance. Lods are ignored when "M2UseLOD" CVar is set to 0.<br />
<br />
===AFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${anim_id}-${sub_anim_id}.anim</tt>}}<br />
struct<br />
{<br />
uint16_t anim_id;<br />
uint16_t sub_anim_id;<br />
uint32_t file_id; // might be 0 for "none" (so this is probably not sparse, even if it could be)<br />
} anim_file_ids[];<br />
<br />
===BFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}_${i}.bone</tt>}}<br />
uint32_t boneFileDataIDs[];<br />
<br />
===TXAC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=It is unknown what this replaced. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
char unk[2]; // likely used in CM2SceneRender::SetupTextureTransforms and uploaded to the shader directly. 0 otherwise.<br />
} texture_ac[m2data.header.materials.count + m2data.header.particles.count];<br />
<br />
===EXPT===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
} extended_particle[m2data.header.particles.count];<br />
<br />
Probably outdated chunk after introduction of EXP2 chunk. If EXP2 doesnt exist, client tries to reconstruct it with data from EXPT chunk.<br />
<br />
===EXP2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown}}}}<br />
<br />
<source><br />
struct EXP2<br />
{<br />
M2Array<Exp2Record> content;<br />
};<br />
<br />
struct Exp2Record<br />
{<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
M2PartTrack<fixed16> unk3;<br />
};<br />
</source><br />
<br />
<br />
The length of this M2Array is the same as length of particle_emitters<br />
<br />
===PABC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to replace {{Unverified|[[#Animation_Lookup]]}} from old file. {{Unverified|Exact build unknown}}}}<br />
M2Array<uint16t> m_replacementParentSequenceLookups; // only seen in quillboarbrute*.m2. Contains AnimationIds<br />
<br />
Replaces parentSequencesLookups. But unlike header.sequence_lookups of parent model, this is straight array and not a map. If index with target animation is not found in here, parentSequencesLookups are used instead.<br />
<br />
This chunk called BlacklistAnimData in client.<br />
<br />
Client doesnt seem to use found index and thus whole array is used only to check if the target animation is present.<br />
<br />
===PADC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Moves texture weights from old file to a chunk. {{Unverified|Exact build unknown}}}}<br />
Defines replacement for header.texture_weights (WHY?)<br />
<br />
<source lang="cpp"><br />
struct PADC {<br />
M2Array<M2TextureWeight> texture_weights;<br />
} <br />
</source><br />
<br />
===PSBC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
<br />
Defines ParentSequenceBounds<br />
M2Array<M2Bounds> parentSequenceBounds;<br />
<br />
===PEDC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
M2Array<M2TrackBase> parentEventData;<br />
<br />
===SKID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=This used to be filename based, using <tt>${basename}.skel</tt>. {{Unverified|Exact build unknown}}}}<br />
uint32_t SKeletonfileID; // links to [[M2/.skel]]<br />
<br />
===TXID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629|note=Replaces the filename for {{#Textures}} with hardcoded type.}}<br />
Replaces in-file texture filenames. <br />
struct {<br />
uint32_t fileDataID;<br />
} textureID[]<br />
<br />
===LDV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629}}<br />
<br />
Defines LodData<br />
<br />
<source lang="cpp"><br />
struct LodData<br />
{<br />
uint16 unk0; <br />
uint16 lodCount; //maxLod = lodCount-1; <br />
float unk2_f;<br />
uint8_t particleBoneLod[4]; //lod serves as indes into this array<br />
_DWORD unk4;<br />
};<br />
</source><br />
<br />
Somehow defines _lod%0d.skin files. On pandarenfemale.m2, lodCount == 4. SFID has 7 files first 4 are ordinary .skin files and last 3 are _lod%0d.skin files. Enumeration for _lod%0d.skin files for that model starts from 1, and last file in SFID is pandarenfemale_lod03.skin So technically maxLod indeed represents maximum Lod<br />
<br />
unk2_f is used in formula, but it's purpose is unknown<br />
fmaxf(fminf(740.0 / unk2_f, 5.0), 0.5);<br />
<br />
LodData.particleBoneLod works this way: Each model has current lod which is [0..3]. Next:<br />
<source lang="cpp"><br />
if ( lod < 1 )<br />
result = 0;<br />
<br />
if ( LodData)<br />
result = (0x10000 << LodData->particleBoneLod[lod]);<br />
else<br />
result = (0x10000 << (lod- 1));<br />
<br />
...<br />
//For each ParticleEmitter and related M2Particle record<br />
if ( result & M2CompBone[M2Particle->old.boneIndex].flags ) {<br />
//Do not animate this emitter<br />
}<br />
</source><br />
<br />
<br />
[[User:Deamon|Deamon]] ([[User talk:Deamon|talk]])<br />
<br />
===RPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>recursion_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} recursive_particle_models[particle count];<br />
<br />
===GPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>geometry_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} geometry_particle_models[particle count];<br />
<br />
===WFV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV1 {<br />
// unknown<br />
};<br />
<br />
===WFV2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV2 {<br />
// unknown<br />
};<br />
<br />
===PGD1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=1|min_build=1.13.2.30172}}<br />
struct PGD1Entry {<br />
char _0x00[2];<br />
};<br />
M2Array<PGD1Entry> p_g_d_v1; // count appears to be equivalent to particle count<br />
<br />
=Skeleton and animation=<br />
==Global sequences==<br />
A list of timestamps that act as upper limits for global sequence ranges.<br />
struct M2Loop<br />
{<br />
uint32_t timestamp;<br />
} loops[];<br />
<br />
==Standard animation block==<br />
* {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} uses a single-timeline approach, chaining all animations into one long piece and separating them via begin and end given in animation data. {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, each animation has an own timeline.<br />
* Animation blocks contain a list of lists of timestamps and a list of lists of values, where the first list is by animation and the second one by timestamp-entry.<br />
*'''Many values that change with time are specified using blocks like the following.'''<br />
<br />
template<typename T><br />
struct M2Array<br />
{<br />
/*0x00*/ uint32_t number;<br />
/*0x04*/ uint32_t offset_elements;<br />
/*0x08*/<br />
};<br />
<br />
struct M2TrackBase<br />
{<br />
/*0x00*/ uint16_t interpolation_type;<br />
/*0x02*/ uint16_t global_sequence;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}}<br />
M2Array<pair<uint32_t>> interpolation_ranges; // no longer required {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, as implicit by minimum and maximum timestamp per sequence.<br />
M2Array<uint32_t> timestamps;<br />
#else<br />
/*0x04*/ M2Array<M2Array<uint32_t>> timestamps;<br />
#endif<br />
/*0x0C*/<br />
};<br />
<br />
template<typename T><br />
struct M2Track : M2TrackBase<br />
{<br />
/*0x00*/ // base <br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} <br />
M2Array<T> values;<br />
#else<br />
/*0x0C*/ M2Array<M2Array<T>> values;<br />
#endif<br />
/*0x14*/<br />
};<br />
<br />
* Thus, as example, with<br />
<br />
{{Template:Type|M2CompBone|link=M2#Bones}} b;<br />
<br />
one may get the number of animations having translation information with<br />
<br />
b.translation.timestamps.number<br />
<br />
and the number of timestamps in the first animation using<br />
<br />
b.translation.timestamps.elements[0].number<br />
<br />
and the first timestamp value of the first animation via<br />
<br />
b.translation.timestamps.elements[0].elements[0]<br />
<br />
The actual translation vector for animation 0 at timestamp 0 is at<br />
<br />
b.translation.values.elements[0].elements[0]<br />
<br />
* Some timestamps/values.elements entries may have number/elements = 0, if for that animation id no animation is given.<br />
* [[#.anim_files|.anim]] files are just a blob of data which may as well be in the main model file, that is pointed to by the first array_ref layer.<br />
* [model file name][animation id]-[animation sub-id][[#.anim_files|.anim]]<br />
* it seems like it is possible to detect if animation data is stored in-m2 or externally via<br />
** All animations which have flags & 0x20 are stored internally. <br />
** Animations which do not have flags & 0x20 are not stored internally. <br />
** Animations which do not have flags & 0x20 AND do not have flags & 0x40 are in [[#.anim_files|.anim]] files<br />
** Animations which do not have flags & 0x20 AND DO have flags 0x40 are stored... somewhere. I have no clue.<br />
<br />
===Global Sequences===<br />
If a block has a sequence >= 0, the block has a completely separate max timestamp. This is the value in the model's ofsGlobalSequences table; index into that table with this sequence value and use that as the block's max timestamp. Blocks that use these global sequences also only have one track, so at the same time as clipping the current timestamp to the max time above, interpolated value should always be taken from track 0 in the block. <br />
<br />
A global sequence is completely unrelated to animations. It just always loops. This way, the sequence is not interrupted when an animation is launched.<br />
<br />
This appears to be frequently used by models that don't have more conventional animations (login screen animations, items/weapons with animated effects, etc).<br />
<br />
-- Rour, additionally, these sequences can be longer or shorter than whatever animation is running for a given model, so I recommend taking a global scene timestamp and then clipping that value into the given max timestamp range. Otherwise animations will appear to reset when the regular animation loops, which is not good.<br />
<br />
===Interpolation===<br />
* If the interpolation type is 0, then values will change instantly at the timestamp, with no interpolation whatsoever.<br />
* If the interpolation type is 1, then the block linearly interpolates between keyframe values (lerp for vectors/colours, nlerp for quaternions).<br />
* If the interpolation type is 2, then cubic bezier spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the first control point is the first spline key's <code>value</code>, the second control point is the first spline key's <code>tanOut</code>, the third control point is the second spline key's <code>tanIn</code>, and the fourth control point is the second spline key's <code>value</code>.<br />
* If the interpolation type is 3, then cubic hermite spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the starting point is the first spline key's <code>value</code>, the starting tangent is the first spline key's <code>tanOut</code>, the ending tangent is the second spline key's <code>tanIn</code>, and the ending point is the second spline key's <code>value</code>.<br />
<br />
'''NOTE:''' There is confusion about type <code>2</code> and <code>3</code> being <code>hermite/bezier</code> or <code>bezier/hermite</code>. Alpha says that <code>2 = hermite</code>, WoD says that <code>2 = bezier</code>. This was changed when the format went from MDL to M2. --[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 01:53, 4 September 2017 (CEST), --[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) 19:15, 14 November 2018 (GMT)<br />
<br />
In WotLK 2 appears to be bezier, which pretty much confirms it being bezier for wotlk+. TBC and classic need further checking though. -- [[User:Skarn|Skarn]] ([[User talk:Skarn|talk]]) 01:38, 16 June 2018 (CEST)<br />
<br />
==.anim files==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=3}}<br />
<br />
Low priority sequences (e.g. emotes, one shot animations) are in extra files to allow for lazy loading. These files are raw data of timestamps and values for animation blocks. Blizzard's exporter prefers to align blocks to a 16 byte boundary, but that's not required.<br />
<br />
'''The client loads .anim files if (M2Sequence.flags & 0x130 ) == 0.''' The .anim file to use is <code>"%s%04d-%02d.anim" % (model_filename_without_extension, anim.id, anim.sub_anim_id)</code>.<br />
<br />
===Legion 24500===<br />
In Legion, these files are optionally chunked now. They are chunked either<br />
* if M2 header's 0x200000 flag is set and thus the new mid expansion format change is used<br />
* if the M2 has a .skel file<br />
<br />
For new format M2s, .anim is pretty much unchanged except that there is the AFM2 chunk header. The AFSA and AFSB chunks do not appear in that case. If it is a .skel file based model, the chunks are present and animation data is split into bone and attach data. The AFM2 chunk then contains the animation data for ????, the AFSA chunk that for attachments and the AFSB chunk that for bones. See .skel files for that.<br />
<br />
====AFM2====<br />
The same content as an old anim file would have. In fact, files that were just converted to the new format are bit identical except for the chunk header.<br />
====AFSA====<br />
skeleton data for attachments<br />
====AFSB====<br />
skeleton data for bones<br />
<br />
==Animation sequences==<br />
List of animations present in the model. <br />
struct M2Sequence<br />
{<br />
uint16_t id; // Animation id in [[AnimationData.dbc]]<br />
uint16_t variationIndex; // Sub-animation id: Which number in a row of animations this one is.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
uint32_t start_timestamp;<br />
uint32_t end_timestamp;<br />
#else<br />
uint32_t duration; // The length of this animation sequence in milliseconds.<br />
#endif<br />
float movespeed; // This is the speed the character moves with in this animation.<br />
uint32_t flags; // See below.<br />
int16_t frequency; // This is used to determine how often the animation is played. For all animations of the same type, this adds up to 0x7FFF (32767).<br />
uint16_t _padding;<br />
M2Range replay; // May both be 0 to not repeat. Client will pick a random number of repetitions within bounds if given.<br />
#if version < ??? < 6.0.1<br />
uint32_t blendTime;<br />
#else<br />
uint16_t blendTimeIn; // The client blends (lerp) animation states between animations where the end and start values differ. This specifies how long that blending takes. Values: 0, 50, 100, 150, 200, 250, 300, 350, 500.<br />
uint16_t blendTimeOut; // The client blends between this sequence and the next sequence for blendTimeOut milliseconds.<br />
#endif<br />
// For both blendTimeIn and blendTimeOut, the client plays both sequences simultaneously while interpolating between their animation transforms.<br />
{{Template:Type|M2Bounds}} bounds;<br />
int16_t variationNext; // id of the following animation of this AnimationID, points to an Index or is -1 if none.<br />
uint16_t aliasNext; // id in the list of animations. Used to find actual animation if this sequence is an alias (flags & 0x40)<br />
} sequences[];<br />
<br />
--[[User:Koward|Koward]] ([[User talk:Koward|talk]]) 09:50, 18 December 2015 (UTC) In M2 v274 (Legion), it looks like blend_time has been divided in two uint16_t, and for standard animations the old blend_time is duplicated in both fields (ex : uint32 150 becomes two uint16 150). Maybe start and end blend_time values ? See Creature/GennGreymane/GennGreymane.m2 .<br />
===Flags===<br />
''One thing I saw in the source is that "-1 animationblocks" in bones wont get parsed if 0x20 is not set.''<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="500" | Description<br />
|-<br />
| 0x01 || Sets 0x80 when loaded. (M2Init)<br />
|-<br />
| 0x02 ||<br />
|-<br />
| 0x04 ||<br />
|-<br />
| 0x08 ||<br />
|-<br />
| 0x10 || apparently set during runtime in CM2Shared::LoadLowPrioritySequence for all entries of a loaded sequence (including aliases)<br />
|-<br />
| 0x20 || primary bone sequence -- If set, the animation data is in the .m2 file. If not set, the animation data is in an .anim file. {{Template:Unverified|Was named 'looped animation' by schlumpf years ago, without source.}}<br />
|-<br />
| 0x40 || has next / is alias (To find the animation data, the client skips these by following aliasNext until an animation without 0x40 is found.)<br />
|-<br />
| 0x80 || Blended animation (if either side of a transition has 0x80, lerp between end->start states, unless end==start by comparing bone values)<br />
|-<br />
| 0x100 || sequence stored in model ?<br />
|-<br />
| 0x200 ||<br />
|-<br />
| 0x400 ||<br />
|-<br />
| 0x800 || seen in Legion 24500 models<br />
|}<br />
<br />
-- Rour, some animations rely on blending to look right. The MoP mage CM shoulders only animate half of their movement and rely on lerping back to the start position to look correct.<br />
<br />
=== Animation Lookup===<br />
Hash table for Animations in [[AnimationData.dbc]].<br />
<br />
struct<br />
{<br />
uint16_t animation_index; // Index at [[#Animation_sequences|ofsAnimations]] which represents the animation in [[AnimationData.dbc]]. -1 if none.<br />
} animation_lookups[];<br />
<br />
The hash used is <tt>anim_id % num_buckets</tt>. If a bucket is used, a stride of <tt>n^2</tt> is added with <tt>n = 1, 2, …</tt> until the entry is matching:<br />
<br />
<source lang="c++"><br />
M2Sequence* find_entry (uint32_t anim_id)<br />
{<br />
size_t i (anim_id % animation_lookups.count);<br />
<br />
for (size_t stride (1); true; ++stride)<br />
{<br />
if (animation_lookups[i] == -1)<br />
{<br />
return nullptr;<br />
}<br />
if (animation_sequences[animation_lookups[i]].id == anim_id)<br />
{<br />
return &animation_sequences[i];<br />
}<br />
<br />
i = (i + stride * stride) % animation_lookups.count;<br />
// so original_i + 1, original_i + 1 + 4, original_i + 1 + 4 + 9, …<br />
}<br />
<br />
[[unreachable]];<br />
}<br />
</source><br />
<br />
The entry referenced is the first in the `nextAlias` chain of a given animation id. Thus, <tt>num_buckets < num_animations</tt>, even if a model would have all animations multiple times.<br />
<br />
=== Playable Animation Lookup ===<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=2|note=Partially inlined into M2Sequences}}<br />
<br />
Lookup table for Playable Animation in [[AnimationData.dbc]]<br />
<br />
'''Offset Type Description'''<br />
0x00 int16 Fallback Animation ID in [[AnimationData.dbc]]<br />
0x02 int16 Flags (0, 1, 3 seen)<br />
<br />
Models don't present all animations sequences. This table maps from global animation list to available animation for the current model. The engine uses it to know which animation clip to play. That's why there are a lot of zeros ("Stand") for creatures.<br />
<br />
Flags are used to modify how the clip should be played:<br />
'''Value Meaning'''<br />
0 Play normal<br />
1 Play backwards?<br />
3 Freeze<br />
<br />
For instance, the "HumanMale/HumanMale.m2" model has just one "Loot" animation sequence. "LootHold" and "LootUp" are obtained with flags 3 and 1.<br />
<br />
== Bones ==<br />
struct M2CompBone // probably M2Bone {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
{<br />
int32_t key_bone_id; // Back-reference to [[#Key-Bone_Lookup|the key bone lookup table]]. -1 if this is no key bone.<br />
enum<br />
{<br />
spherical_billboard = 0x8,<br />
cylindrical_billboard_lock_x = 0x10,<br />
cylindrical_billboard_lock_y = 0x20,<br />
cylindrical_billboard_lock_z = 0x40,<br />
transformed = 0x200,<br />
kinematic_bone = 0x400, // MoP+: allow [[PHYS|physics]] to influence this bone<br />
helmet_anim_scaled = 0x1000, // set blend_modificator to [[HelmetAnimScaling.dbc|helmetAnimScalingRec]].m_amount for this bone<br />
};<br />
uint32_t flags; <br />
int16_t parent_bone; // Parent bone ID or -1 if there is none.<br />
uint16_t [[M2/.skin#Submeshes|submesh_id]]; // Mesh part ID OR uDistToParent?<br />
union { // only {{Template:Sandbox/VersionRange|min_expansionlevel=2}}?<br />
struct {<br />
uint16_t uDistToFurthDesc;<br />
uint16_t uZRatioOfChain;<br />
} CompressData; // {{Template:Unverified|No model has ever had this part of the union used.}}<br />
uint32_t boneNameCRC; // these are for debugging only. their bone names match those in key bone lookup.<br />
};<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation;<br />
#else<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|M2CompQuat|link=Quaternion_values_and_2.x}}> rotation; // compressed values, default is (32767,32767,32767,65535) == (0,0,0,1) == identity<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scale;<br />
{{Template:Type|C3Vector}} pivot; // The pivot point of that bone.<br />
} bones[];<br />
The bone indices in the vertex definitions seem to index into this data.<br />
<br />
===Billboards===<br />
The billboarding bits are used for various things:<br />
* Light halos around lamps must always face the viewer<br />
* The cannonball stack model (in the Deadmines or Booty Bay), where each cannonball is a crude hemisphere, they always face the viewer to create the illusion of actual cannonballs.<br />
<br />
Spherical and cylindrical billboard bits are mutually exclusive. Only one of them can be used for the bone.<br />
<br />
===Bone Lookup Table===<br />
Lookup table for bones referenced from [[M2/.skin#Submeshes|M2SkinSection]]. <br />
struct <br />
{<br />
uint16_t bone;<br />
} bone_lookup[];<br />
<br />
===Key-Bone Lookup===<br />
Its a lookup table for key skeletal bones like hands, arms, legs, etc. nKeyBoneLookup is 27 for the most models. At static models it is mostly 1.<br />
struct <br />
{<br />
uint16_t bone; // -1 if none<br />
} key_bone_lookup[];<br />
<br />
Official list:<br />
*00 "ArmL"<br />
*01 "ArmR"<br />
*02 "ShoulderL"<br />
*03 "ShoulderR"<br />
*04 "SpineLow"<br />
*05 "Waist"<br />
*06 "Head"<br />
*07 "Jaw"<br />
*08 "IndexFingerR"<br />
*09 "MiddleFingerR"<br />
*10 "PinkyFingerR"<br />
*11 "RingFingerR"<br />
*12 "ThumbR"<br />
*13 "IndexFingerL"<br />
*14 "MiddleFingerL"<br />
*15 "PinkyFingerL"<br />
*16 "RingFingerL"<br />
*17 "ThumbL"<br />
*18 "$BTH"<br />
*19 "$CSR"<br />
*20 "$CSL"<br />
*21 "_Breath"<br />
*22 "_Name"<br />
*23 "_NameMount"<br />
*24 "$CHD"<br />
*25 "$CCH"<br />
*26 "Root"<br />
*27 "Wheel1" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*28 "Wheel2" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*29 "Wheel3" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*30 "Wheel4" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*31 "Wheel5" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*32 "Wheel6" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*33 "Wheel7" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*34 "Wheel8" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
<br />
=Geometry and rendering=<br />
==Vertices==<br />
struct M2Vertex<br />
{<br />
{{Template:Type|C3Vector}} pos;<br />
uint8 bone_weights[4];<br />
uint8 bone_indices[4];<br />
{{Template:Type|C3Vector}} normal;<br />
{{Template:Type|C2Vector}} tex_coords[2]; // two textures, depending on shader used<br />
};<br />
<br />
Models, too, use a Z-up coordinate systems, so in order to convert to Y-up, the X, Y, Z values become (X, -Z, Y).<br />
<br />
-- Rour, the WoW vertex shaders all follow the same pattern, "Diffuse_XX_YY" (or sometimes XX, YY and Env). The particular vertex shader that is used chooses which set of texture coordinates to use. So Diffuse_T1 sends T1 texcoords to the fragment shader. Where Diffuse_T1_T2 sends both (for textures 0 and 1) but Diffuse_T1_T1 sends the same coords for both textures. Etc.<br />
<br />
==Views (LOD)==<br />
Skin profiles describe LOD views onto the model. They use all those lookup tables to be able to reference only parts of the lists while not being dynamically sized within the profile data.<br />
<br />
{{Template:Sandbox/VersionRange|max_expansionlevel=2}} they were stored in the [[M2]] itself, {{Template:Sandbox/VersionRange|min_expansionlevel=3}} they have been moved to [[M2/.skin|.skin]] files. The offsets are relative to the file the skin profile header is defined in. There is one [[M2/.skin|.skin]] file per profile, each with a separate header, while in the inlined version, all headers are sequential. See the [[M2/.skin|.skin]] file page for formats of both versions.<br />
<br />
==Render flags==<br />
struct M2Material<br />
{<br />
uint16_t flags;<br />
uint16_t blending_mode; // apparently a bitfield<br />
} materials[];<br />
*'''Flags:'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="1000" | Meaning<br />
|-<br />
| 0x01 || Unlit<br />
|-<br />
| 0x02 || Unfogged<br />
|-<br />
| 0x04 || Two-sided (no backface culling if set)<br />
|-<br />
| 0x08 || depthTest<br />
|-<br />
| 0x10 || depthWrite<br />
|-<br />
| 0x40 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x80 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x400 || ??? (seen in WoD)<br />
|-<br />
| 0x800 || prevent alpha for custom elements. if set, use (fully) opaque or transparent. (litSphere, shadowMonk) (MoP+)<br />
|}<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! source !! dest !! notes<br />
|-<br />
| 0 || GL_ONE || GL_ZERO || blending disabled (GxBlendStateDesc: 0)<br />
|-<br />
| 1 || GL_ONE || GL_ZERO || Mod (1)<br />
|-<br />
| 2 || GL_SRC_ALPHA || GL_ONE_MINUS_SRC_ALPHA || Decal (2)<br />
|-<br />
| 3 || GL_ONE || GL_ONE || Add (10)<br />
|-<br />
| 4 || GL_SRC_ALPHA || GL_ONE || Mod2x (3)<br />
|-<br />
| 5 || GL_DST_COLOR || GL_ZERO || Fade (4)<br />
|-<br />
| 6 || GL_DST_COLOR || GL_SRC_COLOR || Deeprun Tram glass (5)<br />
|-<br />
| 7 || GL_ONE || GL_ONE_MINUS_SRC_ALPHA || WoD+ (13)<br />
|}<br />
<br />
(blend equation is always GL_FUNC_ADD. Values are retrieved via GxBlendStateDesc's lower 5 bits. no separate blend func for alpha.)<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="100" | Mapped to !! width="1500" | Meaning<br />
|-<br />
| 0 || 0 || Combiners_Opaque (Blend disabled)<br />
|-<br />
| 1 || 1 || Combiners_Mod (Blend enabled, Src = ONE, Dest = ZERO, SrcAlpha = ONE, DestAlpha = ZERO)<br />
|-<br />
| 2 || 1 || Combiners_Decal (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 3 || 1 || Combiners_Add (Blend enabled, Src = SRC_COLOR, Dest = DEST_COLOR, SrcAlpha = SRC_ALPHA, DestAlpha = DEST_ALPHA )<br />
|-<br />
| 4 || 1 || Combiners_Mod2x (Blend enabled, Src = SRC_ALPHA, Dest = ONE, SrcAlpha = SRC_ALPHA, DestAlpha = ONE )<br />
|-<br />
| 5 || 4 || Combiners_Fade (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 6 || 4 || Used in the Deeprun Tram subway glass, supposedly (Blend enabled, Src = DEST_COLOR, Dest = SRC_COLOR, SrcAlpha = DEST_ALPHA, DestAlpha = SRC_ALPHA )<br />
|-<br />
| 7 ||? || New in WoD, needs research! ''Example model: World\Expansion05\Doodads\Shadowmoon\Doodads\6FX_Fire_Grassline_Doodad_blue_LARGE.m2''<br />
|}<br />
''*Blend values are taken from D3D11 debugging of the client<br />
===Blend mode overrides===<br />
If this block is present (globalflags&8) and the "shading" flags of a textureunit wont be &0x8000, blending modes wont get mapped to the values above but to the ones in this block.<br />
<br />
Instead of Mapping[renderflags->blendingmode] it will be UnknownBlock[textureunit->Shading].<br />
<br />
As shading is not &0x8000 and (in their code) needs to be above 0, this may only touch Diffuse_T1.<br />
<br />
According to wod, if the M2 Header has flag 0x08, instead of reading blend mode from M2 [[#Render_flags|RenderFlags]], blendMode is read from the raw blend maps referenced in header.<br />
<br />
var flags = renderFlags[texUnit.renderFlags];<br />
var blendMode = flags >> 16;<br />
if ((header.GlobalModelFlags & 0x08) != 0 && texUnit.shader_id < mBlendMap.Length)<br />
blendMode = mBlendMap[texUnit.shader_id];<br />
<br />
==Texture unit lookup table==<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=4|max_exclusive=1|note=Still present but unused in Cataclysm}}<br />
struct <br />
{<br />
uint16_t unit; // -1, 0, or 1. see below<br />
} texture_units[];<br />
<br />
For models that use multitexturing, this maps given texture unit numbers into actual texture unit numbers (0 or 1).<br />
<br />
Values of -1 seem to mean environment mapping.<br />
<br />
One model is of special interest, Creature/KelThuzad/KelThuzad.m2, which is the only one that has an nTexUnits of 3, and has three texture units specified for some of its submeshes. Sure enough, two of those map to 0 and 1, and one maps to -1.<br />
<br />
More confusion thanks to my favorite "weird" model, World/Generic/Gnome/Passive Doodads/GnomeMachine/GnomeSubwayGlass.m2, which is the translucent, environment mapped glass tunnel in the Deeprun Tram. It only has a single value in this block, -1, which is used for the single texture layer in both render operations in the model. This and the magic with rendering flags/blend modes make up the neat transparent-reflective glass effect, but confuse me even more about how envmapping and such is handled. (and where it seems to get the bluish color from - is it in the model (no color blocks in this particular model), the wmo, a solid background color, or simply the result of the blending used?)<br />
<br />
As a side note, on my (dated) system WoW does every texture unit in a single pass.<br />
<br />
==Colors and transparency==<br />
===Colors===<br />
struct M2Color<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> color; // vertex colors in rgb order<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alpha; // 0 - transparent, 0x7FFF - opaque. Normaly NonInterp<br />
} colors[];<br />
<br />
This block is the M2 equivalent to the GEOA chunk in MDX files, it represents the vertex color and visibility animations for meshes. Referenced from the Texture Unit blocks in the [[M2/.skin|*.skin]]. If a texunit belonging to a submesh has a value of -1 then the submesh doesnot use this block. Contains a separate timeline for transparency values. If no animation is used, the given value is constant.<br />
<br />
===Transparency===<br />
struct M2TextureWeight<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> weight;<br />
} textureWeights[];<br />
<br />
Specifies global transparency values in addition to the values given in the Color block. I assume these are multiplied together eventually.<br />
====Transparency lookup table====<br />
struct <br />
{<br />
uint16_t transparency;<br />
} transparency_lookup[];<br />
Contains indices into the Transparency block. Used by the texture unit definitions in the LOD block.<br />
<br />
==Textures==<br />
*'''Textures are defined globally in a list''', additionally, a lookup table is given, referenced during rendering, to select textures.<br />
struct M2Texture<br />
{<br />
uint32_t type; // see below<br />
uint32_t flags; // see below<br />
M2Array<char> filename; // for non-hardcoded textures (type != 0), this still points to a zero-sized string<br />
} textures[];<br />
<br />
====Texture Types====<br />
Texture type is 0 for regular textures, nonzero for skinned textures (filename not referenced in the M2 file!) For instance, in the NightElfFemale model, her eye glow is a type 0 texture and has a file name, the other 3 textures have types of 1, 2 and 6. The texture filenames for these come from client database files:<br />
*DBFilesClient\[[CharSections.dbc]]<br />
*DBFilesClient\[[CreatureDisplayInfo.dbc]]<br />
*DBFilesClient\[[ItemDisplayInfo.dbc]]<br />
*(possibly more)<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="800" | Meaning<br />
|-<br />
| 0 || - NONE - -- Texture given in filename<br />
|-<br />
| 1 || TEX_COMPONENT_SKIN -- Skin -- Body + clothes<br />
|-<br />
| 2 || TEX_COMPONENT_OBJECT_SKIN -- Object Skin -- Item, Capes ("Item\ObjectComponents\Cape\*.blp")<br />
|-<br />
| 3 || TEX_COMPONENT_WEAPON_BLADE -- Weapon Blade -- Used on several models but not used in the client as far as I see. Armor Reflect?<br />
|-<br />
| 4 || TEX_COMPONENT_WEAPON_HANDLE -- Weapon Handle<br />
|-<br />
| 5 || TEX_COMPONENT_ENVIRONMENT -- (OBSOLETE) Environment (Please remove from source art)<br />
|-<br />
| 6 || TEX_COMPONENT_CHAR_HAIR -- Character Hair<br />
|-<br />
| 7 || TEX_COMPONENT_CHAR_FACIAL_HAIR -- (OBSOLETE) Character Facial Hair (Please remove from source art)<br />
|-<br />
| 8 || TEX_COMPONENT_SKIN_EXTRA -- Skin Extra<br />
|-<br />
| 9 || TEX_COMPONENT_UI_SKIN -- UI Skin -- Used on inventory art M2s (1): inventoryartgeometry.m2 and inventoryartgeometryold.m2<br />
|-<br />
| 10 || TEX_COMPONENT_TAUREN_MANE -- (OBSOLETE) Tauren Mane (Please remove from source art) -- Only used in quillboarpinata.m2. I can't even find something referencing that file. Oo Is it used?<br />
|-<br />
| 11 || TEX_COMPONENT_MONSTER_1 -- Monster Skin 1 -- Skin for creatures or gameobjects #1<br />
|-<br />
| 12 || TEX_COMPONENT_MONSTER_2 -- Monster Skin 2 -- Skin for creatures or gameobjects #2<br />
|-<br />
| 13 || TEX_COMPONENT_MONSTER_3 -- Monster Skin 3 -- Skin for creatures or gameobjects #3<br />
|-<br />
| 14 || TEX_COMPONENT_ITEM_ICON -- Item Icon -- Used on inventory art M2s (2): ui-button.m2 and forcedbackpackitem.m2 (CSimpleModel_ReplaceIconTexture("texture"))<br />
|-<br />
| 15 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Background Color<br />
|-<br />
| 16 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem Color<br />
|-<br />
| 17 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Border Color<br />
|-<br />
| 18 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem<br />
|}<br />
<br />
====Flags====<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Meaning<br />
|-<br />
| 1 || Texture wrap X<br />
|-<br />
| 2 || Texture wrap Y<br />
|}<br />
<br />
====Texture lookup table====<br />
struct <br />
{<br />
uint16_t texture;<br />
} texture_lookup[];<br />
<br />
====Replacable texture lookup====<br />
struct <br />
{<br />
uint16_t replacement;<br />
} texture_replacements[];<br />
<br />
A reverse lookup table for 'replaced' textures, mapping replacable ids to texture indices or -1. Only goes up to the maximum id used in the model.<br />
<br />
Its strange, that HARDCODED is in the list, as a model can have more than one of course. Its just the last one written to the file.<br />
<br />
=Effects=<br />
==Texture Transforms==<br />
*'''This block contains definitions for texture animations,''' for example, flowing water or lava in some models. The keyframe values are used in the texture transform matrix.<br />
<br />
struct M2TextureTransform<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation; // rotation center is texture center (0.5, 0.5)<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scaling;<br />
} textureTransforms[];<br />
<br />
Seems like UV rotation in some models are made against (0.5, 0.5) point instead of (0, 0). At least it's the case for world\goober\g_scourgerunecirclecrystal.m2 <br />
<br />
So to get the proper UV rotation it would be necessary apply rotation this way:<br />
<br />
* Translate UV anim matrix to point (0.5, 0.5)<br />
* Apply rotation mat from quaternion<br />
* UV anim matrix to point (-0.5, -0.5)<br />
<br />
====Texture Transforms lookup table ====<br />
struct <br />
{<br />
uint16_t anim_texture_id; // -1 for static<br />
} anim_texture_lookup[];<br />
<br />
== Ribbon emitters ==<br />
struct M2Ribbon<br />
{<br />
uint32_t ribbonId; // Always (as I have seen): -1.<br />
uint32_t boneIndex; // A bone to attach to.<br />
{{Template:Type|C3Vector}} position; // And a position, relative to that bone.<br />
M2Array<uint16_t> textureIndices; // into [[#Textures|textures]]<br />
M2Array<uint16_t> materialIndices; // into [[#Render_flags|materials]]<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> colorTrack;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alphaTrack; // And an alpha value in a short, where: 0 - transparent, 0x7FFF - opaque.<br />
[[#Standard_animation_block|M2Track]]<float> heightAboveTrack;<br />
[[#Standard_animation_block|M2Track]]<float> heightBelowTrack; // do not set to same!<br />
float edgesPerSecond; // this defines how smooth the ribbon is. A low value may produce a lot of edges.<br />
float edgeLifetime; // the length aka Lifespan. in seconds<br />
float gravity; // use arcsin(val) to get the emission angle in degree<br />
uint16_t textureRows; // tiles in texture<br />
uint16_t textureCols;<br />
[[#Standard_animation_block|M2Track]]<uint16_t> texSlotTrack;<br />
[[#Standard_animation_block|M2Track]]<uchar> visibilityTrack;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}} // TODO: verify version<br />
int16_t priorityPlane;<br />
uint16_t padding;<br />
#endif<br />
} ribbons[];<br />
Some models that contain ribbon emitters and are viewable in the game world are: Wisps in BFD, Al'ar the Phoenix in Tempest Keep and any other phoenix models and the energy trails in the COT (not the actual instance, but the entrance cave in Tanaris Desert). Other models with ribbon emitters are spells and effects.<br />
<br />
''Parameters from the MDL format that are probably in here somewhere: emission rate, rows, cols ...?''<br />
<br />
== Particle emitters ==<br />
'''This is partly wrong as hell!''' Do not rely on this block, at all. It might even be wrong for WotLK.<br />
===M2ParticleOld===<br />
struct M2ParticleOld {<br />
uint32 particleId; // Always (as I have seen): -1.<br />
uint32 flags; // See Below<br />
{{Template:Type|C3Vector}} Position; // The position. Relative to the following bone.<br />
uint16 bone; // The [[#Bones|bone]] its attached to.<br />
union<br />
{<br />
uint16 texture; // And the [[#Texture_definitions|textures]] that are used. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
struct // For multi-textured particles actually three ids<br />
{<br />
uint16_t texture_0 : 5;<br />
uint16_t texture_1 : 5;<br />
uint16_t texture_2 : 5;<br />
uint16_t : 1;<br />
};<br />
#endif<br />
};<br />
M2Array<char> geometry_model_filename; // if given, this emitter spawns models<br />
M2Array<char> recursion_model_filename; // if given, this emitter is an '''alias''' for the (maximum 4) emitters of the given model<br />
<br />
#if >= 263 (late Burning Crusade)<br />
uint8 blendingType; // A blending type for the particle. See Below<br />
uint8 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
uint16 particleColorIndex; // This one is used for [[ParticleColor.dbc]]. See below.<br />
#else<br />
uint16 blendingType; // A blending type for the particle. See Below<br />
uint16 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
#endif<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
{{Template:Type|fixed_point}}<uint8_t, 2, 5> multiTextureParamX[2];<br />
#else<br />
uint8 particleType; // Found below.<br />
uint8 headorTail; // 0 - Head, 1 - Tail, 2 - Both <br />
#endif<br />
uint16 textureTileRotation; // Rotation for the texture tile. (Values: -1,0,1) -- priorityPlane<br />
uint16 textureDimensions_rows; // for tiled textures<br />
uint16 textureDimensions_columns;<br />
[[#Standard_animation_block|M2Track]]<float> emissionSpeed; // Base velocity at which particles are emitted.<br />
[[#Standard_animation_block|M2Track]]<float> speedVariation; // Random variation in particle emission speed. (range: 0 to 1)<br />
[[#Standard_animation_block|M2Track]]<float> verticalRange; // longitude; Drifting away vertically. (range: 0 to pi) For plane generators, this is the maximum polar angle of the initial velocity; <br />
// 0 makes the velocity straight up (+z). For sphere generators, this is the maximum elevation of the initial position; <br />
// 0 makes the initial position entirely in the x-y plane (z=0).<br />
[[#Standard_animation_block|M2Track]]<float> horizontalRange; // latitude; They can do it horizontally too! (range: 0 to 2*pi) For plane generators, this is the maximum azimuth angle of the initial velocity; <br />
// 0 makes the velocity have no sideways (y-axis) component. <br />
// For sphere generators, this is the maximum azimuth angle of the initial position.<br />
[[#Standard_animation_block|M2Track]]<float> gravity; // Not necessarily a float; [[#Compressed Particle Gravity|see below]].<br />
[[#Standard_animation_block|M2Track]]<float> lifespan;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float lifespanVary; // An individual particle's lifespan is added to by <code>lifespanVary * random(-1, 1)</code><br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionRate; <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float emissionRateVary; // This adds to the base emissionRate value the same way as lifespanVary. The random value is different every update.<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaLength; // For plane generators, this is the width of the plane in the x-axis.<br />
// For sphere generators, this is the minimum radius.<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaWidth; // For plane generators, this is the width of the plane in the y-axis.<br />
// For sphere generators, this is the maximum radius.<br />
[[#Standard_animation_block|M2Track]]<float> zSource; // When greater than 0, the initial velocity of the particle is <code>(particle.position - C3Vector(0, 0, zSource)).Normalize()</code><br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C3Vector}}> colorTrack; // Most likely they all have 3 timestamps for {start, middle, end}.<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|fixed16}}> alphaTrack;<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C2Vector}}> scaleTrack;<br />
{{Template:Type|C2Vector}} scaleVary; // A percentage amount to randomly vary the scale of each particle<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> headCellTrack; // Some kind of intensity values seen: 0,16,17,32 (if set to different it will have high intensity)<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> tailCellTrack;<br />
#else<br />
float midPoint; // middleTime; Middle point in lifespan (0 to 1).<br />
{{Type|CImVector}}[3] colorValues; // start, middle, end<br />
float[3] scaleValues;<br />
uint16[3] lifespanUVAnim;<br />
uint16[3] decayUVAnim;<br />
int16[2] tailUVAnim; // start, end<br />
int16[2] tailDecayUVAnim;<br />
#endif<br />
float tailLength;<br />
float twinkleSpeed; // twinkleFPS; has something to do with the spread<br />
float twinklePercent; // same mechanic as MDL twinkleOnOff but non-binary in 0.11.0<br />
{{Template:Type|CRange}} twinkleScale; // min, max<br />
float burstMultiplier; // ivelScale; requires (flags & 0x40)<br />
float drag; // For a non-zero values, instead of travelling linearly the particles seem to slow down sooner. Speed is multiplied by exp( -drag * t ).<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float baseSpin; // Initial rotation of the particle quad<br />
float baseSpinVary;<br />
float spin; // Rotation of the particle quad per second<br />
float spinVary;<br />
#else<br />
float spin; // 0.0 for none, 1.0 to rotate the particle 360 degrees throughout its lifetime.<br />
#endif<br />
<br />
[[#M2Box|M2Box]] tumble;<br />
{{Template:Type|C3Vector}} windVector;<br />
float windTime;<br />
<br />
float followSpeed1;<br />
float followScale1;<br />
float followSpeed2;<br />
float followScale2;<br />
M2Array<C3Vector> splinePoints; // Set only for spline praticle emitter. Contains array of points for spline<br />
[[#Standard_animation_block|M2Track]]<uchar> enabledIn; // (boolean) Appears to be used sparely now, probably there's a flag that links particles to animation sets where they are enabled.<br />
} particles[];<br />
<br />
Spin can be a float value greater or less one. Results look better if use it as a "phase shift": particle_rotate = randfloat(-sys->rotation * pi, sys->rotation * pi); --Igor<br />
<br />
===M2Particle (Cata+)===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
* Cata+ has multi texture support<br />
using fp_6_9 = {{Template:Type|fixed_point}}<uint16_t, 6, 9>;<br />
struct vector_2fp_6_9 { fp_6_9 x; fp_6_9 y; };<br />
struct M2Particle<br />
{<br />
M2ParticleOld old;<br />
vector_2fp_6_9 multiTextureParam0[2];<br />
vector_2fp_6_9 multiTextureParam1[2];<br />
} particles[];<br />
<br />
In addition to these two parameters, <code>ParticleType</code> and <code>HeadOrTail</code> got reused (as in replaced at their current position) as <code>multiTextureParamX[2]</code> where all arrays are one entry per additional texture.<br />
<br />
I don't know if the previous meaning of the two parameters still exists, got moved, or was just never used to begin with. ParticleType appears to be implicit by having <code>flags & 0x10100000</code> (→ multi texture), a model (→ model) or neither (→ default).--[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 23:47, 29 October 2015 (UTC)<br />
<br />
===Particle Flags===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="600" | Description<br />
|-<br />
| 0x1 || Particles are affected by lighting; <br />
|-<br />
| 0x2 || <br />
|-<br />
| 0x4 || <br />
|-<br />
| 0x8 || Particles travel "up" in world space, rather than model.<br />
|-<br />
| 0x10 || Do not Trail<br />
|-<br />
| 0x20 || Unlightning<br />
|-<br />
| 0x40 || Use Burst Multiplier<br />
|-<br />
| 0x80 || Particles in Model Space<br />
|-<br />
| 0x100 || <br />
|-<br />
| 0x200 || spawn position randomized in some way?<br />
|-<br />
| 0x400 || STYLE: Pinned Particles, their quad enlarges from their creation position to where they expand.<br />
|-<br />
| 0x800 || <br />
|-<br />
| 0x1000 || XYQuad Particles. They align to XY axis facing Z axis direction.<br />
|-<br />
| 0x2000 || clamp to ground; call CParticleEmitter2::ProjectParticle<br />
|-<br />
| 0x4000 || <br />
|-<br />
| 0x8000 || <br />
|-<br />
| 0x10000 || ChooseRandomTexture<br />
|-<br />
| 0x20000 || STYLE: "Outward" particles, most emitters have this and their particles move away from the origin, when they don't the particles start at origin+(speed*life) and move towards the origin.<br />
|-<br />
| 0x40000 || STYLE: unknown. In a large proportion of particles this seems to be simply the opposite of the above flag, but in some (e.g. voidgod.m2 or wingedlionmount.m2) both flags are true.<br />
|-<br />
| 0x80000 || If set, ScaleVary affects x and y independently; if not set, ScaleVary.x affects x and y uniformly, and ScaleVary.y is not used.<br />
|-<br />
| 0x200000 || Random FlipBookStart<br />
|-<br />
| 0x400000 || Ignores Distance (or 0x4000000?!, CMapObjDef::SetDoodadEmittersIgnoresDistance has this one)<br />
|-<br />
| 0x800000 || gravity values are compressed vectors instead of z-axis values (see Compressed Particle Gravity below)<br />
|-<br />
| 0x1000000 || bone generator = bone, not joint<br />
|-<br />
| 0x4000000 || do not throttle emission rate based on distance<br />
|-<br />
| 0x10000000 || Particle uses multi-texturing (could be one of the other WoD-specific flags), see multi-textured section.<br />
|}<br />
<br />
--[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) A comparison of 0.10.0's MDX files to 0.11.0's and 1.12.1's M2 files indicates that the [[MDX#Flags|MDX PRE2 flags]] (≥0x8000) were probably unchanged during the switch from MDL to M2, albeit separated into their own enum.<br />
<br />
===ParticleColorIndex===<br />
This is used in conjunction with [[ParticleColor.dbc]] to alter the default colour of particles. If the particle colour is not meant to be changed then its ParticleColorIndex will have a value of zero. If the particle colour may be changed then the value will be 11, 12 or 13, indicating whether the first, second or third Start, Mid and End colours are to be used, respectively. The row of [[ParticleColor.dbc]] to be used is determined its ID value, which should correspond to the ParticleColorID value supplied by [[CreatureDisplayInfo.dbc]] or [[ItemDisplayInfo.dbc]] for creatures or items.<br />
===Particle types===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || "normal" particle<br />
|-<br />
| 1 || large quad from the particle's origin to its position (used in Moonwell water effects)<br />
|-<br />
| 2 || seems to be the same as 0 (found some in the Deeprun Tram blinky-lights-sign thing)<br />
|}<br />
''ParticleType is always 0 and, maybe, now (Flags & 0x40000) != 0 means "particles from origin to position". --Igor''<br />
''Checked and verified --BlinkHawk''<br />
===Particle Blendings===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || glDisable(GL_BLEND); glDisable(GL_ALPHA_TEST);<br />
|-<br />
| 1 || glBlendFunc(GL_SRC_COLOR, GL_ONE);<br />
|-<br />
| 2 || glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);<br />
|-<br />
| 3 || glDisable(GL_BLEND); glEnable(GL_ALPHA_TEST);<br />
|-<br />
| 4 || glBlendFunc(GL_SRC_ALPHA, GL_ONE);<br />
|}<br />
from Modelviewer source<br />
-- Rour, some WoD particle effects are using blend mode 0x7 here.<br />
===The Fake-AnimationBlock===<br />
*Its pretty much like the real one but without the "header".<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Offset !! width="90" | Type !! width="120" | Name !! width="500" | Description<br />
|-<br />
| 0x000 || uint32 || nTimestamps || The number of timestamps.<br />
|-<br />
| 0x004 || uint32 || ofsTimestamps || And the offset to them. The timestamps are shorts! (?)<br />
|-<br />
| 0x008 || uint32 || nKeys || The same number again. This time its the number of Keys / Values.<br />
|-<br />
| 0x00C || uint32 || ofsKeys || And their offset.<br />
|}<br />
But they're unable to change between different animations, so they directly point to the data.<br />
<br />
===Compressed Particle Gravity===<br />
Key values in the gravity track are decompressed at load time from a 4-byte value to a C3Vector.<br />
<br />
<nowiki><br />
struct CompressedParticleGravity {<br />
int8_t x, y;<br />
int16_t z;<br />
};<br />
for (/* each 4-byte value in the particle gravity track */) {<br />
float *pValue;<br />
C3Vector *pDst;<br />
if (particle.flags & 0x800000) {<br />
// interpret the 4 bytes at pValue as CompressedParticleGravity:<br />
CompressedParticleGravity v = new (pValue) CompressedParticleGravity();<br />
C3Vector dir = C3Vector(v.x, v.y, 0) * (1.0f / 128.0f);<br />
float z = sqrtf(1.0f - dir.Dot(dir));<br />
float mag = v.z * 0.04238648f;<br />
if (mag < 0) {<br />
z = -z;<br />
mag = -mag;<br />
}<br />
dir.z = z;<br />
dir *= mag;<br />
*pDst = dir;<br />
} else {<br />
*pDst = C3Vector(0, 0, -(*pValue));<br />
}<br />
}</nowiki><br />
<br />
<br />
===M2Box===<br />
struct M2Box {<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMin;<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMax;<br />
}<br />
<br />
=Miscellaneous=<br />
==Name==<br />
char name[];<br />
<br />
Informative name used for debugging purposes. Not used in retail clients.<br />
<br />
==Bounding volumes==<br />
These blocks give a simplified bounding volume for the model. Characters and creatures have just a simple box.<br />
===Vertices===<br />
This block defines the possible points used for the model. They are referenced in the triangles block later.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} position;<br />
} bounding_vertices[];<br />
<br />
===Triangles===<br />
The number in the header tells us how many uint16s there are, not how many triangles. To use this better, you should group three of them into an array. The nBoundingTriangles/3 indices will tell you which vertices are used for the triangle then. <br />
<br />
struct <br />
{<br />
uint16_t index; // three entries pointing to vertices per triangle<br />
} bounding_triangles[];<br />
<br />
The number nBoundingTriangles once again contains the number of indices used, so divide by 3 to get the number of triangles.<br />
<br />
===Normals===<br />
This one defines a normal per triangle. The vectors are normalized, but Blizzard seems to have some problems getting a simple vector normalized leading in several 0,0,0.999999999 ones. Whatever.<br />
<br />
As each vertex has a corresponding normal vector, it should be true that nBoundingNormals = nBoundingTriangles / 3.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} normal;<br />
} bounding_normals[];<br />
<br />
== Lights ==<br />
struct M2Light<br />
{<br />
/*0x00*/ uint16_t type; // Types are listed below.<br />
/*0x02*/ int16_t bone; // -1 if not attached to a bone<br />
/*0x04*/ {{Template:Type|C3Vector}} position; // relative to bone, if given<br />
/*0x10*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> ambient_color;<br />
/*0x24*/ [[#Standard_animation_block|M2Track]]<float> ambient_intensity; // defaults to 1.0<br />
/*0x38*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> diffuse_color;<br />
/*0x4C*/ [[#Standard_animation_block|M2Track]]<float> diffuse_intensity; // defaults to 1.0<br />
/*0x60*/ [[#Standard_animation_block|M2Track]]<float> attenuation_start;<br />
/*0x74*/ [[#Standard_animation_block|M2Track]]<float> attenuation_end;<br />
/*0x88*/ [[#Standard_animation_block|M2Track]]<uint8_t> visibility; // enabled?<br />
/*0x9C*/<br />
} lights[];<br />
<br />
Two light types:<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Description<br />
|-<br />
| 0 || Directional<br />
|-<br />
| 1 || Point light<br />
|}<br />
<br />
Directional light type is not used (at least in 3.3.5) outside login screen, and doesn't seem to be taken into account in world.<br />
<br />
==Cameras==<br />
These blocks are present in the "flyby" camera models which completely lack geometry and the main menu backdrop models which are supposed to have a fixed camera. Additionally, characters and monsters also have this block. The reason that non-mainmenu and non-geometry M2s have cameras was is you can see the unit's portrait and the character info tab.<br />
<br />
struct M2Camera<br />
{<br />
uint32_t type; // 0: portrait, 1: characterinfo; -1: else (flyby etc.); referenced backwards in the lookup table.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=4|max_exclusive=1}}<br />
float fov; // Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
float far_clip;<br />
float near_clip;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> positions; // How the camera's position moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> target_position; // How the target moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} target_position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> roll; // The camera can have some roll-effect. Its 0 to 2*Pi. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> FoV; //Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
} cameras[];<br />
<br />
===Camera field of view===<br />
<br />
The <tt>fov</tt> included in <tt>M2Camera</tt> is a diagonal field of view (in radians). The client converts it to a vertical field of view at runtime using the following formula:<br />
<br />
<syntaxhighlight lang="cpp"><br />
float vfov = dfov / sqrt(1.0 + pow(aspect, 2.0));<br />
</syntaxhighlight><br />
<br />
The aspect ratio used is determined by the rect being presented on (eg. the game window).<br />
<br />
Note that this formula makes an assumption that the screen being projected to is planar (ie a monitor).<br />
<br />
===Camera lookup table===<br />
This block lists the different cameras existing in the model. The index in the array is also the type. CameraLookupTable[1] is always the character tab camera.<br />
<br />
"-1" type cameras are not referenced.<br />
<br />
If nCameras >= 1, then nCameraLookup will be >= 1 regardless of whether any camera will be actually referenced in it. See interface/glues/models/ui_mainmenu_warlords/ui_mainmenu_warlords.m2. A valid block thus may be -1s only. This appears to be an exporter-quirk rather than a requirement.<br />
<br />
struct<br />
{<br />
uint16_t camera;<br />
} camera_lookup[];<br />
<br />
== Attachments ==<br />
This block specifies a bunch of locations on the body - hands, shoulders, head, back, knees etc. It is used to put items on a character. This seems very likely as this block also contains positions for sheathed weapons, a shield, etc.<br />
struct M2Attachment<br />
{<br />
uint32_t id; // Referenced in the [[#Attachment_Lookup|lookup-block]] below.<br />
uint16_t bone; // attachment base<br />
uint16_t unknown; // see BogBeast.m2 in vanilla for a model having values here<br />
{{Template:Type|C3Vector}} position; // relative to bone; Often this value is the same as bone's pivot point <br />
[[#Standard_animation_block|M2Track]]<uchar> animate_attached; // whether or not the attached model is animated when this model is. only a bool is used. default is true.<br />
} attachments[];<br />
<br />
Meaning depends on type of model. The following are for creatures/characters mainly:<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="50" | ID !! width="250" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
|-<br />
| 0 || Shield / MountMain / ItemVisual0<br />
| 12 || Back<br />
| 24 || Special2<br />
| 36 || Bullet (version: somewhen after alpha)<br />
| 48 || RightFoot<br />
|-<br />
| 1 || HandRight / ItemVisual1<br />
| 13 || ShoulderFlapRight<br />
| 25 || Special3<br />
| 37 || SpellHandOmni (version: somewhen after alpha)<br />
| 49 || ShieldNoGlove<br />
|-<br />
| 2 || HandLeft / ItemVisual2<br />
| 14 || ShoulderFlapLeft<br />
| 26 || SheathMainHand<br />
| 38 || SpellHandDirected (version: somewhen after alpha)<br />
| 50 || SpineLow<br />
|-<br />
| 3 || ElbowRight / ItemVisual3<br />
| 15 || ChestBloodFront<br />
| 27 || SheathOffHand<br />
| 39 || VehicleSeat1 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 51 || AlteredShoulderR<br />
|-<br />
| 4 || ElbowLeft / ItemVisual4<br />
| 16 || ChestBloodBack<br />
| 28 || SheathShield<br />
| 40 || VehicleSeat2 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 52 || AlteredShoulderL<br />
|-<br />
| 5 || ShoulderRight<br />
| 17 || Breath<br />
| 29 || PlayerNameMounted<br />
| 41 || VehicleSeat3 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 53 || BeltBuckle {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
|-<br />
| 6 || ShoulderLeft<br />
| 18 || PlayerName<br />
| 30 || LargeWeaponLeft<br />
| 42 || VehicleSeat4 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 54 || SheathCrossbow<br />
|-<br />
| 7 || KneeRight<br />
| 19 || Base<br />
| 31 || LargeWeaponRight<br />
| 43 || VehicleSeat5 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 55 || HeadTop {{Template:Sandbox/VersionRange|min_expansionlevel=7}}<br />
|-<br />
| 8 || KneeLeft<br />
| 20 || Head<br />
| 32 || HipWeaponLeft<br />
| 44 || VehicleSeat6 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 9 || HipRight<br />
| 21 || SpellLeftHand<br />
| 33 || HipWeaponRight<br />
| 45 || VehicleSeat7 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 10 || HipLeft<br />
| 22 || SpellRightHand<br />
| 34 || Chest<br />
| 46 || VehicleSeat8 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 11 || Helm<br />
| 23 || Special1<br />
| 35 || HandArrow<br />
| 47 || LeftFoot<br />
| || <br />
|}<br />
For weapons, usually 5 of these points are present, which correspond to the 5 columns in [[ItemVisuals.dbc]], which in turn has 5 models from [[ItemVisualEffects.dbc]]. This is for the weapon glowy effects and such. The effect ID is the last column in [[ItemDisplayInfo.dbc]]. They take the ids 0 to 4. Mounts take the id 0 for their rider. Breath (17) is used by CGCamera::FinishLoadingTarget() aswell as some other one. The name above the head of a Unit (CGUnit_C::GetNamePosition) looks for PlayerNameMounted (29), then PlayerName (18).<br />
===Attachment Lookup===<br />
The index of the array defines, which type that attachment is of. Its the same as the list above. The lookups and the id of the animations point in a circle.<br />
struct <br />
{<br />
uint16_t attachment;<br />
} attachment_lookup[];<br />
<br />
==Events==<br />
These events are used for timing sounds for example. You can find the $DTH (death) event on nearly every model. It will play the death sound for the unit.<br />
<br />
The events you can use depend on the way, the model is used. Dynamic objects can shake the camera, doodads shouldn't. Units can do a lot more than other objects.<br />
<br />
Somehow there are some entries, that don't use the $... names but identifiers like "DEST" (destination), "POIN" (point) or "WHEE" (wheel). How they are used? Idk.<br />
<br />
struct M2Event<br />
{<br />
uint32_t identifier; // mostly a 3 character name prefixed with '$'.<br />
uint32_t data; // This data is passed when the event is fired. <br />
uint32_t bone; // Somewhere it has to be attached.<br />
{{Template:Type|C3Vector}} position; // Relative to that bone of course, animated. Pivot without animating.<br />
[[#Standard_animation_block|M2TrackBase]] enabled; // This is a timestamp-only animation block. It is built up the same as a normal [[#Standard_animation_block|AnimationBlocks]], but is missing values, as every timestamp is an implicit "fire now".<br />
} events[];<br />
===Possible Events===<br />
There are a lot more of them. I did not list all up to now.<br />
{| style="background:#FCFCFC || color:black"<br />
! width="50" | ID !! width="40" | data !! width="60" | what !! width="60" | Type !! width="200" | seen to be fired on !! width="600" | Description<br />
|-<br />
| $AH[0-3] || — || PlaySoundKit (customAttack[x]) || || || soundEffect ID is defined by CreatureSoundDataRec::m_customAttack[x]<br />
|-<br />
| $BMD || — || BowMissleDestination || RangedWeapon || ||<br />
|-<br />
| $AIM || || || Vehicles || CGUnit_C::ComputeMissileTrajectory || Position used as MissileFirePos.<br />
|-<br />
| $ALT || || anim_swap_event / DisplayTransition || Unit || || CUnitDisplayTransition_C::UpdateState(1) or CUnitDisplayTransition_C::HandleAnimSwapEvent<br />
|-<br />
| $BL[0-3] || — || FootstepAnimEventHit (left) || Unit || || Backwards<br />
|-<br />
| $BR[0-3] || — || FootstepAnimEventHit (right) || Unit || || Backwards<br />
|-<br />
| $BRT || — || PlaySoundKit (birth) || || || soundEffect ID is defined by CreatureSoundDatarec::m_birthSoundID<br />
|-<br />
| $BTH || — || Breath || Unit || All situations, where nothing happens or breathing. || Adds Special Unit Effect based on unit state (under water, in-snow, …)<br />
|-<br />
| $BWP || — || PlayRangedItemPull (Bow Pull) || Unit || LoadRifle, LoadBow || <br />
|-<br />
| $BWR || — || BowRelease || Unit || AttackRifle, AttackBow, AttackThrown || <br />
|-<br />
| $CAH || — || || Unit || Attack*, *Unarmed, ShieldBash, Special* || attack hold? CGUnit_C::HandleCombatAnimEvent<br />
|-<br />
| $CCH || — || || Unit || mostly not fired, AttackThrown || CEffect::DrawFishingString needs this on the model for getting the string attachments.<br />
|-<br />
| $CFM || — || || Unit || CGCamera::UpdateMountHeightOrOffset || CGCamera::UpdateMountHeightOrOffset: Only z is used. Non-animated. Not used if $CMA<br />
|-<br />
| $CHD || || || Unit || not fired || probably does not exist?!<br />
|-<br />
| $CMA || — || || Unit || || CGCamera::UpdateMountHeightOrOffset: Position for camera<br />
|-<br />
| $CPP || || PlayCombatActionAnimKit || || || parry, anims, depending on some state, also some callback which might do more<br />
|-<br />
| $CSD || soundEntryId || PlayEmoteSound || Unit || Emote* || <br />
|-<br />
| $CSL || — || release_missiles_on_next_update if has_pending_missiles (left) || Unit || AttackRifle, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSR || — || release_missiles_on_next_update if has_pending_missiles (right) || Unit || AttackBow, AttackRifle, AttackThrown, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSS || — || PlayWeaponSwooshSound || || sound played depends on CGUnit_C::GetWeaponSwingType<br />
|-<br />
| $CST || — || release_missiles_on_next_update if has_pending_missiles || Unit || Attack*, *Unarmed, ShieldBash, Special*, SpellCast, Parry*, EmoteEat, EmoteRoar, Kick, ... || $CSL/R/T are also used in CGUnit_C::ComputeDefaultMissileFirePos.<br />
|-<br />
| $CVS || || ||  ? || || Data: SoundEntriesAdvanced.dbc, Sound — Not present in 6.0.1.18179<br />
|-<br />
| $DSE || — || DestroyEmitter || MapObj || || <br />
|-<br />
| $DSL || soundEntryId || DoodadSoundLoop (low priority) | GO || || <br />
|-<br />
| $DSO || soundEntryId || DoodadSoundOneShot || GO || || <br />
|-<br />
| $DTH || — || DeathThud + LootEffect || Unit || Death, Drown, Knockdown || """I'm dead now!"", UnitCombat_C, this plays death sounds and more." Note that this is NOT triggering [[CreatureSoundData.dbc|CreatureSoundDataRec]]::m_soundDeathID, but that is just always triggered as soon as the death animation plays.<br />
|-<br />
| $EAC || — || object package state enter 3, exit 2, 4, 5 || || || <br />
|-<br />
| $EDC || — || object package state enter 5, exit 3, 4, 2 || || || <br />
|-<br />
| $EMV || — || object package state enter 4, exit 3, 2, 5 || || || <br />
|-<br />
| $ESD || — || PlayEmoteStateSound || Unit || || soundEffect ID is implicit by currently played emote<br />
|-<br />
| $EWT || — || object package state enter 2, exit 3, 4, 5 || || || <br />
|-<br />
| $FD[1-9] || — || PlayFidgetSound || || || CreatureSoundDataRec::m_soundFidget (only has 5 entries, so don’t use 6-9)<br />
|-<br />
| $FDX || — || PlayUnitSound (stand) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundStandID<br />
|-<br />
| $FL[0-3] || — || FootstepAnimEventHit (left) || || || Forward<br />
|-<br />
| $FR[0-3] || — || FootstepAnimEventHit (right) || || || Forward<br />
|-<br />
| $FSD || — || HandleFootfallAnimEvent || Unit || Walk, Run (multiple times), ... || Plays some sound. Footstep? Also seen at several emotes etc. where feet are moved. CGUnit_C::HandleFootfallAnimEvent<br />
|-<br />
| $GC[0-3] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x + 6] ({Custom0, Custom1, Custom2, Custom3})<br />
|-<br />
| $GO[0-5] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x] ({Stand, Open, Loop, Close, Destroy, Opened})<br />
|-<br />
| $HIT || — || PlayWoundAnimKit || Unit || Attack*, *Unarmed, ShieldBash, Special* || soundEntryId depends on SpellVisualKit<br />
|-<br />
| $KVS || || || Â ? || || MapLoad.cpp -- not found in 6.0.1.18179<br />
|-<br />
| $RL[0-3] || — || FootstepAnimEventHit (left) || || || Running<br />
|-<br />
| $RR[0-3] || — || FootstepAnimEventHit (right) || || || Running<br />
|-<br />
| $SCD || — || PlaySoundKit (spellCastDirectedSound) || || || soundEffect ID is defined by CreatureSoundDataRec::m_spellCastDirectedSoundID<br />
|-<br />
| $SHK || spellEffectCameraShakesID || AddShake || GO || || <br />
|-<br />
| $SHL || — || ExchangeSheathedWeapon (left) || || Sheath, HipSheath || <br />
|-<br />
| $SHR || — || ExchangeSheathedWeapon (right) || || Sheath, HipSheath || <br />
|-<br />
| $SL[0-3] || — || FootstepAnimEventHit (left) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $SMD || — || PlaySoundKit (submerged) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergedSoundID<br />
|-<br />
| $SMG || — || PlaySoundKit (submerge) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergeSoundID<br />
|-<br />
| $SND || soundEntryId || PlaySoundKit (custom) || GO || || <br />
|-<br />
| $SR[0-3] || — || FootstepAnimEventHit (right) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $STx || || || Mounts || MountTransitionObject::UpdateCharacterData || Not seen in 6.0.1.18179 -- x is {E and B} , sequence time is taken of both, pivot of $STB. (Also, attachment info for attachment 0)<br />
|-<br />
| $TRD || — || HandleSpellEventSound || Unit || EmoteWork*, UseStanding* || soundEffect ID is implicit by SpellRec<br />
|-<br />
| $VG[0-8] || — || HandleBoneAnimGrabEvent || || || <br />
|-<br />
| $VT[0-8] || — || HandleBoneAnimThrowEvent || || || <br />
|-<br />
| $WGG || — || PlayUnitSound (wingGlide) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingGlideID<br />
|-<br />
| $WL[0-3] || — || FootstepAnimEventHit (left) || || || <br />
|-<br />
| $WNG || — || PlayUnitSound (wingFlap) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingFlapID<br />
|-<br />
| $WR[0-3] || — || FootstepAnimEventHit (right) || || || <br />
|-<br />
| $WTB || — || || Weapons || || Weapon Trail Bottom position, also used for Bow String<br />
|-<br />
| $WTT || — || || Weapons || || Weapon Trail Top position<br />
|-<br />
| $WWG || || || Â ? || || Calls some function in the Object VMT. -- Not seen in 6.0.1.18179<br />
|-<br />
| DEST || || || Â ? || || exploding ballista, that one has a really fucked up block. Oo<br />
|-<br />
| POIN || || || Unit || not fired || Data:Â ?, seen on multiple models. Basilisk for example. (6801)<br />
|-<br />
| WHEE || || || Â ? || || Data: 601+, Used on wheels at vehicles.<br />
|-<br />
| BOTT || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|-<br />
| TOP || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|}<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=M2&diff=26853M22019-06-01T08:36:14Z<p>Simca: /* M2ParticleOld */ Added a few comments from furl's notes.</p>
<hr />
<div>[[M2]] files (also called [[MDX]]) contain model objects. Each [[M2]] file describes the vertices, faces, materials, texture names, animations and properties of one model. [[M2]] files don't have a chunked format like most other WoW formats (except in Legion). Since it is chunked in Legion, all offsets are relative to beginning of the MD21 chunk's data rather than the beginning of the file. <br />
<br />
Models are used for doodads (decoration objects), players, monsters and really everything in the game except for [[ADT|Terrain]] and [[WMO]]s.<br />
<br />
==Header==<br />
The header has mostly the layout of number-offset pairs, containing the number of a particular record in the file, and the offset. These appear at fixed places in the header. Record sizes are not specified in the file.<br />
<br />
struct <br />
{<br />
// note: Offsets are for {{Template:Sandbox/VersionRange|min_expansionlevel=3}}!<br />
/*0x000*/ uint32_t magic; // "MD20". [[M2#Legion|Legion]] uses a chunked file format starting with MD21.<br />
/*0x004*/ uint32_t [[#Versions|version]];<br />
/*0x008*/ {{Template:Type/M2Array|char}} name; // should be globally unique, used to reload by name in internal clients<br />
<br />
/*0x010*/ struct<br />
{<br />
uint32_t flag_tilt_x : 1;<br />
uint32_t flag_tilt_y : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
uint32_t flag_use_texture_combiner_combos : 1; // add textureCombinerCombos array to end of data<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
uint32_t flag_load_phys_data : 1;<br />
uint32_t : 1;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=6}}<br />
uint32_t flag_unk_0x80 : 1; // with this flag unset, demon hunter tattoos stop glowing<br />
// since {{Template:Unverified|{{Template:Sandbox/PrettyVersion|expansionlevel=4|build=4.0.1.12911}}}} {{Template:Unverified|every}} model now has this flag<br />
uint32_t flag_camera_related : 1; // TODO: verify version<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=7}} // TODO: verify version, these are just added based on where I first saw them -- schlumpf.<br />
uint32_t flag_new_particle_record : 1; // In CATA: new version of ParticleEmitters. By default, length of M2ParticleOld is 476. <br />
// But if 0x200 is set or if version is bigger than 271, length of M2ParticleOld is 492.<br />
uint32_t flag_unk_0x400 : 1;<br />
uint32_t flag_texture_transforms_use_bone_sequences : 1; // {{Template:Sandbox/VersionRange|min_expansionlevel=6}} 0x800 -- When set, texture transforms are animated using the sequence being played on the bone found by index in tex_unit_lookup_table[textureTransformIndex], instead of using the sequence being played on the model's first bone. Example model: 6DU_HellfireRaid_FelSiege03_Creature<br />
uint32_t flag_unk_0x1000 : 1;<br />
uint32_t flag_unk_0x2000 : 1; // seen in various legion models<br />
uint32_t flag_unk_0x4000 : 1;<br />
uint32_t flag_unk_0x8000 : 1; // seen in UI_MainMenu_Legion<br />
uint32_t flag_unk_0x10000 : 1;<br />
uint32_t flag_unk_0x20000 : 1;<br />
uint32_t flag_unk_0x40000 : 1;<br />
uint32_t flag_unk_0x80000 : 1;<br />
uint32_t flag_unk_0x100000 : 1;<br />
uint32_t flag_unk_0x200000 : 1; // apparently: use 24500 upgraded model format: chunked .anim files, change in the exporter reordering sequence+bone blocks before name<br />
#endif<br />
#endif<br />
#endif<br />
#endif<br />
} global_flags;<br />
<br />
/*0x014*/ {{Template:Type/M2Array|M2Loop|section=Global_sequences}} global_loops; // Timestamps used in [[#Global_Sequences_2|global looping animations]].<br />
/*0x01C*/ {{Template:Type/M2Array|M2Sequence|section=Animation_sequences}} sequences; // Information about the animations in the model.<br />
/*0x024*/ {{Template:Type/M2Array|uint16_t|section=Animation_Lookup}} sequenceIdxHashById; // Mapping of [[#Animation_sequences|sequence IDs]] to the entries in the [[#Animation_sequences|Animation sequences]] block.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?|section=Playable_Animation_Lookup}} playable_animation_lookup;<br />
#endif<br />
/*0x02C*/ {{Template:Type/M2Array|M2CompBone|section=Bones}} bones; // MAX_BONES = 0x100 => Creature\SlimeGiant\GiantSlime.M2 has 312 bones ({{Template:Sandbox/PrettyVersion|expansionlevel=3}})<br />
/*0x034*/ {{Template:Type/M2Array|uint16_t|section=Key-Bone_Lookup}} boneIndicesById; //Lookup table for key skeletal bones. (alt. name: key_bone_lookup)<br />
/*0x03C*/ {{Template:Type/M2Array|M2Vertex|section=Vertices}} vertices;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|M2SkinProfile|section=Views_.28LOD.29}} skin_profiles;<br />
#else<br />
/*0x044*/ uint32_t num_skin_profiles; // [[#Views_.28LOD.29|Views (LOD)]] are now in [[M2/.skin|.skins]].<br />
#endif<br />
<br />
/*0x048*/ {{Template:Type/M2Array|M2Color|section=Colors}} colors; // Color and alpha animations definitions.<br />
/*0x050*/ {{Template:Type/M2Array|M2Texture|section=Textures}} textures;<br />
/*0x058*/ {{Template:Type/M2Array|M2TextureWeight|section=Transparency}} texture_weights; // Transparency of textures.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
{{Template:Type/M2Array|?}} unknown;<br />
#endif<br />
/*0x060*/ {{Template:Type/M2Array|M2TextureTransform|section=Texture_Transforms}} texture_transforms;<br />
/*0x068*/ {{Template:Type/M2Array|uint16_t|section=Replacable_texture_lookup}} textureIndicesById; // (alt. name: replacable_texture_lookup)<br />
/*0x070*/ {{Template:Type/M2Array|M2Material|section=Render_flags}} materials; // Blending modes / render flags.<br />
/*0x078*/ {{Template:Type/M2Array|uint16_t|section=Bone_Lookup_Table}} boneCombos; // (alt. name: bone_lookup_table)<br />
/*0x080*/ {{Template:Type/M2Array|uint16_t|section=Texture_lookup_table}} textureCombos; // (alt. name: texture_lookup_table)<br />
/*0x088*/ {{Template:Type/M2Array|uint16_t|section=Texture_unit_lookup_table}} textureTransformBoneMap; // (alt. name: tex_unit_lookup_table)<br />
/*0x090*/ {{Template:Type/M2Array|uint16_t|section=Transparency_lookup_table}} textureWeightCombos; // (alt. name: transparency_lookup_table)<br />
/*0x098*/ {{Template:Type/M2Array|uint16_t|section=Texture_Transforms_lookup_table}} textureTransformCombos; // (alt. name: texture_transforms_lookup_table)<br />
<br />
/*0x0A0*/ {{Template:Type|CAaBox}} bounding_box; // min/max( [1].z, 2.0277779f ) - 0.16f seems to be the maximum camera height<br />
/*0x0B8*/ float bounding_sphere_radius; // detail doodad draw dist = clamp (bounding_sphere_radius * detailDoodadDensityFade * detailDoodadDist, …)<br />
/*0x0BC*/ {{Template:Type|CAaBox}} collision_box;<br />
/*0x0D4*/ float collision_sphere_radius;<br />
<br />
/*0x0D8*/ {{Template:Type/M2Array|uint16_t|section=Triangles}} collisionIndices; // (alt. name: collision_triangles)<br />
/*0x0E0*/ {{Template:Type/M2Array|C3Vector|section=Vertices_2}} collisionPositions; // (alt. name: collision_vertices)<br />
/*0x0E8*/ {{Template:Type/M2Array|C3Vector|section=Normals}} collisionFaceNormals; // (alt. name: collision_normals) <br />
/*0x0F0*/ {{Template:Type/M2Array|M2Attachment|section=Attachments}} attachments; // position of equipped weapons or effects<br />
/*0x0F8*/ {{Template:Type/M2Array|uint16_t|section=Attachment_Lookup}} attachmentIndicesById; // (alt. name: attachment_lookup_table)<br />
/*0x100*/ {{Template:Type/M2Array|M2Event|section=Events}} events; // Used for playing sounds when dying and a lot else.<br />
/*0x108*/ {{Template:Type/M2Array|M2Light|section=Lights}} lights; // Lights are mainly used in loginscreens but in wands and some doodads too.<br />
/*0x110*/ {{Template:Type/M2Array|M2Camera|section=Cameras}} cameras; // The cameras are present in most models for having a model in the character tab. <br />
/*0x118*/ {{Template:Type/M2Array|uint16_t|section=Camera_lookup_table}} cameraIndicesById; // (alt. name: camera_lookup_table)<br />
/*0x120*/ {{Template:Type/M2Array|M2Ribbon|section=Ribbon_emitters}} ribbon_emitters; // Things swirling around. See the CoT-entrance for light-trails.<br />
/*0x128*/ {{Template:Type/M2Array|M2Particle|section=Particle_emitters}} particle_emitters;<br />
<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=2}} // TODO: verify version<br />
if (flag_use_texture_combiner_combos)<br />
{<br />
/*0x130*/ {{Template:Type/M2Array|uint16_t|section=Blend_mode_overrides}} textureCombinerCombos; // When set, textures blending is overriden by the associated array.<br />
}<br />
#endif<br />
} header;<br />
<br />
==Types==<br />
struct M2Bounds {<br />
CAaBox extent;<br />
float radius;<br />
};<br />
template<typename T><br />
struct M2Array {<br />
uint32_t size;<br />
uint32_t offset; // pointer to T, relative to begin of m2 data block (i.e. MD21 chunk content or begin of file)<br />
};<br />
struct M2TrackBase {<br />
uint16_t trackType;<br />
uint16_t loopIndex;<br />
M2Array<M2SequenceTimes> sequenceTimes;<br />
};<br />
template<typename T> <br />
struct M2PartTrack {<br />
M2Array<fixed16> times;<br />
M2Array<T> values;<br />
};<br />
template<typename T> <br />
struct M2SplineKey {<br />
T value;<br />
T inTan;<br />
T outTan;<br />
};<br />
struct M2Range {<br />
uint32_t minimum;<br />
uint32_t maximum;<br />
};<br />
<br />
==Versions==<br />
Files get handled differently depending on this! Ranges are inclusive.<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Version !! width="190" | Extension<br />
|-<br />
| 272-274 || Legion<br />
|-<br />
| 272 || Warlords of Draenor<br />
|-<br />
| 272 || Mists of Pandaria<br />
|-<br />
| 265-272 || Cataclysm<br />
|-<br />
| 264 || Wrath of the Lich King<br />
|-<br />
| 260-263 || The Burning Crusade<br />
|-<br />
| 256-257 || Classic<br />
|-<br />
| 256 || Pre-Release<br />
|}<br />
<br />
==Chunks==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
<br />
From Legion and up, the file might be [[Chunk|chunked]] instead. If this is the case, the magic will be anything but 'MD20' and the m2 data will be in the 'MD21' chunk. If the first magic is 'MD20', it will be loaded just fine like it did previously. Note that the chunks can be in any order with MD21 often being first. <br />
<br />
'''NOTE''': Unlike all other chunked formats in WoW, chunk names in M2 are '''NOT''' reversed. Example: AFID == AFID in file.<br />
<br />
===MD21===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740}}<br />
The MD21 chunk contains the not yet chunked data, just like in the old format. The content literally is an pre legion file, starting with the MD20 magic. This also implies that all offsets inside this chunk are relative to the ''chunk'', not the ''file''.<br />
M2Data pre_legion_style_data;<br />
<br />
===PFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}.phys</tt>}}<br />
uint32_t phys_file_id;<br />
<br />
===SFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${view}.skin</tt> and <tt>${basename}_lod${lodband}.skin</tt>}}<br />
uint32_t skinFileDataIDs[header.nViews];<br />
uint32_t lod_skinFileDataIDs[lodBands /* 2? */];<br />
Some model files, for example 'Creature\NightborneFemaleCitizen\NightborneFemaleCitizen.m2' have 4 skin files and 2 lod files but only 20 bytes are in chunk. In chunk there are 4 skins and 1 lod present.<br />
<br />
Lod skins are selected based on distance to entity/doodad and chosen based on GetCVar("entityLodDist")/X and GetCVar("doodadLodDist")/X where X - distance. Lods are ignored when "M2UseLOD" CVar is set to 0.<br />
<br />
===AFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}${anim_id}-${sub_anim_id}.anim</tt>}}<br />
struct<br />
{<br />
uint16_t anim_id;<br />
uint16_t sub_anim_id;<br />
uint32_t file_id; // might be 0 for "none" (so this is probably not sparse, even if it could be)<br />
} anim_file_ids[];<br />
<br />
===BFID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|note=This used to be filename based, using <tt>${basename}_${i}.bone</tt>}}<br />
uint32_t boneFileDataIDs[];<br />
<br />
===TXAC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=It is unknown what this replaced. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
char unk[2]; // likely used in CM2SceneRender::SetupTextureTransforms and uploaded to the shader directly. 0 otherwise.<br />
} texture_ac[m2data.header.materials.count + m2data.header.particles.count];<br />
<br />
===EXPT===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown, not the first one though}}}}<br />
struct {<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
} extended_particle[m2data.header.particles.count];<br />
<br />
Probably outdated chunk after introduction of EXP2 chunk. If EXP2 doesnt exist, client tries to reconstruct it with data from EXPT chunk.<br />
<br />
===EXP2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to partially replace M2ParticleOld's variables. {{Unverified|Exact build unknown}}}}<br />
<br />
<source><br />
struct EXP2<br />
{<br />
M2Array<Exp2Record> content;<br />
};<br />
<br />
struct Exp2Record<br />
{<br />
_DWORD zSource;<br />
_DWORD unk1;<br />
_DWORD unk2;<br />
M2PartTrack<fixed16> unk3;<br />
};<br />
</source><br />
<br />
<br />
The length of this M2Array is the same as length of particle_emitters<br />
<br />
===PABC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Appears to replace {{Unverified|[[#Animation_Lookup]]}} from old file. {{Unverified|Exact build unknown}}}}<br />
M2Array<uint16t> m_replacementParentSequenceLookups; // only seen in quillboarbrute*.m2. Contains AnimationIds<br />
<br />
Replaces parentSequencesLookups. But unlike header.sequence_lookups of parent model, this is straight array and not a map. If index with target animation is not found in here, parentSequencesLookups are used instead.<br />
<br />
This chunk called BlacklistAnimData in client.<br />
<br />
Client doesnt seem to use found index and thus whole array is used only to check if the target animation is present.<br />
<br />
===PADC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=Moves texture weights from old file to a chunk. {{Unverified|Exact build unknown}}}}<br />
Defines replacement for header.texture_weights (WHY?)<br />
<br />
<source lang="cpp"><br />
struct PADC {<br />
M2Array<M2TextureWeight> texture_weights;<br />
} <br />
</source><br />
<br />
===PSBC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
<br />
Defines ParentSequenceBounds<br />
M2Array<M2Bounds> parentSequenceBounds;<br />
<br />
===PEDC===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note={{Unverified|Exact build unknown}}}}<br />
M2Array<M2TrackBase> parentEventData;<br />
<br />
===SKID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.???|note=This used to be filename based, using <tt>${basename}.skel</tt>. {{Unverified|Exact build unknown}}}}<br />
uint32_t SKeletonfileID; // links to [[M2/.skel]]<br />
<br />
===TXID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629|note=Replaces the filename for {{#Textures}} with hardcoded type.}}<br />
Replaces in-file texture filenames. <br />
struct {<br />
uint32_t fileDataID;<br />
} textureID[]<br />
<br />
===LDV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26629}}<br />
<br />
Defines LodData<br />
<br />
<source lang="cpp"><br />
struct LodData<br />
{<br />
uint16 unk0; <br />
uint16 lodCount; //maxLod = lodCount-1; <br />
float unk2_f;<br />
uint8_t particleBoneLod[4]; //lod serves as indes into this array<br />
_DWORD unk4;<br />
};<br />
</source><br />
<br />
Somehow defines _lod%0d.skin files. On pandarenfemale.m2, lodCount == 4. SFID has 7 files first 4 are ordinary .skin files and last 3 are _lod%0d.skin files. Enumeration for _lod%0d.skin files for that model starts from 1, and last file in SFID is pandarenfemale_lod03.skin So technically maxLod indeed represents maximum Lod<br />
<br />
unk2_f is used in formula, but it's purpose is unknown<br />
fmaxf(fminf(740.0 / unk2_f, 5.0), 0.5);<br />
<br />
LodData.particleBoneLod works this way: Each model has current lod which is [0..3]. Next:<br />
<source lang="cpp"><br />
if ( lod < 1 )<br />
result = 0;<br />
<br />
if ( LodData)<br />
result = (0x10000 << LodData->particleBoneLod[lod]);<br />
else<br />
result = (0x10000 << (lod- 1));<br />
<br />
...<br />
//For each ParticleEmitter and related M2Particle record<br />
if ( result & M2CompBone[M2Particle->old.boneIndex].flags ) {<br />
//Do not animate this emitter<br />
}<br />
</source><br />
<br />
<br />
[[User:Deamon|Deamon]] ([[User talk:Deamon|talk]])<br />
<br />
===RPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>recursion_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} recursive_particle_models[particle count];<br />
<br />
===GPID===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.27826|note=Replaces [[#M2ParticleOld]]'s <tt>geometry_model_filename</tt>}}<br />
<br />
struct {<br />
uint32_t fileDataID;<br />
} geometry_particle_models[particle count];<br />
<br />
===WFV1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV1 {<br />
// unknown<br />
};<br />
<br />
===WFV2===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.2.0.30080}}<br />
struct WFV2 {<br />
// unknown<br />
};<br />
<br />
===PGD1===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=1|min_build=1.13.2.30172}}<br />
struct PGD1Entry {<br />
char _0x00[2];<br />
};<br />
M2Array<PGD1Entry> p_g_d_v1; // count appears to be equivalent to particle count<br />
<br />
=Skeleton and animation=<br />
==Global sequences==<br />
A list of timestamps that act as upper limits for global sequence ranges.<br />
struct M2Loop<br />
{<br />
uint32_t timestamp;<br />
} loops[];<br />
<br />
==Standard animation block==<br />
* {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} uses a single-timeline approach, chaining all animations into one long piece and separating them via begin and end given in animation data. {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, each animation has an own timeline.<br />
* Animation blocks contain a list of lists of timestamps and a list of lists of values, where the first list is by animation and the second one by timestamp-entry.<br />
*'''Many values that change with time are specified using blocks like the following.'''<br />
<br />
template<typename T><br />
struct M2Array<br />
{<br />
/*0x00*/ uint32_t number;<br />
/*0x04*/ uint32_t offset_elements;<br />
/*0x08*/<br />
};<br />
<br />
struct M2TrackBase<br />
{<br />
/*0x00*/ uint16_t interpolation_type;<br />
/*0x02*/ uint16_t global_sequence;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}}<br />
M2Array<pair<uint32_t>> interpolation_ranges; // no longer required {{Template:Sandbox/VersionRange|min_expansionlevel=3}}, as implicit by minimum and maximum timestamp per sequence.<br />
M2Array<uint32_t> timestamps;<br />
#else<br />
/*0x04*/ M2Array<M2Array<uint32_t>> timestamps;<br />
#endif<br />
/*0x0C*/<br />
};<br />
<br />
template<typename T><br />
struct M2Track : M2TrackBase<br />
{<br />
/*0x00*/ // base <br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=3|max_exclusive=1}} <br />
M2Array<T> values;<br />
#else<br />
/*0x0C*/ M2Array<M2Array<T>> values;<br />
#endif<br />
/*0x14*/<br />
};<br />
<br />
* Thus, as example, with<br />
<br />
{{Template:Type|M2CompBone|link=M2#Bones}} b;<br />
<br />
one may get the number of animations having translation information with<br />
<br />
b.translation.timestamps.number<br />
<br />
and the number of timestamps in the first animation using<br />
<br />
b.translation.timestamps.elements[0].number<br />
<br />
and the first timestamp value of the first animation via<br />
<br />
b.translation.timestamps.elements[0].elements[0]<br />
<br />
The actual translation vector for animation 0 at timestamp 0 is at<br />
<br />
b.translation.values.elements[0].elements[0]<br />
<br />
* Some timestamps/values.elements entries may have number/elements = 0, if for that animation id no animation is given.<br />
* [[#.anim_files|.anim]] files are just a blob of data which may as well be in the main model file, that is pointed to by the first array_ref layer.<br />
* [model file name][animation id]-[animation sub-id][[#.anim_files|.anim]]<br />
* it seems like it is possible to detect if animation data is stored in-m2 or externally via<br />
** All animations which have flags & 0x20 are stored internally. <br />
** Animations which do not have flags & 0x20 are not stored internally. <br />
** Animations which do not have flags & 0x20 AND do not have flags & 0x40 are in [[#.anim_files|.anim]] files<br />
** Animations which do not have flags & 0x20 AND DO have flags 0x40 are stored... somewhere. I have no clue.<br />
<br />
===Global Sequences===<br />
If a block has a sequence >= 0, the block has a completely separate max timestamp. This is the value in the model's ofsGlobalSequences table; index into that table with this sequence value and use that as the block's max timestamp. Blocks that use these global sequences also only have one track, so at the same time as clipping the current timestamp to the max time above, interpolated value should always be taken from track 0 in the block. <br />
<br />
A global sequence is completely unrelated to animations. It just always loops. This way, the sequence is not interrupted when an animation is launched.<br />
<br />
This appears to be frequently used by models that don't have more conventional animations (login screen animations, items/weapons with animated effects, etc).<br />
<br />
-- Rour, additionally, these sequences can be longer or shorter than whatever animation is running for a given model, so I recommend taking a global scene timestamp and then clipping that value into the given max timestamp range. Otherwise animations will appear to reset when the regular animation loops, which is not good.<br />
<br />
===Interpolation===<br />
* If the interpolation type is 0, then values will change instantly at the timestamp, with no interpolation whatsoever.<br />
* If the interpolation type is 1, then the block linearly interpolates between keyframe values (lerp for vectors/colours, nlerp for quaternions).<br />
* If the interpolation type is 2, then cubic bezier spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the first control point is the first spline key's <code>value</code>, the second control point is the first spline key's <code>tanOut</code>, the third control point is the second spline key's <code>tanIn</code>, and the fourth control point is the second spline key's <code>value</code>.<br />
* If the interpolation type is 3, then cubic hermite spline interpolation is used. This is only valid for M2SplineKey tracks. When interpolating between two spline keys, the starting point is the first spline key's <code>value</code>, the starting tangent is the first spline key's <code>tanOut</code>, the ending tangent is the second spline key's <code>tanIn</code>, and the ending point is the second spline key's <code>value</code>.<br />
<br />
'''NOTE:''' There is confusion about type <code>2</code> and <code>3</code> being <code>hermite/bezier</code> or <code>bezier/hermite</code>. Alpha says that <code>2 = hermite</code>, WoD says that <code>2 = bezier</code>. This was changed when the format went from MDL to M2. --[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 01:53, 4 September 2017 (CEST), --[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) 19:15, 14 November 2018 (GMT)<br />
<br />
In WotLK 2 appears to be bezier, which pretty much confirms it being bezier for wotlk+. TBC and classic need further checking though. -- [[User:Skarn|Skarn]] ([[User talk:Skarn|talk]]) 01:38, 16 June 2018 (CEST)<br />
<br />
==.anim files==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=3}}<br />
<br />
Low priority sequences (e.g. emotes, one shot animations) are in extra files to allow for lazy loading. These files are raw data of timestamps and values for animation blocks. Blizzard's exporter prefers to align blocks to a 16 byte boundary, but that's not required.<br />
<br />
'''The client loads .anim files if (M2Sequence.flags & 0x130 ) == 0.''' The .anim file to use is <code>"%s%04d-%02d.anim" % (model_filename_without_extension, anim.id, anim.sub_anim_id)</code>.<br />
<br />
===Legion 24500===<br />
In Legion, these files are optionally chunked now. They are chunked either<br />
* if M2 header's 0x200000 flag is set and thus the new mid expansion format change is used<br />
* if the M2 has a .skel file<br />
<br />
For new format M2s, .anim is pretty much unchanged except that there is the AFM2 chunk header. The AFSA and AFSB chunks do not appear in that case. If it is a .skel file based model, the chunks are present and animation data is split into bone and attach data. The AFM2 chunk then contains the animation data for ????, the AFSA chunk that for attachments and the AFSB chunk that for bones. See .skel files for that.<br />
<br />
====AFM2====<br />
The same content as an old anim file would have. In fact, files that were just converted to the new format are bit identical except for the chunk header.<br />
====AFSA====<br />
skeleton data for attachments<br />
====AFSB====<br />
skeleton data for bones<br />
<br />
==Animation sequences==<br />
List of animations present in the model. <br />
struct M2Sequence<br />
{<br />
uint16_t id; // Animation id in [[AnimationData.dbc]]<br />
uint16_t variationIndex; // Sub-animation id: Which number in a row of animations this one is.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=2}}<br />
uint32_t start_timestamp;<br />
uint32_t end_timestamp;<br />
#else<br />
uint32_t duration; // The length of this animation sequence in milliseconds.<br />
#endif<br />
float movespeed; // This is the speed the character moves with in this animation.<br />
uint32_t flags; // See below.<br />
int16_t frequency; // This is used to determine how often the animation is played. For all animations of the same type, this adds up to 0x7FFF (32767).<br />
uint16_t _padding;<br />
M2Range replay; // May both be 0 to not repeat. Client will pick a random number of repetitions within bounds if given.<br />
#if version < ??? < 6.0.1<br />
uint32_t blendTime;<br />
#else<br />
uint16_t blendTimeIn; // The client blends (lerp) animation states between animations where the end and start values differ. This specifies how long that blending takes. Values: 0, 50, 100, 150, 200, 250, 300, 350, 500.<br />
uint16_t blendTimeOut; // The client blends between this sequence and the next sequence for blendTimeOut milliseconds.<br />
#endif<br />
// For both blendTimeIn and blendTimeOut, the client plays both sequences simultaneously while interpolating between their animation transforms.<br />
{{Template:Type|M2Bounds}} bounds;<br />
int16_t variationNext; // id of the following animation of this AnimationID, points to an Index or is -1 if none.<br />
uint16_t aliasNext; // id in the list of animations. Used to find actual animation if this sequence is an alias (flags & 0x40)<br />
} sequences[];<br />
<br />
--[[User:Koward|Koward]] ([[User talk:Koward|talk]]) 09:50, 18 December 2015 (UTC) In M2 v274 (Legion), it looks like blend_time has been divided in two uint16_t, and for standard animations the old blend_time is duplicated in both fields (ex : uint32 150 becomes two uint16 150). Maybe start and end blend_time values ? See Creature/GennGreymane/GennGreymane.m2 .<br />
===Flags===<br />
''One thing I saw in the source is that "-1 animationblocks" in bones wont get parsed if 0x20 is not set.''<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="500" | Description<br />
|-<br />
| 0x01 || Sets 0x80 when loaded. (M2Init)<br />
|-<br />
| 0x02 ||<br />
|-<br />
| 0x04 ||<br />
|-<br />
| 0x08 ||<br />
|-<br />
| 0x10 || apparently set during runtime in CM2Shared::LoadLowPrioritySequence for all entries of a loaded sequence (including aliases)<br />
|-<br />
| 0x20 || primary bone sequence -- If set, the animation data is in the .m2 file. If not set, the animation data is in an .anim file. {{Template:Unverified|Was named 'looped animation' by schlumpf years ago, without source.}}<br />
|-<br />
| 0x40 || has next / is alias (To find the animation data, the client skips these by following aliasNext until an animation without 0x40 is found.)<br />
|-<br />
| 0x80 || Blended animation (if either side of a transition has 0x80, lerp between end->start states, unless end==start by comparing bone values)<br />
|-<br />
| 0x100 || sequence stored in model ?<br />
|-<br />
| 0x200 ||<br />
|-<br />
| 0x400 ||<br />
|-<br />
| 0x800 || seen in Legion 24500 models<br />
|}<br />
<br />
-- Rour, some animations rely on blending to look right. The MoP mage CM shoulders only animate half of their movement and rely on lerping back to the start position to look correct.<br />
<br />
=== Animation Lookup===<br />
Hash table for Animations in [[AnimationData.dbc]].<br />
<br />
struct<br />
{<br />
uint16_t animation_index; // Index at [[#Animation_sequences|ofsAnimations]] which represents the animation in [[AnimationData.dbc]]. -1 if none.<br />
} animation_lookups[];<br />
<br />
The hash used is <tt>anim_id % num_buckets</tt>. If a bucket is used, a stride of <tt>n^2</tt> is added with <tt>n = 1, 2, …</tt> until the entry is matching:<br />
<br />
<source lang="c++"><br />
M2Sequence* find_entry (uint32_t anim_id)<br />
{<br />
size_t i (anim_id % animation_lookups.count);<br />
<br />
for (size_t stride (1); true; ++stride)<br />
{<br />
if (animation_lookups[i] == -1)<br />
{<br />
return nullptr;<br />
}<br />
if (animation_sequences[animation_lookups[i]].id == anim_id)<br />
{<br />
return &animation_sequences[i];<br />
}<br />
<br />
i = (i + stride * stride) % animation_lookups.count;<br />
// so original_i + 1, original_i + 1 + 4, original_i + 1 + 4 + 9, …<br />
}<br />
<br />
[[unreachable]];<br />
}<br />
</source><br />
<br />
The entry referenced is the first in the `nextAlias` chain of a given animation id. Thus, <tt>num_buckets < num_animations</tt>, even if a model would have all animations multiple times.<br />
<br />
=== Playable Animation Lookup ===<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=2|note=Partially inlined into M2Sequences}}<br />
<br />
Lookup table for Playable Animation in [[AnimationData.dbc]]<br />
<br />
'''Offset Type Description'''<br />
0x00 int16 Fallback Animation ID in [[AnimationData.dbc]]<br />
0x02 int16 Flags (0, 1, 3 seen)<br />
<br />
Models don't present all animations sequences. This table maps from global animation list to available animation for the current model. The engine uses it to know which animation clip to play. That's why there are a lot of zeros ("Stand") for creatures.<br />
<br />
Flags are used to modify how the clip should be played:<br />
'''Value Meaning'''<br />
0 Play normal<br />
1 Play backwards?<br />
3 Freeze<br />
<br />
For instance, the "HumanMale/HumanMale.m2" model has just one "Loot" animation sequence. "LootHold" and "LootUp" are obtained with flags 3 and 1.<br />
<br />
== Bones ==<br />
struct M2CompBone // probably M2Bone {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
{<br />
int32_t key_bone_id; // Back-reference to [[#Key-Bone_Lookup|the key bone lookup table]]. -1 if this is no key bone.<br />
enum<br />
{<br />
spherical_billboard = 0x8,<br />
cylindrical_billboard_lock_x = 0x10,<br />
cylindrical_billboard_lock_y = 0x20,<br />
cylindrical_billboard_lock_z = 0x40,<br />
transformed = 0x200,<br />
kinematic_bone = 0x400, // MoP+: allow [[PHYS|physics]] to influence this bone<br />
helmet_anim_scaled = 0x1000, // set blend_modificator to [[HelmetAnimScaling.dbc|helmetAnimScalingRec]].m_amount for this bone<br />
};<br />
uint32_t flags; <br />
int16_t parent_bone; // Parent bone ID or -1 if there is none.<br />
uint16_t [[M2/.skin#Submeshes|submesh_id]]; // Mesh part ID OR uDistToParent?<br />
union { // only {{Template:Sandbox/VersionRange|min_expansionlevel=2}}?<br />
struct {<br />
uint16_t uDistToFurthDesc;<br />
uint16_t uZRatioOfChain;<br />
} CompressData; // {{Template:Unverified|No model has ever had this part of the union used.}}<br />
uint32_t boneNameCRC; // these are for debugging only. their bone names match those in key bone lookup.<br />
};<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=1}}<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation;<br />
#else<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|M2CompQuat|link=Quaternion_values_and_2.x}}> rotation; // compressed values, default is (32767,32767,32767,65535) == (0,0,0,1) == identity<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scale;<br />
{{Template:Type|C3Vector}} pivot; // The pivot point of that bone.<br />
} bones[];<br />
The bone indices in the vertex definitions seem to index into this data.<br />
<br />
===Billboards===<br />
The billboarding bits are used for various things:<br />
* Light halos around lamps must always face the viewer<br />
* The cannonball stack model (in the Deadmines or Booty Bay), where each cannonball is a crude hemisphere, they always face the viewer to create the illusion of actual cannonballs.<br />
<br />
Spherical and cylindrical billboard bits are mutually exclusive. Only one of them can be used for the bone.<br />
<br />
===Bone Lookup Table===<br />
Lookup table for bones referenced from [[M2/.skin#Submeshes|M2SkinSection]]. <br />
struct <br />
{<br />
uint16_t bone;<br />
} bone_lookup[];<br />
<br />
===Key-Bone Lookup===<br />
Its a lookup table for key skeletal bones like hands, arms, legs, etc. nKeyBoneLookup is 27 for the most models. At static models it is mostly 1.<br />
struct <br />
{<br />
uint16_t bone; // -1 if none<br />
} key_bone_lookup[];<br />
<br />
Official list:<br />
*00 "ArmL"<br />
*01 "ArmR"<br />
*02 "ShoulderL"<br />
*03 "ShoulderR"<br />
*04 "SpineLow"<br />
*05 "Waist"<br />
*06 "Head"<br />
*07 "Jaw"<br />
*08 "IndexFingerR"<br />
*09 "MiddleFingerR"<br />
*10 "PinkyFingerR"<br />
*11 "RingFingerR"<br />
*12 "ThumbR"<br />
*13 "IndexFingerL"<br />
*14 "MiddleFingerL"<br />
*15 "PinkyFingerL"<br />
*16 "RingFingerL"<br />
*17 "ThumbL"<br />
*18 "$BTH"<br />
*19 "$CSR"<br />
*20 "$CSL"<br />
*21 "_Breath"<br />
*22 "_Name"<br />
*23 "_NameMount"<br />
*24 "$CHD"<br />
*25 "$CCH"<br />
*26 "Root"<br />
*27 "Wheel1" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*28 "Wheel2" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*29 "Wheel3" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*30 "Wheel4" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*31 "Wheel5" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*32 "Wheel6" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*33 "Wheel7" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
*34 "Wheel8" {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
<br />
=Geometry and rendering=<br />
==Vertices==<br />
struct M2Vertex<br />
{<br />
{{Template:Type|C3Vector}} pos;<br />
uint8 bone_weights[4];<br />
uint8 bone_indices[4];<br />
{{Template:Type|C3Vector}} normal;<br />
{{Template:Type|C2Vector}} tex_coords[2]; // two textures, depending on shader used<br />
};<br />
<br />
Models, too, use a Z-up coordinate systems, so in order to convert to Y-up, the X, Y, Z values become (X, -Z, Y).<br />
<br />
-- Rour, the WoW vertex shaders all follow the same pattern, "Diffuse_XX_YY" (or sometimes XX, YY and Env). The particular vertex shader that is used chooses which set of texture coordinates to use. So Diffuse_T1 sends T1 texcoords to the fragment shader. Where Diffuse_T1_T2 sends both (for textures 0 and 1) but Diffuse_T1_T1 sends the same coords for both textures. Etc.<br />
<br />
==Views (LOD)==<br />
Skin profiles describe LOD views onto the model. They use all those lookup tables to be able to reference only parts of the lists while not being dynamically sized within the profile data.<br />
<br />
{{Template:Sandbox/VersionRange|max_expansionlevel=2}} they were stored in the [[M2]] itself, {{Template:Sandbox/VersionRange|min_expansionlevel=3}} they have been moved to [[M2/.skin|.skin]] files. The offsets are relative to the file the skin profile header is defined in. There is one [[M2/.skin|.skin]] file per profile, each with a separate header, while in the inlined version, all headers are sequential. See the [[M2/.skin|.skin]] file page for formats of both versions.<br />
<br />
==Render flags==<br />
struct M2Material<br />
{<br />
uint16_t flags;<br />
uint16_t blending_mode; // apparently a bitfield<br />
} materials[];<br />
*'''Flags:'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Flag !! width="1000" | Meaning<br />
|-<br />
| 0x01 || Unlit<br />
|-<br />
| 0x02 || Unfogged<br />
|-<br />
| 0x04 || Two-sided (no backface culling if set)<br />
|-<br />
| 0x08 || depthTest<br />
|-<br />
| 0x10 || depthWrite<br />
|-<br />
| 0x40 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x80 || shadow batch related ??? (seen in WoD)<br />
|-<br />
| 0x400 || ??? (seen in WoD)<br />
|-<br />
| 0x800 || prevent alpha for custom elements. if set, use (fully) opaque or transparent. (litSphere, shadowMonk) (MoP+)<br />
|}<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! source !! dest !! notes<br />
|-<br />
| 0 || GL_ONE || GL_ZERO || blending disabled (GxBlendStateDesc: 0)<br />
|-<br />
| 1 || GL_ONE || GL_ZERO || Mod (1)<br />
|-<br />
| 2 || GL_SRC_ALPHA || GL_ONE_MINUS_SRC_ALPHA || Decal (2)<br />
|-<br />
| 3 || GL_ONE || GL_ONE || Add (10)<br />
|-<br />
| 4 || GL_SRC_ALPHA || GL_ONE || Mod2x (3)<br />
|-<br />
| 5 || GL_DST_COLOR || GL_ZERO || Fade (4)<br />
|-<br />
| 6 || GL_DST_COLOR || GL_SRC_COLOR || Deeprun Tram glass (5)<br />
|-<br />
| 7 || GL_ONE || GL_ONE_MINUS_SRC_ALPHA || WoD+ (13)<br />
|}<br />
<br />
(blend equation is always GL_FUNC_ADD. Values are retrieved via GxBlendStateDesc's lower 5 bits. no separate blend func for alpha.)<br />
*'''Blending mode'''<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="100" | Mapped to !! width="1500" | Meaning<br />
|-<br />
| 0 || 0 || Combiners_Opaque (Blend disabled)<br />
|-<br />
| 1 || 1 || Combiners_Mod (Blend enabled, Src = ONE, Dest = ZERO, SrcAlpha = ONE, DestAlpha = ZERO)<br />
|-<br />
| 2 || 1 || Combiners_Decal (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 3 || 1 || Combiners_Add (Blend enabled, Src = SRC_COLOR, Dest = DEST_COLOR, SrcAlpha = SRC_ALPHA, DestAlpha = DEST_ALPHA )<br />
|-<br />
| 4 || 1 || Combiners_Mod2x (Blend enabled, Src = SRC_ALPHA, Dest = ONE, SrcAlpha = SRC_ALPHA, DestAlpha = ONE )<br />
|-<br />
| 5 || 4 || Combiners_Fade (Blend enabled, Src = SRC_ALPHA, Dest = INV_SRC_ALPHA, SrcAlpha = SRC_ALPHA, DestAlpha = INV_SRC_ALPHA )<br />
|-<br />
| 6 || 4 || Used in the Deeprun Tram subway glass, supposedly (Blend enabled, Src = DEST_COLOR, Dest = SRC_COLOR, SrcAlpha = DEST_ALPHA, DestAlpha = SRC_ALPHA )<br />
|-<br />
| 7 ||? || New in WoD, needs research! ''Example model: World\Expansion05\Doodads\Shadowmoon\Doodads\6FX_Fire_Grassline_Doodad_blue_LARGE.m2''<br />
|}<br />
''*Blend values are taken from D3D11 debugging of the client<br />
===Blend mode overrides===<br />
If this block is present (globalflags&8) and the "shading" flags of a textureunit wont be &0x8000, blending modes wont get mapped to the values above but to the ones in this block.<br />
<br />
Instead of Mapping[renderflags->blendingmode] it will be UnknownBlock[textureunit->Shading].<br />
<br />
As shading is not &0x8000 and (in their code) needs to be above 0, this may only touch Diffuse_T1.<br />
<br />
According to wod, if the M2 Header has flag 0x08, instead of reading blend mode from M2 [[#Render_flags|RenderFlags]], blendMode is read from the raw blend maps referenced in header.<br />
<br />
var flags = renderFlags[texUnit.renderFlags];<br />
var blendMode = flags >> 16;<br />
if ((header.GlobalModelFlags & 0x08) != 0 && texUnit.shader_id < mBlendMap.Length)<br />
blendMode = mBlendMap[texUnit.shader_id];<br />
<br />
==Texture unit lookup table==<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=4|max_exclusive=1|note=Still present but unused in Cataclysm}}<br />
struct <br />
{<br />
uint16_t unit; // -1, 0, or 1. see below<br />
} texture_units[];<br />
<br />
For models that use multitexturing, this maps given texture unit numbers into actual texture unit numbers (0 or 1).<br />
<br />
Values of -1 seem to mean environment mapping.<br />
<br />
One model is of special interest, Creature/KelThuzad/KelThuzad.m2, which is the only one that has an nTexUnits of 3, and has three texture units specified for some of its submeshes. Sure enough, two of those map to 0 and 1, and one maps to -1.<br />
<br />
More confusion thanks to my favorite "weird" model, World/Generic/Gnome/Passive Doodads/GnomeMachine/GnomeSubwayGlass.m2, which is the translucent, environment mapped glass tunnel in the Deeprun Tram. It only has a single value in this block, -1, which is used for the single texture layer in both render operations in the model. This and the magic with rendering flags/blend modes make up the neat transparent-reflective glass effect, but confuse me even more about how envmapping and such is handled. (and where it seems to get the bluish color from - is it in the model (no color blocks in this particular model), the wmo, a solid background color, or simply the result of the blending used?)<br />
<br />
As a side note, on my (dated) system WoW does every texture unit in a single pass.<br />
<br />
==Colors and transparency==<br />
===Colors===<br />
struct M2Color<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> color; // vertex colors in rgb order<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alpha; // 0 - transparent, 0x7FFF - opaque. Normaly NonInterp<br />
} colors[];<br />
<br />
This block is the M2 equivalent to the GEOA chunk in MDX files, it represents the vertex color and visibility animations for meshes. Referenced from the Texture Unit blocks in the [[M2/.skin|*.skin]]. If a texunit belonging to a submesh has a value of -1 then the submesh doesnot use this block. Contains a separate timeline for transparency values. If no animation is used, the given value is constant.<br />
<br />
===Transparency===<br />
struct M2TextureWeight<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> weight;<br />
} textureWeights[];<br />
<br />
Specifies global transparency values in addition to the values given in the Color block. I assume these are multiplied together eventually.<br />
====Transparency lookup table====<br />
struct <br />
{<br />
uint16_t transparency;<br />
} transparency_lookup[];<br />
Contains indices into the Transparency block. Used by the texture unit definitions in the LOD block.<br />
<br />
==Textures==<br />
*'''Textures are defined globally in a list''', additionally, a lookup table is given, referenced during rendering, to select textures.<br />
struct M2Texture<br />
{<br />
uint32_t type; // see below<br />
uint32_t flags; // see below<br />
M2Array<char> filename; // for non-hardcoded textures (type != 0), this still points to a zero-sized string<br />
} textures[];<br />
<br />
====Texture Types====<br />
Texture type is 0 for regular textures, nonzero for skinned textures (filename not referenced in the M2 file!) For instance, in the NightElfFemale model, her eye glow is a type 0 texture and has a file name, the other 3 textures have types of 1, 2 and 6. The texture filenames for these come from client database files:<br />
*DBFilesClient\[[CharSections.dbc]]<br />
*DBFilesClient\[[CreatureDisplayInfo.dbc]]<br />
*DBFilesClient\[[ItemDisplayInfo.dbc]]<br />
*(possibly more)<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="800" | Meaning<br />
|-<br />
| 0 || - NONE - -- Texture given in filename<br />
|-<br />
| 1 || TEX_COMPONENT_SKIN -- Skin -- Body + clothes<br />
|-<br />
| 2 || TEX_COMPONENT_OBJECT_SKIN -- Object Skin -- Item, Capes ("Item\ObjectComponents\Cape\*.blp")<br />
|-<br />
| 3 || TEX_COMPONENT_WEAPON_BLADE -- Weapon Blade -- Used on several models but not used in the client as far as I see. Armor Reflect?<br />
|-<br />
| 4 || TEX_COMPONENT_WEAPON_HANDLE -- Weapon Handle<br />
|-<br />
| 5 || TEX_COMPONENT_ENVIRONMENT -- (OBSOLETE) Environment (Please remove from source art)<br />
|-<br />
| 6 || TEX_COMPONENT_CHAR_HAIR -- Character Hair<br />
|-<br />
| 7 || TEX_COMPONENT_CHAR_FACIAL_HAIR -- (OBSOLETE) Character Facial Hair (Please remove from source art)<br />
|-<br />
| 8 || TEX_COMPONENT_SKIN_EXTRA -- Skin Extra<br />
|-<br />
| 9 || TEX_COMPONENT_UI_SKIN -- UI Skin -- Used on inventory art M2s (1): inventoryartgeometry.m2 and inventoryartgeometryold.m2<br />
|-<br />
| 10 || TEX_COMPONENT_TAUREN_MANE -- (OBSOLETE) Tauren Mane (Please remove from source art) -- Only used in quillboarpinata.m2. I can't even find something referencing that file. Oo Is it used?<br />
|-<br />
| 11 || TEX_COMPONENT_MONSTER_1 -- Monster Skin 1 -- Skin for creatures or gameobjects #1<br />
|-<br />
| 12 || TEX_COMPONENT_MONSTER_2 -- Monster Skin 2 -- Skin for creatures or gameobjects #2<br />
|-<br />
| 13 || TEX_COMPONENT_MONSTER_3 -- Monster Skin 3 -- Skin for creatures or gameobjects #3<br />
|-<br />
| 14 || TEX_COMPONENT_ITEM_ICON -- Item Icon -- Used on inventory art M2s (2): ui-button.m2 and forcedbackpackitem.m2 (CSimpleModel_ReplaceIconTexture("texture"))<br />
|-<br />
| 15 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Background Color<br />
|-<br />
| 16 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem Color<br />
|-<br />
| 17 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Border Color<br />
|-<br />
| 18 || {{Template:Sandbox/VersionRange|min_expansionlevel=4}} Guild Emblem<br />
|}<br />
<br />
====Flags====<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Meaning<br />
|-<br />
| 1 || Texture wrap X<br />
|-<br />
| 2 || Texture wrap Y<br />
|}<br />
<br />
====Texture lookup table====<br />
struct <br />
{<br />
uint16_t texture;<br />
} texture_lookup[];<br />
<br />
====Replacable texture lookup====<br />
struct <br />
{<br />
uint16_t replacement;<br />
} texture_replacements[];<br />
<br />
A reverse lookup table for 'replaced' textures, mapping replacable ids to texture indices or -1. Only goes up to the maximum id used in the model.<br />
<br />
Its strange, that HARDCODED is in the list, as a model can have more than one of course. Its just the last one written to the file.<br />
<br />
=Effects=<br />
==Texture Transforms==<br />
*'''This block contains definitions for texture animations,''' for example, flowing water or lava in some models. The keyframe values are used in the texture transform matrix.<br />
<br />
struct M2TextureTransform<br />
{<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> translation;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C4Quaternion}}> rotation; // rotation center is texture center (0.5, 0.5)<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> scaling;<br />
} textureTransforms[];<br />
<br />
Seems like UV rotation in some models are made against (0.5, 0.5) point instead of (0, 0). At least it's the case for world\goober\g_scourgerunecirclecrystal.m2 <br />
<br />
So to get the proper UV rotation it would be necessary apply rotation this way:<br />
<br />
* Translate UV anim matrix to point (0.5, 0.5)<br />
* Apply rotation mat from quaternion<br />
* UV anim matrix to point (-0.5, -0.5)<br />
<br />
====Texture Transforms lookup table ====<br />
struct <br />
{<br />
uint16_t anim_texture_id; // -1 for static<br />
} anim_texture_lookup[];<br />
<br />
== Ribbon emitters ==<br />
struct M2Ribbon<br />
{<br />
uint32_t ribbonId; // Always (as I have seen): -1.<br />
uint32_t boneIndex; // A bone to attach to.<br />
{{Template:Type|C3Vector}} position; // And a position, relative to that bone.<br />
M2Array<uint16_t> textureIndices; // into [[#Textures|textures]]<br />
M2Array<uint16_t> materialIndices; // into [[#Render_flags|materials]]<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> colorTrack;<br />
[[#Standard_animation_block|M2Track]]<{{Template:Type|fixed16}}> alphaTrack; // And an alpha value in a short, where: 0 - transparent, 0x7FFF - opaque.<br />
[[#Standard_animation_block|M2Track]]<float> heightAboveTrack;<br />
[[#Standard_animation_block|M2Track]]<float> heightBelowTrack; // do not set to same!<br />
float edgesPerSecond; // this defines how smooth the ribbon is. A low value may produce a lot of edges.<br />
float edgeLifetime; // the length aka Lifespan. in seconds<br />
float gravity; // use arcsin(val) to get the emission angle in degree<br />
uint16_t textureRows; // tiles in texture<br />
uint16_t textureCols;<br />
[[#Standard_animation_block|M2Track]]<uint16_t> texSlotTrack;<br />
[[#Standard_animation_block|M2Track]]<uchar> visibilityTrack;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}} // TODO: verify version<br />
int16_t priorityPlane;<br />
uint16_t padding;<br />
#endif<br />
} ribbons[];<br />
Some models that contain ribbon emitters and are viewable in the game world are: Wisps in BFD, Al'ar the Phoenix in Tempest Keep and any other phoenix models and the energy trails in the COT (not the actual instance, but the entrance cave in Tanaris Desert). Other models with ribbon emitters are spells and effects.<br />
<br />
''Parameters from the MDL format that are probably in here somewhere: emission rate, rows, cols ...?''<br />
<br />
== Particle emitters ==<br />
'''This is partly wrong as hell!''' Do not rely on this block, at all. It might even be wrong for WotLK.<br />
===M2ParticleOld===<br />
struct M2ParticleOld {<br />
uint32 particleId; // Always (as I have seen): -1.<br />
uint32 flags; // See Below<br />
{{Template:Type|C3Vector}} Position; // The position. Relative to the following bone.<br />
uint16 bone; // The [[#Bones|bone]] its attached to.<br />
union<br />
{<br />
uint16 texture; // And the [[#Texture_definitions|textures]] that are used. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
struct // For multi-textured particles actually three ids<br />
{<br />
uint16_t texture_0 : 5;<br />
uint16_t texture_1 : 5;<br />
uint16_t texture_2 : 5;<br />
uint16_t : 1;<br />
};<br />
#endif<br />
};<br />
M2Array<char> geometry_model_filename; // if given, this emitter spawns models<br />
M2Array<char> recursion_model_filename; // if given, this emitter is an '''alias''' for the (maximum 4) emitters of the given model<br />
<br />
#if >= 263 (late Burning Crusade)<br />
uint8 blendingType; // A blending type for the particle. See Below<br />
uint8 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
uint16 particleColorIndex; // This one is used for [[ParticleColor.dbc]]. See below.<br />
#else<br />
uint16 blendingType; // A blending type for the particle. See Below<br />
uint16 emitterType; // 1 - Plane (rectangle), 2 - Sphere, 3 - Spline, 4 - Bone<br />
#endif<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
{{Template:Type|fixed_point}}<uint8_t, 2, 5> multiTextureParamX[2];<br />
#else<br />
uint8 particleType; // Found below.<br />
uint8 headorTail; // 0 - Head, 1 - Tail, 2 - Both <br />
#endif<br />
uint16 textureTileRotation; // Rotation for the texture tile. (Values: -1,0,1) -- priorityPlane<br />
uint16 textureDimensions_rows; // for tiled textures<br />
uint16 textureDimensions_columns;<br />
[[#Standard_animation_block|M2Track]]<float> emissionSpeed; // Base velocity at which particles are emitted.<br />
[[#Standard_animation_block|M2Track]]<float> speedVariation; // Random variation in particle emission speed. (range: 0 to 1)<br />
[[#Standard_animation_block|M2Track]]<float> verticalRange; // longitude; Drifting away vertically. (range: 0 to pi) For plane generators, this is the maximum polar angle of the initial velocity; <br />
// 0 makes the velocity straight up (+z). For sphere generators, this is the maximum elevation of the initial position; <br />
// 0 makes the initial position entirely in the x-y plane (z=0).<br />
[[#Standard_animation_block|M2Track]]<float> horizontalRange; // latitude; They can do it horizontally too! (range: 0 to 2*pi) For plane generators, this is the maximum azimuth angle of the initial velocity; <br />
// 0 makes the velocity have no sideways (y-axis) component. <br />
// For sphere generators, this is the maximum azimuth angle of the initial position.<br />
[[#Standard_animation_block|M2Track]]<float> gravity; // Not necessarily a float; [[#Compressed Particle Gravity|see below]].<br />
[[#Standard_animation_block|M2Track]]<float> lifespan;<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float lifespanVary; // An individual particle's lifespan is added to by <code>lifespanVary * random(-1, 1)</code><br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionRate; <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float emissionRateVary; // This adds to the base emissionRate value the same way as lifespanVary. The random value is different every update.<br />
#endif<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaLength; // For plane generators, this is the width of the plane in the x-axis.<br />
// For sphere generators, this is the minimum radius.<br />
[[#Standard_animation_block|M2Track]]<float> emissionAreaWidth; // For plane generators, this is the width of the plane in the y-axis.<br />
// For sphere generators, this is the maximum radius.<br />
[[#Standard_animation_block|M2Track]]<float> zSource; // When greater than 0, the initial velocity of the particle is <code>(particle.position - C3Vector(0, 0, zSource)).Normalize()</code><br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C3Vector}}> colorTrack; // Most likely they all have 3 timestamps for {start, middle, end}.<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|fixed16}}> alphaTrack;<br />
[[#The_Fake-AnimationBlock|FBlock]]<{{Template:Type|C2Vector}}> scaleTrack;<br />
{{Template:Type|C2Vector}} scaleVary; // A percentage amount to randomly vary the scale of each particle<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> headCellTrack; // Some kind of intensity values seen: 0,16,17,32 (if set to different it will have high intensity)<br />
[[#The_Fake-AnimationBlock|FBlock]]<uint16> tailCellTrack;<br />
#else<br />
float midPoint; // middleTime; Middle point in lifespan (0 to 1).<br />
{{Type|CImVector}}[3] colorValues; // start, middle, end<br />
float[3] scaleValues;<br />
uint16[3] lifespanUVAnim;<br />
uint16[3] decayUVAnim;<br />
int16[2] tailUVAnim; // start, end<br />
int16[2] tailDecayUVAnim;<br />
#endif<br />
float tailLength;<br />
float twinkleSpeed; // twinkleFPS; has something to do with the spread<br />
float twinklePercent; // same mechanic as MDL twinkleOnOff but non-binary in 0.11.0<br />
{{Template:Type|CRange}} twinkleScale; // min, max<br />
float burstMultiplier; // ivelScale; requires (flags & 0x40)<br />
float drag; // For a non-zero values, instead of travelling linearly the particles seem to slow down sooner. Speed is multiplied by exp( -drag * t ).<br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
float baseSpin; // Initial rotation of the particle quad<br />
float baseSpinVary;<br />
float spin; // Rotation of the particle quad per second<br />
float spinVary;<br />
#else<br />
float spin; // 0.0 for none, 1.0 to rotate the particle 360 degrees throughout its lifetime.<br />
#endif<br />
<br />
[[#M2Box|M2Box]] tumble;<br />
{{Template:Type|C3Vector}} windVector;<br />
float windTime;<br />
<br />
float followSpeed1;<br />
float followScale1;<br />
float followSpeed2;<br />
float followScale2;<br />
M2Array<C3Vector> splinePoints; // Set only for spline praticle emitter. Contains array of points for spline<br />
[[#Standard_animation_block|M2Track]]<uchar> enabledIn; // (boolean) Appears to be used sparely now, probably there's a flag that links particles to animation sets where they are enabled.<br />
} particles[];<br />
<br />
Spin can be a float value greater or less one. Results look better if use it as a "phase shift": particle_rotate = randfloat(-sys->rotation * pi, sys->rotation * pi); --Igor<br />
<br />
===M2Particle (Cata+)===<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
* Cata+ has multi texture support<br />
using fp_6_9 = {{Template:Type|fixed_point}}<uint16_t, 6, 9>;<br />
struct vector_2fp_6_9 { fp_6_9 x; fp_6_9 y; };<br />
struct M2Particle<br />
{<br />
M2ParticleOld old;<br />
vector_2fp_6_9 multiTextureParam0[2];<br />
vector_2fp_6_9 multiTextureParam1[2];<br />
} particles[];<br />
<br />
In addition to these two parameters, <code>ParticleType</code> and <code>HeadOrTail</code> got reused (as in replaced at their current position) as <code>multiTextureParamX[2]</code> where all arrays are one entry per additional texture.<br />
<br />
I don't know if the previous meaning of the two parameters still exists, got moved, or was just never used to begin with. ParticleType appears to be implicit by having <code>flags & 0x10100000</code> (→ multi texture), a model (→ model) or neither (→ default).--[[User:Schlumpf|Schlumpf]] ([[User talk:Schlumpf|talk]]) 23:47, 29 October 2015 (UTC)<br />
<br />
===Particle Flags===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="600" | Description<br />
|-<br />
| 0x1 || Particles are affected by lighting; <br />
|-<br />
| 0x2 || <br />
|-<br />
| 0x4 || <br />
|-<br />
| 0x8 || Particles travel "up" in world space, rather than model.<br />
|-<br />
| 0x10 || Do not Trail<br />
|-<br />
| 0x20 || Unlightning<br />
|-<br />
| 0x40 || <br />
|-<br />
| 0x80 || Particles in Model Space<br />
|-<br />
| 0x100 || <br />
|-<br />
| 0x200 || spawn position randomized in some way?<br />
|-<br />
| 0x400 || STYLE: Pinned Particles, their quad enlarges from their creation position to where they expand.<br />
|-<br />
| 0x800 || <br />
|-<br />
| 0x1000 || XYQuad Particles. They align to XY axis facing Z axis direction.<br />
|-<br />
| 0x2000 || clamp to ground; call CParticleEmitter2::ProjectParticle<br />
|-<br />
| 0x4000 || <br />
|-<br />
| 0x8000 || <br />
|-<br />
| 0x10000 || ChooseRandomTexture<br />
|-<br />
| 0x20000 || STYLE: "Outward" particles, most emitters have this and their particles move away from the origin, when they don't the particles start at origin+(speed*life) and move towards the origin.<br />
|-<br />
| 0x40000 || STYLE: unknown. In a large proportion of particles this seems to be simply the opposite of the above flag, but in some (e.g. voidgod.m2 or wingedlionmount.m2) both flags are true.<br />
|-<br />
| 0x80000 || If set, ScaleVary affects x and y independently; if not set, ScaleVary.x affects x and y uniformly, and ScaleVary.y is not used.<br />
|-<br />
| 0x200000 || Random FlipBookStart<br />
|-<br />
| 0x400000 || Ignores Distance (or 0x4000000?!, CMapObjDef::SetDoodadEmittersIgnoresDistance has this one)<br />
|-<br />
| 0x800000 || gravity values are compressed vectors instead of z-axis values (see Compressed Particle Gravity below)<br />
|-<br />
| 0x1000000 || bone generator = bone, not joint<br />
|-<br />
| 0x4000000 || do not throttle emission rate based on distance<br />
|-<br />
| 0x10000000 || Particle uses multi-texturing (could be one of the other WoD-specific flags), see multi-textured section.<br />
|}<br />
<br />
--[[User:Barncastle|Barncastle]] ([[User talk:Barncastle|talk]]) A comparison of 0.10.0's MDX files to 0.11.0's and 1.12.1's M2 files indicates that the [[MDX#Flags|MDX PRE2 flags]] (≥0x8000) were probably unchanged during the switch from MDL to M2, albeit separated into their own enum.<br />
<br />
===ParticleColorIndex===<br />
This is used in conjunction with [[ParticleColor.dbc]] to alter the default colour of particles. If the particle colour is not meant to be changed then its ParticleColorIndex will have a value of zero. If the particle colour may be changed then the value will be 11, 12 or 13, indicating whether the first, second or third Start, Mid and End colours are to be used, respectively. The row of [[ParticleColor.dbc]] to be used is determined its ID value, which should correspond to the ParticleColorID value supplied by [[CreatureDisplayInfo.dbc]] or [[ItemDisplayInfo.dbc]] for creatures or items.<br />
===Particle types===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || "normal" particle<br />
|-<br />
| 1 || large quad from the particle's origin to its position (used in Moonwell water effects)<br />
|-<br />
| 2 || seems to be the same as 0 (found some in the Deeprun Tram blinky-lights-sign thing)<br />
|}<br />
''ParticleType is always 0 and, maybe, now (Flags & 0x40000) != 0 means "particles from origin to position". --Igor''<br />
''Checked and verified --BlinkHawk''<br />
===Particle Blendings===<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="500" | Description<br />
|-<br />
| 0 || glDisable(GL_BLEND); glDisable(GL_ALPHA_TEST);<br />
|-<br />
| 1 || glBlendFunc(GL_SRC_COLOR, GL_ONE);<br />
|-<br />
| 2 || glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);<br />
|-<br />
| 3 || glDisable(GL_BLEND); glEnable(GL_ALPHA_TEST);<br />
|-<br />
| 4 || glBlendFunc(GL_SRC_ALPHA, GL_ONE);<br />
|}<br />
from Modelviewer source<br />
-- Rour, some WoD particle effects are using blend mode 0x7 here.<br />
===The Fake-AnimationBlock===<br />
*Its pretty much like the real one but without the "header".<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Offset !! width="90" | Type !! width="120" | Name !! width="500" | Description<br />
|-<br />
| 0x000 || uint32 || nTimestamps || The number of timestamps.<br />
|-<br />
| 0x004 || uint32 || ofsTimestamps || And the offset to them. The timestamps are shorts! (?)<br />
|-<br />
| 0x008 || uint32 || nKeys || The same number again. This time its the number of Keys / Values.<br />
|-<br />
| 0x00C || uint32 || ofsKeys || And their offset.<br />
|}<br />
But they're unable to change between different animations, so they directly point to the data.<br />
<br />
===Compressed Particle Gravity===<br />
Key values in the gravity track are decompressed at load time from a 4-byte value to a C3Vector.<br />
<br />
<nowiki><br />
struct CompressedParticleGravity {<br />
int8_t x, y;<br />
int16_t z;<br />
};<br />
for (/* each 4-byte value in the particle gravity track */) {<br />
float *pValue;<br />
C3Vector *pDst;<br />
if (particle.flags & 0x800000) {<br />
// interpret the 4 bytes at pValue as CompressedParticleGravity:<br />
CompressedParticleGravity v = new (pValue) CompressedParticleGravity();<br />
C3Vector dir = C3Vector(v.x, v.y, 0) * (1.0f / 128.0f);<br />
float z = sqrtf(1.0f - dir.Dot(dir));<br />
float mag = v.z * 0.04238648f;<br />
if (mag < 0) {<br />
z = -z;<br />
mag = -mag;<br />
}<br />
dir.z = z;<br />
dir *= mag;<br />
*pDst = dir;<br />
} else {<br />
*pDst = C3Vector(0, 0, -(*pValue));<br />
}<br />
}</nowiki><br />
<br />
<br />
===M2Box===<br />
struct M2Box {<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMin;<br />
{{Template:Type|C3Vector}} ModelRotationSpeedMax;<br />
}<br />
<br />
=Miscellaneous=<br />
==Name==<br />
char name[];<br />
<br />
Informative name used for debugging purposes. Not used in retail clients.<br />
<br />
==Bounding volumes==<br />
These blocks give a simplified bounding volume for the model. Characters and creatures have just a simple box.<br />
===Vertices===<br />
This block defines the possible points used for the model. They are referenced in the triangles block later.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} position;<br />
} bounding_vertices[];<br />
<br />
===Triangles===<br />
The number in the header tells us how many uint16s there are, not how many triangles. To use this better, you should group three of them into an array. The nBoundingTriangles/3 indices will tell you which vertices are used for the triangle then. <br />
<br />
struct <br />
{<br />
uint16_t index; // three entries pointing to vertices per triangle<br />
} bounding_triangles[];<br />
<br />
The number nBoundingTriangles once again contains the number of indices used, so divide by 3 to get the number of triangles.<br />
<br />
===Normals===<br />
This one defines a normal per triangle. The vectors are normalized, but Blizzard seems to have some problems getting a simple vector normalized leading in several 0,0,0.999999999 ones. Whatever.<br />
<br />
As each vertex has a corresponding normal vector, it should be true that nBoundingNormals = nBoundingTriangles / 3.<br />
struct <br />
{<br />
{{Template:Type|C3Vector}} normal;<br />
} bounding_normals[];<br />
<br />
== Lights ==<br />
struct M2Light<br />
{<br />
/*0x00*/ uint16_t type; // Types are listed below.<br />
/*0x02*/ int16_t bone; // -1 if not attached to a bone<br />
/*0x04*/ {{Template:Type|C3Vector}} position; // relative to bone, if given<br />
/*0x10*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> ambient_color;<br />
/*0x24*/ [[#Standard_animation_block|M2Track]]<float> ambient_intensity; // defaults to 1.0<br />
/*0x38*/ [[#Standard_animation_block|M2Track]]<{{Template:Type|C3Vector}}> diffuse_color;<br />
/*0x4C*/ [[#Standard_animation_block|M2Track]]<float> diffuse_intensity; // defaults to 1.0<br />
/*0x60*/ [[#Standard_animation_block|M2Track]]<float> attenuation_start;<br />
/*0x74*/ [[#Standard_animation_block|M2Track]]<float> attenuation_end;<br />
/*0x88*/ [[#Standard_animation_block|M2Track]]<uint8_t> visibility; // enabled?<br />
/*0x9C*/<br />
} lights[];<br />
<br />
Two light types:<br />
{| style="background:#FCFCFC; color:black"<br />
! width="70" | Value !! width="200" | Description<br />
|-<br />
| 0 || Directional<br />
|-<br />
| 1 || Point light<br />
|}<br />
<br />
Directional light type is not used (at least in 3.3.5) outside login screen, and doesn't seem to be taken into account in world.<br />
<br />
==Cameras==<br />
These blocks are present in the "flyby" camera models which completely lack geometry and the main menu backdrop models which are supposed to have a fixed camera. Additionally, characters and monsters also have this block. The reason that non-mainmenu and non-geometry M2s have cameras was is you can see the unit's portrait and the character info tab.<br />
<br />
struct M2Camera<br />
{<br />
uint32_t type; // 0: portrait, 1: characterinfo; -1: else (flyby etc.); referenced backwards in the lookup table.<br />
#if {{Template:Sandbox/VersionRange|max_expansionlevel=4|max_exclusive=1}}<br />
float fov; // Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
float far_clip;<br />
float near_clip;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> positions; // How the camera's position moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<{{Template:Type|C3Vector}}>> target_position; // How the target moves. Should be 3*3 floats.<br />
{{Template:Type|C3Vector}} target_position_base;<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> roll; // The camera can have some roll-effect. Its 0 to 2*Pi. <br />
#if {{Template:Sandbox/VersionRange|min_expansionlevel=4}}<br />
[[#Standard_animation_block|M2Track]]<M2SplineKey<float>> FoV; //Diagonal FOV in radians. See below for conversion.<br />
#endif<br />
} cameras[];<br />
<br />
===Camera field of view===<br />
<br />
The <tt>fov</tt> included in <tt>M2Camera</tt> is a diagonal field of view (in radians). The client converts it to a vertical field of view at runtime using the following formula:<br />
<br />
<syntaxhighlight lang="cpp"><br />
float vfov = dfov / sqrt(1.0 + pow(aspect, 2.0));<br />
</syntaxhighlight><br />
<br />
The aspect ratio used is determined by the rect being presented on (eg. the game window).<br />
<br />
Note that this formula makes an assumption that the screen being projected to is planar (ie a monitor).<br />
<br />
===Camera lookup table===<br />
This block lists the different cameras existing in the model. The index in the array is also the type. CameraLookupTable[1] is always the character tab camera.<br />
<br />
"-1" type cameras are not referenced.<br />
<br />
If nCameras >= 1, then nCameraLookup will be >= 1 regardless of whether any camera will be actually referenced in it. See interface/glues/models/ui_mainmenu_warlords/ui_mainmenu_warlords.m2. A valid block thus may be -1s only. This appears to be an exporter-quirk rather than a requirement.<br />
<br />
struct<br />
{<br />
uint16_t camera;<br />
} camera_lookup[];<br />
<br />
== Attachments ==<br />
This block specifies a bunch of locations on the body - hands, shoulders, head, back, knees etc. It is used to put items on a character. This seems very likely as this block also contains positions for sheathed weapons, a shield, etc.<br />
struct M2Attachment<br />
{<br />
uint32_t id; // Referenced in the [[#Attachment_Lookup|lookup-block]] below.<br />
uint16_t bone; // attachment base<br />
uint16_t unknown; // see BogBeast.m2 in vanilla for a model having values here<br />
{{Template:Type|C3Vector}} position; // relative to bone; Often this value is the same as bone's pivot point <br />
[[#Standard_animation_block|M2Track]]<uchar> animate_attached; // whether or not the attached model is animated when this model is. only a bool is used. default is true.<br />
} attachments[];<br />
<br />
Meaning depends on type of model. The following are for creatures/characters mainly:<br />
<br />
{| style="background:#FCFCFC; color:black"<br />
! width="50" | ID !! width="250" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
! width="50" | ID !! width="150" | Description<br />
|-<br />
| 0 || Shield / MountMain / ItemVisual0<br />
| 12 || Back<br />
| 24 || Special2<br />
| 36 || Bullet (version: somewhen after alpha)<br />
| 48 || RightFoot<br />
|-<br />
| 1 || HandRight / ItemVisual1<br />
| 13 || ShoulderFlapRight<br />
| 25 || Special3<br />
| 37 || SpellHandOmni (version: somewhen after alpha)<br />
| 49 || ShieldNoGlove<br />
|-<br />
| 2 || HandLeft / ItemVisual2<br />
| 14 || ShoulderFlapLeft<br />
| 26 || SheathMainHand<br />
| 38 || SpellHandDirected (version: somewhen after alpha)<br />
| 50 || SpineLow<br />
|-<br />
| 3 || ElbowRight / ItemVisual3<br />
| 15 || ChestBloodFront<br />
| 27 || SheathOffHand<br />
| 39 || VehicleSeat1 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 51 || AlteredShoulderR<br />
|-<br />
| 4 || ElbowLeft / ItemVisual4<br />
| 16 || ChestBloodBack<br />
| 28 || SheathShield<br />
| 40 || VehicleSeat2 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 52 || AlteredShoulderL<br />
|-<br />
| 5 || ShoulderRight<br />
| 17 || Breath<br />
| 29 || PlayerNameMounted<br />
| 41 || VehicleSeat3 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 53 || BeltBuckle {{Template:Sandbox/VersionRange|min_expansionlevel=5}}<br />
|-<br />
| 6 || ShoulderLeft<br />
| 18 || PlayerName<br />
| 30 || LargeWeaponLeft<br />
| 42 || VehicleSeat4 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 54 || SheathCrossbow<br />
|-<br />
| 7 || KneeRight<br />
| 19 || Base<br />
| 31 || LargeWeaponRight<br />
| 43 || VehicleSeat5 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| 55 || HeadTop {{Template:Sandbox/VersionRange|min_expansionlevel=7}}<br />
|-<br />
| 8 || KneeLeft<br />
| 20 || Head<br />
| 32 || HipWeaponLeft<br />
| 44 || VehicleSeat6 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 9 || HipRight<br />
| 21 || SpellLeftHand<br />
| 33 || HipWeaponRight<br />
| 45 || VehicleSeat7 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 10 || HipLeft<br />
| 22 || SpellRightHand<br />
| 34 || Chest<br />
| 46 || VehicleSeat8 {{Template:Sandbox/VersionRange|min_expansionlevel=3}}<br />
| || <br />
|-<br />
| 11 || Helm<br />
| 23 || Special1<br />
| 35 || HandArrow<br />
| 47 || LeftFoot<br />
| || <br />
|}<br />
For weapons, usually 5 of these points are present, which correspond to the 5 columns in [[ItemVisuals.dbc]], which in turn has 5 models from [[ItemVisualEffects.dbc]]. This is for the weapon glowy effects and such. The effect ID is the last column in [[ItemDisplayInfo.dbc]]. They take the ids 0 to 4. Mounts take the id 0 for their rider. Breath (17) is used by CGCamera::FinishLoadingTarget() aswell as some other one. The name above the head of a Unit (CGUnit_C::GetNamePosition) looks for PlayerNameMounted (29), then PlayerName (18).<br />
===Attachment Lookup===<br />
The index of the array defines, which type that attachment is of. Its the same as the list above. The lookups and the id of the animations point in a circle.<br />
struct <br />
{<br />
uint16_t attachment;<br />
} attachment_lookup[];<br />
<br />
==Events==<br />
These events are used for timing sounds for example. You can find the $DTH (death) event on nearly every model. It will play the death sound for the unit.<br />
<br />
The events you can use depend on the way, the model is used. Dynamic objects can shake the camera, doodads shouldn't. Units can do a lot more than other objects.<br />
<br />
Somehow there are some entries, that don't use the $... names but identifiers like "DEST" (destination), "POIN" (point) or "WHEE" (wheel). How they are used? Idk.<br />
<br />
struct M2Event<br />
{<br />
uint32_t identifier; // mostly a 3 character name prefixed with '$'.<br />
uint32_t data; // This data is passed when the event is fired. <br />
uint32_t bone; // Somewhere it has to be attached.<br />
{{Template:Type|C3Vector}} position; // Relative to that bone of course, animated. Pivot without animating.<br />
[[#Standard_animation_block|M2TrackBase]] enabled; // This is a timestamp-only animation block. It is built up the same as a normal [[#Standard_animation_block|AnimationBlocks]], but is missing values, as every timestamp is an implicit "fire now".<br />
} events[];<br />
===Possible Events===<br />
There are a lot more of them. I did not list all up to now.<br />
{| style="background:#FCFCFC || color:black"<br />
! width="50" | ID !! width="40" | data !! width="60" | what !! width="60" | Type !! width="200" | seen to be fired on !! width="600" | Description<br />
|-<br />
| $AH[0-3] || — || PlaySoundKit (customAttack[x]) || || || soundEffect ID is defined by CreatureSoundDataRec::m_customAttack[x]<br />
|-<br />
| $BMD || — || BowMissleDestination || RangedWeapon || ||<br />
|-<br />
| $AIM || || || Vehicles || CGUnit_C::ComputeMissileTrajectory || Position used as MissileFirePos.<br />
|-<br />
| $ALT || || anim_swap_event / DisplayTransition || Unit || || CUnitDisplayTransition_C::UpdateState(1) or CUnitDisplayTransition_C::HandleAnimSwapEvent<br />
|-<br />
| $BL[0-3] || — || FootstepAnimEventHit (left) || Unit || || Backwards<br />
|-<br />
| $BR[0-3] || — || FootstepAnimEventHit (right) || Unit || || Backwards<br />
|-<br />
| $BRT || — || PlaySoundKit (birth) || || || soundEffect ID is defined by CreatureSoundDatarec::m_birthSoundID<br />
|-<br />
| $BTH || — || Breath || Unit || All situations, where nothing happens or breathing. || Adds Special Unit Effect based on unit state (under water, in-snow, …)<br />
|-<br />
| $BWP || — || PlayRangedItemPull (Bow Pull) || Unit || LoadRifle, LoadBow || <br />
|-<br />
| $BWR || — || BowRelease || Unit || AttackRifle, AttackBow, AttackThrown || <br />
|-<br />
| $CAH || — || || Unit || Attack*, *Unarmed, ShieldBash, Special* || attack hold? CGUnit_C::HandleCombatAnimEvent<br />
|-<br />
| $CCH || — || || Unit || mostly not fired, AttackThrown || CEffect::DrawFishingString needs this on the model for getting the string attachments.<br />
|-<br />
| $CFM || — || || Unit || CGCamera::UpdateMountHeightOrOffset || CGCamera::UpdateMountHeightOrOffset: Only z is used. Non-animated. Not used if $CMA<br />
|-<br />
| $CHD || || || Unit || not fired || probably does not exist?!<br />
|-<br />
| $CMA || — || || Unit || || CGCamera::UpdateMountHeightOrOffset: Position for camera<br />
|-<br />
| $CPP || || PlayCombatActionAnimKit || || || parry, anims, depending on some state, also some callback which might do more<br />
|-<br />
| $CSD || soundEntryId || PlayEmoteSound || Unit || Emote* || <br />
|-<br />
| $CSL || — || release_missiles_on_next_update if has_pending_missiles (left) || Unit || AttackRifle, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSR || — || release_missiles_on_next_update if has_pending_missiles (right) || Unit || AttackBow, AttackRifle, AttackThrown, SpellCast*, ChannelCast* || "x is {L or R} (""Left/right hand"") (?)"<br />
|-<br />
| $CSS || — || PlayWeaponSwooshSound || || sound played depends on CGUnit_C::GetWeaponSwingType<br />
|-<br />
| $CST || — || release_missiles_on_next_update if has_pending_missiles || Unit || Attack*, *Unarmed, ShieldBash, Special*, SpellCast, Parry*, EmoteEat, EmoteRoar, Kick, ... || $CSL/R/T are also used in CGUnit_C::ComputeDefaultMissileFirePos.<br />
|-<br />
| $CVS || || ||  ? || || Data: SoundEntriesAdvanced.dbc, Sound — Not present in 6.0.1.18179<br />
|-<br />
| $DSE || — || DestroyEmitter || MapObj || || <br />
|-<br />
| $DSL || soundEntryId || DoodadSoundLoop (low priority) | GO || || <br />
|-<br />
| $DSO || soundEntryId || DoodadSoundOneShot || GO || || <br />
|-<br />
| $DTH || — || DeathThud + LootEffect || Unit || Death, Drown, Knockdown || """I'm dead now!"", UnitCombat_C, this plays death sounds and more." Note that this is NOT triggering [[CreatureSoundData.dbc|CreatureSoundDataRec]]::m_soundDeathID, but that is just always triggered as soon as the death animation plays.<br />
|-<br />
| $EAC || — || object package state enter 3, exit 2, 4, 5 || || || <br />
|-<br />
| $EDC || — || object package state enter 5, exit 3, 4, 2 || || || <br />
|-<br />
| $EMV || — || object package state enter 4, exit 3, 2, 5 || || || <br />
|-<br />
| $ESD || — || PlayEmoteStateSound || Unit || || soundEffect ID is implicit by currently played emote<br />
|-<br />
| $EWT || — || object package state enter 2, exit 3, 4, 5 || || || <br />
|-<br />
| $FD[1-9] || — || PlayFidgetSound || || || CreatureSoundDataRec::m_soundFidget (only has 5 entries, so don’t use 6-9)<br />
|-<br />
| $FDX || — || PlayUnitSound (stand) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundStandID<br />
|-<br />
| $FL[0-3] || — || FootstepAnimEventHit (left) || || || Forward<br />
|-<br />
| $FR[0-3] || — || FootstepAnimEventHit (right) || || || Forward<br />
|-<br />
| $FSD || — || HandleFootfallAnimEvent || Unit || Walk, Run (multiple times), ... || Plays some sound. Footstep? Also seen at several emotes etc. where feet are moved. CGUnit_C::HandleFootfallAnimEvent<br />
|-<br />
| $GC[0-3] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x + 6] ({Custom0, Custom1, Custom2, Custom3})<br />
|-<br />
| $GO[0-5] || — || GameObject_C_PlayAnimatedSound || || || soundEffect ID is defined by GameObjectDisplayInfoRec::m_Sound[x] ({Stand, Open, Loop, Close, Destroy, Opened})<br />
|-<br />
| $HIT || — || PlayWoundAnimKit || Unit || Attack*, *Unarmed, ShieldBash, Special* || soundEntryId depends on SpellVisualKit<br />
|-<br />
| $KVS || || || Â ? || || MapLoad.cpp -- not found in 6.0.1.18179<br />
|-<br />
| $RL[0-3] || — || FootstepAnimEventHit (left) || || || Running<br />
|-<br />
| $RR[0-3] || — || FootstepAnimEventHit (right) || || || Running<br />
|-<br />
| $SCD || — || PlaySoundKit (spellCastDirectedSound) || || || soundEffect ID is defined by CreatureSoundDataRec::m_spellCastDirectedSoundID<br />
|-<br />
| $SHK || spellEffectCameraShakesID || AddShake || GO || || <br />
|-<br />
| $SHL || — || ExchangeSheathedWeapon (left) || || Sheath, HipSheath || <br />
|-<br />
| $SHR || — || ExchangeSheathedWeapon (right) || || Sheath, HipSheath || <br />
|-<br />
| $SL[0-3] || — || FootstepAnimEventHit (left) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $SMD || — || PlaySoundKit (submerged) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergedSoundID<br />
|-<br />
| $SMG || — || PlaySoundKit (submerge) || || || soundEffect ID is defined by CreatureSoundDatarec::m_submergeSoundID<br />
|-<br />
| $SND || soundEntryId || PlaySoundKit (custom) || GO || || <br />
|-<br />
| $SR[0-3] || — || FootstepAnimEventHit (right) || || Stop, (JumpEnd), (Shuffle*) || Stop<br />
|-<br />
| $STx || || || Mounts || MountTransitionObject::UpdateCharacterData || Not seen in 6.0.1.18179 -- x is {E and B} , sequence time is taken of both, pivot of $STB. (Also, attachment info for attachment 0)<br />
|-<br />
| $TRD || — || HandleSpellEventSound || Unit || EmoteWork*, UseStanding* || soundEffect ID is implicit by SpellRec<br />
|-<br />
| $VG[0-8] || — || HandleBoneAnimGrabEvent || || || <br />
|-<br />
| $VT[0-8] || — || HandleBoneAnimThrowEvent || || || <br />
|-<br />
| $WGG || — || PlayUnitSound (wingGlide) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingGlideID<br />
|-<br />
| $WL[0-3] || — || FootstepAnimEventHit (left) || || || <br />
|-<br />
| $WNG || — || PlayUnitSound (wingFlap) || || || soundEffect ID is defined by CreatureSoundDataRec::m_soundWingFlapID<br />
|-<br />
| $WR[0-3] || — || FootstepAnimEventHit (right) || || || <br />
|-<br />
| $WTB || — || || Weapons || || Weapon Trail Bottom position, also used for Bow String<br />
|-<br />
| $WTT || — || || Weapons || || Weapon Trail Top position<br />
|-<br />
| $WWG || || || Â ? || || Calls some function in the Object VMT. -- Not seen in 6.0.1.18179<br />
|-<br />
| DEST || || || Â ? || || exploding ballista, that one has a really fucked up block. Oo<br />
|-<br />
| POIN || || || Unit || not fired || Data:Â ?, seen on multiple models. Basilisk for example. (6801)<br />
|-<br />
| WHEE || || || Â ? || || Data: 601+, Used on wheels at vehicles.<br />
|-<br />
| BOTT || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|-<br />
| TOP || || || Â ? || || Purpose unknown. Seen in well_vortex01.m2<br />
|}<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/HelmetGeosetVisData&diff=26851DB/HelmetGeosetVisData2019-06-01T06:20:21Z<p>Simca: Removed old obvious observations, keeping only the important notes. Also mentioned that this was replaced.</p>
<hr />
<div>Used for hiding certain elements of the face on certain races for certain helmets.<br />
<br />
Flag fields are all race bitmask. If ((flag & (1 << race)) > 0) is true, the mesh group will reset to default value: 1, 101, 201 etc(the connection between field and mesh group is given below)<br />
Every id is universal for both genders. That's why every helmet in ItemDisplayInfo has 2 values for HelmetGeosetVisData.<br />
<br />
In Patch 8.1.5, it was replaced with [[DB/HelmetGeosetData|HelmetGeosetData.db2]], which contains the same data but in a much more straightforward way. The main reason they switched is because they were no longer able to fit ChrRaces' rows into a 32-bit bitmask.<br />
<br />
==0.5.3.3368==<br />
struct HelmetGeosetVisDataRec {<br />
uint32_t m_ID;<br />
uint32_t m_DefaultFlags[32];<br />
uint32_t m_PreferredFlags[32];<br />
uint32_t m_HideFlags[32];<br />
};<br />
==1.12.1.5875==<br />
struct HelmetGeosetVisDataRec {<br />
uint32_t m_ID;<br />
uint32_t hairFlags;<br />
uint32_t facialFlags[3];<br />
uint32_t earsFlags;<br />
};<br />
==Structure==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180 " | Field <br />
! width="80" | Type <br />
! width="600" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || HairFlags || Integer ||<br />
|- <br />
| 3 || Facial1Flags || Integer || (Beard, Tusks)<br />
|- <br />
| 4 || Facial2Flags || Integer || (Earrings)<br />
|- <br />
| 5 || Facial3Flags || Integer || See [[ChrRaces.dbc|ChrRaces]], column 24 to 26 for information on what is what.<br />
|- <br />
| 6 || EarsFlags || Integer || <br />
|- <br />
| 7 || Unknown || Integer || <br />
|- <br />
| 8 || Unknown || Integer || <br />
|}<br />
<br />
==6.0.1.18179==<br />
struct HelmetGeosetVisDataRec {<br />
uint32_t m_ID;<br />
uint32_t m_hideGeoset[7]; // groups 0, 100, 200, 300, 700, 1600, 1700, Legion: +2400, 2500<br />
};<br />
<br />
==8.0.1.25902==<br />
struct HelmetGeosetVisDataRec {<br />
// uint32_t m_ID;<br />
uint32_t m_hideGeoset[9];<br />
};<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/HelmetGeosetData&diff=26850DB/HelmetGeosetData2019-06-01T06:19:15Z<p>Simca: Added more information about the file's past.</p>
<hr />
<div>{{SectionBox|This database was introduced in {{Sandbox/PrettyVersion|expansionlevel=8}}.}}<br />
Used for hiding certain elements of the face on certain races for certain helmets. <br />
<br />
If table contains data for current race, the mesh group will reset to default value: 1, 101, 201 etc.<br />
<br />
This file serves as a replacement for [[DB/HelmetGeosetVisData]] and was added in the same build where that DB was removed.<br />
<br />
----<br />
<br />
==8.1.5.29484==<br />
struct HelmetGeosetDataRec {<br />
uint32_t m_ID;<br />
uint32_t m_RaceId;<br />
uint8_t m_GeosetGroup;<br />
uint32_t m_GeosetVisDataId; // ref, has values equal to the old HelmetGeosetVisDataRec.m_ID; used in [[DB/ItemDisplayInfo]]<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_BfA]]</div>Simcahttps://wowdev.wiki/index.php?title=ADB&diff=26831ADB2019-05-24T00:48:00Z<p>Simca: /* DBCache.bin */ Added version 6 and version 7.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
The structures described here are used in [[ADB|ADB files]], which are a cache of dynamically streamed database entries, typically used for hotfixes and content that Blizzard wants to hide from dataminers until the last minute.<br />
<br />
For more information, please check out the page on [[DB2|DB2 files]]. This article assumes you already have a basic understanding of how they function.<br />
<br />
=Table content structures=<br />
This page describes the structure of ADB files. For a list of existing DB2 files (and therefore ADB files) and their contents, see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=WCH3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WCH4 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[DB2#WDB2]] for how to adapt this structure for its .DB2 counterpart.<br />
struct adb_header<br />
{<br />
uint32_t magic; // 'WCH3'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero for ADBs<br />
};<br />
<br />
template<typename record_type><br />
struct adb_file<br />
{<br />
adb_header header;<br />
// static_assert (header.record_size == sizeof (record_type));<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WCH4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings and an optional block after the string block that contains non-inline IDs.<br />
<br />
It is worth noting that min_id and max_id fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present.<br />
<br />
Note that, in most cases, the non-inline IDs structure still exists, even if it is made completely redundant by the existence of the offset map. After all, why do work once when you could do it twice?<br />
<br />
==Structure==<br />
See [[DB2#WDB3]] for how to adapt this structure for its .DB2 counterpart.<br />
template<typename record_type><br />
struct wch4_file<br />
{<br />
adb_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WCH5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.22345|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and was phased out in favor of WCH6 in Legion (Patch 7.0.3 Build 21414). As such, support for WCH5 is likely important for almost nobody.<br />
<br />
While not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
It's worth noting that WCH5 is missing two header fields compared to its .DB2 counterpart. Since the deduplication technology we call 'copy_table' is not employed in any of the .ADB formats, copy_table_size variable was always zero anyway. As for flags, Blizzard always loads the .DB2 file before loading the .ADB file, so the flags field being in both files is just redundant (for them); any .ADB file will be read using the flags field from the .DB2 file of the same name. This is inconvenient for us, as most (if not all) of us load .ADB files standalone at the moment, but there are workarounds (such as having a configuration file to remember the last used flags value for each named .DB2 file).<br />
<br />
After Legion build 21737, an additional header field was added (right before 'build') called 'layout_hash'. This field's appearance is similar to the change to the [[DB2#WDB5|WDB5]] header made in the same build, except in that case, the 'build' field was removed when the 'layout_hash' field was added. For the post-21737 WCH5 header, both fields are present.<br />
<br />
==Structure==<br />
See [[DB2#WDB4]] or [[DB2#WDB5]] for how to adapt this structure for its .DB2 counterpart.<br />
struct wch5_adb_pre_21737_header<br />
{<br />
uint32_t magic; // 'WCH5'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count only count as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
struct wch5_adb_post_21737_header<br />
{<br />
uint32_t magic; // 'WCH5' for .adb (cache)<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4 but the entire array only counts as '1' field for WCH5<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // from build 21737 onward, this field is present<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
};<br />
<br />
template<typename record_type><br />
struct wch5_file<br />
{<br />
wch5_adb_post_21737_header header;<br />
struct offset_map_entry // Optional - can be detected by pulling 'flags' from relevant DB2<br />
{<br />
uint32_t ID;<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count]; // Optional - can be detected by pulling 'flags' from relevant DB2<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WCH6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22345|max_expansionlevel=7|max_build=7.0.3.22451|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22345) and was phased out in favor of WCH7 in Legion (Patch 7.0.3 Build 22451).<br />
<br />
Literally nothing changed in WCH6. The introduction of this incremented signature was probably a temporary countermeasure to combat a problem Blizzard was having with 'build' not being updated successfully. This would leave ADBs around basically forever (since they are only wiped when current build is greater than the one in the ADBs being loaded), which could cause conflicts if you downloaded an updated client with new DB2s.<br />
<br />
=WCH7=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22451|max_expansionlevel=7|max_build=7.0.3.22484|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22451) and was phased out in favor of WCH8 in Legion in the next build (Patch 7.0.3 Build 22484).<br />
<br />
A new header field has been added between 'record_count' and 'field_count'. This field designates the size of a new table which is located between the string table and the non-inline IDs table. All records are padded to 'record_size'.<br />
<br />
=WCH8=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.22484|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 22484) and is still in use today. It was used alongside [[DB2#WDB5|WDB5]].<br />
<br />
The only difference to WCH7 is the padding of files with an offset map. These are padded as per the WCH5 specification again; presumably as this was unnecessarily changed in WCH7.<br />
<br />
=DBCache.bin=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7}}<br />
[[DB2#WDB5|WDB5]] and [[ADB#WCH8|WCH8]] were retired simultaneously in Patch 7.2.0 Build 23436. All files in the 'ADB' folder were consolidated into a single new file, DBCache.bin. It contains server-provided new and changed records for all client-side database tables. It even sometimes contains server-side database changes (entries with an unknown 'table_hash'). It is theorized that the file is a sort of 'packet replayer' for the WoW executable, used for updating client-side DB2s to latest hotfixes on boot up, before confirming with the server about the newest changes.<br />
<br />
The file uses the same record structures as the WoW binary requires but with none of the mechanics found in newer [[DB2]] formats. There is no offset map, field compression of any kind, and no deduplication. Strings are inline, and IDs are always provided before the record, regardless of whether or not there is an ID value in the record data additionally.<br />
<br />
==Version 1==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.2.0.23780|max_exclusive=1}}<br />
<br />
struct dbcache_file<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry entry;<br />
}<br />
};<br />
<br />
==Version 2 through Version 4==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23780|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
<br />
Version 2 added a new field directly after 'magic' in 'dbcache_entry'. At first, it was thought that the new field was just an expansion of the signature, as the field looked like 'XFTH2' for a long time. However, it appears to be something different.<br />
<br />
There were no structure changes in Version 3 or Version 4.<br />
<br />
struct dbcache_file_v2<br />
{<br />
struct dbcache_file_header<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 5 and Version 6==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436}}<br />
<br />
Version 5 added a new field directly after 'build_id' in 'dbcache_file_header'. It is a simple SHA256 hash for the file's contents (minus the header or with the header having 00s in that area, not sure which).<br />
<br />
There were no structure changes in Version 6.<br />
<br />
struct dbcache_file_v5<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t sha256[32];<br />
};<br />
<br />
struct dbcache_entry_v2<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t region_ish; // 3 for eu, 4 for ptr, apparently 1 for us<br />
int32_t index;<br />
uint32_t data_size;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v2 entry;<br />
}<br />
};<br />
<br />
==Version 7==<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8}}<br />
<br />
Version 7 removed the 'region_ish' field and moved the 'data_size' two fields back towards the end of the entry structure.<br />
<br />
struct dbcache_file_v7<br />
{<br />
struct dbcache_file_header_v5<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
uint32_t version;<br />
uint32_t build_id;<br />
uint8_t sha256[32];<br />
};<br />
<br />
struct dbcache_entry_v7<br />
{<br />
uint32_t magic; // 'XFTH' ('HoTFiX')<br />
int32_t index;<br />
uint32_t table_hash;<br />
uint32_t record_id;<br />
uint32_t data_size; // Moved from before 'table_hash' to after 'record_id' in Version 7<br />
uint8_t is_valid; // A boolean describing the 'validity' of the file. This may also control 'deletion' of the entry in question.<br />
uint8_t padd[3];<br />
if (data_size) uint8_t record[data_size];<br />
};<br />
<br />
dbcache_file_header_v5 header;<br />
while (!FEof()) // This is code for '010 Editor' that means 'keep reading until the end of the file'<br />
{<br />
dbcache_entry_v7 entry;<br />
}<br />
};<br />
<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=TACT&diff=26572TACT2019-01-22T18:31:41Z<p>Simca: /* World of Warcraft */ Added 243's key.</p>
<hr />
<div>TACT ('''T'''rusted '''A'''pplication '''C'''ontent '''T'''ransfer) is the name of the content transfer part of [[NGDP]]. It is often used to install games that use [[CASC]] filesystems, but also supports non-CASC based products.<br />
<br />
=Product Information=<br />
Upon initial release, TACT used (and still uses) regular HTTP to retrieve information for its products. In early 2018, Blizzard started working on a new system that will likely eventually replace this. More information can be found on the [[Ribbit]] page. For now, the information retrieved via HTTP and the newer system are mostly in sync (with minor differences). Over time it is likely the older HTTP system will become deprecated. <br />
<br />
==HTTP URLs==<br />
Keep in mind these might be replaced with [[Ribbit]] at any time. Implement/rely on these at your own risk! <br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="width: 200px" | URL<br />
! style="width: 550px" | Description<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/cdns || a table of CDN domains available with game data per region<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/versions || current version, buildconfig, cdnconfig, productconfig and optionally keyring per region<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/bgdl || similar to versions, but tailored for use by the Battle.net App background downloader<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blobs || contains InstallBlobMD5 and GameBlobMD5<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blob/game || a blob file that regulates game functionality for the Battle.net App<br />
|-<br />
| http://us.patch.battle.net:1119/(product)/blob/install || a blob file that regulates installer functionality for the game in the Battle.net App<br />
|}<br />
<br />
==Products==<br />
The following products are known to have existed at one point. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Non-Games<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| agent || Battle.net Agent || style="color: green;" | Active<br />
|-<br />
| agent_test || Probably Agent test || <br />
|-<br />
| bna || Battle.net App || style="color: green;" | Active<br />
|-<br />
| bts || Bootstrapper || style="color: orange;" | Partial (only versions as of now, cdnpath=tpr/bnt004)<br />
|-<br />
| catalogs || Catalog || style="color: green;" | Active<br />
|-<br />
| clnt || Client || style="color: red;" | Deprecated<br />
|-<br />
| demo || || style="color: orange;" | Partial<br />
|-<br />
| test || || style="color: red;" | Deprecated<br />
|}<br />
<br />
Most games have multiple products. Tables for these are collapsed by default to not affect page length. Click Expand to see all known products for a game. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Diablo III <br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| d3 || Diablo 3 Retail || style="color: green;" | Active<br />
|-<br />
| d3b || Diablo 3 Beta (2013) || style="color: orange;" | Partial<br />
|-<br />
| d3cn || Diablo 3 China || style="color: green;" | Active<br />
|-<br />
| d3cnt || Diablo 3 China Test (?) || style="color: orange;" | Unused (everything empty)<br />
|-<br />
| d3t || Diablo 3 Test || style="color: green;" | Active<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Destiny 2<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| dst2 || Destiny 2 || style="color: green;" | Active<br />
|-<br />
| dst2a || Destiny 2 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| dst2dev || Destiny 2 "takehome" development || style="color: orange;" | Partial (encrypted?)<br />
|-<br />
| dst2e1 || Destiny 2 Event || Active; probably 1..9<br />
|-<br />
| dst2igr || Destiny 2 Internet Game Room || style="color: green;" | Active<br />
|-<br />
| dst2t || Destiny 2 Public Test || style="color: green;" | Active<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Heroes of the Storm<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| bnt || Heroes of the Storm Alpha || style="color: red;" | Deprecated<br />
|-<br />
| hero || Heroes of the Storm Retail || style="color: green;" | Active<br />
|-<br />
| heroc || Heroes of the Storm Tournament || style="color: green;" | Active<br />
|-<br />
| herot || Heroes of the Storm Test || style="color: green;" | Active<br />
|-<br />
| storm || Heroes of the Storm || style="color: red;" | Deprecated<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Hearthstone<br />
|-<br />
| hsb || Hearthstone Retail || style="color: green;" | Active<br />
|-<br />
| hsc || Hearthstone Chournament || style="color: green;" | Active<br />
|-<br />
| hst || Hearthstone Test || style="color: orange;" | Partial<br />
|-<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Overwatch (aka Prometheus)<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| pro || Overwatch Retail || style="color: green;" | Active<br />
|-<br />
| proc || Overwatch Tournament US || style="color: green;" | Active<br />
|-<br />
| proc_cn || Overwatch Tournament China || style="color: green;" | Active<br />
|-<br />
| proc_eu || Overwatch Tournament Europe || style="color: green;" | Active<br />
|-<br />
| proc_kr || Overwatch Tournament Korea || style="color: green;" | Active<br />
|-<br />
| proc2 || Overwatch Professional 2 (prometheus_tournament_2) || style="color: green;" | Active<br />
|-<br />
| proc2_cn || Overwatch Professional 2 China || style="color: green;" | Active<br />
|-<br />
| proc2_eu || Overwatch Professional 2 Europe || style="color: green;" | Active<br />
|-<br />
| proc2_kr || Overwatch Professional 2 Korea || style="color: green;" | Active<br />
|-<br />
| proc3 || Overwatch Tournament (Dev) || <br />
|-<br />
| prodev || Overwatch Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| proe || || Not on public CDNs<br />
|-<br />
| prot || Overwatch Test || style="color: green;" | Active<br />
|-<br />
| prov || Overwatch Vendor || style="color: green;" | Active<br />
|-<br />
| proms || Overwatch World Cup Viewer || style="color: orange;"| Partial<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|StarCraft 1<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| s1 || StarCraft 1 || style="color: green;" | Active<br />
|-<br />
| s1a || Starcraft 1 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| s1t || StarCraft 1 Test || style="color: green;" | Active<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|StarCraft II<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| s2 || StarCraft II Retail || style="color: green;" | Active<br />
|-<br />
| s2b || StarCraft II Beta || style="color: red;" | Deprecated<br />
|-<br />
| s2t || StarCraft II Test || style="color: red;" | Deprecated<br />
|-<br />
| sc2 || StarCraft II || style="color: red;" | Deprecated<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Call of Duty Black Ops 4<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| viper || Call of Duty Black Ops 4 || style="color: green;" | Active<br />
|-<br />
| viperdev || Call of Duty Black Ops 4 - Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| viperv1 || Call of Duty Black Ops 4 Vendor || <br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|Warcraft III<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| w3 || Warcraft III || style="color: green;"| Active<br />
|-<br />
| w3t || Warcraft III Public Test || style="color: green;"| Active<br />
|-<br />
| war3 || Warcraft III (old) || style="color: orange;"| Partial<br />
|}<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black" class="mw-collapsible mw-collapsed wikitable"<br />
! style="background:grey; color:white; border-bottom:1.5px solid black; width: 668px;" colspan="3"|World of Warcraft<br />
|-<br />
! width="90" | Product<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| wow || World of Warcraft Retail || style="color: green; | Active<br />
|-<br />
| wow_beta || World of Warcraft Alpha/Beta || style="color: green; | Active<br />
|-<br />
| wow_classic || World of Warcraft Classic || Not on public CDNs (yet?)<br />
|- <br />
| wowdev || World of Warcraft Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowdemo || World of Warcraft (Classic) Demo || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowe1 || World of Warcraft Event 1 || style="color: green;" | Active<br />
|-<br />
| wowe2 || World of Warcraft Event 2 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowe3 || World of Warcraft Event 3 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowt || World of Warcraft Test || style="color: green;" | Active<br />
|-<br />
| wowv || World of Warcraft Vendor || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowz || World of Warcraft Submission (previously Vendor) || style="color: green; | Active<br />
|}<br />
<br />
=Glossary=<br />
This page uses many terms. Here's an overview of stuff you need to know to be able to understand most of the explanations below. <br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! style="width: 100px"| Term<br />
! style="width: 550px" | Description<br />
|-<br />
|Filename || The file's real filename. Note that one file in TACT can have many names referring to it - essentially, one encoding key can map to many different name hashes.<br />
|-<br />
|Name Hash/Lookup || The filename after being hashed with the [[#hashpath|Jenkins Hash]].<br />
|-<br />
|Content Hash/CKey || The MD5 of the entire file in its uncompressed state; the purest representation of the data.<br />
|-<br />
|Encoding Hash/EKey || MD5 hash of the potentially encoded file. For unencoded files, the content hash/CKey. For chunkless [[BLTE]] files lacking a chunk table, this hash covers the entire encoded file. For chunked [[BLTE]] files, this hash covers only the [[BLTE]] headers including chunk table, as the chunk table contains hashes of the content of each chunk. A given file can be encoded in many ways, and a single content hash may potentially have multiple encoding keys.<br />
|-<br />
| CDN Key || The key used to lookup a file on the CDN. Same as encoding hash/EKey.<br />
|}<br />
<br />
=File types=<br />
Files are stored on CDNs in the following format: <tt>http://(cdnsHost)/(cdnsPath)/(pathType)/(FirstTwoHexOfHash)/(SecondTwoHexOfHash)/(FullHash)</tt><br />
<br />
Current list of CDNs for a specific product can be found in the <tt>http://us.patch.battle.net:1119/(product)/cdns</tt> file or via [[Ribbit]] in the <tt>v1/products/(product/)cdns</tt> endpoint. Blizzard regularly shuffles around CDNs and sometimes even adds/removes CDNs, so be sure to parse <tt>cdns</tt> to stay up to date. The CDN path found in the <tt>cdns</tt> file very rarely changes, the only known occurence is Overwatch changing from <tt>tpr/pro</tt> to <tt>tpr/ovw</tt>.<br />
<br />
Known path types are:<br />
* config - contains the three types of config files: Build configs, CDN configs, and Patch configs<br />
* data - contains archives, indexes, and unarchived standalone files (typically binaries, mp3s, and movies and files mentioned in buildconfig like root, install and download)<br />
* patch - contains patch manifests, files, archives, indexes<br />
<br />
Blizzard regularly cleans old builds from the CDN so any example files mentioned in this article might be unavailable at the time of reading (feel free to update them to something more recent if this is the case).<br />
<br />
Product configs referred to in the versions file are generally saved in <tt>tpr/configs</tt> for all games, but to be sure this path should be found by parsing the cdns file for the product in question.<br />
==Config Files==<br />
<br />
===Build Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/22/38/2238ab9c57b672457a2fa6fe2107b388</tt><br />
<br />
Some of the files listed in this file are explained later on in this article.<br />
<br />
What follows is a table for all known variables that have been seen in a buildconfig.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Root|root]] || Content hash of the '''decoded''' root file, look this up in encoding to get the '''encoded''' hash.<br />
|-<br />
| [[#Install|install]] || First key is the content hash of the '''decoded''' install file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash/CDN key.<br />
|-<br />
| install-size || Install size(s) corresponding to the install hash(es). Absent in older WoW builds.<br />
|-<br />
| [[#Download|download]] || First key is the content hash of the '''decoded''' download file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash.<br />
|-<br />
| download-size || Download size(s) corresponding to the download hash(es). Absent in older WoW builds.<br />
|-<br />
| [[#Download Size|size]] || CKey and EKey of the download size file, respectively. Introduced in WoW build 27547.<br />
|-<br />
| size-size || Download size sizes corresponding to the download size keys.<br />
|-<br />
| build-partial-priority || Content hash:block size pairs for priority non-archived files, ordered by priority. The block size appears to be the amount to request in each partial HTTP request. These files are [[BLTE]]-encoded and usually but not always split into blocks of fixed uncompressed size where the partial priority block size is a power-of-two multiple of the file's uncompressed block size. First seen in build 24116. Optional.<br />
|-<br />
| partial-priority || Content hash of a partial download file containing priority files to download first; note that the sizes in the download file entries are download block sizes, '''not''' file sizes. First seen in build 22231, replaced by build-partial-priority in build 24116. Optional.<br />
|-<br />
| partial-priority-size || Unknown: always 0 if present. Present if partial-priority is present.<br />
|-<br />
| [[#Encoding|encoding]] || First key is the content hash of the '''decoded''' encoding file. Second key is the '''encoded''' hash. If either none or 1 is found, the game (at least Wow) switches to plain-data (?) . Seen in build 20173<br />
|-<br />
| encoding-size || Encoding sizes corresponding to the encoding hashes.<br />
|-<br />
| [[#Patch|patch]] || The manifest of patchable data files. Does not include the metadata files e.g. encoding, whose patches are specified in the patch config file. Optional.<br />
|-<br />
| patch-size || Size of the patch manifest, if any. Optional.<br />
|-<br />
| [[#Patch_Config|patch-config]] || Content hash of non-encoded patch config (see Patch Config)<br />
|-<br />
| build-attributes || Optional. Seen in agent.<br />
|-<br />
| build-branch || Optional. Presumably the SCM branch built.<br />
|-<br />
| build-comments || Optional.<br />
|-<br />
| build-creator || Optional. Presumably the user who submitted the build.<br />
|-<br />
| build-fixed-hash || Optional. Seen in S2.<br />
|-<br />
| build-replay-hash || Optional. Seen in S2.<br />
|-<br />
| build-name || Optional? Name of the build<br />
|-<br />
| build-playbuild-installer || Optional? Type of installer for the Battle.net app to use<br />
|-<br />
| build-product || Optional? Product name<br />
|-<br />
| build-t1-manifest-version || Optional.<br />
|-<br />
| build-uid || Optional? Program code (see Products)<br />
|}<br />
<br />
===CDN Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/42/33/423364147752a596911aa1de2ff1f6a4</tt><br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Archives|archives]] || CDN keys of all archives (and by appending .index to the hash their indexes)<br />
|-<br />
| [[#Archive-Group_Index_.28.index.29|archive-group]] || CDN key of the the combined index file (see Archive-Group Index)<br />
|-<br />
| file-index || CDN key of .index file for unarchived files. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| file-index-size || Size of unarchived file .index.<br />
|-<br />
| [[#patch-archives|patch-archives]] || CDN keys of patch archives (needs research)<br />
|-<br />
| patch-archive-group || CDN key of probably the combined patch index file (needs research)<br />
|-<br />
| [[#Build_Config|builds]] || List of build configs this config supports (optional)<br />
|-<br />
| patch-file-index || CDN key of .index file for unarchived patches. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| patch-file-index-size || Size of unarchived patch .index.<br />
|}<br />
<br />
===Patch Config===<br />
Example file: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/config/20/f0/20f0593dd1b9fdaeaa7a808f83d48f1d</tt><br />
<br />
This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 in build 19027 (October 10th, 2014).<br />
The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material.<br />
The structure and purpose of all of the fields of this file requires further research.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| patch-entry || Repeats 3 times with patch entries for install, download and encoding file. Builds using the partial-priority (not build-partial-priority) mechanism may additionally have a partial-priority entry.<br />
|-<br />
| patch || Patch manifest file<br />
|-<br />
| patch-size || Size of patch file (optional?)<br />
|}<br />
<br />
====patch-entry====<br />
<br />
The format of these strings is:<br />
<br />
<pre>patch-entry = <type> <content hash> <content size> <BLTE-encoding key> <BLTE-encoded size> <encoding string></pre><br />
<br />
followed by sets of<br />
<br />
<pre><old BLTE-encoding key> <old content size> <patch hash> <patch size></pre><br />
<br />
This serves to pair old files to the patch needed to bring them up to date, as well as providing the information required to detect if the file is already up to date.<br />
<br />
==Data Files==<br />
<br />
Example index: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef.index</tt><br />
<br />
Example archive: <tt>http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef</tt><br />
<br />
==Patch Files==<br />
<br />
==Encoding table==<br />
The encoding file maps content hashes <tt>C-Key</tt>s to encoded-file hashes <tt>E-Key</tt>s. In addition, there is information on how the files are [[BLTE]]-encoded by <tt>E-Spec</tt>s.<br />
<br />
Blocks in this file are, in this order<br />
<br />
* header<br />
* encoding specification data <tt>ESpec</tt><br />
* content key → encoding key table <tt>CEKeyPageTable</tt><br />
* encoding key → encoding spec table <tt>EKeySpecPageTable</tt><br />
* encoding specification data for the encoding file itself<br />
<br />
An incomplete/outdated 010 Editor template can be found at [https://gist.github.com/heksesang/fdda3e4f8a5ed53b71ed this gist] which can be used to understand page handling.<br />
<br />
====Header====<br />
Header is a constant 0x16 bytes giving size information for the other blocks, mostly.<br />
<br />
struct {<br />
/*0x00*/ char signature[2]; // "EN"<br />
enum {<br />
encoding_version_1 = 1, // {{Sandbox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125}}<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ckey;<br />
/*0x04*/ uint8_BE_t hash_size_ekey;<br />
/*0x05*/ uint16_BE_t CEKeyPageTable_page_size_kb; // in kilo bytes. e.g. 4 in here → 4096 byte pages (default)<br />
/*0x07*/ uint16_BE_t EKeySpecPageTable_page_size_kb; // ^<br />
/*0x09*/ uint32_BE_t CEKeyPageTable_page_count;<br />
/*0x0D*/ uint32_BE_t EKeySpecPageTable_page_count;<br />
/*0x11*/ uint8_BE_t _unknown_x11; // 0 -- sometimes assumed to be part of ESpec_block_size, but actually asserted to be zero by agent<br />
/*0x12*/ uint32_BE_t ESpec_block_size;<br />
/*0x16*/<br />
} header;<br />
<br />
====ESpec====<br />
Encoding specification strings are just a blob of zero terminated strings referenced by <tt>EKeySpecPageTable</tt> with the accumulated size of <tt>header.ESpec_block_size</tt> (including zero terminators).<br />
<br />
The definition of the format for these strings is described on the [[BLTE#Encoding_Specification_.28ESpec.29|BLTE]] page.<br />
<br />
====Page Tables====<br />
The format of the two page tables is the same: <br />
<br />
* an index for fast key → page access, followed by<br />
* the actual pages with specific content<br />
<br />
In both cases, the entries in the lists have the same count, and semi-dynamic size, depending on <tt>header.hash_size_ckey</tt> and <tt>header.hash_size_ekey</tt>. Note that the page checksum size is fixed to MD5's 16 bytes. The position comments below assume the standard key size of 16 bytes.<br />
<br />
struct page_index_t {<br />
/*0x00*/ char first_Xkey[header.hash_size_Xkey]; // where X is c for CEKeyPageTable and e for EKeySpecPageTable<br />
/*0x10*/ char page_md5[0x10];<br />
/*0x20 usually*/<br />
};<br />
<br />
The pages themselves are filled as much possible with actual entry structs. They are padded to the end for alignment of pages. Pages don't actually have to be full.<br />
<br />
=====CEKeyPageTable=====<br />
This table maps one <tt>ckey</tt> to one or more <tt>ekey</tt>s. This means that there can be multiple representations, e.g. an encrypted and an unencrypted version. This isn't usually the case and on reading any of them can be picked, since they represent the same file content. It can be used to pick an already downloaded archive, or unencrypted one, or handle deleted archives.<br />
<br />
struct ckey_ekey_entry_t {<br />
/*0x00*/ uint8_BE_t keyCount;<br />
/*0x01*/ uint40_BE_t file_size; // of the non-encoded version of the file<br />
/*0x06*/ char ckey[header.hash_size_ckey]; // this ckey is represented by…<br />
/*0x16*/ char ekey[header.hash_size_ekey][keyCount]; // …these ekeys<br />
/*0x26 usually*/<br />
} page_entries[];<br />
<br />
=====EKeySpecPageTable=====<br />
This table maps one <tt>ekey</tt> to the corresponding <tt>espec</tt> describing how the encoding happened.<br />
<br />
struct ekey_espec_entry_t {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint32_BE_t espec_index; // not an offset but an index, assuming zero-terminated espec strings<br />
/*0x14*/ uint40_BE_t file_size; // of the encoded version of the file<br />
/*0x19 usually*/<br />
} page_entries[];<br />
<br />
==Install manifest==<br />
File signature: "IN"<br />
<br />
The install file lists files installed on disk. Since the install file is shared by architectures and OSs, there are also tags to select a subset of files. When using multiple tags, a binary combination of the bitfields of files to be installed can be created. <br />
<br />
====Header Structure====<br />
The file begins with a 10 byte header describing the number of tags and files listed. '''Structure names were invented by the author of this page.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[2] || FileSignature || "IN"<br />
|-<br />
| 0x02 || uint8_t || Version? || 1<br />
|-<br />
| 0x03 || uint8_t || hash_size || size of hashes used for files (usually md5 -> 16)<br />
|-<br />
| 0x04 || uint16_BE_t || num_tags || number of tags in header of file<br />
|- <br />
| 0x06 || uint32_BE_t || num_entries || The number of entries in the body of the file<br />
|}<br />
<br />
====Tags Structure====<br />
After the header, an array with information about available tags follows. Each tag has a bitfield listing the files installed when the given tag is chosen. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || name || <br />
|- <br />
| uint16_BE_t || type || A number shared amongst specific flags. Actual meaning is [[#Product_Specific|specific to products]].<br />
|- <br />
| char[divru (header.entries, CHAR_BIT)] || files || A bitfield that lists which files are installed when the specified tag is installed.<br />
|}<br />
<br />
====Files Structure====<br />
The remainder of the file is populated by a list of files with their content hash, each a variable size (due to the strings). Structure names were invented by the author of this page.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || FileName || The name of the file.<br />
|- <br />
| char[header.hash_size] || hash || The hash of the uncompressed file. Usually MD5.<br />
|- <br />
| uint32_BE_t || Size || The size of the file.<br />
|}<br />
<br />
====C-like structure====<br />
char I; char N;<br />
uint8_BE_t _unk3;<br />
uint8_BE_t hash_size;<br />
uint16_BE_t num_tags;<br />
uint32_BE_t num_files;<br />
<br />
struct {<br />
string name;<br />
uint16_BE_t type;<br />
char flags[divru (num_files, CHAR_BIT)];<br />
} tags[num_tags];<br />
<br />
struct {<br />
string name;<br />
char hash[hash_size];<br />
uint32_BE_t size;<br />
} files[num_files];<br />
<br />
==Download manifest==<br />
The download file lists all files stored in the data archives.<br />
The client uses this to download files ahead of time, without it, the client will download on demand which can lead to issues. <br />
The download priority is set inside the entries with 0 being the highest and 2 the lowest however, if the game is running, missing assets in the player's vicinity take precedence.<br />
<br />
Just like the install file, the download file is shared across all architectures and locales so utilizes the same bitfield-tag system to assess what subset of files are needed.<br />
<br />
NOTE: partial-priority download files do not contain the actual file sizes but redefine the FileSize field to ChunkSize.<br />
<br />
This file has this structure:<br />
<br />
* Header<br />
* Entries[Header.EntryCount]<br />
* Tags[Header.TagsCount]<br />
<br />
====Download Header====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "DL")<br />
|-<br />
| char || Version || 1 < 7.3.0, 2<br />
|-<br />
| char || ChecksumSize || Always 0x10<br />
|-<br />
| char || unk || ??? Always 1<br />
|-<br />
| int [BE] || EntryCount || The amount of file entries in this file<br />
|-<br />
| short [BE] || TagCount || The amount of tag entries in this file<br />
|}<br />
<br />
====Download Entry====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="120" | Version Added<br />
! width="810" | Description<br />
|-<br />
| char || Unk || 2 || ??? Appears to be a boolean. Currently only set to 1 on 4 specific records<br />
|-<br />
| char[16] || Hash || 1 || This hash is found in every node of the encoding file. (Reverse lookup) MD5<br />
|-<br />
| uint40_t [BE] || FileSize || 1 || The compressed size of the file<br />
|-<br />
| char || DownloadPriority || 1 || 0 = highest, 2 = lowest; -1 for the Install since {{Unverified|{{Sandbox/PrettyVersion|expansionlevel=8}}}}<br />
|-<br />
| char[4] || Unk || 1 || ???<br />
|}<br />
<br />
====Download Tag====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| string || Name || A C-String indicating this tag's Name.<br />
|-<br />
| short [BE] || Type || Hash type<br />
|-<br />
| char[N] || Bits || an array of size N = Header.EntryCount / 8 + (Header.EntryCount % 8 > 0 ? 1 : 0); that is basically a massive bit mess. Use Schroeppel's 8 bits reverse function on it to have bits.<br />
|}<br />
<br />
====code-ish====<br />
struct {<br />
/*0x00*/ char signature[2]; // "DL"<br />
enum {<br />
download_version_1 = 1,<br />
download_version_2 = 2, // {{Unverified|{{Sandbox/VersionRange|min_expansionlevel=7|min_build=7.3.0.???}}}}<br />
download_version_3 = 3,<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ekey;<br />
/*0x04*/ uint8_BE_t has_checksum_in_entry;<br />
/*0x05*/ uint32_BE_t entry_count;<br />
/*0x09*/ uint16_BE_t tag_count;<br />
/*0x0b*/<br />
<br />
#if version ≥ download_version_2<br />
/*0x0b*/ uint8_BE_t {{Unverified|number_of_flag_bytes}}; // defaults to 0, up to {{Unverified|4}}<br />
/*0x0c*/<br />
<br />
#if version >= download_version_3<br />
/*0x0c*/ uint8_BE_t {{Unverified|base_priority}}; // defaults to 0<br />
/*0x0d*/ char _unknown_0d[3]; // As of 1.15.6.2-test4, this is explicitly 0. It is ignored on reading.<br />
/*0x10*/<br />
#endif<br />
#endif<br />
} header;<br />
<br />
struct {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint40_BE_t file_size;<br />
/*0x15*/ uint8_BE_t priority; // {{Unverified|header.base_priority is subtracted on parse}}<br />
/*0x16*/<br />
<br />
#if header.has_checksum_in_entry<br />
/*0x16*/ uint32_BE_t checksum;<br />
/*0x1a*/<br />
#endif<br />
<br />
#if header.version ≥ download_version_2<br />
enum {<br />
download_flag_plugin = 1, // "plugin"<br />
download_flag_plugin_data = 2, // "plugin-data"<br />
}; <br />
uint8_BE_t flags[header.number_of_flag_bytes]; // {{Unverified|defaults to 0 if no flag bytes present}}<br />
#endif<br />
<br />
} entries[header.entry_count];<br />
<br />
struct {<br />
char const name[]; // this string is zero terminated, no fixed size<br />
// thus for readability we start offset at 0 here.<br />
/*0+00*/ uint16_BE_t type; // game specific. usually architecture, category, locale, os, region or alike.<br />
/*0+02*/ char mask[divru (header.entry_count._, CHAR_BIT)]; // if bit is set, entries[bit] is part of this tag<br />
/*0x??*/<br />
} tag[header.tag_count];<br />
<br />
==Download Size ==<br />
Build 27547 introduced the Download Size file of unknown purpose. The file is a stripped-down Download file with partial EKeys and files sorted by encoded files size. The purpose of this file is not clear.<br />
<br />
struct Header<br />
{<br />
char signature[2]; // "DS"<br />
uint8_t version;<br />
uint8_t ekeySize; // 9<br />
uint32_BE_t numFiles;<br />
uint16_BE_t numTags;<br />
uint40_BE_t totalSize; // Size of all files combined<br />
};<br />
<br />
struct TagEntry<br />
{<br />
char name[]; // Null-terminated<br />
uint16_BE_t type;<br />
char fileMask[(hdr.numFiles + 7) / 8];<br />
}<br />
<br />
struct FileEntry<br />
{<br />
char ekey[hdr.ekeySize];<br />
uint32_BE_t esize;<br />
};<br />
<br />
SizeHeader hdr;<br />
TagEntry tags[hdr.numTags];<br />
FileEntries files[hdr.numFiles]; // Sorted descending by esize<br />
<br />
==Patch==<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "PA")<br />
|-<br />
| char || version || 1 or 2<br />
|-<br />
| char || file_key_size || <= 0x10<br />
|-<br />
| char || size_b || <= 0x10<br />
|-<br />
| char || patch_key_size || <= 0x10<br />
|-<br />
| char || block_size_bits || 2 <= block_size_bits <= 24. block size == 2^block_size_bits.<br />
|-<br />
| short || block_count ||<br />
|-<br />
| char || flags ||<br />
|-<br />
| char[16] || EncodingCkey || ckey for encoding file<br />
|-<br />
| char[16] || EncodingEkey || ekey for encoding file<br />
|-<br />
| int || DecodedSize || Decoded encoding file size in bytes<br />
|-<br />
| int || EncodedSize || Encoded encoding file size in bytes<br />
|-<br />
| char || EspecLength || Length of the following string<br />
|-<br />
| char[EspecLength] || EncodingEspec || espec of encoding file<br />
|-<br />
| char[] || ??? || byte array containing blocks entries, blocks, and optional patch tail<br />
|}<br />
<br />
header+entries needs to be less than 0x10000 bytes (at least in wow-18179). md5sum is only checked for header+entries, file might be larger thus.<br />
<br />
struct PatchManifest_Header<br />
{<br />
uint16_BE_t magic; // 'PA'<br />
uint8_t version; // 1 or 2<br />
uint8_t file_key_size; // <= 0x10<br />
uint8_t size_b; // <= 0x10<br />
uint8_t patch_key_size; // <= 0x10<br />
uint8_t block_size_bits; // 12 <= block_size_bits <= 24. max block size == 2^block_size_bits<br />
uint16_BE_t block_count; // (file_key_size + 20) * entry_count + sizeof (PatchManifest_Header) < 0x10000<br />
uint8_t unk2; // flags<br />
<br />
#if encoding_information_apparently_added_after_18179<br />
uint8_t encoding_ckey[16];<br />
uint8_t encoding_ekey[16]; // probably since PA2<br />
uint32_BE_t decoded_size;<br />
uint32_BE_t encoded_size;<br />
uint8_t encoding_espec_length;<br />
char encoding_format[encoding_espec_length];<br />
#endif<br />
} header;<br />
<br />
struct PatchManifest_Block<br />
{<br />
uint8_t last_file_ckey[header.file_key_size];<br />
uint8_t md5_of_block[16];<br />
uint32_BE_t block_offset; // in this file<br />
} blocks[header.block_count]; // sorted ascending by key<br />
<br />
// at positions given in PatchManifest_Block<br />
struct block<br />
{<br />
struct<br />
{<br />
uint8_t num_patches; // <= 0x10.<br />
uint8_t target_file_ckey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
struct<br />
{<br />
uint8_t source_file_ekey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
uint8_t patch_ekey[header.patch_key_size];<br />
uint32_BE_t patch_size;<br />
uint8_t unk; // some sort of patch index number. first entry seems to always be 1<br />
} patches[num_patches];<br />
} files[]; // count unspecified: read until the next file num_patches would be 0 <br />
// OR block would exceed max block size<br />
};<br />
<br />
// some files have a block of data after the last block of the patch manifest (which may be shorter than max block size). this block appears to be a patch of encoding, but the format is not understood.<br />
<br />
=Product Specific=<br />
In this section, the game/usage specific parts of CASC are describe. While CASC is a generic format, a lot of stuff is hardcoded, like hash sizes. Other parts are actually left up to the implementation, like root files or download tags.<br />
<br />
==World of Warcraft==<br />
===Root===<br />
struct CASRecord {<br />
char content_key[16]; // MD5 hash of the file's raw data<br />
uint64 name_hash; // Jenkins96 (lookup3) hash of the file's path <br />
};<br />
<br />
enum locale_flags : uint32 {<br />
enUS = 0x2,<br />
koKR = 0x4,<br />
frFR = 0x10,<br />
deDE = 0x20,<br />
zhCN = 0x40,<br />
esES = 0x80,<br />
zhTW = 0x100,<br />
enGB = 0x200,<br />
enCN = 0x400,<br />
enTW = 0x800,<br />
esMX = 0x1000,<br />
ruRU = 0x2000,<br />
ptBR = 0x4000,<br />
itIT = 0x8000,<br />
ptPT = 0x10000,<br />
};<br />
enum content_flags : uint32 {<br />
LowViolence = 0x80,<br />
Bundle = 0x40000000,<br />
NoCompression = 0x80000000,<br />
};<br />
<br />
struct CASBlock {<br />
int32 num_records;<br />
content_flags flags;<br />
locale_flags locale;<br />
int32 fileDataIDDeltas[num_records]; // each block starts with 0, +1 is implicit per entry, so consecutive ids will have delta=0<br />
CASRecord records[num_records];<br />
<br />
int32 file_data_id (size_t index) const<br />
{<br />
return index == 0<br />
? fileDataIDDeltas[index]<br />
: file_data_id (index - 1) + 1 + fileDataIDDeltas[index];<br />
}<br />
};<br />
<br />
while (FTell() < FileSize())<br />
CASBlock blocks;<br />
====hashpath====<br />
This function is used in the root file by WoW and other older MPQ-based games to calculate filename lookups.<br />
<br />
hashpath (string path) → uint32_t<br />
{<br />
string normalized = toupper (path).replace (from: '/', to: '\\')<br />
uint32_t pc = 0, pb = 0;<br />
hashlittle2 (normalized, strlen (normalized), &pc, &pb);<br />
return pc;<br />
}<br />
<br />
===Tags===<br />
Values depend on versions, semantic categories are cross version.<br />
<br />
* '''Platform''': The deployment target, i.e. Windows or OSX<br />
* '''Architecture''': Sub-division of the deployment target, i.e. x86_32 or x86_64<br />
* '''Locale''': The same as in [[Localization]]: Files specific to a single localisation of the game.<br />
* '''Region''': Equivalent to the patch server regions, i.e. us, eu, kr, tw, cn.<br />
* '''Category''': A replacement for the MPQ system to tag low priority downloads: speech, text<br />
* '''Alternate''': A special category for censored content.<br />
<br />
====Version specific values====<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125|max_expansionlevel=6|max_build=6.0.1.18761}}<br />
<br />
Architecture = 1, Locale = 2, Platform = 3<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18764|max_expansionlevel=6|max_build=6.2.2.20426}}<br />
<br />
Architecture = 1, Category = 2, Locale = 3, Platform = 4, Region = 5<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.2.2.20438|max_expansionlevel=7}}<br />
<br />
Platform = 1, Architecture = 2, Locale = 3, Region = 4, Category = 5<br />
<br />
{{SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26604|note={{Unverified|Actually alternate was just missing above but schlumpf doesn't want to verify since when}}}}<br />
<br />
enum {<br />
platform = 1,<br />
architecture = 2,<br />
locale = 3,<br />
region = 4,<br />
category = 5,<br />
alternate = 0x4000,<br />
};<br />
<br />
=Armadillo=<br />
If <tt>$program/blob/game</tt> or <tt>productconfig</tt> contains <tt>decryption_key_name</tt>, data on CDN is encrypted. The key is retrieved from the keychain or manually entered (a base32 encoded version that contains key and checksum is used). It is stored in <tt>$key_name.ak</tt>. The key file contains the key followed by a the first four bytes of the md5 hash of the key for verification purposes.<br />
<br />
struct {<br />
/*0x00*/ unsigned char key[0x10];<br />
/*0x10*/ unsigned char md5_of_key[0x4]; // first four bytes only<br />
/*0x14*/<br />
} ak_file;<br />
<br />
Encryption uses Salsa20. The IV is the last 8 bytes (16 characters) of the cdn hash.<br />
<br />
==Known keys==<br />
decryption_key_name used_by key checksum base32<br />
<br />
# Starcraft 1<br />
sc1Dev s1a F6 79 DC 38 E0 C3 65 FB 48 2E 48 A7 48 90 9D 29 19 F5 BB 88 6Z45YOHAYNS7WSBOJCTUREE5FEM7LO4I<br />
<br />
# Overwatch<br />
pro pro (pre-launch) <br />
proc proc3<br />
prolivedev prodev <br />
provendor prov<br />
<br />
# World of Warcraft<br />
wowdemo wowdemo<br />
wowdevalpha wowdev<br />
wowvendor wowv<br />
<br />
# Destiny 2<br />
destiny2_openbeta dst2<br />
destiny_dev dst2a (pre-alpha)<br />
destiny_event dst2e1<br />
destiny_live dst2a<br />
dst2livedev dst2dev<br />
<br />
# Call of Duty: Black Ops 4<br />
viperlivedev viperdev<br />
<br />
=CDN File Organization=<br />
Data for every CASC-based game exists on the CDN in one of two places at any given time. To reduce file-system and download overhead many files are packed into archives and indexed by archive indices, both of a different format than the CASC installations found on client systems; other "unarchived", "standalone", or "loose" files are stored as separate files on the CDN that must be downloaded independently. There are at least three different reasons why a particular file is found in one or the other:<br />
# Small files typically incur larger filesystem overhead and benefit most from being packed into archives. A rough rule of thumb appears to be that files smaller than 2 MB or so are put in archives. Presumably larger files are not archived because they make it more difficult to minimize the number of archive files, which are limited in size (a 2 MB file limit would limit unused space in an archive to under 0.8%, given enough data to form a full archive to begin with).<br />
# Key files such as encoding, partial-priority, TVFS (Warcraft 3 root), and the download, install, and patch manifests, as well as their respective patches, are typically stored loose for quick access and are rarely ever found in archives.<br />
# As games evolve old files become obsolete and are removed from the CDN. However, the archive system means that archives can only be removed when every file in the archive is no longer needed, potentially wasting large amounts of space on the CDN - the exact opposite of the purpose of bundling files into archives to begin with. Thus as the amount of unused data in an archive grows over time, files still in use may be converted to loose files to allow the archive to be purged from the CDN, even when the files are unusually small to be found independently.<br />
<br />
Previously, no official indices/manifests of loose files existed, and they could only be found by subtracting archived files from file lists in encoding or manifest files. Beginning with Warcraft 3 and subsequently being deployed more widely on 7/24/2018, new fields in the CDN config file link to index files containing "all" loose data or patch files on the current CDNs.<br />
<br />
===Archives===<br />
Archives are extensionless 256 MB files that are usually only stored on the Blizzard CDNs. Their naming follows the standard URL hash format using the '/data/' path type.<br />
<br />
The structure of the archives is presumably just file fragment after file fragment. You will never need to parse it because you can just look up offset + size of your file fragment in the index files and then take the piece directly out of the archive.<br />
<br />
The fragments are all [[BLTE]] encoded.<br />
<br />
The filename is '''NOT''' the hash of the archive content but the hash of the '''index''''s footer.<br />
<br />
===Archive Indexes (.index)===<br />
These '.index' files reveal to the user where the compressed game files are located within the archives. All indexes (except the Archive-Group index, see below) are named after their archive (only difference is these have an extension).<br />
'.index' files are stored on the CDN using the standard hash naming scheme (remember they have an extension though). They are also located in the directory 'INSTALL_DIR/Data/indices/' for a WoW install.<br />
<br />
====Normal Index Entry Structure====<br />
*'''The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null EKey fields. The last chunk is a table-of-contents, listing the LAST EKey in each chunk and checksum of each block. All checksums are the first checksumSize bytes of the MD5 of the respective data. Structure names were invented by the author of this page.'''<br />
<br />
struct index_entry {<br />
char EKey[footer.keySizeInBytes];<br />
uint_BE_t(footer.sizeBytes) blte_encoded_size;<br />
uint_BE_t(footer.offsetBytes) offset_to_blte_encoded_data_in_archive;<br />
};<br />
<br />
struct index_block {<br />
static constexpr const block_size = footer.blockSizeKb << 10;<br />
index_entry entries[block_size / sizeof (index_entry)];<br />
char padding[block_size - sizeof (entries)];<br />
} blocks[];<br />
<br />
struct {<br />
struct {<br />
char last_EKey[footer.keySizeInBytes]; // last EKey of a block<br />
} entries[num_blocks];<br />
<br />
struct {<br />
char lower_part_of_md5_of_block[footer.checksumSize]; <br />
} blocks_hash[num_blocks];<br />
} toc;<br />
<br />
struct {<br />
char toc_hash[checksumSize]; // client tries to read with 0x10, then backs off when smaller<br />
char version?; // always 1<br />
char _11; // 0<br />
char _12; // 0<br />
char blockSizeKb?; // Normally 4. Left-shifted by 10. Believed to be block size in KB.<br />
char offsetBytes; // Normally 4 for archive indices, 6 for group indices, and 0 for loose file indices<br />
char sizeBytes; // Normally 4<br />
char keySizeInBytes; // Normally 16<br />
char checksumSize; // Normally 8, <= 0x10<br />
uint32_t numElements; // BigEndian in _old_ versions (e.g. 18179)<br />
char footerChecksum[checksumSize];<br />
} footer;<br />
<br />
* footerChecksum is calculated over the footer beginning with version ''when footerChecksum is zeroed''<br />
* The archive/index name is the MD5 of the footer beginning with toc_hash<br />
<br />
===archive-group===<br />
<tt>archive-group</tt> is actually a very special <tt>.index</tt> file that is typically much larger than other indices. There is '''no''' archive attached, it is only the index. Also, the index is '''not''' found on the CDN! It is assembled by combining all index files given in the CDN config, on client side. The hash is just for verification purposes. Client side a file is assembled using the normal index file format, for caching.<br />
<br />
It has a single difference in format to normal indices: While other indices have their <tt>offsetBytes</tt> long <tt>offset</tt> field point into the archive, for archive groups, the field also has an <tt>archiveIndex</tt>:<br />
<br />
struct {<br />
uint16_BE_t archiveIndex; // Index of the archive in the CDN config's archive list<br />
uint32_BE_t offsetBytes; // The offset within the specified archive<br />
};<br />
<br />
It does not only having the offset, but also an index into a file. Semantically that's still an offset, so no further field for size is used.<br />
<br />
It is suggested you do not just parse indices by <tt>.index</tt> filename locally but take the config files into account. An easy heuristic is that if <tt>offsetBytes</tt> is not 4, it is a special index, either loose files or a group.<br />
<br />
===patch-archives===<br />
Patch archives are the /patch/ equivalent to the regular (data) archives on the CDN. Like archives, these are binary blobs of fragments indexed by an accompanying .index file with the same name. Again, the index is a <tt>hash, size, offset</tt> tuple, but the hash is the content hash rather than an encoding hash.<br />
<br />
Most files in patch archives are [[Patching Files|ZBSDIFF1]] blobs, though in principle any file that might be in the /patch/ namespace may be found in patch archives and must be handled accordingly.<br />
<br />
===patch-archive-group===<br />
See [[#archive-group|archive-group]]. There is no known difference other than the combined data being <tt>patch-archives</tt>.<br />
<br />
=TACT keys=<br />
TACT keys are used to encrypt data in files. In most cases, TACT keys are streamed to clients when content (such as cinematics in WoW or skins in Overwatch) is unlocked.<br />
<br />
== Battle.net app==<br />
key_name key type seen in<br />
2C547F26A2613E01 37C50C102D4C9E3A5AC069F072B1417D salsa20 Battle.net App Alpha 1.5.0<br />
<br />
== Overwatch ==<br />
Overwatch has keys included in the client/keyring, but they can also be streamed from the server.<br />
<div class="mw-collapsible mw-collapsed toccolours "><br />
The key table is collapsed by default to reduce page length. Click expand on the right to expand. <br />
<div class="mw-collapsible-content" id="mw-customcollapsible-myDivision"><br />
key_name key type seen in method used for<br />
FB680CB6A8BF81F3 62D90EFA7F36D71C398AE2F1FE37BDB9 salsa20 0.8.0.24919 keyring<br />
402CD9D8D6BFED98 AEB0EADEA47612FE6C041A03958DF241 salsa20 0.8.0.24919 keyring<br />
DBD3371554F60306 34E397ACE6DD30EEFDC98A2AB093CD3C salsa20 0.8.0.24919 keyring<br />
11A9203C9881710A 2E2CB8C397C2F24ED0B5E452F18DC267 salsa20 0.8.0.24919 keyring<br />
A19C4F859F6EFA54 0196CB6F5ECBAD7CB5283891B9712B4B salsa20 0.8.0.24919 keyring<br />
87AEBBC9C4E6B601 685E86C6063DFDA6C9E85298076B3D42 salsa20 0.8.0.24919 keyring<br />
DEE3A0521EFF6F03 AD740CE3FFFF9231468126985708E1B9 salsa20 0.8.0.24919 keyring<br />
8C9106108AA84F07 53D859DDA2635A38DC32E72B11B32F29 salsa20 0.8.0.24919 keyring<br />
49166D358A34D815 667868CD94EA0135B9B16C93B1124ABA salsa20 0.8.0.24919 keyring<br />
1463A87356778D14 69BD2A78D05C503E93994959B30E5AEC salsa20 ≤ 1.0.3.0 keyring<br />
5E152DE44DFBEE01 E45A1793B37EE31A8EB85CEE0EEE1B68 salsa20 ≤ 1.0.3.0 keyring<br />
9B1F39EE592CA415 54A99F081CAD0D08F7E336F4368E894C salsa20 ≤ 1.0.3.0 keyring<br />
24C8B75890AD5917 31100C00FDE0CE18BBB33F3AC15B309F salsa20 ≤ 1.0.3.0 keyring<br />
EA658B75FDD4890F DEC7A4E721F425D133039895C36036F8 salsa20 ≤ 1.0.3.0 keyring<br />
026FDCDF8C5C7105 8F41809DA55366AD416D3C337459EEE3 salsa20 keyring<br />
CAE3FAC925F20402 98B78E8774BF275093CB1B5FC714511B salsa20 keyring<br />
061581CA8496C80C DA2EF5052DB917380B8AA6EF7A5F8E6A salsa20 keyring<br />
BE2CB0FAD3698123 902A1285836CE6DA5895020DD603B065 salsa20 keyring<br />
57A5A33B226B8E0A FDFC35C99B9DB11A326260CA246ACB41 salsa20 1.1.0.0.30200 keyring Ana<br />
42B9AB1AF5015920 C68778823C964C6F247ACC0F4A2584F8 salsa20 1.2.0.1.30684 keyring Summer Games<br />
4F0FE18E9FA1AC1A 89381C748F6531BBFCD97753D06CC3CD salsa20 1.2.0.1.30684 keyring<br />
7758B2CF1E4E3E1B 3DE60D37C664723595F27C5CDBF08BFA salsa20 1.2.0.1.30684 keyring<br />
E5317801B3561125 7DD051199F8401F95E4C03C884DCEA33 salsa20 1.4.0.2.32143 keyring Halloween Terror<br />
16B866D7BA3A8036 1395E882BF25B481F61A4D621141DA6E salsa20 1.4.1.0.31804 keyring Bastion Blizzcon 2016 skin<br />
11131FFDA0D18D30 C32AD1B82528E0A456897B3CE1C2D27E salsa20 1.5.0.1.32795 keyring Sombra<br />
CAC6B95B2724144A 73E4BEA145DF2B89B65AEF02F83FA260 salsa20 1.5.0.1.32795 keyring Ecopoint: Antarctica<br />
B7DBC693758A5C36 BC3A92BFE302518D91CC30790671BF10 salsa20 1.5.0.1.32795 keyring Genji Oni Skin (HotS Nexus Challenge)<br />
90CA73B2CDE3164B 5CBFF11F22720BACC2AE6AAD8FE53317 salsa20 1.6.1.0.33236 keyring Oasis map<br />
6DD3212FB942714A E02C1643602EC16C3AE2A4D254A08FD9 salsa20 1.6.1.0.33236 keyring<br />
11DDB470ABCBA130 66198766B1C4AF7589EFD13AD4DD667A salsa20 1.6.1.0.33236 keyring Winter Wonderland<br />
5BEF27EEE95E0B4B 36BCD2B551FF1C84AA3A3994CCEB033E salsa20 keyring<br />
9359B46E49D2DA42 173D65E7FCAE298A9363BD6AA189F200 salsa20 keyring Diablo's 20th anniversary<br />
1A46302EF8896F34 8029AD5451D4BC18E9D0F5AC449DC055 salsa20 1.7.0.2.34156 keyring Year of the Rooster<br />
693529F7D40A064C CE54873C62DAA48EFF27FCC032BD07E3 salsa20 1.8.0.0.34470 keyring CTF Maps<br />
388B85AEEDCB685D D926E659D04A096B24C19151076D379A salsa20 1.8.0.0.34470 keyring Numbani Update (Doomfist teaser)<br />
E218F69AAC6C104D F43D12C94A9A528497971F1CBE41AD4D salsa20 1.9.0.0.34986 keyring Orisa<br />
F432F0425363F250 BA69F2B33C2768F5F29BFE78A5A1FAD5 salsa20 1.10.0.0.35455 keyring Uprising<br />
061D52F86830B35D D779F9C6CC9A4BE103A4E90A7338F793 salsa20 1.10.0.0.35455 keyring D.Va Officer Skin (HotS Nexus Challenge 2)<br />
1275C84CF113EF65 CF58B6933EAF98AF53E76F8426CC7E6C salsa20 keyring<br />
D9C7C7AC0F14C868 3AFDF68E3A5D63BABA1E6821883F067D salsa20 keyring<br />
BD4E42661A432951 6DE8E28C8511644D5595FC45E5351472 salsa20 1.11.0.0.36376 keyring Anniversary event<br />
C43CB14355249451 0EA2B44F96A269A386856D049A3DEC86 salsa20 1.12.0.0.37104 keyring Horizon Lunar Colony<br />
E6D914F8E4744953 C8477C289DCE66D9136507A33AA33301 salsa20 1.13.0.0.37646 keyring Doomfist<br />
5694C503F8C80178 7F4CF1C1FBBAD92B184336D677EBF937 salsa20 1.13.0.0.37646 keyring Doomfist<br />
21DBFD65F3E54269 AB580C3837CAF8A461F243A566B2AE4D salsa20 1.13.0.0.37646 keyring Summer Games 2017<br />
27ABA5F88DD8D078 ???????????????????????????????? salsa20 1.13.0.0.37646 ??????? ??????<br />
21E1F90E71D33C71 328742339162B32676C803C2255931A6 salsa20 1.14.1.0.39083 keyring Deathmatch<br />
D9CB055BCDD40B6E 49FB4477A4A0825327E9A73682BECD0C salsa20 1.15.0.0.????? keyring Junkertown<br />
8175CE3C694C6659 E3F3FA7726C70D26AE130D969DDDF399 salsa20 1.16.0.0.40011 keyring Halloween 2017<br />
B8DE51690075435A C07E9260BB711217E7DE6FED911F4296 salsa20 1.16.0.0.????? keyring Winston Blizzcon 2017 skin<br />
F6CF23955B5D437D AEBA227328A5B0AA9F51DAE3F6A7DFE4 salsa20 1.17.0.2.41350 keyring Moira<br />
0E4D9426F2891F5C 9FF064C38BE52CCDF73748180F628205 salsa20 1.18.1.2.42076 keyring Winter Wonderland 2017<br />
9240BA6A2A0CF684 DF2E37D78B43108FA6242068B70D1F65 salsa20 1.19.1.3.42563 keyring Overwatch League<br />
82297FBAB7F5EB80 B534C20965852FB15AECAC17E381B417 salsa20 1.19.1.3.42563 keyring Jan 2017 Lootbox Update<br />
9ADF00AA1A174A69 9A4AC899261A2F1C6969F39397C358E7 salsa20 1.19.1.3.42563 keyring Blizzard World<br />
CFA05AA76B49F881 526DDDEF19BF373C25B629A334CD7237 salsa20 1.19.1.3.42563 keyring WoW's Battle For Azeroth Preorder<br />
493455579DA0B18E C0BABF72AD2C05DFC14017D1ADBF5977 salsa20 1.19.3.1.43036 stream Inaugral Season Spray/Icon ??<br />
6362C5AD65DAE686 62F603D5390F763ED51773F0164FEDB5 salsa20 1.19.3.1.43036 stream White/Gray OWL Skins ??<br />
8162E5313A9C135D F407834D9521587C5012B0A59D7E064B salsa20 1.20.0.2.43435 stream Lunar New Year 2018 (Dog) / Ayutthaya / Comp CTF<br />
68EAE8FDC008C381 ???????????????????????????????? salsa20 1.20.0.2.43435 n/a ??<br />
F412C6327C4BF091 6FAFC648CBF1C2115B769593C170E732 salsa20 1.20.0.2.43435 stream SC2 20th Anniversary (Kerrigan Skin)<br />
3B3ED0874091B174 5D09C2688B1D9F1A4DB64602C1661D24 salsa20 1.21.?.?.????? ??????? Brigitte<br />
37FD04E05D2A6292 F06455E56CD144914295F2EF153D23BA salsa20 1.21.?.?.????? ??????? Brigitte Cosmetics<br />
C0DDC77552BE5794 F2BB7F35990E2900CBD877B4D3A7139C salsa20 1.22.?.?.????? ??????? OWL away skins<br />
68D8EB839DC15D75 29AFFECA5299C4140A12A66F954EF1E3 salsa20 1.22.?.?.????? ??????? Archives 2018 (Retribution)<br />
209F33BBAC9D1295 BD535438D0CDEE0E9567E0EF671C809F salsa20 1.23.?.?.????? ??????? Rialto<br />
A55F8C6F20454D94 42D76285412461B0C75AB75FB52F596E salsa20 1.23.?.?.????? ??????? Mercy BCRF Items<br />
3EEEDB8E7C29A09B 837AC79305E4BFBA3B2226F1B98200BF salsa20 1.24.?.?.????? ??????? Anniversary 2018 Map/Items<br />
22C1AF6758F8449E 28AA1BD2B9A1E3633989B1BBF64AEAC4 salsa20 1.26.?.?.????? ??????? Emily / Comp Season 11 / Comp 3v3 Items<br />
11B2E01B9331799E 9A1C99303D6A58978E29C98873327A6A salsa20 1.26.?.?.????? ??????? Wrecking Ball<br />
8498337C740329B3 EC73D663E3FC72416B3E467915708B4F salsa20 1.26.?.?.????? ??????? Wrecking Ball Cosmetics<br />
E6E8BCABE3CC96C1 57727E52600665EAC02BD87F0692898F salsa20 1.26.?.?.????? ??????? OWL Grand Finals Unlocks / Lucio Emote<br />
5D4AC0DC6F3113BA D1C9F1FF69585D1398EBA0463481F5CF salsa20 1.27.?.?.????? ??????? Summer Games 2018<br />
27F9D85973DCD5AF C5FE1015BCE0B7848F022868AFF654D1 salsa20 1.27.?.?.????? ??????? Summer Games 2018<br />
67AAD845CC0F03BD 6CD8AD3F37F54ABEC7630294D49041BF salsa20 1.27.?.?.????? ??????? D.Va Nano Cola Challenge<br />
CA13F0C79042A1A0 FCD89CE8812E6346076FC82DD7A92487 salsa20 1.28.?.?.????? ??????? Busan<br />
C4D84093A32684BD 38E182423EEDD8E3F57AC1D407B470D0 salsa20 1.28.?.?.????? ??????? Blizzcon 2018 Sombra Demon Hunter Skin<br />
0BFE5A2B3C606BA1 D6419B8E42820B0B24E08DA444A06822 salsa20 1.29.?.?.????? ??????? Halloween Terror 2018<br />
402CD9D8D6BFED98 AEB0EADEA47612FE6C041A03958DF241 salsa20 1.29.?.?.????? ??????? ??<br />
F1CBDF48147D26C6 4B6944695157D43D33E40B5692445ADB salsa20 1.30.?.?.????? ??????? 1.30 + World Cup Viewer<br />
01A48CFAE25F85CD 60D77A58062D03DC693C01954DD18021 salsa20 1.30.?.?.????? ??????? 1.30 + World Cup Viewer<br />
FEBBF66DAEF6C9BE 4A220AE3A6808ED2697410C94C1CE970 salsa20 1.30.0.1.????? ??????? Ashe<br />
</div></div><br />
<br />
== World of Warcraft ==<br />
WoW's [[DB/TactKey|TactKey.db2]] and [[DB/TactKeyLookup|TactKeyLookup.db2]] contain keys and key_names (called lookups there) respectively. Either can be streamed from server.<br />
These files cannot be bruteforced by requesting hotfix data (Blizzard has guards in place).<br />
<br />
The id field in the table below corresponds to the id field in the db2's.<br />
<br />
key_name key type id seen in used for<br />
FA505078126ACB3E BDC51862ABED79B2DE48C8E7E66C6200 salsa20 15 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
FF813F7D062AC0BC AA0B5C77F088CCC2D39049BD267F066D salsa20 25 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
D1E9B5EDF9283668 8E4A2579894E38B4AB9058BA5C7328EE salsa20 39 WOW-20740patch7.0.1_Beta Enchanted Torch pet<br />
B76729641141CB34 9849D1AA7B1FD09819C5C66283A326EC salsa20 40 WOW-20740patch7.0.1_Beta Enchanted Pen pet<br />
FFB9469FF16E6BF8 D514BD1909A9E5DC8703F4B8BB1DFD9A salsa20 41 WOW-20740patch7.0.1_Beta not used between 7.0 and 7.3<br />
23C5B5DF837A226C 1406E2D873B6FC99217A180881DA8D62 salsa20 42 WOW-20740patch7.0.1_Beta Enchanted Cauldron pet<br />
3AE403EF40AC3037 ???????????????????????????????? salsa20 51 WOW-21249patch7.0.3_Beta not used between 7.0 and 7.3<br />
E2854509C471C554 433265F0CDEB2F4E65C0EE7008714D9E salsa20 52 WOW-21249patch7.0.3_Beta Warcraft movie items<br />
8EE2CB82178C995A DA6AFC989ED6CAD279885992C037A8EE salsa20 55 WOW-21531patch7.0.3_Beta BlizzCon 2016 Murlocs<br />
5813810F4EC9B005 01BE8B43142DD99A9E690FAD288B6082 salsa20 56 WOW-21531patch7.0.3_Beta Fel Kitten<br />
7F9E217166ED43EA 05FC927B9F4F5B05568142912A052B0F salsa20 57 WOW-21531patch7.0.3_Beta Legion music <br />
C4A8D364D23793F7 D1AC20FD14957FABC27196E9F6E7024A salsa20 58 WOW-21691patch7.0.3_Beta Demon Hunter #1 cinematic (legion_dh1)<br />
40A234AEBCF2C6E5 C6C5F6C7F735D7D94C87267FA4994D45 salsa20 59 WOW-21691patch7.0.3_Beta Demon Hunter #2 cinematic (legion_dh2)<br />
9CF7DFCFCBCE4AE5 72A97A24A998E3A5500F3871F37628C0 salsa20 60 WOW-21691patch7.0.3_Beta Val'sharah #1 cinematic (legion_val_yd)<br />
4E4BDECAB8485B4F 3832D7C42AAC9268F00BE7B6B48EC9AF salsa20 61 WOW-21691patch7.0.3_Beta Val'sharah #2 cinematic (legion_val_yx)<br />
94A50AC54EFF70E4 C2501A72654B96F86350C5A927962F7A salsa20 62 WOW-21691patch7.0.3_Beta Sylvanas warchief cinematic (legion_org_vs)<br />
BA973B0E01DE1C2C D83BBCB46CC438B17A48E76C4F5654A3 salsa20 63 WOW-21691patch7.0.3_Beta Stormheim Sylvanas vs Greymane cinematic (legion_sth)<br />
494A6F8E8E108BEF F0FDE1D29B274F6E7DBDB7FF815FE910 salsa20 64 WOW-21691patch7.0.3_Beta Harbingers Gul'dan video (legion_hrb_g)<br />
918D6DD0C3849002 857090D926BB28AEDA4BF028CACC4BA3 salsa20 65 WOW-21691patch7.0.3_Beta Harbingers Khadgar video (legion_hrb_k)<br />
0B5F6957915ADDCA 4DD0DC82B101C80ABAC0A4D57E67F859 salsa20 66 WOW-21691patch7.0.3_Beta Harbingers Illidan video (legion_hrb_i)<br />
794F25C6CD8AB62B 76583BDACD5257A3F73D1598A2CA2D99 salsa20 67 WOW-21846patch7.0.3_Beta Suramar cinematic (legion_su_i)<br />
A9633A54C1673D21 1F8D467F5D6D411F8A548B6329A5087E salsa20 68 WOW-21846patch7.0.3_Beta legion_su_r cinematic<br />
5E5D896B3E163DEA 8ACE8DB169E2F98AC36AD52C088E77C1 salsa20 69 WOW-21846patch7.0.3_Beta Broken Shore intro cinematic (legion_bs_i)<br />
0EBE36B5010DFD7F 9A89CC7E3ACB29CF14C60BC13B1E4616 salsa20 70 WOW-21846patch7.0.3_Beta Alliance Broken Shore cinematic (legion_bs_a)<br />
01E828CFFA450C0F 972B6E74420EC519E6F9D97D594AA37C salsa20 71 WOW-21846patch7.0.3_Beta Horde Broken Shore cinematic (legion_bs_h)<br />
4A7BD170FE18E6AE AB55AE1BF0C7C519AFF028C15610A45B salsa20 72 WOW-21846patch7.0.3_Beta Khadgar & Light's Heart cinematic (legion_iq_lv)<br />
69549CB975E87C4F 7B6FA382E1FAD1465C851E3F4734A1B3 salsa20 73 WOW-21846patch7.0.3_Beta legion_iq_id cinematic<br />
460C92C372B2A166 946D5659F2FAF327C0B7EC828B748ADB salsa20 74 WOW-21952patch7.0.3_Beta Stormheim Alliance cinematic (legion_g_a_sth)<br />
8165D801CCA11962 CD0C0FFAAD9363EC14DD25ECDD2A5B62 salsa20 75 WOW-21952patch7.0.3_Beta Stormheim Horde cinematic (legion_g_h_sth)<br />
A3F1C999090ADAC9 B72FEF4A01488A88FF02280AA07A92BB salsa20 81 WOW-22578patch7.1.0_PTR Firecat Mount<br />
18AFDF5191923610 ???????????????????????????????? salsa20 82 WOW-22578patch7.1.0_PTR not used between 7.1 and 7.3<br />
3C258426058FBD93 ???????????????????????????????? salsa20 91 WOW-23436patch7.2.0_PTR not used between 7.2 and 7.3<br />
094E9A0474876B98 E533BB6D65727A5832680D620B0BC10B salsa20 92 WOW-23910patch7.2.5_PTR shadowstalkerpanthermount, shadowstalkerpantherpet<br />
3DB25CB86A40335E 02990B12260C1E9FDD73FE47CBAB7024 salsa20 93 WOW-23789patch7.2.0_PTR legion_72_ots<br />
0DCD81945F4B4686 1B789B87FB3C9238D528997BFAB44186 salsa20 94 WOW-23789patch7.2.0_PTR legion_72_tst<br />
486A2A3A2803BE89 32679EA7B0F99EBF4FA170E847EA439A salsa20 95 WOW-23789patch7.2.0_PTR legion_72_ars<br />
71F69446AD848E06 E79AEB88B1509F628F38208201741C30 salsa20 97 WOW-24473patch7.3.0_PTR BlizzCon 2017 Mounts (AllianceShipMount and HordeZeppelinMount)<br />
211FCD1265A928E9 A736FBF58D587B3972CE154A86AE4540 salsa20 98 WOW-24473patch7.3.0_PTR Shadow fox pet (store) <br />
0ADC9E327E42E98C 017B3472C1DEE304FA0B2FF8E53FF7D6 salsa20 99 WOW-23910patch7.2.5_PTR legion_72_tsf<br />
BAE9F621B60174F1 38C3FB39B4971760B4B982FE9F095014 salsa20 100 WOW-24727patch7.3.0_PTR Rejection of the Gift cinematic (legion_73_agi)<br />
34DE1EEADC97115E 2E3A53D59A491E5CD173F337F7CD8C61 salsa20 101 WOW-24727patch7.3.0_PTR Resurrection of Alleria Windrunner cinematic (legion_73_avt)<br />
E07E107F1390A3DF 290D27B0E871F8C5B14A14E514D0F0D9 salsa20 102 WOW-25079patch7.3.2_PTR Tottle battle pet, Raptor mount, Horse mount (104 files)<br />
32690BF74DE12530 A2556210AE5422E6D61EDAAF122CB637 salsa20 103 WOW-24781patch7.3.0_PTR legion_73_pan<br />
BF3734B1DCB04696 48946123050B00A7EFB1C029EE6CC438 salsa20 104 WOW-25079patch7.3.2_PTR legion_73_afn<br />
74F4F78002A5A1BE C14EEC8D5AEEF93FA811D450B4E46E91 salsa20 105 WOW-25079patch7.3.2_PTR SilithusPhase01 map<br />
423F07656CA27D23 ???????????????????????????????? salsa20 107 WOW-25600patch7.3.5_PTR bltestmap<br />
0691678F83E8A75D ???????????????????????????????? salsa20 108 WOW-25600patch7.3.5_PTR filedataid 1782602-1782603<br />
324498590F550556 ???????????????????????????????? salsa20 109 WOW-25600patch7.3.5_PTR filedataid 1782615-1782619<br />
C02C78F40BEF5998 ???????????????????????????????? salsa20 110 WOW-25600patch7.3.5_PTR test/testtexture.blp (fdid 1782613)<br />
47011412CCAAB541 ???????????????????????????????? salsa20 111 WOW-25600patch7.3.5_PTR unused in 25600<br />
23B6F5764CE2DDD6 ???????????????????????????????? salsa20 112 WOW-25600patch7.3.5_PTR unused in 25600<br />
8E00C6F405873583 ???????????????????????????????? salsa20 113 WOW-25600patch7.3.5_PTR filedataid 1783470-1783472<br />
78482170E4CFD4A6 768540C20A5B153583AD7F53130C58FE salsa20 114 WOW-25600patch7.3.5_PTR Magni Bronzebeard VO<br />
B1EB52A64BFAF7BF 458133AA43949A141632C4F8596DE2B0 salsa20 115 WOW-25600patch7.3.5_PTR dogmount, 50 files<br />
FC6F20EE98D208F6 57790E48D35500E70DF812594F507BE7 salsa20 117 WOW-25632patch7.3.5_PTR bfa shop stuff<br />
402CFABF2020D9B7 67197BCD9D0EF0C4085378FAA69A3264 salsa20 118 WOW-25678patch7.3.5_PTR bfa ad texture<br />
6FA0420E902B4FBE 27B750184E5329C4E4455CBD3E1FD5AB salsa20 119 WOW-25744patch7.3.5_PTR Legion epilogue cinematics<br />
1076074F2B350A2D 88BF0CD0D5BA159AE7CB916AFBE13865 salsa20 121 WOW-26287patch8.0.1_Beta skiff<br />
816F00C1322CDF52 6F832299A7578957EE86B7F9F15B0188 salsa20 122 WOW-26287patch8.0.1_Beta snowkid<br />
DDD295C82E60DB3C 3429CC5927D1629765974FD9AFAB7580 salsa20 123 WOW-26287patch8.0.1_Beta redbird<br />
83E96F07F259F799 91F7D0E7A02CDE0DE0BD367FABCB8A6E salsa20 124 WOW-26522patch8.0.1_Beta BlizzCon 2018 (Alliance and Horde banners and cloaks)<br />
49FBFE8A717F03D5 C7437770CF153A3135FA6DC5E4C85E65 salsa20 225 WOW-27826patch8.1.0_PTR Meatwagon mount (Warcraft 3: Reforged)<br />
C1E5D7408A7D4484 A7D88E52749FA5459D644523F8359651 salsa20 226 WOW-26871patch8.0.1_Beta Sylvanas Warbringer cinematic<br />
E46276EB9E1A9854 CCCA36E302F9459B1D60526A31BE77C8 salsa20 227 WOW-26871patch8.0.1_Beta ltc_a, ltc_h and ltt cinematics<br />
D245B671DD78648C 19DCB4D45A658B54351DB7DDC81DE79E salsa20 228 WOW-26871patch8.0.1_Beta stz, zia, kta, jnm & ja cinematics<br />
4C596E12D36DDFC3 B8731926389499CBD4ADBF5006CA0391 salsa20 229 WOW-26871patch8.0.1_Beta bar cinematic<br />
0C9ABD5081C06411 25A77CD800197EE6A32DD63F04E115FA salsa20 230 WOW-26871patch8.0.1_Beta zcf cinematic<br />
3C6243057F3D9B24 58AE3E064210E3EDF9C1259CDE914C5D salsa20 231 WOW-26871patch8.0.1_Beta ktf cinematic<br />
7827FBE24427E27D 34A432042073CD0B51627068D2E0BD3E salsa20 232 WOW-26871patch8.0.1_Beta rot cinematic<br />
FAF9237E1186CF66 ???????????????????????????????? salsa20 233 WOW-28048patch8.1.0_PTR encrypted db2 sections (battle pet?)<br />
5DD92EE32BBF9ABD ???????????????????????????????? salsa20 234 WOW-27004patch8.0.1_Subm filedataid 2238294<br />
0B68A7AF5F85F7EE ???????????????????????????????? salsa20 236 WOW-28151patch8.1.0_PTR fdid 2459473, 2459583, 2459601, 2459618, 2459669, 2472143<br />
01531713C83FCC39 ???????????????????????????????? salsa20 237 WOW-28151patch8.1.0_PTR fdid 2460009, 2460732<br />
76E4F6739A35E8D7 ???????????????????????????????? salsa20 238 WOW-28294patch8.1.0_PTR starts at fdid 2492654, total of 8 fdids<br />
66033F28DC01923C 9F9519861490C5A9FFD4D82A6D0067DB salsa20 239 WOW-28294patch8.1.0_PTR vulpine familiar mount<br />
FCF34A9B05AE7E6A ???????????????????????????????? salsa20 240 WOW-28151patch8.1.0_PTR fdid 2468985, 2471011, 2471012, 2471014, 2471016, 2471018<br />
E2F6BD41298A2AB9 ???????????????????????????????? salsa20 241 WOW-28151patch8.1.0_PTR fdid 2468988, 2471019, 2471020, 2471021, 2471022, 2471023<br />
14C4257E557B49A1 ???????????????????????????????? salsa20 242 WOW-28440patch8.1.0_PTR<br />
1254E65319C6EEFF 79D2B3D1CCB015474E7158813864B8E6 salsa20 243 WOW-28440patch8.1.0_PTR<br />
C8753773ADF1174C ???????????????????????????????? salsa20 244 WOW-28938patch8.1.5_PTR<br />
2170BCAA9FA96E22 ???????????????????????????????? salsa20 245 WOW-28938patch8.1.5_PTR<br />
75485627AA225F4D ???????????????????????????????? salsa20 246 WOW-28938patch8.1.5_PTR</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26440DB22018-11-09T23:06:22Z<p>Simca: Fixed typo.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for pallet_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26439DB22018-11-09T23:05:33Z<p>Simca: /* Further Quirks */ Padding garbage bug explained and solutions detailed.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There is a writing bug that has existed since WDB6's common_data padding was added. The bug has expanded slightly in WDC1 and newer formats to also be present in pallet_data blocks. Padding bytes are not properly written when the DB2 uses multiple fields in one block type (either common_data or pallet_data) with different unpadded sizes. For example, if a byte column and a short column both use the common_data block, the byte column's values -can- (but does not always) encounter this bug. The padding difference between the smallest and largest field size using the block will be filled with random garbage bytes. For example, in the above example, a value from the byte column stored in common_data may be '08 02 00 00'. This is invalid. The last three bytes should all be '00', but due to this writing bug, this will not always be true.<br />
<br />
The only perfect way to handle this is to properly mask the data being read with the expected size of the field in order to ward off the 'garbage' data. This would require you to know the proper value size ahead of time (probably from DBCMeta) since that information is not handed out for array_data or common_data fields in any existing DB2 format. Alternatively, you could attempt to detect unexpected patterns in the data in order to proactively detect this bug and mask it. This would likely be very complex, but since the bug seems to manifest in very consistent ways, it may be possible. Because this is a writing bug, it may be fixed by Blizzard at some point in the future without us noticing immediately. Please update the wiki article or contact one of its authors if you believe Blizzard has solved this issue permanently, as it would be of interest.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26436DB22018-11-08T15:29:09Z<p>Simca: Fixed and clarified field_structure sections in WDC* formats.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes (well, actually it reports a value you have to subtract from 32 and divide by 8 first, see the comments in the structure above on the 'size' field in the field structure block). However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4' (technically, reported as value '0', which you subtract from 32 and divide by 8 to get '4'), while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = SizeReportedByFieldStorageInfo / (32 - ValueReportedByFieldStructure)'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26239DB22018-10-12T05:29:47Z<p>Simca: /* WDC3 */ Fixed a minor grammar mistake.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26238DB22018-10-12T03:06:31Z<p>Simca: /* WDC3 */ Fixed a typo.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset map structures, and shed a bit of light of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26237DB22018-10-12T03:05:42Z<p>Simca: /* WDC2 */ Updating comment in structure with encryption details</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset maps structures, and shed a bit of light of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=26236DB22018-10-12T03:02:16Z<p>Simca: Greatly expanded on Alram's initial information about WDC3.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]], [[:Category:DBC_WoD|Warlords of Draenor]] and [[:Category:DBC_Legion|Legion]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1];* // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x02 != 0)<br />
{<br />
uint32_t relationship_IDs[header.max_id - header.min_id + 1]; // only ever used by wmominimaptexture.db2 to record WMOIDs<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231|max_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and was replaced by WDC3 shortly into Battle for Azeroth (Patch 8.1.0 Build 28048).<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
// Pre-WDC2 string value = value read from record + current position - size of field just read - location of string block - size of record data outside this section<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.Locations.Sections[si].StringBlockLocation - ((RecordCount - SectionHeaders[si].RecordCount) * RecordSize);<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). Lastly, it calculates the size of the record data outside this section and subtracts that value out of the result as well. This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
Note that the part where the 'size of record data outside this section' needs to be subtracted out of the string offsets was not actually discovered until WDC3. We suspect that this unknown quirk was part of the issue behind a broken build (Patch 8.1.0 Build 27826), where the strings in the db2s were unreadable by normal methods. This is potentially now explainable if the db2s in that build had unshipped additional sections that were being factored into the calculations, as all of the string offsets in the 'broken' db2s were off by a factor of 'record_size'.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc2_db2_header header;<br />
<br />
// a section = records + string block + offset map + id list + copy table + relationship map<br />
struct wdc2_section_header<br />
{<br />
uint64_t tact_key_id; // encrypted?<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count].<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
// <[TOM_RUS]> last one makes no sense - 6/7/18 (for signed_immediate)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=WDC3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.1.0.28048}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.1.0 Build 28048) and is currently in use today.<br />
<br />
WDC3 consists of some changes to the section header, officially introduces DB2s with multiple sections (as described in WDC2) which seem to be used for storing encrypted data (tact_key_hash in section header is set), finally cleaned up all the empty and unused space in offset maps structures, and shed a bit of light of additional light on string calculation changes made back in WDC2. <br />
<br />
For the section header changes, they removed 'copy_table_size', instead replacing it with a 'copy_table_count' field added at the end of the section header. Additionally, a new field has been added right before that which is 'offset_map_id_count', a field required for parsing the newly slimmed offset map structure. Instead of the size of the offset map structure being an implicit calculation based on MaxID and MinID, it is now an explicit size ('offset_map_id_count' * sizeof(offset_map_entry)). Lastly, the field 'offset_map_offset' is now better called 'offset_records_end'. Instead of pointing at the spot where the offset map structure begins, it points at the position where the offset records end. In WDC2, these would have been the same value, so it's impossible to know if this changed now or back in WDC2, where we wouldn't have been able to tell the difference.<br />
<br />
The reason we can now tell the difference is that the offset map structure has moved back two blocks in the structure. It's now after id_list and copy_table. To get the old 'offset_map_offset' value, you need to do something like:<br />
uint offset_map_offset = offset_records_end + id_list_size + (copy_table_count * 8);<br />
The offset map structure is also now much more compact. Before, it implicitly ran from min_id to max_id, and if the ID didn't exist in the db2, there were 6 bytes of 0s. This was a massive waste of space in large files like ItemSparse.db2. Now, as you read the offset_map entry-by-entry, you also need to read offset_map_id_list entry-by-entry. The first entry in the offset_map has an ID equal to the value of the first entry in offset_map_id_list. Reading the two structures in parallel gives you the same kind of information as before (ID, offset, length) with the only difference being that the ID is explicit, so there does not need to be large gaps everywhere that an ID doesn't exist (in fact, it's impossible for an ID in offset_map_id_list to not exist, as far as I know).<br />
<br />
Note that the WDC2 string changes are still in effect and that when introducing multiple sections, we discovered an additional quirk to these changes that was never noticed before (record table size outside the current section needs to be subtracted from string offsets). This issue only manifests itself on DB2s with multiple sections, no offset map, and non-zero string tables. Re-read the WDC2 section for additional information on the string changes.<br />
<br />
==Structure==<br />
struct wdc3_db2_header<br />
{<br />
uint32_t magic; // 'WDC3'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data<br />
};<br />
wdc3_db2_header header;<br />
<br />
// a section = records + string block + id list + copy table + offset map + offset map id list + relationship map<br />
struct wdc3_section_header<br />
{<br />
uint64_t tact_key_hash; // TactKeyLookup hash<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t offset_records_end; // Offset to the spot where the records end in a file with an offset map structure;<br />
uint32_t id_list_size; // Size of the list of ids present in the section<br />
uint32_t relationship_data_size; // Size of the relationship data in the section<br />
uint32_t offset_map_id_count; // Count of ids present in the offset map in the section<br />
uint32_t copy_table_count; // Count of the number of deduplication entries (you can multiply by 8 to mimic the old 'copy_table_size' field)<br />
};<br />
wdc3_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
// Same as field_compression_bitpacked<br />
field_compression_bitpacked_signed,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
case field_compression.field_compression_bitpacked_signed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte offset+size pairs.<br />
char variable_record_data[section_headers.offset_records_end - section_headers.file_offset];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_count > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[copy_table_count];<br />
}<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[offset_map_id_count];<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
uint32_t offset_map_id_list[offset_map_id_count];<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = {{Template:Unverified|'Has relationship data'}} // This may be 'secondary keys' and is unrelated to WDC1+ relationships<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = {{Template:Unverified|'Is bitpacked'}} // WDC1+<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=CASC&diff=26212CASC2018-09-26T19:32:07Z<p>Simca: /* World of Warcraft */ Added TactKey 124.</p>
<hr />
<div>CASC is the name of the new file system that Blizzard has created to replace the outdated format of [[MPQ]].<br />
<br />
=CASC v1=<br />
The CASC file system made its first debut in the Heroes of the Storm Technical Alpha, which was hosted on Blizzard's servers in late January. The form of CASC that Heroes of the Storm uses is designated by Blizzard as "CASC". In contrast, World of Warcraft's "build-playbuild-installer" config line clearly states it is generated by "ngdptool_casc2" (NGDP stands for Next Generation Download Procotol).<br />
These are the two most substantial changes between CASC v1 and CASC v2:<br />
* Sections of CASC v1 data files are grouped together in collections of files we call "packages". These packages all have the same root folder, and if all of the files are not properly added with the package's base directory, the extraction process will produce an incredibly mangled directory output. This system is completely removed in CASC v2.<br />
* CASC v1's Root file relates content hashes to file names. CASC v2's [[CASC#Root|Root]] file relates content hashes to name hashes. Translating name hashes to file names requires use of the Jenkins Hash function [http://en.wikipedia.org/wiki/Jenkins_hash_function], which in turn requires a listfile to generate the hashes. Essentially CASC v1 has its own listfile (in root). CASC v2 does not, and requires the user to provide names.<br />
The remainder of this article will refer exclusively to the system called CASC v2 as 'CASC'. While many parts of the file system are identical between v1 and v2, there are enough changes to make explaining both formats at once inadvisable.<br />
<br />
=NGDP=<br />
CASC was introduced simultaneously with a new system for managing configuration, blob, and installation files called NGDP, or Next Generation Download Protocol. When the acronym 'NGDP' is used in conjunction with the term CASC, it is typically referring to the hosted components of the CASC file system, and its ability to stream data on the fly.<br />
<br />
==NGDP URLs==<br />
As of October 14th, 2014, the following generic NGDP URLs are known:<br />
* http://us.patch.battle.net:1119/(ProgramCode)/cdns - a table of domains available with game data per region<br />
* http://us.patch.battle.net:1119/(ProgramCode)/versions - a table of the current game version, build config, cdn config and optionally keyring per region<br />
* http://us.patch.battle.net:1119/(ProgramCode)/bgdl - similar to versions, but tailored for use by the Battle.net App background downloader process<br />
* http://us.patch.battle.net:1119/(ProgramCode)/blobs - contains InstallBlobMD5 and GameBlobMD5<br />
* http://us.patch.battle.net:1119/(ProgramCode)/blob/game - a blob file that regulates game functionality for the Battle.net App<br />
* http://us.patch.battle.net:1119/(ProgramCode)/blob/install - a blob file that regulates installer functionality for the game in the Battle.net App<br />
<br />
Keep in mind Blizzard's CDN is pretty shit at caching sometimes so you after an update to the above files it might switch back and forth between the old and new version of the files for a few hours.<br />
<br />
==NGDP Program Codes==<br />
<br />
The following NGDP program codes are known to be served at some point during NGDP's existence. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="90" | Program<br />
! width="250" | Description<br />
! width="200" | Status<br />
|-<br />
| agent || Battle.net Agent || style="color: green;" | Active<br />
|-<br />
| agent_test || Probably Agent test || <br />
|-<br />
| bna || Battle.net App || style="color: green;" | Active<br />
|-<br />
| bnt || Heroes of the Storm Alpha || style="color: red;" | Deprecated<br />
|-<br />
| bts || Bootstrapper (currently for dst2) || style="color: orange;" | Partial (only versions as of now, cdnpath=tpr/bnt004)<br />
|-<br />
| catalogs || Catalog || style="color: green;" | Active<br />
|-<br />
| clnt || Client || style="color: red;" | Deprecated<br />
|-<br />
| d3 || Diablo 3 Retail || style="color: green;" | Active<br />
|-<br />
| d3b || Diablo 3 Beta (2013) || style="color: orange;" | Partial<br />
|-<br />
| d3cn || Diablo 3 China || style="color: green;" | Active<br />
|-<br />
| d3cnt || Diablo 3 China Test (?) || style="color: orange;" | Unused (everything empty)<br />
|-<br />
| d3t || Diablo 3 Test || style="color: green;" | Active<br />
|-<br />
| demo || || style="color: orange;" | Partial<br />
|-<br />
| dst2 || Destiny 2 || style="color: green;" | Active<br />
|-<br />
| dst2a || Destiny 2 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| dst2dev || Destiny 2 "takehome" development || style="color: orange;" | Partial (encrypted?)<br />
|-<br />
| dst2e1 || Destiny 2 Event || Active; probably 1..9<br />
|-<br />
| dst2igr || Destiny 2 ? || style="color: green;" | Active<br />
|-<br />
| dst2t || Destiny 2 Public Test || style="color: green;" | Active<br />
|-<br />
| hero || Heroes of the Storm Retail || style="color: green;" | Active<br />
|-<br />
| heroc || Heroes of the Storm Tournament || style="color: green;" | Active<br />
|-<br />
| herot || Heroes of the Storm Test || style="color: green;" | Active<br />
|-<br />
| hsb || Hearthstone Retail || style="color: green;" | Active<br />
|-<br />
| hsc || Hearthstone Chournament || style="color: green;" | Active<br />
|-<br />
| hst || Hearthstone Test || style="color: orange;" | Partial<br />
|-<br />
| pro || Overwatch Retail || style="color: green;" | Active<br />
|-<br />
| proc || Overwatch Tournament US || style="color: green;" | Active<br />
|-<br />
| proc_cn || Overwatch Tournament China || style="color: green;" | Active<br />
|-<br />
| proc_eu || Overwatch Tournament Europe || style="color: green;" | Active<br />
|-<br />
| proc_kr || Overwatch Tournament Korea || style="color: green;" | Active<br />
|-<br />
| proc2 || Overwatch Professional 2 (prometheus_tournament_2) || style="color: green;" | Active<br />
|-<br />
| proc2_cn || Overwatch Professional 2 China || style="color: green;" | Active<br />
|-<br />
| proc2_eu || Overwatch Professional 2 Europe || style="color: green;" | Active<br />
|-<br />
| proc2_kr || Overwatch Professional 2 Korea || style="color: green;" | Active<br />
|-<br />
| proc3 || Overwatch Tournament (Dev) || <br />
|-<br />
| prodev || Overwatch Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| prot || Overwatch Test || style="color: green;" | Active<br />
|-<br />
| prov || Overwatch Vendor || style="color: green;" | Active<br />
|-<br />
| s1 || StarCraft 1 || style="color: green;" | Active<br />
|-<br />
| s1a || Starcraft 1 Alpha || style="color: blue;" | Active (encrypted)<br />
|-<br />
| s1t || StarCraft 1 Test || style="color: green;" | Active<br />
|-<br />
| s2 || StarCraft II Retail || style="color: green;" | Active<br />
|-<br />
| s2b || StarCraft II Beta || style="color: red;" | Deprecated<br />
|-<br />
| s2t || StarCraft II Test || style="color: red;" | Deprecated<br />
|-<br />
| sc2 || StarCraft II || style="color: red;" | Deprecated<br />
|-<br />
| storm || Heroes of the Storm || style="color: red;" | Deprecated<br />
|-<br />
| test || || style="color: red;" | Deprecated<br />
|-<br />
| viper || Call of Duty Black Ops 4 || style="color: green;" | Active<br />
|-<br />
| viperdev || Call of Duty Black Ops 4 - Alpha ||<br />
|-<br />
| w3 || Warcraft III || style="color: green;"| Active<br />
|-<br />
| w3t || Warcraft III Public Test || style="color: green;"| Active<br />
|-<br />
| war3 || Warcraft III (old) || style="color: orange;"| Partial<br />
|-<br />
| wow || World of Warcraft Retail || style="color: green; | Active<br />
|-<br />
| wow_beta || World of Warcraft Alpha/Beta || style="color: green; | Active<br />
|-<br />
| wowdev || World of Warcraft Dev || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowdemo || World of Warcraft Demo || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowe1 || World of Warcraft Event 1 || style="color: green;" | Active<br />
|-<br />
| wowe2 || World of Warcraft Event 2 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowe3 || World of Warcraft Event 3 || style="color: orange;" | Active (partial)<br />
|-<br />
| wowt || World of Warcraft Test || style="color: green;" | Active<br />
|-<br />
| wowv || World of Warcraft Vendor || style="color: blue;" | Active (encrypted)<br />
|-<br />
| wowz || World of Warcraft Submission (previously Vendor) || style="color: green; | Active<br />
|}<br />
<br />
=CASC Online=<br />
<br />
==Standard URL Hash Format==<br />
URL Format: http://(cdnsHost)/(cdnsPath)/(pathType)/(FirstTwoHexOfHash)/(SecondTwoHexOfHash)/(FullHash)<br />
<br />
Current CDNs for a specific program can be found in the http://us.patch.battle.net:1119/(ProgramCode)/cdns file. Blizzard regularly shuffles around CDNs and sometimes even adds/removes CDNs, so be sure to parse the CDNs file to stay up to date. For WoW, the cdnsPath of "tpr/wow" has never changed.<br />
<br />
Known path types are:<br />
* config - contains the three types of config files: Build configs, CDN configs, and Patch configs<br />
* data - contains archives, indexes, and unarchived standalone files (typically binaries, mp3s, and movies and files mentioned in buildconfig like root, install and download)<br />
* patch - contains patch manifests, files, archives, indexes<br />
<br />
Blizzard regularly cleans old builds from the CDN so any example files mentioned in this article might be unavailable at the time of reading (feel free to update them to something more recent if this is the case).<br />
<br />
==Config Files==<br />
<br />
===Build Config===<br />
Example file: http://blzddist1-a.akamaihd.net/tpr/wow/config/22/38/2238ab9c57b672457a2fa6fe2107b388<br />
<br />
Some of the files listed in this file are explained later on in this article.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Root|root]] || Content hash of the '''decoded''' root file, look this up in encoding to get the '''encoded''' hash.<br />
|-<br />
| [[#Install|install]] || First key is the content hash of the '''decoded''' install file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash/CDN key.<br />
|-<br />
| install-size || Install size(s) corresponding to the install hash(es). Absent in older builds.<br />
|-<br />
| [[#Download|download]] || First key is the content hash of the '''decoded''' download file. Second key, if present, is the '''encoded''' hash; if absent, look up the content hash in encoding to get the encoded hash.<br />
|-<br />
| download-size || Download size(s) corresponding to the download hash(es). Absent in older builds.<br />
|-<br />
| [[#Download Size|size]] || CKey and EKey of the download size file, respectively. Introduced in build 27547.<br />
|-<br />
| size-size || Download size sizes corresponding to the download size keys.<br />
|-<br />
| build-partial-priority || Content hash:block size pairs for priority non-archived files, ordered by priority. The block size appears to be the amount to request in each partial HTTP request. These files are BLTE-encoded and usually but not always split into blocks of fixed uncompressed size where the partial priority block size is a power-of-two multiple of the file's uncompressed block size. First seen in build 24116. Optional.<br />
|-<br />
| partial-priority || Content hash of a partial download file containing priority files to download first; note that the sizes in the download file entries are download block sizes, '''not''' file sizes. First seen in build 22231, replaced by build-partial-priority in build 24116. Optional.<br />
|-<br />
| partial-priority-size || Unknown: always 0 if present. Present if partial-priority is present.<br />
|-<br />
| [[#Encoding|encoding]] || First key is the content hash of the '''decoded''' encoding file. Second key is the '''encoded''' hash. If either none or 1 is found, the game (at least Wow) switches to plain-data (?) . Seen in build 20173<br />
|-<br />
| encoding-size || Encoding sizes corresponding to the encoding hashes.<br />
|-<br />
| [[#Patch|patch]] || Unknown<br />
|-<br />
| patch-size || Unknown<br />
|-<br />
| [[#Patch_Config|patch-config]] || Content hash of non-encoded patch config (see Patch Config)<br />
|-<br />
| build-branch || Optional. Presumably the SCM branch built.<br />
|-<br />
| build-comments || Optional.<br />
|-<br />
| build-creator || Optional. Presumably the user who submitted the build.<br />
|-<br />
| build-name || Optional? Name of the build<br />
|-<br />
| build-playbuild-installer || Optional? Type of installer for the Battle.net app to use<br />
|-<br />
| build-product || Optional? Product name<br />
|-<br />
| build-uid || Optional? Program code (see NGDP Program Codes)<br />
|}<br />
<br />
===CDN Config===<br />
Example file: http://blzddist1-a.akamaihd.net/tpr/wow/config/42/33/423364147752a596911aa1de2ff1f6a4<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| [[#Archives|archives]] || CDN keys of all archives (and by appending .index to the hash their indexes)<br />
|-<br />
| [[#Archive-Group_Index_.28.index.29|archive-group]] || CDN key of the the combined index file (see Archive-Group Index)<br />
|-<br />
| file-index || CDN key of .index file for unarchived files. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| file-index-size || Size of unarchived file .index.<br />
|-<br />
| [[#patch-archives|patch-archives]] || CDN keys of patch archives (needs research)<br />
|-<br />
| patch-archive-group || CDN key of probably the combined patch index file (needs research)<br />
|-<br />
| [[#Build_Config|builds]] || List of build configs this config supports (optional)<br />
|-<br />
| patch-file-index || CDN key of .index file for unarchived patches. These files have 0-byte archive offset fields. Seen in Warcraft III.<br />
|-<br />
| patch-file-index-size || Size of unarchived patch .index.<br />
|}<br />
<br />
===Patch Config===<br />
Example file: http://blzddist1-a.akamaihd.net/tpr/wow/config/20/f0/20f0593dd1b9fdaeaa7a808f83d48f1d<br />
<br />
This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 in build 19027 (October 10th, 2014).<br />
The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material.<br />
The structure and purpose of all of the fields of this file requires further research.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="140" | Value name<br />
! width="650" | Description<br />
|-<br />
| patch-entry || Repeats 3 times with patch entries for install, download and encoding file. Builds using the partial-priority (not build-partial-priority) mechanism may additionally have a partial-priority entry.<br />
|-<br />
| patch || Patch manifest file<br />
|-<br />
| patch-size || Size of patch file (optional?)<br />
|}<br />
<br />
====patch-entry====<br />
<br />
The format of these strings is:<br />
<br />
<pre>patch-entry = <type> <content hash> <content size> <BLTE-encoding key> <BLTE-encoded size> <encoding string></pre><br />
<br />
followed by sets of<br />
<br />
<pre><old BLTE-encoding key> <old content size> <patch hash> <patch size></pre><br />
<br />
This serves to pair old files to the patch needed to bring them up to date, as well as providing the information required to detect if the file is already up to date.<br />
<br />
==Data Files==<br />
<br />
Example index: http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef.index<br />
<br />
Example archive: http://blzddist1-a.akamaihd.net/tpr/wow/data/00/52/0052ea9a56fd7b3b6fe7d1d906e6cdef<br />
<br />
==Patch Files==<br />
<br />
==Armadillo==<br />
If <tt>$program/blob/game</tt> contains <tt>decryption_key_name</tt>, data on CDN is encrypted. The key is retrieved from the keychain or manually entered (a base32 encoded version that contains key and checksum is used). It is stored in <tt>$key_name.ak</tt>. The key file contains the key followed by a the first four bytes of the md5 hash of the key for verification purposes.<br />
<br />
Encryption uses Salsa20. The IV is the last 8 bytes (16 characters) of the cdn hash. <br />
<br />
===Known keys===<br />
decryption_key_name used_by key checksum base32<br />
sc1Dev s1a F6 79 DC 38 E0 C3 65 FB 48 2E 48 A7 48 90 9D 29 19 F5 BB 88 6Z45YOHAYNS7WSBOJCTUREE5FEM7LO4I<br />
prolivedev prodev <br />
pro pro (pre-launch) <br />
destiny_live dst2a<br />
destiny_dev dst2a (pre-alpha)<br />
destiny2_openbeta dst2<br />
viperlivedev viperdev<br />
wowdevalpha wowdev<br />
wowvendor wowv<br />
<br />
=File References=<br />
Files are referred to by many different pieces of data in CASC. A quick summary of them:<br />
* Filename: The file's real name. Note that one file can have many names - essentially, one encoding key can map to many different name hashes.<br />
* Locale Flag: <br />
* Content Flag: <br />
* Name Hash: The file's name, after being hashed with the Jenkins Hash.<br />
* Content Hash: The MD5 of the entire file in its uncompressed state; the purest representation of the data.<br />
* Encoding Hash/Key: MD5 hash of the potentially encoded file. For unencoded files, the content hash. For chunkless BLTE files lacking a chunk table, this hash covers the entire encoded file. For chunked BLTE files, this hash covers only the BLTE headers including chunk table, as the chunk table contains hashes of the content of each chunk. A given file can be encoded in many ways, and a single content hash may potentially have multiple encoding keys.<br />
* CDN Key: The key used to lookup a file on the CDN. Synonym of encoding key.<br />
* Header Hash: Inaccurate synonym of encoding key.<br />
<br />
=BLTE encoded files=<br />
Any files stored inside the data files are BLTE encoded, which means before reading anything in the file, first you have to decode it. The documentation below refers to ''decoded'' files!<br />
<br />
It consists of these chunks in the following order:<br />
* Header<br />
* ChunkInfo (only if Header.headerSize > 0)<br />
* Data<br />
<br />
<br />
To read a BLTE encoded file:<br />
# Read the Header chunk<br />
# Read the ChunkInfo chunk if Header.headerSize > 0<br />
# Read each of the Data chunks and combine them to create the complete file<br />
<br />
'''Note:''' If there is no ChunkInfo struct, there is just one Data chunk.<br />
<br />
<br />
*'''Header'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="200" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[4] || FileSignature || "BLTE"<br />
|-<br />
| 0x04 || uint32_t [BE] || headerSize || Size of the BLTE header (BLTE header = Header + ChunkInfo).<br />
|}<br />
<br />
<br />
*'''ChunkInfoEntry'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="200" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32_t [BE] || compressedSize || Compressed size of the chunk (the compression mode byte is included).<br />
|-<br />
| 0x04 || uint32_t [BE] || decompressedSize || Decompressed chunk of the size.<br />
|-<br />
| 0x08 || char[16] || checksum || The checksum of the compressed chunk (the compression mode byte is included).<br />
|}<br />
<br />
<br />
*'''ChunkInfo'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="200" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint8_t [BE] || flags || Flags of some sort.<br />
|-<br />
| 0x02 || uint24_t [BE] || chunkCount || The number of chunks.<br />
|-<br />
| 0x04 || ChunkInfoEntry[chunkCount] || chunks || The chunk info for the chunks in the file.<br />
|}<br />
<br />
<br />
if either flags != 0xF or chunkCount == 0, the file is deemed badly formatted.<br />
<br />
*'''Data'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="85" | Offset (Hex)<br />
! width="260" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char || encodingMode || Available values: N, Z, F, E<br />
|-<br />
| 0x01 || char[ChunkInfo.compressedSize - 1] || data || The encoded data.<br />
|}<br />
<br />
<br />
<br />
Example implementation as Binary Template can be found here: [[BLTE-Template]]<br />
<br />
<br />
'''Encoding modes:'''<br />
* N: Plain data.<br />
* Z: Zlib encoded data.<br />
* 4: lz4hc encoded data.<br />
<br />
The encoded data is preceded by the two header bytes as specified by the zlib RFC [https://www.ietf.org/rfc/rfc1950.txt] (Section 2.2). 78 DA most of the time.<br />
<br />
Reading bits left-to-right:<br />
<br />
var compressionInfo = reader.ReadBits(4);<br />
var compressionMethod = reader.ReadBits(4);<br />
var flevel = reader.ReadBits(2);<br />
var fdict = reader.ReadBit();<br />
var fcheck = reader.ReadBits(5);<br />
<br />
In .NET you can basically skip these bytes and use DeflateStream. Make sure to wrap around the chunk so it internally does not try to consume bytes from the following chunk.<br />
<br />
* F: Recursively encoded BLTE data.<br />
* E: encrypted: one of salsa20, arc4, rc4.<br />
<br />
struct <br />
{<br />
unsigned char key_name_length; // 0x8<br />
unsigned char key_name[key_name_length];<br />
unsigned char IV_length; // 0x4<br />
unsigned char IV[IV_length];<br />
char type; // 'S': salsa20, 'A': arc4<br />
} E_chunk;<br />
<br />
key_name is resolved by client to the actual key. keys are distributed via keyrings and some keys are hardcoded. <br />
<br />
==Encoding Specification (ESpec)==<br />
ESpecs are string-based representations of the encoding of BLTE-encoded data files that serve as recipes for the patcher to produce a binary-identical encoded output file (as patching operates on the unencoded data). The information they contain is redundant with the information in the BLTE header of the file itself, but due to ESpec shorthand notations multiple ESpecs can encode the same output for the same input (e.g. "b:256*2=z", "b:{256=z,256=z}", etc.), and the same ESpec can result in different output block configurations for inputs of different sizes (e.g. 500 bytes of input results in 2 blocks and 600 bytes 3 blocks with "b:256*=z"). They are used most extensively in [[#Encoding|Encoding files]].<br />
<br />
An example parser can be found [https://gist.github.com/heksesang/b15057fe3f093eebee3a at this gist].<br />
<br />
The strings are not whitespace-tolerant. They use the following [https://en.wikipedia.org/wiki/Extended_Backus–Naur_form#Table_of_symbols EBNF grammar] (concatenations are omitted), where <tt>e-spec</tt> is a top level string:<br />
<br />
e-spec = ( 'n' )<br />
| ( 'z' [ ':' ( zip-level | '{' zip-level ',' zip-bits '}' ) ] )<br />
| ( 'e' ':' '{' encryption-key ',' encryption-iv ',' e-spec '}' )<br />
| ( 'b' ':' ( final-subchunk | '{' ( [{block-subchunk ','}] final-subchunk ) '}' ) )<br />
;<br />
<br />
block-subchunk = block-size-spec '=' e-spec ;<br />
block-size-spec = block-size [ '*' block-count ] ; (* block-count (1 if unspecified) blocks of block-size bytes *)<br />
block-size = number [ block-unit ] ;<br />
block-unit = 'K' (* count * 2^10 *)<br />
| 'M' (* count * 2^20 *)<br />
;<br />
block-count = number ;<br />
final-subchunk = final-size-spec '=' e-spec ;<br />
final-size-spec = block-size-spec<br />
| block-size '*' (* greedy spec of block-size blocks (last block <= block-size) *)<br />
| '*' (* greedy block *)<br />
;<br />
<br />
zip-level = number<br />
;<br />
zip-bits = number | ( 'm' 'p' 'q' )<br />
;<br />
<br />
encryption-key = ? eight byte upper-hex encoded [[#Encryption_keys|key name]] ?<br />
;<br />
encryption-iv = ? four byte upper-hex encoded IV value ?<br />
;<br />
<br />
* where a <tt>greedy</tt> final-size-spec consumes all remaining bytes in the parent block or file<br />
** "all remaining bytes" ''may be 0'', producing no blocks<br />
* where <tt>zip-level</tt> defaults to <tt>9</tt> and <tt>zip-bits</tt> to <tt>15</tt> if not given<br />
** <tt>zip-bits</tt> with value <tt>'mpq'</tt> means <tt>0</tt><br />
<br />
===Examples===<br />
<br />
b:{164=z,16K*565=z,1656=z,140164=z}<br />
<br />
* blocks on top level<br />
** 164 bytes of zip with level=9, bits=15<br />
** 565 blocks of 16Kb zip chunks with level=9, bits=15<br />
** 1656 bytes of zip with level=9, bits=15<br />
** 140164 of zip with level=9, bits=15<br />
<br />
b:{1768=z,66443=n}<br />
<br />
* blocks on top level<br />
** 1768 bytes of zip with level=9, bits=15<br />
** 66443 bytes of raw data<br />
<br />
b:{256K*=e:{237DA26C65073F42,06FC152E,z}}<br />
<br />
* blocks on top level<br />
** unspecified count of 256 kb chunks encrypted with key 237DA26C65073F42 and IV 06FC152E<br />
*** containing zip data with level=9, bits=15<br />
*** last chunk <= 256 kb<br />
<br />
z<br />
<br />
* zipped data on top level with level=9, bits=15<br />
<br />
b:{22=n,31943=z,211232=n,27037696=n,138656=n,17747968=n,*=z}<br />
<br />
* blocks on top level<br />
** 22 bytes of raw data<br />
** 31943 bytes of zip with level=9, bits=15<br />
** 211232 bytes of raw data<br />
** 27037696 bytes of raw data<br />
** 138656 bytes of raw data<br />
** 17747968 bytes of raw data<br />
** an unspecified amount of zipped data with level=9, bits=15<br />
<br />
b:{16K*=z:{6,mpq}}<br />
<br />
* blocks on top level<br />
** unspecified count of 16 kb chunks of zipped data with level=6, bits=0<br />
*** last chunk <= 16 kb<br />
<br />
=Encryption keys=<br />
<br />
== Battle.net app==<br />
key_name key type seen in<br />
2C547F26A2613E01 37C50C102D4C9E3A5AC069F072B1417D salsa20 Battle.net App Alpha 1.5.0<br />
<br />
== Overwatch ==<br />
Overwatch has keys included in the client/keyring, but they can also be streamed from the server.<br />
key_name key type seen in method used for<br />
FB680CB6A8BF81F3 62D90EFA7F36D71C398AE2F1FE37BDB9 salsa20 0.8.0.24919 keyring<br />
402CD9D8D6BFED98 AEB0EADEA47612FE6C041A03958DF241 salsa20 0.8.0.24919 keyring<br />
DBD3371554F60306 34E397ACE6DD30EEFDC98A2AB093CD3C salsa20 0.8.0.24919 keyring<br />
11A9203C9881710A 2E2CB8C397C2F24ED0B5E452F18DC267 salsa20 0.8.0.24919 keyring<br />
A19C4F859F6EFA54 0196CB6F5ECBAD7CB5283891B9712B4B salsa20 0.8.0.24919 keyring<br />
87AEBBC9C4E6B601 685E86C6063DFDA6C9E85298076B3D42 salsa20 0.8.0.24919 keyring<br />
DEE3A0521EFF6F03 AD740CE3FFFF9231468126985708E1B9 salsa20 0.8.0.24919 keyring<br />
8C9106108AA84F07 53D859DDA2635A38DC32E72B11B32F29 salsa20 0.8.0.24919 keyring<br />
49166D358A34D815 667868CD94EA0135B9B16C93B1124ABA salsa20 0.8.0.24919 keyring<br />
1463A87356778D14 69BD2A78D05C503E93994959B30E5AEC salsa20 ≤ 1.0.3.0 keyring<br />
5E152DE44DFBEE01 E45A1793B37EE31A8EB85CEE0EEE1B68 salsa20 ≤ 1.0.3.0 keyring<br />
9B1F39EE592CA415 54A99F081CAD0D08F7E336F4368E894C salsa20 ≤ 1.0.3.0 keyring<br />
24C8B75890AD5917 31100C00FDE0CE18BBB33F3AC15B309F salsa20 ≤ 1.0.3.0 keyring<br />
EA658B75FDD4890F DEC7A4E721F425D133039895C36036F8 salsa20 ≤ 1.0.3.0 keyring<br />
026FDCDF8C5C7105 8F41809DA55366AD416D3C337459EEE3 salsa20 keyring<br />
CAE3FAC925F20402 98B78E8774BF275093CB1B5FC714511B salsa20 keyring<br />
061581CA8496C80C DA2EF5052DB917380B8AA6EF7A5F8E6A salsa20 keyring<br />
BE2CB0FAD3698123 902A1285836CE6DA5895020DD603B065 salsa20 keyring<br />
57A5A33B226B8E0A FDFC35C99B9DB11A326260CA246ACB41 salsa20 1.1.0.0.30200 keyring Ana<br />
42B9AB1AF5015920 C68778823C964C6F247ACC0F4A2584F8 salsa20 1.2.0.1.30684 keyring Summer Games<br />
4F0FE18E9FA1AC1A 89381C748F6531BBFCD97753D06CC3CD salsa20 1.2.0.1.30684 keyring<br />
7758B2CF1E4E3E1B 3DE60D37C664723595F27C5CDBF08BFA salsa20 1.2.0.1.30684 keyring<br />
E5317801B3561125 7DD051199F8401F95E4C03C884DCEA33 salsa20 1.4.0.2.32143 keyring Halloween Terror<br />
16B866D7BA3A8036 1395E882BF25B481F61A4D621141DA6E salsa20 1.4.1.0.31804 keyring Bastion Blizzcon 2016 skin<br />
11131FFDA0D18D30 C32AD1B82528E0A456897B3CE1C2D27E salsa20 1.5.0.1.32795 keyring Sombra<br />
CAC6B95B2724144A 73E4BEA145DF2B89B65AEF02F83FA260 salsa20 1.5.0.1.32795 keyring Ecopoint: Antarctica<br />
B7DBC693758A5C36 BC3A92BFE302518D91CC30790671BF10 salsa20 1.5.0.1.32795 keyring Genji Oni Skin (HotS Nexus Challenge)<br />
90CA73B2CDE3164B 5CBFF11F22720BACC2AE6AAD8FE53317 salsa20 1.6.1.0.33236 keyring Oasis map<br />
6DD3212FB942714A E02C1643602EC16C3AE2A4D254A08FD9 salsa20 1.6.1.0.33236 keyring<br />
11DDB470ABCBA130 66198766B1C4AF7589EFD13AD4DD667A salsa20 1.6.1.0.33236 keyring Winter Wonderland<br />
5BEF27EEE95E0B4B 36BCD2B551FF1C84AA3A3994CCEB033E salsa20 keyring<br />
9359B46E49D2DA42 173D65E7FCAE298A9363BD6AA189F200 salsa20 keyring Diablo's 20th anniversary<br />
1A46302EF8896F34 8029AD5451D4BC18E9D0F5AC449DC055 salsa20 1.7.0.2.34156 keyring Year of the Rooster<br />
693529F7D40A064C CE54873C62DAA48EFF27FCC032BD07E3 salsa20 1.8.0.0.34470 keyring CTF Maps<br />
388B85AEEDCB685D D926E659D04A096B24C19151076D379A salsa20 1.8.0.0.34470 keyring Numbani Update (Doomfist teaser)<br />
E218F69AAC6C104D F43D12C94A9A528497971F1CBE41AD4D salsa20 1.9.0.0.34986 keyring Orisa<br />
F432F0425363F250 BA69F2B33C2768F5F29BFE78A5A1FAD5 salsa20 1.10.0.0.35455 keyring Uprising<br />
061D52F86830B35D D779F9C6CC9A4BE103A4E90A7338F793 salsa20 1.10.0.0.35455 keyring D.Va Officer Skin (HotS Nexus Challenge 2)<br />
1275C84CF113EF65 CF58B6933EAF98AF53E76F8426CC7E6C salsa20 keyring<br />
D9C7C7AC0F14C868 3AFDF68E3A5D63BABA1E6821883F067D salsa20 keyring<br />
BD4E42661A432951 6DE8E28C8511644D5595FC45E5351472 salsa20 1.11.0.0.36376 keyring Anniversary event<br />
C43CB14355249451 0EA2B44F96A269A386856D049A3DEC86 salsa20 1.12.0.0.37104 keyring Horizon Lunar Colony<br />
E6D914F8E4744953 C8477C289DCE66D9136507A33AA33301 salsa20 1.13.0.0.37646 keyring Doomfist<br />
5694C503F8C80178 7F4CF1C1FBBAD92B184336D677EBF937 salsa20 1.13.0.0.37646 keyring Doomfist<br />
21DBFD65F3E54269 AB580C3837CAF8A461F243A566B2AE4D salsa20 1.13.0.0.37646 keyring Summer Games 2017<br />
27ABA5F88DD8D078 ???????????????????????????????? salsa20 1.13.0.0.37646 ??????? ??????<br />
21E1F90E71D33C71 328742339162B32676C803C2255931A6 salsa20 1.14.1.0.39083 keyring Deathmatch<br />
D9CB055BCDD40B6E 49FB4477A4A0825327E9A73682BECD0C salsa20 1.15.0.0.????? keyring Junkertown<br />
8175CE3C694C6659 E3F3FA7726C70D26AE130D969DDDF399 salsa20 1.16.0.0.40011 keyring Halloween 2017<br />
B8DE51690075435A C07E9260BB711217E7DE6FED911F4296 salsa20 1.16.0.0.????? keyring Winston Blizzcon 2017 skin<br />
F6CF23955B5D437D AEBA227328A5B0AA9F51DAE3F6A7DFE4 salsa20 1.17.0.2.41350 keyring Moira<br />
0E4D9426F2891F5C 9FF064C38BE52CCDF73748180F628205 salsa20 1.18.1.2.42076 keyring Winter Wonderland 2017<br />
9240BA6A2A0CF684 DF2E37D78B43108FA6242068B70D1F65 salsa20 1.19.1.3.42563 keyring Overwatch League<br />
82297FBAB7F5EB80 B534C20965852FB15AECAC17E381B417 salsa20 1.19.1.3.42563 keyring Jan 2017 Lootbox Update<br />
9ADF00AA1A174A69 9A4AC899261A2F1C6969F39397C358E7 salsa20 1.19.1.3.42563 keyring Blizzard World<br />
CFA05AA76B49F881 526DDDEF19BF373C25B629A334CD7237 salsa20 1.19.1.3.42563 keyring WoW's Battle For Azeroth Preorder<br />
493455579DA0B18E C0BABF72AD2C05DFC14017D1ADBF5977 salsa20 1.19.3.1.43036 stream Inaugral Season Spray/Icon ??<br />
6362C5AD65DAE686 62F603D5390F763ED51773F0164FEDB5 salsa20 1.19.3.1.43036 stream White/Gray OWL Skins ??<br />
8162E5313A9C135D F407834D9521587C5012B0A59D7E064B salsa20 1.20.0.2.43435 stream Lunar New Year 2018 (Dog) / Ayutthaya / Comp CTF<br />
68EAE8FDC008C381 ???????????????????????????????? salsa20 1.20.0.2.43435 n/a ??<br />
F412C6327C4BF091 6FAFC648CBF1C2115B769593C170E732 salsa20 1.20.0.2.43435 stream SC2 20th Anniversary (Kerrigan Skin)<br />
<br />
== World of Warcraft ==<br />
WoW's [[DB/TactKey|TactKey.db2]] and [[DB/TactKeyLookup|TactKeyLookup.db2]] contain keys and key_names (called lookups there) respectively. Either can be streamed from server.<br />
These files cannot be bruteforced by requesting hotfix data (Blizzard has guards in place)<br />
<br />
The id field in the table below corresponds to the id field in the db2's.<br />
key_name key type id seen in used for<br />
FA505078126ACB3E BDC51862ABED79B2DE48C8E7E66C6200 salsa20 15 WOW-20740patch7.0.1_Beta <not used between 7.0 and 7.3><br />
FF813F7D062AC0BC AA0B5C77F088CCC2D39049BD267F066D salsa20 25 WOW-20740patch7.0.1_Beta <not used between 7.0 and 7.3><br />
D1E9B5EDF9283668 8E4A2579894E38B4AB9058BA5C7328EE salsa20 39 WOW-20740patch7.0.1_Beta Enchanted Torch pet<br />
B76729641141CB34 9849D1AA7B1FD09819C5C66283A326EC salsa20 40 WOW-20740patch7.0.1_Beta Enchanted Pen pet<br />
FFB9469FF16E6BF8 D514BD1909A9E5DC8703F4B8BB1DFD9A salsa20 41 WOW-20740patch7.0.1_Beta <not used between 7.0 and 7.3><br />
23C5B5DF837A226C 1406E2D873B6FC99217A180881DA8D62 salsa20 42 WOW-20740patch7.0.1_Beta Enchanted Cauldron pet<br />
3AE403EF40AC3037 ???????????????????????????????? salsa20 51 WOW-21249patch7.0.3_Beta <not used between 7.0 and 7.3><br />
E2854509C471C554 433265F0CDEB2F4E65C0EE7008714D9E salsa20 52 WOW-21249patch7.0.3_Beta Warcraft movie items<br />
8EE2CB82178C995A DA6AFC989ED6CAD279885992C037A8EE salsa20 55 WOW-21531patch7.0.3_Beta BlizzCon 2016 Murlocs<br />
5813810F4EC9B005 01BE8B43142DD99A9E690FAD288B6082 salsa20 56 WOW-21531patch7.0.3_Beta Fel Kitten<br />
7F9E217166ED43EA 05FC927B9F4F5B05568142912A052B0F salsa20 57 WOW-21531patch7.0.3_Beta Legion music <not used in 7.3><br />
C4A8D364D23793F7 D1AC20FD14957FABC27196E9F6E7024A salsa20 58 WOW-21691patch7.0.3_Beta Demon Hunter #1 cinematic (legion_dh1)<br />
40A234AEBCF2C6E5 C6C5F6C7F735D7D94C87267FA4994D45 salsa20 59 WOW-21691patch7.0.3_Beta Demon Hunter #2 cinematic (legion_dh2)<br />
9CF7DFCFCBCE4AE5 72A97A24A998E3A5500F3871F37628C0 salsa20 60 WOW-21691patch7.0.3_Beta Val'sharah #1 cinematic (legion_val_yd)<br />
4E4BDECAB8485B4F 3832D7C42AAC9268F00BE7B6B48EC9AF salsa20 61 WOW-21691patch7.0.3_Beta Val'sharah #2 cinematic (legion_val_yx)<br />
94A50AC54EFF70E4 C2501A72654B96F86350C5A927962F7A salsa20 62 WOW-21691patch7.0.3_Beta Sylvanas warchief cinematic (legion_org_vs)<br />
BA973B0E01DE1C2C D83BBCB46CC438B17A48E76C4F5654A3 salsa20 63 WOW-21691patch7.0.3_Beta Stormheim Sylvanas vs Greymane cinematic (legion_sth)<br />
494A6F8E8E108BEF F0FDE1D29B274F6E7DBDB7FF815FE910 salsa20 64 WOW-21691patch7.0.3_Beta Harbingers Gul'dan video (legion_hrb_g)<br />
918D6DD0C3849002 857090D926BB28AEDA4BF028CACC4BA3 salsa20 65 WOW-21691patch7.0.3_Beta Harbingers Khadgar video (legion_hrb_k)<br />
0B5F6957915ADDCA 4DD0DC82B101C80ABAC0A4D57E67F859 salsa20 66 WOW-21691patch7.0.3_Beta Harbingers Illidan video (legion_hrb_i)<br />
794F25C6CD8AB62B 76583BDACD5257A3F73D1598A2CA2D99 salsa20 67 WOW-21846patch7.0.3_Beta Suramar cinematic (legion_su_i)<br />
A9633A54C1673D21 1F8D467F5D6D411F8A548B6329A5087E salsa20 68 WOW-21846patch7.0.3_Beta legion_su_r cinematic<br />
5E5D896B3E163DEA 8ACE8DB169E2F98AC36AD52C088E77C1 salsa20 69 WOW-21846patch7.0.3_Beta Broken Shore intro cinematic (legion_bs_i)<br />
0EBE36B5010DFD7F 9A89CC7E3ACB29CF14C60BC13B1E4616 salsa20 70 WOW-21846patch7.0.3_Beta Alliance Broken Shore cinematic (legion_bs_a)<br />
01E828CFFA450C0F 972B6E74420EC519E6F9D97D594AA37C salsa20 71 WOW-21846patch7.0.3_Beta Horde Broken Shore cinematic (legion_bs_h)<br />
4A7BD170FE18E6AE AB55AE1BF0C7C519AFF028C15610A45B salsa20 72 WOW-21846patch7.0.3_Beta Khadgar & Light's Heart cinematic (legion_iq_lv)<br />
69549CB975E87C4F 7B6FA382E1FAD1465C851E3F4734A1B3 salsa20 73 WOW-21846patch7.0.3_Beta legion_iq_id cinematic<br />
460C92C372B2A166 946D5659F2FAF327C0B7EC828B748ADB salsa20 74 WOW-21952patch7.0.3_Beta Stormheim Alliance cinematic (legion_g_a_sth)<br />
8165D801CCA11962 CD0C0FFAAD9363EC14DD25ECDD2A5B62 salsa20 75 WOW-21952patch7.0.3_Beta Stormheim Horde cinematic (legion_g_h_sth)<br />
A3F1C999090ADAC9 B72FEF4A01488A88FF02280AA07A92BB salsa20 81 WOW-22578patch7.1.0_PTR Firecat Mount<br />
18AFDF5191923610 ???????????????????????????????? salsa20 82 WOW-22578patch7.1.0_PTR <not used between 7.1 and 7.3><br />
3C258426058FBD93 ???????????????????????????????? salsa20 91 WOW-23436patch7.2.0_PTR <not used between 7.2 and 7.3><br />
094E9A0474876B98 E533BB6D65727A5832680D620B0BC10B salsa20 92 WOW-23910patch7.2.5_PTR shadowstalkerpanthermount, shadowstalkerpantherpet<br />
3DB25CB86A40335E 02990B12260C1E9FDD73FE47CBAB7024 salsa20 93 WOW-23789patch7.2.0_PTR legion_72_ots<br />
0DCD81945F4B4686 1B789B87FB3C9238D528997BFAB44186 salsa20 94 WOW-23789patch7.2.0_PTR legion_72_tst<br />
486A2A3A2803BE89 32679EA7B0F99EBF4FA170E847EA439A salsa20 95 WOW-23789patch7.2.0_PTR legion_72_ars<br />
71F69446AD848E06 E79AEB88B1509F628F38208201741C30 salsa20 97 WOW-24473patch7.3.0_PTR BlizzCon 2017 Mounts (AllianceShipMount and HordeZeppelinMount)<br />
211FCD1265A928E9 A736FBF58D587B3972CE154A86AE4540 salsa20 98 WOW-24473patch7.3.0_PTR "Shadow" fox pet (store) <br />
0ADC9E327E42E98C 017B3472C1DEE304FA0B2FF8E53FF7D6 salsa20 99 WOW-23910patch7.2.5_PTR legion_72_tsf<br />
BAE9F621B60174F1 38C3FB39B4971760B4B982FE9F095014 salsa20 100 WOW-24727patch7.3.0_PTR Rejection of the Gift cinematic (legion_73_agi)<br />
34DE1EEADC97115E 2E3A53D59A491E5CD173F337F7CD8C61 salsa20 101 WOW-24727patch7.3.0_PTR Resurrection of Alleria Windrunner cinematic (legion_73_avt)<br />
E07E107F1390A3DF 290D27B0E871F8C5B14A14E514D0F0D9 salsa20 102 WOW-25079patch7.3.2_PTR Tottle battle pet, Raptor mount, Horse mount (104 files)<br />
32690BF74DE12530 A2556210AE5422E6D61EDAAF122CB637 salsa20 103 WOW-24781patch7.3.0_PTR legion_73_pan<br />
BF3734B1DCB04696 48946123050B00A7EFB1C029EE6CC438 salsa20 104 WOW-25079patch7.3.2_PTR legion_73_afn<br />
74F4F78002A5A1BE C14EEC8D5AEEF93FA811D450B4E46E91 salsa20 105 WOW-25079patch7.3.2_PTR SilithusPhase01 map<br />
423F07656CA27D23 ???????????????????????????????? salsa20 107 WOW-25600patch7.3.5_PTR bltestmap<br />
0691678F83E8A75D ???????????????????????????????? salsa20 108 WOW-25600patch7.3.5_PTR filedataid 1782602-1782603<br />
324498590F550556 ???????????????????????????????? salsa20 109 WOW-25600patch7.3.5_PTR filedataid 1782615-1782619<br />
C02C78F40BEF5998 ???????????????????????????????? salsa20 110 WOW-25600patch7.3.5_PTR test/testtexture.blp (fdid 1782613)<br />
47011412CCAAB541 ???????????????????????????????? salsa20 111 WOW-25600patch7.3.5_PTR unused in 25600<br />
23B6F5764CE2DDD6 ???????????????????????????????? salsa20 112 WOW-25600patch7.3.5_PTR unused in 25600<br />
8E00C6F405873583 ???????????????????????????????? salsa20 113 WOW-25600patch7.3.5_PTR filedataid 1783470-1783472<br />
78482170E4CFD4A6 768540C20A5B153583AD7F53130C58FE salsa20 114 WOW-25600patch7.3.5_PTR Magni Bronzebeard VO<br />
B1EB52A64BFAF7BF 458133AA43949A141632C4F8596DE2B0 salsa20 115 WOW-25600patch7.3.5_PTR dogmount, 50 files<br />
FC6F20EE98D208F6 57790E48D35500E70DF812594F507BE7 salsa20 117 WOW-25632patch7.3.5_PTR shop stuff<br />
402CFABF2020D9B7 67197BCD9D0EF0C4085378FAA69A3264 salsa20 118 WOW-25678patch7.3.5_PTR filedataid 1854762<br />
6FA0420E902B4FBE 27B750184E5329C4E4455CBD3E1FD5AB salsa20 119 WOW-25744patch7.3.5_PTR legion_735_epa / legion_735_eph<br />
1076074F2B350A2D 88BF0CD0D5BA159AE7CB916AFBE13865 salsa20 121 WOW-26287patch8.0.1_Beta skiff<br />
816F00C1322CDF52 ???????????????????????????????? salsa20 122 WOW-26287patch8.0.1_Beta snowkid<br />
DDD295C82E60DB3C 3429CC5927D1629765974FD9AFAB7580 salsa20 123 WOW-26287patch8.0.1_Beta redbird<br />
83E96F07F259F799 91F7D0E7A02CDE0DE0BD367FABCB8A6E salsa20 124 WOW-26522patch8.0.1_Beta BlizzCon 2018 (Alliance and Horde banners and cloaks)<br />
C1E5D7408A7D4484 A7D88E52749FA5459D644523F8359651 salsa20 226 WOW-26871patch8.0.1_Beta Sylvanas Warbringer cinematic<br />
E46276EB9E1A9854 CCCA36E302F9459B1D60526A31BE77C8 salsa20 227 WOW-26871patch8.0.1_Beta ltc_a, ltc_h and ltt cinematics<br />
D245B671DD78648C 19DCB4D45A658B54351DB7DDC81DE79E salsa20 228 WOW-26871patch8.0.1_Beta stz, zia, kta, jnm & ja cinematics<br />
4C596E12D36DDFC3 B8731926389499CBD4ADBF5006CA0391 salsa20 229 WOW-26871patch8.0.1_Beta bar cinematic<br />
0C9ABD5081C06411 25A77CD800197EE6A32DD63F04E115FA salsa20 230 WOW-26871patch8.0.1_Beta zcf cinematic<br />
3C6243057F3D9B24 58AE3E064210E3EDF9C1259CDE914C5D salsa20 231 WOW-26871patch8.0.1_Beta ktf cinematic<br />
7827FBE24427E27D 34A432042073CD0B51627068D2E0BD3E salsa20 232 WOW-26871patch8.0.1_Beta rot cinematic<br />
5DD92EE32BBF9ABD ???????????????????????????????? salsa20 234 WOW-27004patch8.0.1_Subm filedataid 2238294<br />
<br />
=States of CASC Data=<br />
CASC data comes in all forms and sizes.<br />
<br />
==Key CASC Files==<br />
===Encoding===<br />
The encoding file maps content hashes <tt>C-Key</tt>s to encoded-file hashes <tt>E-Key</tt>s. In addition, there is information on how the files are [[CASC#BLTE_encoded_files|BLTE]]-encoded by <tt>E-Spec</tt>s.<br />
<br />
Blocks in this file are, in this order<br />
<br />
* header<br />
* encoding specification data <tt>ESpec</tt><br />
* content key → encoding key table <tt>CEKeyPageTable</tt><br />
* encoding key → encoding spec table <tt>EKeySpecPageTable</tt><br />
* encoding specification data for the encoding file itself<br />
<br />
An incomplete/outdated 010 Editor template can be found at [https://gist.github.com/heksesang/fdda3e4f8a5ed53b71ed this gist] which can be used to understand page handling.<br />
<br />
====Header====<br />
Header is a constant 0x16 bytes giving size information for the other blocks, mostly.<br />
<br />
struct {<br />
/*0x00*/ char signature[2]; // "EN"<br />
enum {<br />
encoding_version_1 = 1, // {{Sandbox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125}}<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ckey;<br />
/*0x04*/ uint8_BE_t hash_size_ekey;<br />
/*0x05*/ uint16_BE_t CEKeyPageTable_page_size_kb; // in kilo bytes. e.g. 4 in here → 4096 byte pages (default)<br />
/*0x07*/ uint16_BE_t EKeySpecPageTable_page_size_kb; // ^<br />
/*0x09*/ uint32_BE_t CEKeyPageTable_page_count;<br />
/*0x0D*/ uint32_BE_t EKeySpecPageTable_page_count;<br />
/*0x11*/ uint8_BE_t _unknown_x11; // 0 -- sometimes assumed to be part of ESpec_block_size, but actually asserted to be zero by agent<br />
/*0x12*/ uint32_BE_t ESpec_block_size;<br />
/*0x16*/<br />
} header;<br />
<br />
====ESpec====<br />
Encoding specification strings are just a blob of zero terminated strings referenced by <tt>EKeySpecPageTable</tt> with the accumulated size of <tt>header.ESpec_block_size</tt> (including zero terminators).<br />
<br />
The definition of the format for these strings is described in the [[#Encoding_Specification_.28ESpec.29|BLTE]] section.<br />
<br />
====Page Tables====<br />
The format of the two page tables is the same: <br />
<br />
* an index for fast key → page access, followed by<br />
* the actual pages with specific content<br />
<br />
In both cases, the entries in the lists have the same count, and semi-dynamic size, depending on <tt>header.hash_size_ckey</tt> and <tt>header.hash_size_ekey</tt>. Note that the page checksum size is fixed to MD5's 16 bytes. The position comments below assume the standard key size of 16 bytes.<br />
<br />
struct page_index_t {<br />
/*0x00*/ char first_Xkey[header.hash_size_Xkey]; // where X is c for CEKeyPageTable and e for EKeySpecPageTable<br />
/*0x10*/ char page_md5[0x10];<br />
/*0x20 usually*/<br />
};<br />
<br />
The pages themselves are filled as much possible with actual entry structs. They are padded to the end for alignment of pages. Pages don't actually have to be full.<br />
<br />
=====CEKeyPageTable=====<br />
This table maps one <tt>ckey</tt> to one or more <tt>ekey</tt>s. This means that there can be multiple representations, e.g. an encrypted and an unencrypted version. This isn't usually the case and on reading any of them can be picked, since they represent the same file content. It can be used to pick an already downloaded archive, or unencrypted one, or handle deleted archives.<br />
<br />
struct ckey_ekey_entry_t {<br />
/*0x00*/ uint8_BE_t keyCount;<br />
/*0x01*/ uint40_BE_t file_size; // of the non-encoded version of the file<br />
/*0x06*/ char ckey[header.hash_size_ckey]; // this ckey is represented by…<br />
/*0x16*/ char ekey[header.hash_size_ekey][keyCount]; // …these ekeys<br />
/*0x26 usually*/<br />
} page_entries[];<br />
<br />
=====EKeySpecPageTable=====<br />
This table maps one <tt>ekey</tt> to the corresponding <tt>espec</tt> describing how the encoding happened.<br />
<br />
struct ekey_espec_entry_t {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint32_BE_t espec_index; // not an offset but an index, assuming zero-terminated espec strings<br />
/*0x14*/ uint40_BE_t file_size; // of the encoded version of the file<br />
/*0x19 usually*/<br />
} page_entries[];<br />
<br />
===Install===<br />
File signature: "IN"<br />
<br />
The install file lists files installed on disk. Since the install file is shared by architectures and OSs, there are also tags to select a subset of files. When using multiple tags, a binary combination of the bitfields of files to be installed can be created. <br />
<br />
====Header Structure====<br />
The file begins with a 10 byte header describing the number of tags and files listed. '''Structure names were invented by the author of this page.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[2] || FileSignature || "IN"<br />
|-<br />
| 0x02 || uint8_t || Version? || 1<br />
|-<br />
| 0x03 || uint8_t || hash_size || size of hashes used for files (usually md5 -> 16)<br />
|-<br />
| 0x04 || uint16_BE_t || num_tags || number of tags in header of file<br />
|- <br />
| 0x06 || uint32_BE_t || num_entries || The number of entries in the body of the file<br />
|}<br />
<br />
====Tags Structure====<br />
After the header, an array with information about available tags follows. Each tag has a bitfield listing the files installed when the given tag is chosen. <br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || name || <br />
|- <br />
| uint16_BE_t || type || A number shared amongst specific flags. Actual meaning is [[#Product_Specific|specific to products]].<br />
|- <br />
| char[divru (header.entries, CHAR_BIT)] || files || A bitfield that lists which files are installed when the specified tag is installed.<br />
|}<br />
<br />
====Files Structure====<br />
The remainder of the file is populated by a list of files with their content hash, each a variable size (due to the strings). Structure names were invented by the author of this page.<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| char[] || FileName || The name of the file.<br />
|- <br />
| char[header.hash_size] || hash || The hash of the uncompressed file. Usually MD5.<br />
|- <br />
| uint32_BE_t || Size || The size of the file.<br />
|}<br />
<br />
====C-like structure====<br />
char I; char N;<br />
uint8_BE_t _unk3;<br />
uint8_BE_t hash_size;<br />
uint16_BE_t num_tags;<br />
uint32_BE_t num_files;<br />
<br />
struct {<br />
string name;<br />
uint16_BE_t type;<br />
char flags[divru (num_files, CHAR_BIT)];<br />
} tags[num_tags];<br />
<br />
struct {<br />
string name;<br />
char hash[hash_size];<br />
uint32_BE_t size;<br />
} files[num_files];<br />
<br />
===Download===<br />
The download file lists all files stored in the data archives.<br />
The client uses this to download files ahead of time, without it, the client will download on demand which can lead to issues. <br />
The download priority is set inside the entries with 0 being the highest and 2 the lowest however, if the game is running, missing assets in the player's vicinity take precedence.<br />
<br />
Just like the install file, the download file is shared across all architectures and locales so utilizes the same bitfield-tag system to assess what subset of files are needed.<br />
<br />
NOTE: partial-priority download files do not contain the actual file sizes but redefine the FileSize field to ChunkSize.<br />
<br />
This file has this structure:<br />
<br />
* Header<br />
* Entries[Header.EntryCount]<br />
* Tags[Header.TagsCount]<br />
<br />
====Download Header====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "DL")<br />
|-<br />
| char || Version || 1 < 7.3.0, 2<br />
|-<br />
| char || ChecksumSize || Always 0x10<br />
|-<br />
| char || unk || ??? Always 1<br />
|-<br />
| int [BE] || EntryCount || The amount of file entries in this file<br />
|-<br />
| short [BE] || TagCount || The amount of tag entries in this file<br />
|}<br />
<br />
====Download Entry====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="120" | Version Added<br />
! width="810" | Description<br />
|-<br />
| char || Unk || 2 || ??? Appears to be a boolean. Currently only set to 1 on 4 specific records<br />
|-<br />
| char[16] || Hash || 1 || This hash is found in every node of the encoding file. (Reverse lookup) MD5<br />
|-<br />
| uint40_t [BE] || FileSize || 1 || The compressed size of the file<br />
|-<br />
| char || DownloadPriority || 1 || 0 = highest, 2 = lowest<br />
|-<br />
| char[4] || Unk || 1 || ???<br />
|}<br />
<br />
====Download Tag====<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| string || Name || A C-String indicating this tag's Name.<br />
|-<br />
| short [BE] || Type || Hash type<br />
|-<br />
| char[N] || Bits || an array of size N = Header.EntryCount / 8 + (Header.EntryCount % 8 > 0 ? 1 : 0); that is basically a massive bit mess. Use Schroeppel's 8 bits reverse function on it to have bits.<br />
|}<br />
<br />
====code-ish====<br />
struct {<br />
/*0x00*/ char signature[2]; // "DL"<br />
enum {<br />
download_version_1 = 1,<br />
download_version_2 = 2, // {{Unverified|{{Sandbox/VersionRange|min_expansionlevel=7|min_build=7.3.0.???}}}}<br />
download_version_3 = 3,<br />
};<br />
/*0x02*/ uint8_BE_t version;<br />
/*0x03*/ uint8_BE_t hash_size_ekey;<br />
/*0x04*/ uint8_BE_t has_checksum_in_entry;<br />
/*0x05*/ uint32_BE_t entry_count;<br />
/*0x09*/ uint16_BE_t tag_count;<br />
/*0x0b*/<br />
<br />
#if version ≥ download_version_2<br />
/*0x0b*/ uint8_BE_t {{Unverified|number_of_flag_bytes}}; // defaults to 0, up to {{Unverified|4}}<br />
/*0x0c*/<br />
<br />
#if version >= download_version_3<br />
/*0x0c*/ uint8_BE_t {{Unverified|base_priority}}; // defaults to 0<br />
/*0x0d*/ char _unknown_0d[3]; // As of 1.15.6.2-test4, this is explicitly 0. It is ignored on reading.<br />
/*0x10*/<br />
#endif<br />
#endif<br />
} header;<br />
<br />
struct {<br />
/*0x00*/ char ekey[header.hash_size_ekey];<br />
/*0x10*/ uint40_BE_t file_size;<br />
/*0x15*/ uint8_BE_t priority; // {{Unverified|header.base_priority is subtracted on parse}}<br />
/*0x16*/<br />
<br />
#if header.has_checksum_in_entry<br />
/*0x16*/ uint32_BE_t checksum;<br />
/*0x1a*/<br />
#endif<br />
<br />
#if header.version ≥ download_version_2<br />
enum {<br />
download_flag_plugin = 1, // "plugin"<br />
download_flag_plugin_data = 2, // "plugin-data"<br />
}; <br />
uint8_BE_t flags[header.number_of_flag_bytes]; // {{Unverified|defaults to 0 if no flag bytes present}}<br />
#endif<br />
<br />
} entries[header.entry_count];<br />
<br />
struct {<br />
char const name[]; // this string is zero terminated, no fixed size<br />
// thus for readability we start offset at 0 here.<br />
/*0+00*/ uint16_BE_t type; // game specific. usually architecture, category, locale, os, region or alike.<br />
/*0+02*/ char mask[divru (header.entry_count._, CHAR_BIT)]; // if bit is set, entries[bit] is part of this tag<br />
/*0x??*/<br />
} tag[header.tag_count];<br />
<br />
===Download Size===<br />
Build 27547 introduced the Download Size file of unknown purpose. The file is a stripped-down Download file with partial EKeys and files sorted by encoded files size. The purpose of this file is not clear.<br />
<br />
struct Header<br />
{<br />
char signature[2]; // "DS"<br />
uint8_t version;<br />
uint8_t ekeySize; // 9<br />
uint32_BE_t numFiles;<br />
uint16_BE_t numTags;<br />
uint40_BE_t totalSize; // Size of all files combined<br />
};<br />
<br />
struct TagEntry<br />
{<br />
char name[]; // Null-terminated<br />
uint16_BE_t type;<br />
char fileMask[(hdr.numFiles + 7) / 8];<br />
}<br />
<br />
struct FileEntry<br />
{<br />
char ekey[hdr.ekeySize];<br />
uint32_BE_t esize;<br />
};<br />
<br />
SizeHeader hdr;<br />
TagEntry tags[hdr.numTags];<br />
FileEntries[hdr.numFiles]; // Sorted descending by esize<br />
<br />
===Patch===<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="120" | Type<br />
! width="150" | Name<br />
! width="810" | Description<br />
|-<br />
| char[2] || Signature || The signature for this file (always "PA")<br />
|-<br />
| char || version || 1 or 2<br />
|-<br />
| char || file_key_size || <= 0x10<br />
|-<br />
| char || size_b || <= 0x10<br />
|-<br />
| char || patch_key_size || <= 0x10<br />
|-<br />
| char || block_size_bits || 2 <= block_size_bits <= 24. block size == 2^block_size_bits.<br />
|-<br />
| short || block_count ||<br />
|-<br />
| char || flags ||<br />
|-<br />
| char[16] || EncodingCkey || ckey for encoding file<br />
|-<br />
| char[16] || EncodingEkey || ekey for encoding file<br />
|-<br />
| int || DecodedSize || Decoded encoding file size in bytes<br />
|-<br />
| int || EncodedSize || Encoded encoding file size in bytes<br />
|-<br />
| char || EspecLength || Length of the following string<br />
|-<br />
| char[EspecLength] || EncodingEspec || espec of encoding file<br />
|-<br />
| char[] || ??? || byte array containing blocks entries, blocks, and optional patch tail<br />
|}<br />
<br />
<br />
header+entries needs to be less than 0x10000 bytes (at least in wow-18179). md5sum is only checked for header+entries, file might be larger thus.<br />
<br />
struct PatchManifest_Header<br />
{<br />
uint16_BE_t magic; // 'PA'<br />
uint8_t version; // 1 or 2<br />
uint8_t file_key_size; // <= 0x10<br />
uint8_t size_b; // <= 0x10<br />
uint8_t patch_key_size; // <= 0x10<br />
uint8_t block_size_bits; // 12 <= block_size_bits <= 24. max block size == 2^block_size_bits<br />
uint16_BE_t block_count; // (file_key_size + 20) * entry_count + sizeof (PatchManifest_Header) < 0x10000<br />
uint8_t unk2; // flags<br />
<br />
#if encoding_information_apparently_added_after_18179<br />
uint8_t encoding_ckey[16];<br />
uint8_t encoding_ekey[16]; // probably since PA2<br />
uint32_BE_t decoded_size;<br />
uint32_BE_t encoded_size;<br />
uint8_t encoding_espec_length;<br />
char encoding_format[encoding_espec_length];<br />
#endif<br />
} header;<br />
<br />
struct PatchManifest_Block<br />
{<br />
uint8_t last_file_ckey[header.file_key_size];<br />
uint8_t md5_of_block[16];<br />
uint32_BE_t block_offset; // in this file<br />
} blocks[header.block_count]; // sorted ascending by key<br />
<br />
// at positions given in PatchManifest_Block<br />
struct block<br />
{<br />
struct<br />
{<br />
uint8_t num_patches; // <= 0x10.<br />
uint8_t target_file_ckey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
struct<br />
{<br />
uint8_t source_file_ekey[header.file_key_size];<br />
uint40_BE_t decoded_size;<br />
uint8_t patch_ekey[header.patch_key_size];<br />
uint32_BE_t patch_size;<br />
uint8_t unk; // some sort of patch index number. first entry seems to always be 1<br />
} patches[num_patches];<br />
} files[]; // count unspecified: read until the next file num_patches would be 0 <br />
// OR block would exceed max block size<br />
};<br />
<br />
// some files have a block of data after the last block of the patch manifest (which may be shorter than max block size). this block appears to be a patch of encoding, but the format is not understood.<br />
<br />
==CDN File Organization==<br />
Data for every CASC-based game exists on the CDN in one of two places at any given time. To reduce file-system and download overhead many files are packed into archives and indexed by archive indices, both of a different format than the CASC installations found on client systems; other "unarchived", "standalone", or "loose" files are stored as separate files on the CDN that must be downloaded independently. There are at least three different reasons why a particular file is found in one or the other:<br />
# Small files typically incur larger filesystem overhead and benefit most from being packed into archives. A rough rule of thumb appears to be that files smaller than 2 MB or so are put in archives. Presumably larger files are not archived because they make it more difficult to minimize the number of archive files, which are limited in size (a 2 MB file limit would limit unused space in an archive to under 0.8%, given enough data to form a full archive to begin with).<br />
# Key files such as encoding, partial-priority, TVFS (Warcraft 3 root), and the download, install, and patch manifests, as well as their respective patches, are typically stored loose for quick access and are rarely ever found in archives.<br />
# As games evolve old files become obsolete and are removed from the CDN. However, the archive system means that archives can only be removed when every file in the archive is no longer needed, potentially wasting large amounts of space on the CDN - the exact opposite of the purpose of bundling files into archives to begin with. Thus as the amount of unused data in an archive grows over time, files still in use may be converted to loose files to allow the archive to be purged from the CDN, even when the files are unusually small to be found independently.<br />
<br />
Previously, no official indices/manifests of loose files existed, and they could only be found by subtracting archived files from file lists in encoding or manifest files. Beginning with Warcraft 3 and subsequently being deployed more widely on 7/24/2018, new fields in the CDN config file link to index files containing "all" loose data or patch files on the current CDNs.<br />
<br />
===Archives===<br />
Archives are extensionless 256 MB files that are usually only stored on the Blizzard CDNs. Their naming follows the standard URL hash format using the '/data/' path type.<br />
<br />
The structure of the archives is presumably just file fragment after file fragment. You will never need to parse it because you can just look up offset + size of your file fragment in the index files and then take the piece directly out of the archive.<br />
<br />
The fragments are all BLTE encoded.<br />
<br />
The filename is '''NOT''' the hash of the archive content but the hash of the '''index''''s footer.<br />
<br />
===Archive Indexes (.index)===<br />
These '.index' files reveal to the user where the compressed game files are located within the archives. All indexes (except the Archive-Group index, see below) are named after their archive (only difference is these have an extension).<br />
'.index' files are stored on the CDN using the standard hash naming scheme (remember they have an extension though). They are also located in the directory 'INSTALL_DIR/Data/indices/' for a WoW install.<br />
<br />
====Normal Index Entry Structure====<br />
*'''The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null blte_header_hash fields. The last chunk is a table-of-contents, listing the LAST blte_header_hash in each chunk and checksum of each block. All checksums are the first checksumSize bytes of the MD5 of the respective data. Structure names were invented by the author of this page.'''<br />
<br />
struct index_entry {<br />
char blte_header_hash[footer.keySizeInBytes];<br />
uint_BE_t(footer.sizeBytes) blte_encoded_size;<br />
uint_BE_t(footer.offsetBytes) offset_to_blte_encoded_data_in_archive;<br />
};<br />
<br />
struct index_block {<br />
static constexpr const block_size = footer.blockSizeKb << 10;<br />
index_entry entries[block_size / sizeof (index_entry)];<br />
char padding[block_size - sizeof (entries)];<br />
} blocks[];<br />
<br />
struct {<br />
struct {<br />
char last_hash[footer.keySizeInBytes]; // last hash of a block<br />
} entries[num_blocks];<br />
<br />
struct {<br />
char lower_part_of_md5_of_block[footer.checksumSize]; <br />
} blocks_hash[num_blocks];<br />
} toc;<br />
<br />
struct {<br />
char toc_hash[checksumSize]; // client tries to read with 0x10, then backs off when smaller<br />
char version?; // always 1<br />
char _11; // 0<br />
char _12; // 0<br />
char blockSizeKb?; // Normally 4. Left-shifted by 10. Believed to be block size in KB.<br />
char offsetBytes; // Normally 4 for archive indices, 6 for group indices, and 0 for loose file indices<br />
char sizeBytes; // Normally 4<br />
char keySizeInBytes; // Normally 16<br />
char checksumSize; // Normally 8, <= 0x10<br />
uint32_t numElements; // BigEndian in _old_ versions (e.g. 18179)<br />
char footerChecksum[checksumSize];<br />
} footer;<br />
<br />
* footerChecksum is calculated over the footer beginning with version ''when footerChecksum is zeroed''<br />
* The archive/index name is the MD5 of the footer beginning with toc_hash<br />
<br />
===archive-group===<br />
<tt>archive-group</tt> is actually a very special <tt>.index</tt> file that is typically much larger than other indices. There is '''no''' archive attached, it is only the index. Also, the index is '''not''' found on the CDN! It is assembled by combining all index files given in the CDN config, on client side. The hash is just for verification purposes. Client side a file is assembled using the normal index file format, for caching.<br />
<br />
It has a single difference in format to normal indices: While other indices have their <tt>offsetBytes</tt> long <tt>offset</tt> field point into the archive, for archive groups, the field also has an <tt>archiveIndex</tt>:<br />
<br />
struct {<br />
uint16_BE_t archiveIndex; // Index of the archive in the CDN config's archive list<br />
uint32_BE_t offsetBytes; // The offset within the specified archive<br />
};<br />
<br />
It does not only having the offset, but also an index into a file. Semantically that's still an offset, so no further field for size is used.<br />
<br />
It is suggested you do not just parse indices by <tt>.index</tt> filename locally but take the config files into account. An easy heuristic is that if <tt>offsetBytes</tt> is not 4, it is a special index, either loose files or a group.<br />
<br />
===patch-archives===<br />
Patch archives are the /patch/ equivalent to the regular (data) archives on the CDN. Like archives, these are binary blobs of fragments indexed by an accompanying .index file with the same name. Again, the index is a <tt>hash, size, offset</tt> tuple, but the hash is the content hash rather than an encoding hash.<br />
<br />
Most files in patch archives are [[Patching Files|ZBSDIFF1]] blobs, though in principle any file that might be in the /patch/ namespace may be found in patch archives and must be handled accordingly.<br />
<br />
===patch-archive-group===<br />
See [[#archive-group|archive-group]]. There is no known difference other than the combined data being <tt>patch-archives</tt>.<br />
<br />
==Journal-based Data Files==<br />
During the installation process for a Blizzard game, the program will download the required files as requested by root, encoding, download, and install. It stores the downloaded data fragments in data files in "INSTALL_DIR\Data\data\". The program will record the content hash (BLTE-compressed hash), size, and position of the file as well as the number of the data file that it is in. It places those four parameters into journal files with the extension '.idx'.<br />
<br />
===Shared Memory===<br />
The shared memory file is called 'shmem' and is usually located in the same folder as the data and .IDX journals. This file contains the path where the data files are stored, which is the current version of each of the .IDX files, and which areas of the data files have unused space. The file is recreated every time a client is started.<br />
<br />
====Shared Memory Header Structure====<br />
*'''The first part of the header.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32_t || BlockType || A value indicating what type of block this is. For this block, the value is 4.<br />
|- <br />
| 0x04 || uint32_t || NextBlock || The offset of the next block.<br />
|- <br />
| 0x08 || char[0x100] || DataPath || The path of the data files. This is prefixed with "Global\" if the path is an absolute path.<br />
|}<br />
<br />
<br />
*'''Followed by a number of these entries. The count can be calculated like this: (NextBlock - 264 - idxFileCount * 4) / 8'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32_t || Size || The size of the block.<br />
|- <br />
| 0x04 || uint32_t || Offset || The offset of the block.<br />
|}<br />
<br />
<br />
*'''Followed by a number of these entries. The count is equal to number of .IDX files (usually 16).'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32_t || Version || The version number. Used to identify the .IDX filename.<br />
|}<br />
<br />
<br />
====Shared Memory Free Space Structure====<br />
After a small header, this structure is split up into two equal parts.<br />
The first part contains entries with the number of unused bytes.<br />
The second part contains entries with the position of the unused bytes.<br />
<br />
There can be up to 1090 entries. Each of the two parts will always be 5450 bytes, so if there are fewer than 1090 entries, the rest of the bytes will be padded with '\0'.<br />
<br />
*'''The header part of the structure.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32_t || BlockType || A value indicating what type of block this is. For this block, the value is 1.<br />
|- <br />
| 0x04 || uint32_t || NextBlock || The offset of the next block.<br />
|- <br />
| 0x08 || char[0x18] || Padding || Padding at the end of the header.<br />
|}<br />
<br />
<br />
*'''This is the number of unused bytes. There can be up to 1090 entries of these. If there are fewer, the rest of the area is padded.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|- <br />
| 0x00 || uint10* || DataNumber || This is always set to 0 in this part of the block.<br />
|- <br />
| 0x01 || uint30* || Count || The number of unused bytes.<br />
|}<br />
<br />
<br />
*'''This is the position of the unused bytes. There can be up to 1090 entries of these. If there are fewer, the rest of the area is padded.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|- <br />
| 0x00 || uint10* || DataNumber || The number of the data file where the unused bytes are located.<br />
|- <br />
| 0x01 || uint30* || Offset || The position within the data file where the unused bytes are located.<br />
|}<br />
<br />
===.IDX Journals===<br />
Example file path: INSTALL_DIR\Data\data\0e00000054.idx<br />
<br />
.IDX journals contain a mapping from keys to the location of their data in the local CASC archives. There used to be one .IDX file per journal, and the naming scheme used to have two separate meanings. The '0e' part of the file name used to designate which archive the .IDX file was associated with. This changed halfway through the Warlords Beta. Now there are 16 indices total, and the first byte of the hex filename says which of the 16 indices it is, while the remainder of the hex filename is just a version number that increments when a new set of files is added to the local archives.<br />
<br />
To determine which of the 16 indices a key is bucketed in, the key is hashed by xoring together each 4-bit nibble in the first 9 bytes of the key:<br />
<br />
uint8_t cascGetBucketIndex(const uint8_t k[16]) {<br />
uint8_t i = k[0] ^ k[1] ^ k[2] ^ k[3] ^ k[4] ^ k[5] ^ k[6] ^ k[7] ^ k[8];<br />
return (i & 0xf) ^ (i >> 4);<br />
}<br />
<br />
<br />
====.IDX Header Structure====<br />
<br />
The header is little-endian:<br />
<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || uint32 || HeaderHashSize || The number of bytes to use for the hash at +04; usually 0x10.<br />
|- <br />
| 0x04 || uint32 || HeaderHash || This should equal the value of pc after calling hashlittle2 on the following HeaderHashSize bytes of the file with an initial value of 0 for pb and pc.<br />
|- <br />
| 0x08 || uint16 || Unk0 || Must be 7<br />
|- <br />
| 0x0a || uint8 || BucketIndex || The bucket index of this file; should be the same as the first byte of the hex filename.<br />
|- <br />
| 0x0b || uint8 || Unk1 || Must be 0<br />
|- <br />
| 0x0c || uint8 || EntrySizeBytes || Must be 4<br />
|- <br />
| 0x0d || uint8 || EntryOffsetBytes || Must be 5<br />
|- <br />
| 0x0e || uint8 || EntryKeyBytes || Must be 9<br />
|- <br />
| 0x0f || uint8 || ArchiveFileHeaderBytes || Must be 30<br />
|- <br />
| 0x10 || uint64 || ArchiveTotalSizeMaximum || The maximum size of a casc installation; 0x4000000000, or 256GiB.<br />
|- <br />
| 0x18 || char[8] || padding || The header is padded with zeroes to the next 0x10-byte boundary.<br />
|- <br />
| 0x20 || uint32 || EntriesSize || This is the length in bytes of the entries in the index file.<br />
|- <br />
| 0x24 || uint32 || EntriesHash || This should equal the value of pc after calling hashlittle2 on the following EntriesSize bytes of the file with an initial value of 0 for pb and pc.<br />
|}<br />
<br />
====.IDX Entry Structure====<br />
*'''The rest of the file is populated by these normal entries, each 0x12 bytes in size. Structure names were invented by the author of this section because official names were not available.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Dec)<br />
! width="70" | Type<br />
! width="90" | Name<br />
! width="900" | Description<br />
|-<br />
| 00 || char[9] || Key || The first 9 bytes of the key for this entry.<br />
|- <br />
| 09 || uint40* || Offset || Unlike the other little-endian integers in this file, this is a big-endian 5-byte integer. The top 10 bits are the number of the archive (data.%03d), and the bottom 30 bits are the offset in that archive to the file data.<br />
|- <br />
| 14 || uint32 || Size || The length of the file in bytes.<br />
|}<br />
*'''* designates unusual data types. In C#, you can read the Offset by reading a Byte, reading a big-endian UInt32, shifting the byte left 32 bits, and ORing them together. Use a 30-bit mask (0x3fffffff) to get the file offset, and right shift the value 30 bits to get the archive number.<br />
<br />
===.XXX Data Files===<br />
Example file path: INSTALL_DIR\Data\data\data.015<br />
<br />
These files consist of a sequence of headers with corresponding BLTE data.<br />
<br />
Most .xxx archives begin with 16 special index cross-linking files. These files have no data and have encoding keys of XXYYbba1af16c50e1900000000000000, where XX is the index number and YY is the .xxx number. The purpose of these files is unclear.<br />
<br />
*'''The data header.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="200" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[0x10] || BlteHash || Encoding key of the file, in reversed byte order. Note that only as many bytes (final bytes in this reversed order) of this key as are contained in the .idx files (9) must be accurate, and the remaining 7 bytes may be 0s or otherwise altered.<br />
|- <br />
| 0x10 || uint32_t || Size || The size of this header + the following data.<br />
|- <br />
| 0x14 || char[0x02] || Flags?? || Unknown. Mostly 0. Set to 1,0 by Agent.exe on index cross-linking files, possibly indicating data-less metadata files.<br />
|- <br />
| 0x16 || uint32_t || ChecksumA || hashlittle(first 0x16 bytes of the header, 0x3D6BE971)<br />
|- <br />
| 0x1A || uint32_t || ChecksumB || Checksum of the first 0x1A bytes of the header. The exact algorithm seems to vary over time.<br />
|}<br />
<br />
<br />
*'''The BLTE data.'''<br />
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"<br />
! width="80" | Offset (Hex)<br />
! width="200" | Type<br />
! width="150" | Name<br />
! width="900" | Description<br />
|-<br />
| 0x00 || char[Header.Size - 30] || Data || The BLTE file data. See the BLTE section above.<br />
|}<br />
<br />
=Product Specific=<br />
In this section, the game/usage specific parts of CASC are describe. While CASC is a generic format, a lot of stuff is hardcoded, like hash sizes. Other parts are actually left up to the implementation, like root files or download tags.<br />
<br />
==World of Warcraft==<br />
===Root===<br />
struct CASRecord {<br />
char content_key[16]; // MD5 hash of the file's raw data<br />
uint64 name_hash; // Jenkins96 (lookup3) hash of the file's path <br />
};<br />
<br />
enum locale_flags : uint32 {<br />
enUS = 0x2,<br />
koKR = 0x4,<br />
frFR = 0x10,<br />
deDE = 0x20,<br />
zhCN = 0x40,<br />
esES = 0x80,<br />
zhTW = 0x100,<br />
enGB = 0x200,<br />
enCN = 0x400,<br />
enTW = 0x800,<br />
esMX = 0x1000,<br />
ruRU = 0x2000,<br />
ptBR = 0x4000,<br />
itIT = 0x8000,<br />
ptPT = 0x10000,<br />
};<br />
enum content_flags : uint32 {<br />
LowViolence = 0x80,<br />
Bundle = 0x40000000,<br />
NoCompression = 0x80000000,<br />
};<br />
<br />
struct CASBlock {<br />
int32 num_records;<br />
content_flags flags;<br />
locale_flags locale;<br />
int32 fileDataIDDeltas[num_records]; // each block starts with 0, +1 is implicit per entry, so consecutive ids will have delta=0<br />
CASRecord records[num_records];<br />
<br />
int32 file_data_id (size_t index) const<br />
{<br />
return index == 0<br />
? fileDataIDDeltas[index]<br />
: file_data_id (index - 1) + 1 + fileDataIDDeltas[index];<br />
}<br />
};<br />
<br />
while (FTell() < FileSize())<br />
CASBlock blocks;<br />
<br />
===Tags===<br />
Values depend on versions, semantic categories are cross version.<br />
<br />
* '''Platform''': The deployment target, i.e. Windows or OSX<br />
* '''Architecture''': Sub-division of the deployment target, i.e. x86_32 or x86_64<br />
* '''Locale''': The same as in [[Localization]]: Files specific to a single localisation of the game.<br />
* '''Region''': Equivalent to the patch server regions, i.e. us, eu, kr, tw, cn.<br />
* '''Category''': A replacement for the MPQ system to tag low priority downloads: speech, text<br />
* '''Alternate''': A special category for censored content.<br />
<br />
====Version specific values====<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125|max_expansionlevel=6|max_build=6.0.1.18761}}<br />
<br />
Architecture = 1, Locale = 2, Platform = 3<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18764|max_expansionlevel=6|max_build=6.2.2.20426}}<br />
<br />
Architecture = 1, Category = 2, Locale = 3, Platform = 4, Region = 5<br />
<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.2.2.20438|max_expansionlevel=7}}<br />
<br />
Platform = 1, Architecture = 2, Locale = 3, Region = 4, Category = 5<br />
<br />
{{SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26604|note={{Unverified|Actually alternate was just missing above but schlumpf doesn't want to verify since when}}}}<br />
<br />
enum {<br />
platform = 1,<br />
architecture = 2,<br />
locale = 3,<br />
region = 4,<br />
category = 5,<br />
alternate = 0x4000,<br />
};<br />
<br />
=hashpath=<br />
hashpath (string path) → uint32_t<br />
{<br />
string normalized = toupper (path).replace (from: '/', to: '\\')<br />
uint32_t pc = 0, pb = 0;<br />
hashlittle2 (normalized, strlen (normalized), &pc, &pb);<br />
return pc;<br />
}<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=Known_Server-Side_Tables&diff=26080Known Server-Side Tables2018-08-06T23:12:38Z<p>Simca: Capitalized all the obvious table names.</p>
<hr />
<div>There's currently no purpose to keeping this list up to date, but we can do it anyway, just for fun. Obviously, we can't ever get any of these tables unless Blizzard does something very stupid, but it's neat to have a glimpse of how things work behind the scenes.<br />
<br />
* AI_GroupAction<br />
* AI_GroupActionSet<br />
* AI_RandomActions<br />
* AI_TriggerAction<br />
* AI_TriggerActionSet<br />
* AreaShareInfo<br />
* AreaTriggerAction<br />
* AreaXAuras<br />
* AzeriteTierUnlockSet<br />
* Cfg_Battlegrps<br />
* Cfg_Databases<br />
* Cfg_Realms<br />
* Cfg_Sites<br />
* CharacterLoadoutSpell<br />
* CharacterTemplateData<br />
* CharacterTemplates<br />
* CharRestrictionXDeliverable<br />
* CharStartKit<br />
* CharStartQuest<br />
* ConsoleScripts (previously Client-Side)<br />
* Conversation<br />
* CreatureLabel<br />
* CreatureSpellData<br />
* DesignerContextForBI<br />
* DestructibleHitPoint<br />
* FactionAdjust<br />
* GameEvents<br />
* GameMilestoneHistory<br />
* Gossip<br />
* InternalRecord<br />
* ItemBonusList<br />
* LevelBandedRewardEntry<br />
* MapTransferLocations<br />
* NamedPoint<br />
* OutfitAppearance<br />
* PathEdge<br />
* PetScalingNode<br />
* PointsModifier<br />
* PointsModifierSet<br />
* ProceduralObject<br />
* ProceduralObjectInteractible<br />
* ProceduralResourceBudget<br />
* ProceduralResourceBudgetList<br />
* ProgressiveEvent<br />
* proresbudgetlistxpromapspaconf<br />
* PvpSeasonReward<br />
* QuestAreaTrigger<br />
* QuestCompletionNpc<br />
* Questgiver<br />
* QuestObjectiveCompletionEffect<br />
* QuestObjectiveXEffect<br />
* RandomText<br />
* ScalingTreasureDrop<br />
* SceneScriptOld<br />
* SceneScriptOldText<br />
* SpawnEvent<br />
* SpawnGroup<br />
* SpawnLoc<br />
* SpawnObject<br />
* SpawnPathNode<br />
* SpawnRegion<br />
* SpawnSquad<br />
* SpawnTracking<br />
* SpawnTrackingAction<br />
* SpawnTrackingState<br />
* SpellClass<br />
* SpellHealPrediction<br />
* SpellSandboxScaling<br />
* SpellScript<br />
* SpellScriptPackage<br />
* SpellScriptText<br />
* SpellTargetFilter<br />
* SpellTargetFilterRule<br />
* SpellVisualMissileSet<br />
* SpellXDifficulty<br />
* StringID<br />
* TrainerList<br />
* TrainerServicesList<br />
* TrainerServicesV2<br />
* TrainerV2<br />
* Treasure<br />
* TreasureItem<br />
* TreasurePicker<br />
* VendorItem<br />
* VendorList<br />
* VendorListMap<br />
* WorldStateAction<br />
* WorldStateActionSet</div>Simcahttps://wowdev.wiki/index.php?title=WDB&diff=26066WDB2018-08-06T03:59:56Z<p>Simca: /* Header */ Uhh, headers are 20 bytes again so there must have been some point where it switched back.</p>
<hr />
<div>The WDB files are found inside the WDB folder. The client caches data it receives from the server. This is mainly done to reduce network traffic.<br />
<br />
Not all WDB caches are saved to disk. This is defined in the client by the <tt>DBCache</tt>'s constructor which contains a persistent parameter.<br />
<br />
== Header ==<br />
<br />
*WDB files < 1.6: Header length is 16 bytes<br />
*WDB files >=1.6: Header length is 20 bytes (Verified till 1.9.4)<br />
*WDB files >=3.0.8-9506: Header length is 24 bytes <br />
*WDB files ??? (Legion and up at the very least): Header length is 20 bytes<br />
<br />
'''Offset Type Description'''<br />
0x00 char[4] Identifier, depends on the WDB file, will be explained later<br />
0x04 uint32 Client Version - Version of the client (lo -> hi encoding)<br />
0x08 char[4] Client Local - The Locale of the client<br />
0x0C ???? 4 bytes unknown<br />
0x10 ???? 4 bytes unknown<br />
0x15 ???? 4 bytes unknown (Beginning with version 1.6 and later)<br />
0x18 ???? 4 bytes unknown (Beginning with version 3.0.8-9506 )<br />
<br />
'''Column Field Type Notes''' <br />
1 Signature String 4-byte string identifying the file (reversed!) <br />
2 Client Version Integer 4-byte integer identifying the client version <br />
3 Language String 4-byte string identifying the language (reversed!) {{Template:Sandbox/VersionRange|min_expansionlevel=1|min_build=1.6.0}}<br />
4 Record Size Integer Something to do with row length, there is no consistent way of retrieving it as of yet.<br />
In Alpha this is row length with strings treated as 4 bytes.<br />
5 Record Version Integer A manually updated versioning field - except for WoWCache.wdb which is read from <code>WardenCachedModule::Version</code><br />
As of 'Cache Version' this is only validated by the client for WoWCache.wdb<br />
6 Cache Version Integer 4-bytes A packet based versioning field set via SMSG_CLIENTCACHE_VERSION {{Template:Sandbox/VersionRange|min_expansionlevel=3|min_build=3.0.8}}<br />
<br />
== WDB files ==<br />
<br />
'''File Signature''' <br />
[[ArenaTeamCache.wdb]] WATM {{Template:Sandbox/VersionRange|min_expansionlevel=3|max_expansionlevel=4}}<br />
[[BattlePetNameCache.wdb]] WBPN {{Template:Unverified|{{Template:Sandbox/VersionRange|min_expansionlevel=5|max_expansionlevel=6}} Not seen > 6.2.4}}<br />
[[CreatureCache.wdb]] WMOB <br />
[[DanceCache.wdb]] WDAN {{Template:Sandbox/VersionRange|min_expansionlevel=3|max_expansionlevel=4}}<br />
[[GameObjectCache.wdb]] WGOB <br />
[[GuildStatsCache.wdb]] WGLD<br />
[[ItemCache.wdb]] WIDB {{Template:Sandbox/VersionRange|max_expansionlevel=3}}<br />
[[ItemNameCache.wdb]] WNDB {{Template:Sandbox/VersionRange|max_expansionlevel=3}}<br />
[[ItemTextCache.wdb]] WITX <br />
[[NameCache.wdb]] WNAM<br />
[[NPCCache.wdb]] WNPC <br />
[[PageTextCache.wdb]] WPTX <br />
[[PetitionCache.wdb]] WPTN<br />
[[PetNameCache.wdb]] WPNM<br />
[[QuestCache.wdb]] WQST <br />
[[RealmCache.wdb]] WRLM {{Template:Unverified|{{Template:Sandbox/VersionRange|min_expansionlevel=5|max_expansionlevel=6|max_build=6.0.1.18179}} Not seen ≥ 6.2.3}}<br />
[[WOWCache.wdb]] WRDN<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=QuestCache.wdb&diff=26065QuestCache.wdb2018-08-05T23:56:42Z<p>Simca: Added full structure with comments for version 8.0.1.27075.</p>
<hr />
<div>The [[QuestCache.wdb]] holds most of the information for quest you have seen in game. <br />
*Signature: WQST <br />
<br />
==Structure==<br />
<br />
===Version 8.0.1.27075===<br />
typedef struct {<br />
int ID;<br />
int Length;<br />
if (Length > 0)<br />
{<br />
int QuestID;<br />
int QuestType;<br />
int QuestLevel; // Recommended level to complete the quest<br />
int Quest_UNK_27075; // Unknown. Almost always 0, but sometimes 1, 3, or 5.<br />
int QuestMaxScalingLevel; // Maximum level that the quest reward and difficulty will scale to<br />
int QuestPackageID; // FK to QuestPackageItem.db2<br />
int QuestMinLevel; // Required level to pick up the quest<br />
int QuestSortID; // When QuestSortID is greater than 0, FK to AreaTable.db2; otherwise, FK to QuestSort.db2<br />
int QuestInfoID; // FK to QuestInfo.db2<br />
int SuggestedGroupNum;<br />
int RewardNextQuest; // Next QuestID in the chain; sometimes blank when it shouldn't be because chains are often not linear and require multiple quests to continue at certain points<br />
int RewardXPDifficulty; // The column of QuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardMoney; // Precomputed final money value based on player level at the time of caching; not very useful unless you can ensure consistent player levels<br />
int RewardMoneyDifficulty; // The column of QuestMoneyReward to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardMoneyMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardBonusMoney; // Bonus money rewarded if completed at max level<br />
int RewardDisplaySpell[3];<br />
int RewardSpell;<br />
int RewardHonor; // Amount of honor rewarded by the quest<br />
float RewardHonorKill; // Multiplier applied to honor rewarded by the quest (or to kills during it? unknown exactly)<br />
int RewardArtifactXPDifficulty; // The column of ArtifactQuestXp to use. Use Player Level as the ID if it is within the range of QuestMinLevel to QuestMaxScalingLevel, otherwise QuestLevel.<br />
float RewardArtifactXPMultiplier; // Multiplier applied to the value retrieved from the field above<br />
int RewardArtifactCategoryID;<br />
int ProvidedItem; // Item linked to the quest, usually destroying it will force the quest to abandon<br />
uint Flags;<br />
uint Flags2;<br />
uint Flags3; // Only 95% sure about this, can confirm for sure when they add new flags in the future (values right now are always 0 or 2)<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} RewardFixedItems[4]; // Rewards always given<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
} ItemDrop[4]; // Rewards forced on the player outside the quest dialog(?); rarely used now<br />
struct {<br />
int ItemID;<br />
int Quantity;<br />
int DisplayID;<br />
} RewardChoiceItems[6]; // Reward choices - player can pick one<br />
int POIContinent; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIx; // Very rarely used these days as POIs are provided by a different system usually<br />
float POIy; // Very rarely used these days as POIs are provided by a different system usually<br />
int POIPriority; // Very rarely used these days as POIs are provided by a different system usually<br />
int RewardTitle; // Very rarely used (mainly for TBC Isle of Quel'Danas money title); Blizzard prefers to reward titles from quests via RewardSpell these days instead<br />
int RewardArenaPoints; // Not used since TBC<br />
int RewardSkillLineID;<br />
int RewardNumSkillUps;<br />
int PortraitGiverDisplayID;<br />
int BFA_UnkDisplayID; // Purpose of the field is unknown and it is used significantly less often than the fields around it, but it appears to also be a CreatureDisplayInfoID<br />
int PortraitTurnInDisplayID;<br />
struct {<br />
int FactionID;<br />
int FactionValue; // The column of QuestFactionReward to use<br />
int FactionOverride; // An override used when Blizzard wants to reward a non-standard amount of reputation, like '1500' or '3000'<br />
int FactionGainMaxRank; // The reputation threshold where you stop being able to apply the reputation reward. For example, '7' means that the rep counts all the way through Exalted.<br />
} RewardFaction[5];<br />
int RewardFactionFlags;<br />
struct {<br />
int CurrencyID;<br />
int Quantity;<br />
} RewardCurrency[4];<br />
int AcceptedSoundKitID;<br />
int CompleteSoundKitID;<br />
int AreaGroupID;<br />
int TimeAllowed;<br />
int NumObjectives;<br />
uint64 RaceFlags;<br />
uint QuestRewardID;<br />
uint ExpansionID;<br />
<br />
BitfieldDisablePadding();<br />
BitfieldLeftToRight();<br />
uint LogTitleLength : 9;<br />
uint LogDescriptionLength : 12;<br />
uint QuestDescriptionLength : 12;<br />
uint AreaDescriptionLength : 9;<br />
uint PortraitGiverTextLength : 10;<br />
uint PortraitGiverNameLength : 8;<br />
uint PortraitTurnInTextLength : 10;<br />
uint PortraitTurnInNameLength : 8;<br />
uint QuestCompletionLogLength : 11;<br />
BitfieldEnablePadding();<br />
<br />
struct<br />
{<br />
int ID;<br />
ubyte Type;<br />
byte StorageIndex;<br />
int ObjectID;<br />
int Amount;<br />
int Flags;<br />
int Flags2;<br />
float PercentAmount;<br />
int NumVisualEffects;<br />
int VisualEffects[NumVisualEffects];<br />
ubyte DescriptionLength;<br />
char Description[DescriptionLength];<br />
} Objectives[NumObjectives] <optimize=false>;<br />
<br />
char LogTitle[LogTitleLength];<br />
char LogDescription[LogDescriptionLength];<br />
char QuestDescription[QuestDescriptionLength];<br />
char AreaDescription[AreaDescriptionLength];<br />
char PortraitGiverText[PortraitGiverTextLength];<br />
char PortraitGiverName[PortraitGiverNameLength];<br />
char PortraitTurnInText[PortraitTurnInTextLength];<br />
char PortraitTurnInName[PortraitTurnInNameLength];<br />
char QuestCompletionLog[QuestCompletionLogLength];<br />
}<br />
} QuestCacheRow;<br />
<br />
===Version 3.3.5?===<br />
'''Column Field Type Notes''' <br />
1 questID Integer <br />
2 entryLength Integer WDB Files <br />
3 DuplicatedQuestID Integer This is a duplicate of the quest ID <br />
4 QuestType Integer 2 or 0, 0 appears to be when it's a quick click-through quest like repeatable quests (but not exclusively) <br />
5 QuestLevel Integer The level of the quest. This is not the level at which the quest is atainable at. <br />
6 areaID/sortID Integer AreaTable.dbc if negative the value points to: [[QuestSort.dbc]] <br />
7 infoID Integer [[QuestInfo.dbc]] <br />
8 SuggestedPlayers Integer <br />
9 FactionID Integer [[Faction.dbc]] <br />
10 FactionAmount Integer e.g. 3000 <br />
11 Unknown Integer Always 0? <br />
12 Unknown Integer Always 0? <br />
13 nextQuestID Integer The quest that follows this quest <br />
14 coins Integer Value is in Copper - Coins rewarded on completion <br />
15 SubExp70 Integer Value is in Copper - Coins rewarded on lvl 70 instead of experience (guess) <br />
16 RewardSpellID Integer [[Spell.dbc]]: Spell or ability that is added to players spellbook upon completion. This is probably an argument passed into EffectOnPlayer somehow, as this is sometimes a duplicated value as the EffectOnPlayer field, and it's not a spell that the player learns. (Such as the Razorhide quest turnin). <br />
17 EffectOnPlayer Integer [[Spell.dbc]]: Spell/effect cast on player when completing. <br />
18 startingItemID Integer [[ItemCache.wdb]] The item that you are given when you start the quest, such as a package to deliver. <br />
19 QuestFlags BitMask<br />
20 givenItem1 Integer [[ItemCache.wdb]] <br />
21 givenItem1Amount Integer <br />
22 givenItem2 Integer [[ItemCache.wdb]]<br />
23 givenItem2Amount Integer <br />
24 givenItem3 Integer [[ItemCache.wdb]]<br />
25 givenItem3Amount Integer <br />
26 givenItem4 Integer [[ItemCache.wdb]] <br />
27 givenItem4Amount Integer <br />
28 choiceItem1 Integer [[ItemCache.wdb]] <br />
29 choiceItem1Amount Integer <br />
30 choiceItem2 Integer [[ItemCache.wdb]] <br />
31 choiceItem2Amount Integer <br />
32 choiceItem3 Integer [[ItemCache.wdb]] <br />
33 choiceItem3Amount Integer <br />
34 choiceItem4 Integer [[ItemCache.wdb]] <br />
35 choiceItem4Amount Integer <br />
36 choiceItem5 Integer [[ItemCache.wdb]] <br />
37 choiceItem5Amount Integer <br />
38 choiceItem6 Integer [[ItemCache.wdb]] <br />
39 choiceItem6Amount Integer <br />
40 Unknown Integer <br />
41 Unknown Integer <br />
42 Unknown Integer <br />
43 Unknown Integer <br />
44 name String <br />
45 description String <br />
46 details String <br />
47 subdescription String <br />
48 killCreature1 Integer [[CreatureCache.wdb]] <br />
49 killCreature1Amount Integer <br />
50 collectItem1 Integer [[ItemCache.wdb]] <br />
51 collectItem1Amount Integer <br />
52 killCreature2 Integer [[CreatureCache.wdb]] <br />
53 killCreature2Amount Integer <br />
54 collectItem2 Integer [[ItemCache.wdb]] <br />
55 collectItem2Amount Integer <br />
56 killCreature3 Integer [[CreatureCache.wdb]]<br />
57 killCreature3Amount Integer <br />
58 collectItem3 Integer [[ItemCache.wdb]] <br />
59 collectItem3Amount Integer <br />
60 killCreature4 Integer [[CreatureCache.wdb]] <br />
61 killCreature4Amount Integer <br />
62 collectItem4 Integer [[ItemCache.wdb]] <br />
63 collectItem4Amount Integer <br />
64 Objective1 String If not killCreature1 or collectItem1 this is the objective if set <br />
65 Objective2 String If not killCreature2 or collectItem2 this is the objective if set <br />
66 Objective3 String If not killCreature3 or collectItem3 this is the objective if set <br />
67 Objective4 String If not killCreature4 or collectItem4 this is the objective if set <br />
<br />
===Version 0.5.3.3368===<br />
For {{Template:Sandbox/PrettyVersion|expansionlevel=0|build=0.5.3.3368}} client version<br />
'''Column Field Type''' <br />
1 QuestId Integer<br />
2 QuestType Integer<br />
3 QuestLevel Integer<br />
4 QuestSortID Integer<br />
5 QuestInfoID Integer<br />
6 RewardNextQuest Integer<br />
7 RewardMoney Integer<br />
8 StartItem Integer<br />
9 RewardItems Integer[4]<br />
10 RewardAmount Integer[4]<br />
11 RewardChoiceItems Integer[6]<br />
12 RewardChoiceAmount Integer[6]<br />
13 POIContinent Integer<br />
14 POIx Float<br />
15 POIy Float<br />
16 POIPriority Integer<br />
17 LogTitle Char[128]<br />
18 LogDescription Char[1024]<br />
19 QuestDescription Char[1024]<br />
20 AreaDescription Char[128]<br />
21 MonsterToKill Integer[4]<br />
22 MonsterToKillQuantity Integer[4]<br />
23 ItemToGet Integer[4]<br />
24 ItemToGetQuantity Integer[4]<br />
25 GetDescription Char[4][64]<br />
<br />
==Quest Flags==<br />
1 (1) Deliver <br />
2 (2) Kill <br />
3 (4) Speak To <br />
4 (8) Repeatable? Q:Shareable?<br />
5 (16) Exploration <br />
6 (32) Timed Quest <br />
7 (64) Raid Quest Tagged as "(Raid)" (to verify)<br />
8 (128) Reputation <br />
9 (256) Unknown <br />
10 (512) Unknown <br />
11 (1024) Unknown <br />
12 (2048) Unknown <br />
13 (4096) Daily This bit is set if it's a Daily Quest<br />
<br />
[[Category:WDB]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25529DB/ItemDisplayInfo2018-04-03T20:25:30Z<p>Simca: /* Geoset Group Field Meaning */ Fix.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {geosetGroup[0] = 2700**, geosetGroup[1] = 2101 }<br />
Shoulder: {geosetGroup[0] = 2601}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Waist: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Wrist: {}<br />
Gloves: {geosetGroup[0] = 401}<br />
Cape: {geosetGroup[0] = 1501}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Weapon: {}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
21: {0: No Geoset; 1: Show Head}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
26: {0: No Geoset; 1: Show Shoulders}<br />
27: {1: Default (No Geoset); 2: Helm 1}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
<nowiki />** This is not a typo. If you are not wearing a helmet, you get geoset 2701. If you are wearing helmet and geosetGroup[0] for your helmet is 0, you get 2702. If you are wearing boots and your geosetGroup[0] is non-zero, you get 2700 + geosetGroup[0]. I'm not sure exactly how a value of '-1' is applied in this scenario, but some of the ItemDisplayInfo entries have this.<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25528DB/ItemDisplayInfo2018-04-03T20:25:11Z<p>Simca: /* Geoset Group Field Meaning */ Fixed this better after a discussion in IRC.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {geosetGroup[0] = 2700**, geosetGroup[1] = 2101 }<br />
Shoulder: {geosetGroup[0] = 2601}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Waist: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Wrist: {}<br />
Gloves: {geosetGroup[0] = 401}<br />
Cape: {geosetGroup[0] = 1501}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Weapon: {}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
21: {0: No Geoset; 1: Show Head}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
26: {0: No Geoset; 1: Show Shoulders}<br />
27: {1: Default (No Geoset); 2: Helm 1}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
<nowiki />* This is not a typo. If you are not wearing a helmet, you get geoset 2701. If you are wearing helmet and geosetGroup[0] for your helmet is 0, you get 2702. If you are wearing boots and your geosetGroup[0] is non-zero, you get 2700 + geosetGroup[0]. I'm not sure exactly how a value of '-1' is applied in this scenario, but some of the ItemDisplayInfo entries have this.<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25527DB/ItemDisplayInfo2018-04-03T20:10:27Z<p>Simca: Unfixed this as it was wrong apparently.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {geosetGroup[0] = 2701, geosetGroup[1] = 2101 }<br />
Shoulder: {geosetGroup[0] = 2601}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Waist: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Wrist: {}<br />
Gloves: {geosetGroup[0] = 401}<br />
Cape: {geosetGroup[0] = 1501}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Weapon: {}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
21: {0: No Geoset; 1: Show Head}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
26: {0: No Geoset; 1: Show Shoulders}<br />
27: {0: Default (No Geoset); 1: Helm 1}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25526DB/ItemDisplayInfo2018-04-03T19:21:06Z<p>Simca: /* Geoset Group Field Meaning */ Fixed 27xx series numbering.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {geosetGroup[0] = 2701, geosetGroup[1] = 2101 }<br />
Shoulder: {geosetGroup[0] = 2601}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Waist: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Wrist: {}<br />
Gloves: {geosetGroup[0] = 401}<br />
Cape: {geosetGroup[0] = 1501}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Weapon: {}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
21: {0: No Geoset; 1: Show Head}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
26: {0: No Geoset; 1: Show Shoulders}<br />
27: {1: Default (No Geoset); 2: Helm 1}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25522DB/ItemDisplayInfo2018-04-02T22:29:45Z<p>Simca: /* 8.0.1.25902 */ Spaces in template fixed.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {geosetGroup[0] = 2700, geosetGroup[1] = 2101 }<br />
Shoulder: {geosetGroup[0] = 2601}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Waist: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Wrist: {}<br />
Gloves: {geosetGroup[0] = 401}<br />
Cape: {geosetGroup[0] = 1501}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Weapon: {}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
21: {0: No Geoset; 1: Show Head}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
26: {0: No Geoset; 1: Show Shoulders}<br />
27: {0: Default (No Geoset); 1: Helm 1}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo/GeosRenderPrep&diff=25516DB/ItemDisplayInfo/GeosRenderPrep2018-03-30T18:13:53Z<p>Simca: Minor update for 8.0.1 code from TOM_RUS.</p>
<hr />
<div> // GeosetRenderPrep.cpp : Defines the entry point for the console application.<br />
//<br />
<br />
#include "stdafx.h"<br />
<br />
int main()<br />
{<br />
return 0;<br />
}<br />
<br />
struct CCharacterComponent<br />
{<br />
uint32_t raceID;<br />
uint32_t classID;<br />
uint32_t skinIndex;<br />
uint32_t genderID;<br />
uint32_t faceIndex;<br />
void *modelPtr;<br />
uint32_t submeshIds[29];<br />
uint32_t itemids[10];<br />
uint32_t gap2[520];<br />
uint64_t shirtRelated[6];<br />
uint32_t flags;<br />
<br />
static void GeosRenderPrep(CCharacterComponent *pCharComponent);<br />
};<br />
<br />
struct CharSectionsRec<br />
{<br />
uint32_t Flags;<br />
};<br />
<br />
struct CM2Model<br />
{<br />
static void SetGeometryVisible(void *model, uint32_t start, uint32_t end, BOOL visible) {}<br />
};<br />
<br />
struct ItemDisplayInfoRec<br />
{<br />
uint32_t Flags;<br />
uint32_t GeosetGroup[4];<br />
};<br />
<br />
CharSectionsRec *ComponentGetSectionsRecord(uint32_t raceID, uint32_t genderID, uint32_t sectionID, uint32_t faceIndex, uint32_t skinIndex, uint32_t unk1)<br />
{<br />
return nullptr;<br />
}<br />
<br />
uint32_t GetConditionalGeoset(uint32_t raceID, uint32_t geosetGroup)<br />
{<br />
return 0;<br />
}<br />
<br />
ItemDisplayInfoRec *GetItemDisplayInfoRec(uint32_t itemID)<br />
{<br />
return nullptr;<br />
}<br />
<br />
void CCharacterComponent::GeosRenderPrep(CCharacterComponent *pCharComponent)<br />
{<br />
BOOL eyeGlowFlag = 0;<br />
<br />
if (pCharComponent->classID == 6) // DK<br />
{<br />
eyeGlowFlag = 1;<br />
}<br />
else<br />
{<br />
// section Face<br />
auto pFaceSection = ComponentGetSectionsRecord(pCharComponent->raceID, pCharComponent->genderID, 1, pCharComponent->faceIndex, pCharComponent->skinIndex, 0);<br />
<br />
if (pFaceSection)<br />
eyeGlowFlag = pFaceSection->Flags & 4;// flags & 0x4<br />
else<br />
eyeGlowFlag = 0;<br />
}<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 0, 3000, 0); // disable all<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 0, 0, 1); // enable skin<br />
<br />
uint32_t i = 0;<br />
do<br />
{<br />
if (eyeGlowFlag && i == 17)<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1703, 1703, 1);<br />
else<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pCharComponent->submeshIds[i], pCharComponent->submeshIds[i], 1);<br />
++i;<br />
} while (i != 29);<br />
<br />
// Tail<br />
BOOL tailFlag = 0;<br />
// section Skin<br />
auto pSkinSection = ComponentGetSectionsRecord(pCharComponent->raceID, pCharComponent->genderID, 0, 0, pCharComponent->skinIndex, 0);<br />
if (pSkinSection)<br />
tailFlag = pSkinSection->Flags & 0x100; // flags & 0x100<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1900, 1999, 0);<br />
<br />
uint32_t tailGeoset = 1;<br />
if (!tailFlag)<br />
tailGeoset = GetConditionalGeoset(pCharComponent->raceID, 19);<br />
if (tailGeoset <= 0)<br />
tailGeoset = 1;<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1900 + tailGeoset, 1900 + tailGeoset, 1);<br />
<br />
// Item ID's<br />
auto shirtID = pCharComponent->itemids[0];<br />
auto chestID = pCharComponent->itemids[1];<br />
auto beltID = pCharComponent->itemids[2];<br />
auto pantsID = pCharComponent->itemids[3];<br />
auto bootsID = pCharComponent->itemids[4];<br />
// 5 wrist?<br />
auto glovesID = pCharComponent->itemids[6];<br />
auto tabardID = pCharComponent->itemids[7];<br />
auto cloakID = pCharComponent->itemids[8];<br />
<br />
// Gloves<br />
if (glovesID)<br />
{<br />
auto pGloves = GetItemDisplayInfoRec(glovesID);<br />
if (pGloves && pGloves->GeosetGroup[0])<br />
{<br />
auto glovesGeoset = 401 + pGloves->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 401, 499, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, glovesGeoset, glovesGeoset, 1);<br />
}<br />
}<br />
else<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[0])<br />
{<br />
auto chesetGeoset = 801 + pChest->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chesetGeoset, chesetGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// unknown shirt pre-check<br />
// possibly !textureArmUpper && !textureArmLower && !textureTorsoUpper && !textureTorsoLower && !textureLegUpper && !textureLegLower ?<br />
if (!pCharComponent->shirtRelated[0] && !pCharComponent->shirtRelated[1] && !pCharComponent->shirtRelated[2] && !pCharComponent->shirtRelated[3] && !pCharComponent->shirtRelated[4] && !pCharComponent->shirtRelated[5])<br />
{<br />
// Shirt<br />
if (shirtID)<br />
{<br />
auto pShirt = GetItemDisplayInfoRec(shirtID);<br />
if (pShirt && pShirt->GeosetGroup[0])<br />
{<br />
auto shirtGeoset = 801 + pShirt->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, shirtGeoset, shirtGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Tabard<br />
auto pTabard = GetItemDisplayInfoRec(tabardID);<br />
if (pTabard)<br />
{<br />
if (!(pTabard->Flags & 0x100000))<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2200, 2299, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2202, 2202, 1);<br />
}<br />
}<br />
else<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[3])<br />
{<br />
auto chestGeoset = 2201 + pChest->GeosetGroup[3];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2200, 2299, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Belt<br />
BOOL hasBulkyBeltFlag = 0;<br />
auto pBelt = GetItemDisplayInfoRec(beltID);<br />
if (pBelt)<br />
hasBulkyBeltFlag = pBelt->Flags & 0x200;<br />
<br />
BOOL dressPants, dressChestpiece;<br />
<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[2])<br />
{<br />
if (pChest->GeosetGroup[2])<br />
{<br />
dressPants = 0;<br />
dressChestpiece = 1;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 902, 999, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1100, 1199, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
auto chestGeoset = 1301 + pChest->GeosetGroup[2];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
}<br />
else if (pantsID)// Pants<br />
{<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
if (pPants && pPants->GeosetGroup[2] && !HIWORD(pCharComponent->flags))<br />
{<br />
dressPants = 1;<br />
dressChestpiece = 0;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 902, 999, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1100, 1199, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
auto pantsGeoset = 1301 + pPants->GeosetGroup[2];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
else<br />
{<br />
dressPants = 0;<br />
dressChestpiece = 0;<br />
// Boots<br />
auto pBoots = GetItemDisplayInfoRec(bootsID);<br />
if (pBoots && pBoots->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 901, 901, 1);<br />
auto bootsGeoset = 501 + pBoots->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, bootsGeoset, bootsGeoset, 1);<br />
}<br />
else<br />
{<br />
// Pants<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
uint32_t pantsGeoset;<br />
if (pPants && pPants->GeosetGroup[1] && !HIWORD(pCharComponent->flags))<br />
{<br />
pantsGeoset = 901 + pPants->GeosetGroup[1];<br />
}<br />
else<br />
{<br />
pantsGeoset = 901;<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
<br />
// Boots<br />
BOOL bootsFlag = 0;<br />
auto pBoots = GetItemDisplayInfoRec(bootsID);<br />
if (pBoots)<br />
bootsFlag = (pBoots->Flags & 0x100000) == 0;<br />
<br />
uint32_t bootsGeoset;<br />
if (pBoots && pBoots->GeosetGroup[1])<br />
{<br />
bootsGeoset = 2000 + pBoots->GeosetGroup[1];<br />
}<br />
else<br />
{<br />
if (bootsFlag)<br />
{<br />
bootsGeoset = 2002;<br />
}<br />
else<br />
{<br />
bootsGeoset = 2001;<br />
}<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, bootsGeoset, bootsGeoset, 1);<br />
<br />
// Tabard<br />
BOOL showsTabard = 0;<br />
BOOL hasDress = dressChestpiece | dressPants;<br />
if (!hasDress && tabardID != 0 && pTabard && pTabard->GeosetGroup[0])<br />
{<br />
showsTabard = 0;<br />
uint32_t tabardGeoset;<br />
if (hasBulkyBeltFlag)<br />
{<br />
showsTabard = 1;<br />
tabardGeoset = 1203;<br />
}<br />
else<br />
{<br />
tabardGeoset = 1201 + pTabard->GeosetGroup[0];<br />
showsTabard = 1;<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, tabardGeoset, tabardGeoset, 1);<br />
}<br />
else if (!(pCharComponent->flags & 0x10))<br />
{<br />
// nop<br />
}<br />
else<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1201, 1201, 1);<br />
<br />
if (!hasDress)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1202, 1202, 1);<br />
showsTabard = 1;<br />
}<br />
}<br />
<br />
if (!(dressChestpiece | showsTabard))<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[1])<br />
{<br />
auto chestGeoset = 1001 + pChest->GeosetGroup[1];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
else if (shirtID)<br />
{<br />
auto pShirt = GetItemDisplayInfoRec(shirtID);<br />
if (pShirt && pShirt->GeosetGroup[1])<br />
{<br />
auto shirtGeoset = 1001 + pShirt->GeosetGroup[1];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, shirtGeoset, shirtGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
if (!dressChestpiece && !HIWORD(pCharComponent->flags))<br />
{<br />
if (pantsID)<br />
{<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
if (pPants && pPants->GeosetGroup[0])<br />
{<br />
auto geosetGroup = pPants->GeosetGroup[0];<br />
auto pantsGeoset = 1101 + geosetGroup;<br />
if (geosetGroup > 2)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
else if (!showsTabard)<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Cloak<br />
if (cloakID)<br />
{<br />
auto pCloak = GetItemDisplayInfoRec(cloakID);<br />
if (pCloak && pCloak->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1500, 1599, 0);<br />
auto cloakGeoset = 1501 + pCloak->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, cloakGeoset, cloakGeoset, 1);<br />
}<br />
}<br />
<br />
// Belt<br />
if (beltID)<br />
{<br />
auto pBelt = GetItemDisplayInfoRec(beltID);<br />
if (pBelt && pBelt->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1800, 1899, 0);<br />
auto beltGeoset = 1801 + pBelt->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, beltGeoset, beltGeoset, 1);<br />
}<br />
}<br />
<br />
BOOL unkBool;<br />
<br />
if (pCharComponent->flags & 8)<br />
{<br />
unkBool = !pantsID && !dressChestpiece && !dressPants && !showsTabard && !tailFlag && !hasBulkyBeltFlag;<br />
//unkBool = ((pantsID | showsTabard | hasDress) == 0) & ((hasBulkyBeltFlag | tailFlag) ^ 1);<br />
if (unkBool)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1401, 1401, 1);<br />
}<br />
else<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1400, 1400, 0);<br />
}<br />
}<br />
else<br />
{<br />
unkBool = 0;<br />
}<br />
<br />
// DH hands<br />
uint32_t v86 = GetConditionalGeoset(pCharComponent->raceID, 23);<br />
if (v86 <= 0)<br />
v86 = 1;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2300 + v86, 2300 + v86, 1);<br />
<br />
//sub_1401BCDC0(pCharComponent, unkBool);<br />
//auto v132 = sub_1401C53B0(pCharComponent->raceID, pCharComponent->genderID);<br />
//auto v133 = sub_14023D380(pCharComponent->raceID, pCharComponent->genderID, pCharComponent->faceIndex, v132 != 0);<br />
//uint32_t v2 = 0;<br />
//if (v133)<br />
// v2 = *v133;<br />
//sub_1412EFA90(pCharComponent->modelPtr, v2);<br />
//sub_1412EE6B0(pCharComponent->modelPtr);<br />
//LOBYTE(pCharComponent->flags) &= 0xFDu;<br />
}</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo/GeosRenderPrep&diff=25515DB/ItemDisplayInfo/GeosRenderPrep2018-03-30T17:40:55Z<p>Simca: Updated code for 8.0.1. It's also more clear.</p>
<hr />
<div> // GeosetRenderPrep.cpp : Defines the entry point for the console application.<br />
//<br />
<br />
#include "stdafx.h"<br />
<br />
int main()<br />
{<br />
return 0;<br />
}<br />
<br />
struct CCharacterComponent<br />
{<br />
uint32_t raceID;<br />
uint32_t classID;<br />
uint32_t skinIndex;<br />
uint32_t genderID;<br />
uint32_t faceIndex;<br />
void *modelPtr;<br />
uint32_t submeshIds[29];<br />
uint32_t itemids[10];<br />
uint32_t gap2[520];<br />
uint64_t shirtRelated[6];<br />
uint32_t flags;<br />
<br />
static void GeosRenderPrep(CCharacterComponent *pCharComponent);<br />
};<br />
<br />
struct CharSectionsRec<br />
{<br />
uint32_t Flags;<br />
};<br />
<br />
struct CM2Model<br />
{<br />
static void SetGeometryVisible(void *model, uint32_t start, uint32_t end, BOOL visible) {}<br />
};<br />
<br />
struct ItemDisplayInfoRec<br />
{<br />
uint32_t Flags;<br />
uint32_t GeosetGroup[4];<br />
};<br />
<br />
CharSectionsRec *ComponentGetSectionsRecord(uint32_t raceID, uint32_t genderID, uint32_t sectionID, uint32_t faceIndex, uint32_t skinIndex, uint32_t unk1)<br />
{<br />
return nullptr;<br />
}<br />
<br />
uint32_t GetConditionalGeoset(uint32_t raceID, uint32_t geosetGroup)<br />
{<br />
return 0;<br />
}<br />
<br />
ItemDisplayInfoRec *GetItemDisplayInfoRec(uint32_t itemID)<br />
{<br />
return nullptr;<br />
}<br />
<br />
void CCharacterComponent::GeosRenderPrep(CCharacterComponent *pCharComponent)<br />
{<br />
BOOL eyeGlowFlag;<br />
<br />
if (pCharComponent->classID == 6) // DK<br />
{<br />
eyeGlowFlag = 1;<br />
}<br />
else<br />
{<br />
// section Face<br />
auto pFaceSection = ComponentGetSectionsRecord(pCharComponent->raceID, pCharComponent->genderID, 1, pCharComponent->faceIndex, pCharComponent->skinIndex, 0);<br />
<br />
if (pFaceSection)<br />
eyeGlowFlag = pFaceSection->Flags & 4;// flags & 0x4<br />
else<br />
eyeGlowFlag = 0;<br />
}<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 0, 3000, 0); // disable all<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 0, 0, 1); // enable skin<br />
<br />
uint32_t i = 0;<br />
do<br />
{<br />
if (eyeGlowFlag && i == 17)<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1703, 1703, 1);<br />
else<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pCharComponent->submeshIds[i], pCharComponent->submeshIds[i], 1);<br />
++i;<br />
} while (i != 29);<br />
<br />
// Tail<br />
BOOL tailFlag = 0;<br />
// section Skin<br />
auto pSkinSection = ComponentGetSectionsRecord(pCharComponent->raceID, pCharComponent->genderID, 0, 0, pCharComponent->skinIndex, 0);<br />
if (pSkinSection)<br />
tailFlag = pSkinSection->Flags & 0x100; // flags & 0x100<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1900, 1999, 0);<br />
<br />
uint32_t tailGeoset = 1;<br />
if (!tailFlag)<br />
tailGeoset = GetConditionalGeoset(pCharComponent->raceID, 19);<br />
if (tailGeoset <= 0)<br />
tailGeoset = 1;<br />
<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1900 + tailGeoset, 1900 + tailGeoset, 1);<br />
<br />
// Item ID's<br />
auto shirtID = pCharComponent->itemids[0];<br />
auto chestID = pCharComponent->itemids[1];<br />
auto beltID = pCharComponent->itemids[2];<br />
auto pantsID = pCharComponent->itemids[3];<br />
auto bootsID = pCharComponent->itemids[4];<br />
// 5?<br />
auto glovesID = pCharComponent->itemids[6];<br />
auto tabardID = pCharComponent->itemids[7];<br />
auto cloakID = pCharComponent->itemids[8];<br />
<br />
// Gloves<br />
if (glovesID)<br />
{<br />
auto pGloves = GetItemDisplayInfoRec(glovesID);<br />
if (pGloves && pGloves->GeosetGroup[0])<br />
{<br />
auto glovesGeoset = 401 + pGloves->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 401, 499, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, glovesGeoset, glovesGeoset, 1);<br />
}<br />
}<br />
else<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[0])<br />
{<br />
auto chesetGeoset = 801 + pChest->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chesetGeoset, chesetGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// unknown shirt pre-check<br />
if (!pCharComponent->shirtRelated[0] && !pCharComponent->shirtRelated[1] && !pCharComponent->shirtRelated[2] && !pCharComponent->shirtRelated[3] && !pCharComponent->shirtRelated[4] && !pCharComponent->shirtRelated[5])<br />
{<br />
// Shirt<br />
if (shirtID)<br />
{<br />
auto pShirt = GetItemDisplayInfoRec(shirtID);<br />
if (pShirt && pShirt->GeosetGroup[0])<br />
{<br />
auto shirtGeoset = 801 + pShirt->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, shirtGeoset, shirtGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Tabard<br />
auto pTabard = GetItemDisplayInfoRec(tabardID);<br />
if (pTabard)<br />
{<br />
if (!(pTabard->Flags & 0x100000))<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2200, 2299, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2202, 2202, 1);<br />
}<br />
}<br />
else<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[3])<br />
{<br />
auto chestGeoset = 2201 + pChest->GeosetGroup[3];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 2200, 2299, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Belt<br />
BOOL hasBulkyBeltFlag = 0;<br />
auto pBelt = GetItemDisplayInfoRec(beltID);<br />
if (pBelt)<br />
hasBulkyBeltFlag = pBelt->Flags & 0x200;<br />
<br />
BOOL dressPants, dressChestpiece;<br />
<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[2])<br />
{<br />
if (pChest->GeosetGroup[2])<br />
{<br />
dressPants = 0;<br />
dressChestpiece = 1;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 902, 999, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1100, 1199, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
auto chestGeoset = pChest->GeosetGroup[2] + 1301;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
}<br />
else if (pantsID)// Pants<br />
{<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
if (pPants && pPants->GeosetGroup[2] && !HIWORD(pCharComponent->flags))<br />
{<br />
dressPants = 1;<br />
dressChestpiece = 0;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 902, 999, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1100, 1199, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
auto pantsGeoset = pPants->GeosetGroup[2] + 1301;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
else<br />
{<br />
dressPants = 0;<br />
dressChestpiece = 0;<br />
// Boots<br />
auto pBoots = GetItemDisplayInfoRec(bootsID);<br />
if (pBoots && pBoots->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 501, 599, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 901, 901, 1);<br />
auto bootsGeoset = 501 + pBoots->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, bootsGeoset, bootsGeoset, 1);<br />
}<br />
else<br />
{<br />
// Pants<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
uint32_t pantsGeoset;<br />
if (pPants && pPants->GeosetGroup[1] && !HIWORD(pCharComponent->flags))<br />
{<br />
pantsGeoset = pPants->GeosetGroup[1] + 901;<br />
}<br />
else<br />
{<br />
pantsGeoset = 901;<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
<br />
// Boots<br />
BOOL bootsFlag = 0;<br />
auto pBoots = GetItemDisplayInfoRec(bootsID);<br />
if (pBoots)<br />
bootsFlag = (pBoots->Flags & 0x100000) == 0;<br />
<br />
uint32_t bootsGeoset;<br />
if (pBoots && pBoots->GeosetGroup[1])<br />
{<br />
bootsGeoset = pBoots->GeosetGroup[1] + 2000;<br />
}<br />
else<br />
{<br />
if (bootsFlag)<br />
{<br />
bootsGeoset = 2002;<br />
}<br />
else<br />
{<br />
bootsGeoset = 2001;<br />
}<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, bootsGeoset, bootsGeoset, 1);<br />
<br />
// Tabard<br />
BOOL showsTabard = 0;<br />
BOOL hasDress = dressChestpiece | dressPants;<br />
if (!hasDress && tabardID != 0 && pTabard && pTabard->GeosetGroup[0])<br />
{<br />
showsTabard = 0;<br />
uint32_t tabardGeoset;<br />
if (hasBulkyBeltFlag)<br />
{<br />
showsTabard = 1;<br />
tabardGeoset = 1203;<br />
}<br />
else<br />
{<br />
tabardGeoset = pTabard->GeosetGroup[0] + 1201;<br />
showsTabard = 1;<br />
}<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, tabardGeoset, tabardGeoset, 1);<br />
}<br />
else if (!(pCharComponent->flags & 0x10))<br />
{<br />
<br />
}<br />
else<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1201, 1201, 1);<br />
<br />
if (!hasDress)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1202, 1202, 1);<br />
showsTabard = 1;<br />
}<br />
}<br />
<br />
if (!(dressChestpiece | showsTabard))<br />
{<br />
// Chest<br />
if (chestID)<br />
{<br />
auto pChest = GetItemDisplayInfoRec(chestID);<br />
if (pChest && pChest->GeosetGroup[1])<br />
{<br />
auto chestGeoset = 1001 + pChest->GeosetGroup[1];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, chestGeoset, chestGeoset, 1);<br />
}<br />
}<br />
else if (shirtID)<br />
{<br />
auto pShirt = GetItemDisplayInfoRec(shirtID);<br />
if (pShirt && pShirt->GeosetGroup[1])<br />
{<br />
auto shirtGeoset = 1001 + pShirt->GeosetGroup[1];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, shirtGeoset, shirtGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
if (!dressChestpiece && !HIWORD(pCharComponent->flags))<br />
{<br />
if (pantsID)<br />
{<br />
auto pPants = GetItemDisplayInfoRec(pantsID);<br />
if (pPants && pPants->GeosetGroup[0])<br />
{<br />
auto geosetGroup = pPants->GeosetGroup[0];<br />
auto pantsGeoset = 1101 + geosetGroup;<br />
if (geosetGroup > 2)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1300, 1399, 0);<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
else if (!showsTabard)<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, pantsGeoset, pantsGeoset, 1);<br />
}<br />
}<br />
}<br />
<br />
// Cloak<br />
if (cloakID)<br />
{<br />
auto pCloak = GetItemDisplayInfoRec(cloakID);<br />
if (pCloak && pCloak->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1500, 1599, 0);<br />
auto cloakGeoset = 1501 + pCloak->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, cloakGeoset, cloakGeoset, 1);<br />
}<br />
}<br />
<br />
// Belt<br />
if (beltID)<br />
{<br />
auto pBelt = GetItemDisplayInfoRec(beltID);<br />
if (pBelt && pBelt->GeosetGroup[0])<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1800, 1899, 0);<br />
auto beltGeoset = 1801 + pBelt->GeosetGroup[0];<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, beltGeoset, beltGeoset, 1);<br />
}<br />
}<br />
<br />
BOOL v76;<br />
<br />
if (pCharComponent->flags & 8)<br />
{<br />
v76 = ((pantsID | showsTabard | hasDress) == 0) & ((hasBulkyBeltFlag | tailFlag) ^ 1);<br />
if (v76)<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1401, 1401, 1);<br />
}<br />
else<br />
{<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, 1400, 1400, 0);<br />
}<br />
}<br />
else<br />
{<br />
v76 = 0;<br />
}<br />
<br />
// DH hands<br />
uint32_t v86 = GetConditionalGeoset(pCharComponent->raceID, 23);<br />
if (v86 <= 0)<br />
v86 = 1;<br />
CM2Model::SetGeometryVisible(pCharComponent->modelPtr, (v86 + 2300), (v86 + 2300), 1);<br />
<br />
//sub_1000990E0(pCharComponent, v76 != 0);<br />
//v87 = pCharComponent->raceID;<br />
//v88 = *genderID;<br />
//if (!byte_101E3C7D0)<br />
//{<br />
// byte_101E3C7D0 = 1;<br />
// v89 = qword_101E3C7C8;<br />
// if (qword_101E3C7C8 || (v89 = sub_1000E0760("hdPlayerModels"), (qword_101E3C7C8 = v89) != 0))<br />
// v90 = *(v89 + 92) != 0;<br />
// else<br />
// v90 = 0;<br />
// sub_100097210(v90, dword_101E3C15C);<br />
//}<br />
//v91 = 0;<br />
//if (v87 > 36)<br />
//{<br />
// v92 = 0;<br />
//}<br />
//else<br />
//{<br />
// v92 = 0;<br />
// if (v88 <= 1)<br />
// v92 = dword_101E3C15C[4 * v87 + 2 * v88] == 1;<br />
//}<br />
//v93 = sub_100148CA0(pCharComponent->raceID, *genderID, pCharComponent->faceIndex, v92);<br />
//if (v93)<br />
// v91 = *v93;<br />
//sub_101436B70(pCharComponent->modelPtr, v91);<br />
//sub_1014387C0(pCharComponent->modelPtr);<br />
//LOBYTE(pCharComponent->flags) &= 0xFDu;<br />
}</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25514DB/ItemDisplayInfo2018-03-30T17:35:24Z<p>Simca: /* Geoset Group Field Meaning */ Thanks to efforts by TOM_RUS, confirmed geosetGroup[3] is hooked up to 2201 series geosets but on chestpieces only (not collections). Also, there is no hookup for 23xx or 27xx series geosets for items.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item.<br />
Head: {}<br />
Shoulder: {}<br />
Cloak: {geosetGroup[0] = 1501}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301, geosetGroup[3] = 2201}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Gloves: {geosetGroup[0] = 401}<br />
Belt: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25512DB/ItemDisplayInfo2018-03-29T21:37:52Z<p>Simca: /* Geoset Group Field Meaning */ Added unverified information about geoset groups 23xx and 27xx.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item (with the exception of collections which can be a part of any slot).<br />
{{Unverified|1=Collections: {geosetGroup[3] = 2201} }}<br />
Head: {}<br />
Shoulder: {}<br />
Cloak: {geosetGroup[0] = 1501}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Gloves: {geosetGroup[0] = 401}<br />
Belt: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
{{Unverified|1=23: {0: No Geoset; 1: Show Shoulders} }}<br />
{{Unverified|1=27: {0: No Geoset; 1: Show Head} }}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25510DB/ItemDisplayInfo2018-03-29T20:42:08Z<p>Simca: /* Geoset Group Field Meaning */ Fixed spacing issue.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item (with the exception of collections which can be a part of any slot).<br />
Collections: {geosetGroup[3] = 2201**}<br />
Head: {}<br />
Shoulder: {}<br />
Cloak: {geosetGroup[0] = 1501}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Gloves: {geosetGroup[0] = 401}<br />
Belt: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<br />
<nowiki />** This is speculation that needs confirming.<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB/ItemDisplayInfo&diff=25509DB/ItemDisplayInfo2018-03-29T20:41:46Z<p>Simca: /* Geoset Group Field Meaning */ Added some information about Collections and geoset series 22XX, some of which is speculation.</p>
<hr />
<div>Contains information about how items display.<br />
<br />
If somebody wants to break the following two sections out into a real 'Geoset' article, go ahead. Just make sure to link M2, SKIN, ItemDisplayInfo, and CreatureDisplayInfoExtra pages to it.<br />
<br />
==Geoset Group Field Meaning==<br />
Each geosetGroup field is a modifier for a specific 'group' of geosets.<br />
For example, on a pair of gloves, geosetGroup[0] refers to the '401' set of geosets. If geosetGroup[0] was '2' on a pair of gloves, it means that you should enable mesh part ID '403' in the M2.<br />
<br />
These vary based on the slot of the item (with the exception of collections which can be a part of any slot).<br />
Collections: {geosetGroup[3] = 2201**}<br />
Head: {}<br />
Shoulder: {}<br />
Cloak: {geosetGroup[0] = 1501}<br />
Chest: {geosetGroup[0] = 801, geosetGroup[1] = 1001, geosetGroup[2] = 1301}<br />
Shirt: {geosetGroup[0] = 801, geosetGroup[1] = 1001}<br />
Tabard: {geosetGroup[0] = 1201}<br />
Gloves: {geosetGroup[0] = 401}<br />
Belt: {geosetGroup[0] = 1801}<br />
Pants: {geosetGroup[0] = 1101, geosetGroup[1] = 901, geosetGroup[2] = 1301}<br />
Boots: {geosetGroup[0] = 501, geosetGroup[1] = 2000*}<br />
Weapon: {}<br />
Shield: {}<br />
Ammo: {}<br />
<br />
group values:<br />
<br />
4: {0: No Geoset; 1: Default; 2: Thin; 3: Folded; 4: Thick}<br />
5: {0: No Geoset; 1: Default; 2: High Boot; 3: Folded Boot; 4: Puffed; 5: Boot 4}<br />
8: {1: Default (No Geoset); 2: Flared Sleeve; 3: Puffy Sleeve; 4: Panda Collar Shirt}<br />
9: {1: Default (No Geoset); 2: Flared Pant Cuff; 3: Knickers; 4: Panda Pants}<br />
10: {1: Default (No Geoset); 2: Doublet; 3: Body 2; 4: Body 3}<br />
11: {1: Default (No Geoset); 2: Mini Skirt; 4: Heavy}<br />
12: {1: Default (No Geoset); 2: Tabard}<br />
13: {0: No Geoset; 1: Default; 2: Long Skirt}<br />
15: {1: Default (No Geoset); 2: Ankle Length; 3: Knee Length; 4: Split Banner; 5: Tapered Waist; 6: Notched Back; 7: Guild Cloak; 8: Split (Long); 9: Tapered (Long); 10: Notched (Long)}<br />
18: {0: No Geoset; 1: Default; 2: Heavy Belt; 3: Panda Cord Belt}<br />
20: {0: No Geoset; 1: Default (Basic Shoes); 2: Toes}<br />
22: {0: No Geoset; 1: Default; 2: Covered Torso}<br />
<br />
<nowiki />* This is not a typo. If you are not wearing boots, you get geoset 2001. If you are wearing boots and geosetGroup[1] for your boots is 0, you get 2002. If you are wearing boots and your geosetGroup[1] is non-zero, you get 2000 + geosetGroup[1].<br />
<nowiki />** This is speculation that needs confirming.<br />
<br />
==Geoset Priorities==<br />
Unfortunately, since many of these are the same across items, you can have items (say, a Shirt and a Chestpiece) with different geosets. The game resolves this using a hardcoded priority that is confusing as hell to follow. I will attempt to simplify it here.<br />
Gloves geosetGroup[0] (401 set) > Chest geosetGroup[0] (801 set) > Shirt geosetGroup[0] (801 set)<br />
Belt geosetGroup[0] (1801 set) > Tabard geosetGroup[0] (1201 set)<br />
Chest geosetGroup[2] (1301 set) > Pants geosetGroup[2] (1301 set) > Boots geosetGroup[0] (501 set) > Pants geosetGroup[1] (901 set)<br />
<br />
As an example (to explain my notation above), if you had Gloves, a Shirt, and a Chestpiece, the game would first look at the gloves and check if your geosetGroup[0] was 0. If it was 0, then you would have normal, fitting gloves, and there would be no conflict with the 801 series geosets that the Chestpiece or Shirt might want to set, and then it would look at the Chestpiece for the same thing (and lastly the Shirt). If it wasn't 0, it would set your appropriate geoset (401 + geosetGroup[0]) and never even look at the geosetGroup[0] entry for your Shirt or Chestpiece (i.e. they would be totally irrelevant, the gloves would be dominant).<br />
<br />
As of last edit, this list is nowhere near comprehensive (unlike the Fields Meaning list), so here is the full geoset enable/disable code that was decompiled from the WoD internal build (and subsequently prettified to make it almost readable): [[DB/ItemDisplayInfo/GeosRenderPrep]]<br />
<br />
==m_texture indices==<br />
enum {<br />
ArmUpperTexture,<br />
ArmLowerTexture,<br />
HandTexture,<br />
TorsoUpperTexture,<br />
TorsoLowerTexture,<br />
LegUpperTexture,<br />
LegLowerTexture,<br />
FootTexture,<br />
#if version > ???? // at least in 6.0.1, probably cata<br />
AccessoryTexture,<br />
#endif<br />
};<br />
<br />
{{Template:SectionBox/VersionRange|max_expansionlevel=0|max_build=0.5.3.3368}}<br />
enum TEXCOMPONENT_SECTIONS<br />
{<br />
TCS_UPPERARM = 0x0,<br />
TCS_LOWERARM = 0x1,<br />
TCS_HAND = 0x2,<br />
TCS_UPPERHEAD = 0x3,<br />
TCS_LOWERHEAD = 0x4,<br />
TCS_UPPERTORSO = 0x5,<br />
TCS_LOWERTORSO = 0x6,<br />
TCS_LEGUPPER = 0x7,<br />
TCS_LEGLOWER = 0x8,<br />
TCS_FEET = 0x9,<br />
NUM_TEXCOMPONENT_SECTIONS = 0xA,<br />
TCS_INVALIDSECTION = 0xB,<br />
};<br />
<br />
==0.5.3.3368==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[4];<br />
uint32_t m_flags; // 1 - Apply guild tabard texture on the char select screen, 2 - Hides underwear<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_itemSize;<br />
uint32_t m_helmetGeosetVisID;<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
==1.12.1.5875==<br />
This sounds utterly wrong based on comparison to other versions.<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
{{Template:Type|stringref}} m_inventoryIcon;<br />
{{Template:Type|stringref}} m_groundModel;<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_spellVisualID;<br />
uint32_t m_groupSoundIndex;<br />
uint32_t m_helmetGeosetVisID[2];<br />
{{Template:Type|stringref}} m_texture[8];<br />
uint32_t m_itemVisual;<br />
};<br />
== 7.0.1.21737 ==<br />
Textured Items now stored in [[DB/ItemDisplayInfoMaterialRes|ItemDisplayInfoMaterialRes]]<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="500" | Notes<br />
|- style="background:#E0E0E0;"<br />
|1 || ModelID || Integer || <br />
|-<br />
| 2 || [[DB/ModelFileData|LeftModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 3 || [[DB/ModelFileData|RightModelID]] || iRefID || References ModelFileData which references FileDataComplete <br />
|-<br />
| 4 ||[[DB/TextureFileData|LeftModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|- style="background:#E0E0E0;"<br />
| 5 ||[[DB/TextureFileData|RightModelTextureID]] || iRefID || References TextureFileData which references FileDataComplete <br />
|-<br />
| 6 || Geoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 7 || Geoset2 || Integer || <br />
|-<br />
| 8 || Geoset3 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 9 || Unknown || Integer || <br />
|-<br />
| 10 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 11 || Unknown || Integer || <br />
|-<br />
| 12 || Flags || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 13 || SpellVisualRef? || Integer || May not be<br />
|-<br />
| 14 || HelmGeoset1 || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 15 || HelmGeoset2 || Integer || <br />
|-<br />
| 16 || ItemVisual || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 17 || ParticleID || Integer || <br />
|-<br />
| 18 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 19 || Unknown || Integer || <br />
|-<br />
| 20 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 21 || Unknown || Integer || <br />
|-<br />
| 22 || Unknown || Integer || <br />
|- style="background:#E0E0E0;"<br />
| 23 || Unknown || Integer || <br />
|}<br />
[[User:Synric|Synric]] 12th July 2016<br />
<br />
==6.0.1.18179==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type|stringref}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type|stringref}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==6.0.3.19243==<br />
struct ItemDisplayInfoRec {<br />
uint32_t m_ID;<br />
{{Template:Type|stringref}} m_modelName[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=TextureItemID}} m_modelTexture[2];<br />
uint32_t m_geosetGroup[3];<br />
uint32_t m_flags;<br />
{{Template:Type/foreign_key|table=SpellVisual}} m_spellVisualID;<br />
{{Template:Type/foreign_key|table=HelmetGeosetVisData}} m_helmetGeosetVis[2];<br />
{{Template:Type/foreign_key|table=TextureFileData|column=m_TextureItemID}} m_texture[9];<br />
{{Template:Type/foreign_key|table=ItemVisuals}} m_itemVisual;<br />
{{Template:Type/foreign_key|table=ParticleColor}} m_particleColorID;<br />
};<br />
<br />
==3.0.2.8905==<br />
{| style="background:#FCFCFC; color:black"<br />
|- <br />
! width="80" | Column <br />
! width="180" | Field <br />
! width="80" | Type <br />
! width="800" | Notes<br />
|- <br />
| 1 || ID || Integer || <br />
|- <br />
| 2 || LeftModel || String || For example at shoulders. Defining the pairs.<br />
|- <br />
| 3 || RightModel || String || <br />
|- <br />
| 4 || LeftModelTexture || String || And of course the textures, if not hardcoded. Replaces texture type 2.<br />
|- <br />
| 5 || RightModelTexture || String || <br />
|- <br />
| 6 || Icon1 || String || The icon displayed in the bags etc.<br />
|-<br />
| 7 || Icon2 || String || This is only used on Cro's apples which can be crushed. Sadly the icon then is a green instead of red apple.<br />
|-<br />
| 8 || geosetGroup || Integer || [[M2/.skin#Mesh_part_ID|explanation]], if non-0, add 1, add group. [item-slot=chest/shirt: wristbands (8)]; [item-slot=glove: glove (4)]; [item-slot=boots: boots (5)]; [item-slot=cape: cape (15)]<br />
|-<br />
| 9 || geosetGroup || Integer || [item-slot=pants: kneepads (9)]<br />
|-<br />
| 10 || geosetGroup || Integer || [item-type=chest.robe: 1 = has_robe]; [geoset trousers == 1: trousers (12)]<br />
|-<br />
| 11 || flags || Integer || [[DB/ItemDisplayInfo#0.5.3.3368|&1 and &2 match 0.5.3]], &4: {{Template:Unverified|event specific items i.e. {{Template:Data/Item|id=22282|name=Purple Dinner Suit}} requires Love is in the Air}}<br />
|-<br />
| 12 || spellVisualID || Integer || <br />
|- style="background:#F0F8FF;"<br />
| 13 || [[ItemGroupSounds.dbc|groupSoundIndex]] || iRefID || <br />
|- style="background:#F0F8FF;"<br />
| 14 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Male<br />
|- style="background:#F0F8FF;"<br />
| 15 || [[HelmetGeosetVisData.dbc|helmetGeosetVis]] || iRefID || For Female<br />
|- <br />
| 16 || UpperArmTexture || String || <br />
|- <br />
| 17 || LowerArmTexture || String || <br />
|- <br />
| 18 || HandsTexture || String || <br />
|- <br />
| 19 || UpperTorsoTexture || String || <br />
|- <br />
| 20 || LowerTorsoTexture || String || <br />
|- <br />
| 21 || UpperLegTexture || String || <br />
|- <br />
| 22 || LowerLegTexture || String || <br />
|- <br />
| 23 || FootTexture || String || <br />
|- style="background:#F0F8FF;"<br />
| 24 || [[ItemVisuals.dbc|itemVisual]] || iRefID || Static enchants. A few values have -1. Probably does not allow additional effects ?<br />
|-<br />
| 25 || particleColorID || Integer || <br />
|}<br />
<br />
==8.0.1.25902==<br />
struct ItemDisplayInfoRec {<br />
// uint32_t m_ID;<br />
<br />
int32_t m_flags;<br />
uint32_t Unk1;<br />
<br />
int32_t m_itemVisual;<br />
int32_t m_particleColorID;<br />
uint32_t Unk2; // only 4 non zero value<br />
int32_t Unk3; // added in build 20810<br />
int32_t Unk4; // added in build 21063<br />
int32_t Unk5; // only 1 non zero value<br />
int32_t Unk6; // added in 21249<br />
uint32_t Unk7; // added in 21249<br />
<br />
uint32_t m_modelName[2];<br />
uint32_t m_modelTexture[2];<br />
uint32_t m_geosetGroup[4];<br />
uint32_t Unk8[4]; // added in 7.0, probably some geoset groups as well, but has much less non zero values than in previous field<br />
uint32_t m_helmetGeosetVis[2];<br />
};<br />
<br />
[[Category:DBC]]<br />
[[Category:DBC_Alpha]]<br />
[[Category:DBC_Vanilla]]<br />
[[Category:3.0.2.8905]][[Category:DBC_WotLK]]<br />
[[Category:DBC_WoD]][[Category:6.0.1.18179]][[Category:6.0.3.19243]]<br />
[[Category:DBC_Legion]][[Category:7.0.1.23717]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=25469DB22018-03-16T19:54:34Z<p>Simca: /* WDC2 */ Updated with newly discovered information about data sections and the new header.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and is currently in use today.<br />
<br />
WDC2 introduces a new system of 'sections' of data. Three chunks of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records / 'sections'. Everything after that was broken into a new form of 'sections' which can potentially occur many times in one file. Note that implementing this system right now appears to be entirely optional as no DB2 file as of the current build has a 'section_count' higher than 1. Also, it appears that the 'offset_map' format (in use for a small handful of DB2s) has a hardcoded limit of 1 section.<br />
<br />
The other major change is to strings and how string offsets are calculated. For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This complicates parsing the string block directly and storing relative locations, but it does slightly simplify reading records.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.StringBlockLocation;<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count; // this is for all sections combined now<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size; // this is for all sections combined now<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t section_count; // new to WDC2, this is number of sections of data (records + copy table + id list + relationship map = a section)<br />
};<br />
wdc2_db2_header header;<br />
<br />
struct wdc2_section_header<br />
{<br />
uint32_t wdc2_unk_header1; // always 0 in {{Template:Sandbox/PrettyVersion|expansionlevel=8|build=8.0.1.26231}} and unnamed in client binary<br />
uint32_t wdc2_unk_header2; // always 0 in {{Template:Sandbox/PrettyVersion|expansionlevel=8|build=8.0.1.26231}} and unnamed in client binary<br />
uint32_t file_offset; // absolute position to the beginning of the section<br />
uint32_t record_count; // 'record_count' for the section<br />
uint32_t string_table_size; // 'string_table_size' for the section<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_section_header section_headers[section_count];<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
struct section<br />
{<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[section_headers.record_size];<br />
};<br />
record_data records[section_headers.record_count];<br />
char string_data[section_headers.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[section_headers.offset_map_offset - section_headers.file_offset];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[section_headers.id_list_size / 4];<br />
if (section_headers.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[section_headers.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (section_headers.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
};<br />
section data_sections[section_count];<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = 'Has secondary key'<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = ? // WDC1 specific. used on all WDC1 files<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=25467DB22018-03-16T10:36:46Z<p>Simca: Fixed a comment in the WDC2 header specifically referencing WDC1.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and is currently in use today.<br />
<br />
WDC2 is WDC1 with sections shuffled around randomly for no apparent reason. There are six new header fields, in addition to the existing fields after 'locale' being shuffled around. Three sections of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records.<br />
<br />
The only actual change that isn't purely reshuffling existing data has to do with strings and how string offsets are calculated (and it's sort of reshuffling existing data, just in a different way). For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This greatly complicates some forms of parsing.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.StringBlockLocation;<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
// Everything past this point has had its position reshuffled since WDC1 - be warned<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // from WDC1 onwards, this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t wdc2_unk_header1;<br />
uint32_t wdc2_unk_header2;<br />
uint32_t wdc2_unk_header3;<br />
uint32_t wdc2_unk_header4;<br />
uint32_t wdc2_unk_header5;<br />
uint32_t wdc2_unk_header6;<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_db2_header header;<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc2_db2_header) - (sizeof(field_structure) * header.total_field_count) - header.field_storage_info_size - header.pallet_data_size - header.common_data_size];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = 'Has secondary key'<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = ? // WDC1 specific. used on all WDC1 files<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=25466DB22018-03-16T10:35:28Z<p>Simca: Added WDC2 documentation.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600|max_expansionlevel=8|max_build=8.0.1.26231}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and was replaced by WDC2 in Battle for Azeroth (Patch 8.0.1 Build 26231).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // in WDC1 this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=WDC2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=8|min_build=8.0.1.26231}}<br />
This file format was introduced in Battle for Azeroth (Patch 8.0.1 Build 26231) and is currently in use today.<br />
<br />
WDC2 is WDC1 with sections shuffled around randomly for no apparent reason. There are six new header fields, in addition to the existing fields after 'locale' being shuffled around. Three sections of the structure ('field_storage_info', 'pallet_data', and 'common_data') were moved up in front of the records.<br />
<br />
The only actual change that isn't purely reshuffling existing data has to do with strings and how string offsets are calculated (and it's sort of reshuffling existing data, just in a different way). For the history of WoW up until WDC2, strings in records were represented as 'the relative position of the referenced string to the start of the string block'. In WDC2, strings in records are represented as 'the relative position from the beginning of the field where this offset was stored to the position of the referenced string in the string block'. This greatly complicates some forms of parsing.<br />
<br />
If before WDC2 you dealt with strings by reading the string block and assigning relative positions to every string, then you will need to either change your approach (to having a constantly-accessible 1:1 representation of the string table available) or mitigate the damage from the change. One method of mitigation is: when reading the values of the string offsets in the records, replace the data at that point in time with 'old-style' string offsets.<br />
<br />
For example, using code like this when reading the field in question works:<br />
row[field].LongVal = reader.ReadVarInt(Fields[field].Size);<br />
if (IsWDC2() && row[field].Column.Type == ColumnType.String)<br />
{<br />
row[field].LongVal = row[field].LongVal + reader.BaseStream.Position - Fields[field].Size - reader.StringBlockLocation;<br />
}<br />
This sample code reads the field containing the string offset in the record, then recalculates the value it just read by adding in the current position and subtracting the size of the field it just read (to get the absolute position of the referenced string) and then by subtracting the absolute position of the start of the string block (which yields the relative position of the referenced string to the start of the string block). This manipulation ensures that a program converts every string offset back to the pre-WDC2 style.<br />
<br />
==Structure==<br />
struct wdc2_db2_header<br />
{<br />
uint32_t magic; // 'WDC2'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
// Everything past this point has had its position reshuffled since WDC1 - be warned<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // in WDC1 this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t wdc2_unk_header1;<br />
uint32_t wdc2_unk_header2;<br />
uint32_t wdc2_unk_header3;<br />
uint32_t wdc2_unk_header4;<br />
uint32_t wdc2_unk_header5;<br />
uint32_t wdc2_unk_header6;<br />
uint32_t copy_table_size;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t relationship_data_size;<br />
};<br />
wdc2_db2_header header;<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
field_structure fields[header.total_field_count];<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc2_db2_header) - (sizeof(field_structure) * header.total_field_count) - header.field_storage_info_size - header.pallet_data_size - header.common_data_size];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = 'Has secondary key'<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = ? // WDC1 specific. used on all WDC1 files<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=Known_Server-Side_Tables&diff=25408Known Server-Side Tables2018-02-27T23:38:37Z<p>Simca: Lol not WorldState.</p>
<hr />
<div>There's currently no purpose to keeping this list up to date, but we can do it anyway, just for fun. Obviously, we can't ever get any of these tables unless Blizzard does something very stupid, but it's neat to have a glimpse of how things work behind the scenes.<br />
<br />
* AI_GroupAction<br />
* AI_GroupActionSet<br />
* AI_TriggerAction<br />
* AI_TriggerActionSet<br />
* CharacterTemplateData<br />
* CharacterTemplates<br />
* CharStartQuest<br />
* NamedPoint<br />
* ScalingTreasureDrop<br />
* SpawnEvent<br />
* SpawnGroup<br />
* SpawnLoc<br />
* SpawnObject<br />
* SpawnPathNode<br />
* SpawnSquad<br />
* SpawnRegion<br />
* SpellScript<br />
* SpellScriptPackage<br />
* StringID<br />
* Treasure<br />
* TreasureItem<br />
* VendorItem<br />
* VendorList</div>Simcahttps://wowdev.wiki/index.php?title=Known_Server-Side_Tables&diff=25407Known Server-Side Tables2018-02-27T23:38:05Z<p>Simca: Just making this page for fun to keep track of server-side table names we hear about.</p>
<hr />
<div>There's currently no purpose to keeping this list up to date, but we can do it anyway, just for fun. Obviously, we can't ever get any of these tables unless Blizzard does something very stupid, but it's neat to have a glimpse of how things work behind the scenes.<br />
<br />
* AI_GroupAction<br />
* AI_GroupActionSet<br />
* AI_TriggerAction<br />
* AI_TriggerActionSet<br />
* CharacterTemplateData<br />
* CharacterTemplates<br />
* CharStartQuest<br />
* NamedPoint<br />
* ScalingTreasureDrop<br />
* SpawnEvent<br />
* SpawnGroup<br />
* SpawnLoc<br />
* SpawnObject<br />
* SpawnPathNode<br />
* SpawnSquad<br />
* SpawnRegion<br />
* SpellScript<br />
* SpellScriptPackage<br />
* StringID<br />
* Treasure<br />
* TreasureItem<br />
* WorldState<br />
* VendorItem<br />
* VendorList</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=25238DB22017-12-19T20:51:16Z<p>Simca: /* WDC1 */ Updated structure with some information about important and unimportant fields.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBCs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBCMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and is still in use today (as of Patch 7.3.5 Build 25632).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a 4-byte value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you 4-byte values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // in WDC1 this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits; // very important for reading bitpacked fields; size is the sum of all array pieces in bits - for example, uint32[3] will appear here as '96'<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits; // not useful for most purposes; formula they use to calculate is bitpacking_offset_bits = field_offset_bits - (header.bitpacked_data_offset * 8)<br />
uint32_t bitpacking_size_bits; // not useful for most purposes<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
There are several fields in the structure with no apparent purpose, but I believe these are all related. 'bitpacked_data_offset' in the header and 'bitpacking_offset_bits' and 'bitpacking_size_bits' from some variants of 'field_storage_info' (the bitpacked ones) are the fields I am referring to. I believe these three values are used to read from the record starting at the position of the bitpacked values. Our best guess is that Blizzard needs this for some purpose, possibly to integrate with their existing tools.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = 'Has secondary key'<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = ? // WDC1 specific. used on all WDC1 files<br />
<br />
[[Category:Format]]</div>Simcahttps://wowdev.wiki/index.php?title=DB2&diff=25233DB22017-12-18T17:12:40Z<p>Simca: /* WDC1 */ Updated structure with post-field_compression field names (they actually differ fully based on that value) and added a 'further quirks' section with some new info.</p>
<hr />
<div>{{Template:SectionBox/VersionRange|min_expansionlevel=4}}<br />
<br />
DB2 files are the new version of client side databases, introduced in Cataclysm, containing data about items, NPCs, environment, world and a lot more. They are similar in many ways to [[DBC|DBC files]], so you may want to also look at those. They both have headers with similar data, records containing fields of various types with various data, and a string block where 4-byte string references are made. The difference is that DB2 files have become wildly more complex than their simple DBC brethren, especially in recent times. The structure described here was also used in [[ADB|ADB files]] for many years, which are a cache of dynamically streamed database entries, typically used hotfixes and content that Blizzard wants to hide from dataminers until the last minute. Recently the file structures of DB2 files and ADB files have diverged greatly.<br />
<br />
=Table content structures=<br />
This page describes the structure of [[DB2]] files. For a list of existing DB2 files and their contents see the categories [[:Category:DBC|DBC]], [[:Category:DBC_Vanilla|Vanilla]], [[:Category:DBC_BC|Burning Crusade]], [[:Category:DBC_WotLK|Wrath of the Lich King]], [[:Category:DBC_Cataclysm|Cataclysm]], [[:Category:DBC_MoP|Mists of Pandaria]] and [[:Category:DBC_WoD|Warlords of Draenor]]. If you add documentation for a file, please add the correct categories (also the build number) as well.<br />
<br />
=Field Types=<br />
WDB2 began with the following possible field types:<br />
64-bit Integers*<br />
32-bit Integers*<br />
8-bit Integers*<br />
Floats<br />
Strings (strings are represented in the record data as a 32-bit unsigned integer, see the String Block section for more information)<br />
<br />
Additionally, WDB3 added the following possible field type:<br />
16-bit Integers*<br />
<br />
Note that Blizzard does not differentiate between signed and unsigned field types; WoW code simply casts the data around as it needs. Because of this, some fields will make more sense as signed integers (example: a casting time reduction) and some fields will make more sense as unsigned integers (example: bitfields). You will have to make the determination as to which fields should be which on your own. Personally, I default to Signed 32-bit Integers, Unsigned 16-bit Integers, and Unsigned 8-bit Integers.<br />
<br />
As an addendum to the above paragraph, in the binary's DBCMeta for WDB5-ready Legion clients, there is a flag for each field that can designate 'do not sign-extend when decompressing'. However, there are fields which obviously contain unsigned data that are not marked with this flag, so it is not exhaustive.<br />
<br />
Traditionally, 64-bit integers have only made an appearance in a very tiny number of DB2s (usually 1 DB2 at any given time) and only for short periods of time. This meant that 64-bit integer support was not necessary. However, the addition of Allied Races in Legion's Patch 7.3.5 Build 25600 caused this to change. There were no longer enough bits for 'RaceMask', a field used in half a dozen different DB2s, and the field was expanded to be a 64-bit integer.<br />
<br />
==Determining Field Types==<br />
In WDB2, you can mostly just assume that every field will be four bytes. Three out of the five possibilities are four bytes, and the 8-bit integers are only used very, very rarely (literally like 3-4 files out of hundreds). The 64-bit Integers are actually only used ONCE so far (in CriteriaTree it was needed to store the number 2,500,000,000, which is 250k gold in copper). Deciding whether or not a four-byte value is a float, an integer, or a string is not terribly difficult (floats will basically always have certain bits set and every value in a string field will be an offset into the string block, which you can check), and this approach gives you ~98% compatibility with the WDB2 format with minimal effort.<br />
<br />
In WDB3 and WDB4, things become much, much harder. Determining field types on the fly is virtually impossible, and the majority of DBCs (80%+) have at least one field that is not four bytes. The only proper solution is to read the WoW binary executable and parse it for the DBCMeta structure. In that structure you will find the field types for all fields.<br />
<br />
In the newer formats such as WDB5 and WDB6, determining field types on the fly is much more doable. Any field being compressed cannot be a string or a float as they are always 4 bytes, meaning you can just assume they are integers. You can use the 'field structure block' present in WDB5+ in order to determine the size of the fields and the distance between the offsets in the field structure block in order to determine array length for all fields except the last one (the last field will be padded out to 'record_size').<br />
<br />
=String Block=<br />
[[DB2]] records can contain localized strings. In contrast to [[DBC|DBCs]], a [[DB2]] file only contains localized values for a given locale (header.locale).<br />
<br />
Since Cataclysm, all DB files contain only localized string values, including [[DBC|DBCs]].<br />
<br />
The rest of the string block is equivalent to [[DBC]] version. See [[DBC#String_Block|documentation there]].<br />
<br />
=WDB2=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=4|max_expansionlevel=7|max_build=7.0.1.20740|max_exclusive=1}}<br />
This file format was introduced in Cataclysm. It was phased out in favor of WDB3 in Legion (Patch 7.0.1 build 20740).<br />
<br />
==Structure==<br />
See [[ADB#WCH3]] for how to adapt this structure for its .ADB counterpart.<br />
struct db2_header<br />
{<br />
uint32_t magic; // 'WDB2'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB2<br />
uint32_t record_size;<br />
uint32_t string_table_size; // string block almost always contains at least one zero-byte<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size; // always zero in WDB2 (?) - see WDB3 for information on how to parse this<br />
};<br />
<br />
template<typename record_type><br />
struct db2_file<br />
{<br />
db2_header header;<br />
<br />
if (header.max_id != 0)<br />
{<br />
int indices[header.max_id - header.min_id + 1]; // maps from id to row index in records[] below<br />
short string_lengths[header.max_id - header.min_id + 1]; // sum of lengths of all strings in row<br />
}<br />
<br />
record_type records[header.record_count];*<br />
char string_table[header.string_table_size];<br />
};<br />
<br />
*Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).<br />
<br />
=WDB3=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20740|max_expansionlevel=7|max_build=7.0.1.20810|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WDB4/WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WDB3 and WCH4 is likely important for almost nobody.<br />
<br />
The major changes are the addition of an optional offset map which forces records to have inline strings, an optional block after the string block that contains non-inline IDs, and an optional block after that block which contains the ID numbers of rows that have been deduplicated to save space.<br />
<br />
It is worth noting that min_id, max_id, and copy_table_size fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present. Additionally, the copy_table_size field is now non-zero sometimes, requiring action.<br />
<br />
==Structure==<br />
See [[ADB#WCH4]] for how to adapt this structure for its .ADB counterpart.<br />
template<typename record_type><br />
struct wdb3_file<br />
{<br />
db2_header header;<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // This offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // This is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];*<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
uint32_t IDs[header.record_count];*<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.<br />
<br />
*This part of the structure is optional.<br />
<br />
=WDB4=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.1.20810|max_expansionlevel=7|max_build=7.0.3.21414|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.1 build 20810) and is still in use today. It was partially phased out in favor of WDB5 just a few months later, also in Legion (Patch 7.0.3 build 21414). As such, support for WDB4 is likely important for almost nobody.<br />
<br />
The offset_map structure has moved back farther in the file and is now located in between the string table and the non-inline IDs table. Additionally, while not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.<br />
<br />
The header has changed from the original format; it gained one new field ('flags'). This flags field allows for easy detection of the optional structures.<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb4_db2_header<br />
{<br />
uint32_t magic; // 'WDB4'<br />
uint32_t record_count;<br />
uint32_t field_count; // array fields count as the size of array for WDB4<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t build;<br />
uint32_t timestamp_last_written; // set to time(0); when writing in WowClientDB2_Base::Save()<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint32_t flags; // in WDB3, this field was in the WoW executable's DBCMeta instead; possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
};<br />
<br />
template<typename record_type><br />
struct wdb4_file<br />
{<br />
wdb4_db2_header header;<br />
record_type records[header.record_count]; <br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB5=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.0.3.21479|max_expansionlevel=7|max_build=7.2.0.23436|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.0.3 build 21479) and was replaced by WDB6 later in Legion (Patch 7.2.0 build 23436). There have been a variety of ADB formats used simultaneously with WDB5, including WCH5, WCH6, WCH7, and WCH8.<br />
<br />
There are very significant changes to the format in WDB5 which will require substantial effort for existing tools to support. The main changes are that the field_count now counts arrays as '1' (mirroring very old ADB behavior), a field_structure block has been added directly after the header, and the addition of compressed fields (and 24-bit integers). The introduction of the field_structure block, in particular, is hugely positive for us and should improve the accuracy of DB2 parsers or at least reduce their dependence on the WoW binary's DBCMeta. Note that the WoW binary's DBCMeta will often disagree with the 'field_structure' (DBCMeta might say 'int32' but the field_structure block says the size is '3-bytes'). In those cases, the field_structure block takes priority. Additionally, the DBCMeta is still relevant for parsing ADBs, as they do not support compression.<br />
<br />
The header has lost a field compared to WDB4, timestamp_last_written. This field was useless in DB2s and always 0, so its removal is understandable.<br />
<br />
In build 21737, a few builds after the introduction of WDB5, more changes to the header were made. The 'flags' header field was split into two shorts - 'flags' and 'id_index'. 'id_index' is very valuable since the inline ID fields no longer have to be the first field in WDB5; they can appear anywhere in the record. This index lets you know which field is ID, so you can, for example, move it to the beginning of the record for the ease of the viewer. Additionally, the 'build' field was changed to be 'layout_hash'. This hash is unique to the specific column layout, including (at least) position, size, and column name (we don't have most of these names, but we can tell this is true based on layout_hash changes that shouldn't have happened otherwise). It is worth noting that array size is actually not unique per hash. In some cases, the size of an array can change without the hash changing. This replacement for BuildID is often beneficial to us because when layout_hash changes, you know the structure changed (usually - there have been some instances of recalculations).<br />
<br />
==Structure==<br />
See [[ADB#WCH5]] for how to adapt this structure for its .ADB counterpart.<br />
struct wdb5_db2_header<br />
{<br />
uint32_t magic; // 'WDB5' for .db2 (database)<br />
uint32_t record_count;<br />
uint32_t field_count; // for the first time, this counts arrays as '1'; in the past, only the ADB variants have counted arrays as 1 field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but after build 21737, this is a new hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // new in WDB5 (and only after build 21737), this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
};<br />
<br />
template<typename record_type><br />
struct wdb5_file<br />
{<br />
wdb5_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDB6=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.2.0.23436|max_expansionlevel=7|max_build=7.3.5.25600|max_exclusive=1}}<br />
This file format was introduced in Legion (Patch 7.2.0 build 23436) and was replaced by WDC1 later in Legion (Patch 7.3.5 build 25600).<br />
<br />
Two new header fields and a new data block were added in WDB6. Both of the header fields relate to the new block, which we have named 'common_data_table'. Its purpose is to drastically reduce db2 filesize and memory footprint (affected db2s have had their filesizes halved) by letting the client assume that many columns are always 0, unless there is an entry in the proper 'common_data_table' that is mapped to the ID of the row in question. For example, in build 23436, only 10 columns are in the 'normal' row data section for SpellEffect.db2. However, the 'common_data_table' supports up to 26 columns (designated by the new header field named 'total_field_count'). To find the value for one of the latter 16 columns in SpellEffect.db2, you look up the column in the 'common_data_table' and then the ID in the 'common_data_map' for that column. If there is an entry, use the value paired with the ID. If there is not an entry, the value 'defaults'.<br />
<br />
Default values are stored in the WoW binary, in DBMeta. It's worth noting that almost every field's default value is '0', with a few exceptions. For example, the 'Alpha' byte field in CreatureDisplayInfo defaults to 255, and a couple of floats in SpellEffect.db2 default to 1.<br />
<br />
Neither strings or arrays are supported in the 'common_data_table'.<br />
<br />
Starting from Patch 7.3.0 Build 24473, values in the 'common_data_table' are always padded out to 4 bytes. Detecting this change is very annoying as there were no other accompanying changes. If you wish to support WDB6 both before and after this build, you will need to attempt to navigate the common_data_table once without padding. Compare the distance you just navigated against the common_data_table_size field from the header. If they match, it is either not padded (pre-7.3) or does not matter (because all of the common data fields are 4 bytes). If it does not match, then it is padded (post-7.3). After determining this, you can then parse the table again with the knowledge that you are doing it correctly.<br />
<br />
It is worth a minor mention here that from WDB6 onwards, standalone [[ADB#WCH8|ADB files]] were discarded in favor of '[[ADB#DBCache.bin]]', a new format that does not mirror DB2 structure at all.<br />
<br />
==Structure==<br />
struct wdb6_db2_header<br />
{<br />
uint32_t magic; // 'WDB6'<br />
uint32_t record_count;<br />
uint32_t field_count; // this counts arrays as '1' field<br />
uint32_t record_size;<br />
uint32_t string_table_size; // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map<br />
uint32_t table_hash;<br />
uint32_t layout_hash; // used to be 'build', but now this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in [[Loc|TextWowEnum]]<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in [[DB2#Known Flag Meanings|Known Flag Meanings]]<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // new in WDB6, includes columns only expressed in the 'common_data_table', unlike field_count<br />
uint32_t common_data_table_size; // new in WDB6, size of new block called 'common_data_table'<br />
};<br />
<br />
template<typename record_type><br />
struct wdb6_file<br />
{<br />
wdb6_db2_header header;<br />
struct field_structure<br />
{<br />
int16_t size; // size in bits as calculated by: byteSize = (32 - size) / 8; this value can be negative to indicate field sizes larger than 32-bits<br />
uint16_t position; // position of the field within the record, relative to the start of the record<br />
};<br />
field_structure fields[header.field_count];<br />
record_type records[header.record_count];<br />
char string_table[header.string_table_size];<br />
if (flags & 0x01 != 0)<br />
{<br />
struct offset_map_entry<br />
{<br />
uint32_t offset; // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on<br />
uint16_t length; // this is the length of the record located at the specified offset<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
if (flags & 0x04 != 0)<br />
{<br />
uint32_t IDs[header.record_count];<br />
}<br />
if (header.copy_table_size > 0)<br />
{<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
if (header.common_data_table_size > 0)<br />
{<br />
uint32_t num_columns_in_table;<br />
struct common_data_map_entry<br />
{<br />
uint32_t id;<br />
uint32_t value; // Calling this 'uint32_t' is an oversimplification - the size of this field depends on the 'type' from the enum <br />
// (From Patch 7.3.0 Build 24473 onwards, this is no longer true. Values are always padded out to 4 bytes.)<br />
};<br />
struct common_data_table_entry<br />
{<br />
uint32_t count;<br />
uint8_t type; // New enum: string = 0, short = 1, byte = 2, float = 3, int = 4 (int64 = 5??)<br />
common_data_map_entry common_data_map[count];<br />
};<br />
common_data_table_entry common_data_table[num_columns_in_table];<br />
}<br />
};<br />
<br />
Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.<br />
<br />
=WDC1=<br />
{{Template:SectionBox/VersionRange|min_expansionlevel=7|min_build=7.3.5.25600}}<br />
This file format was introduced in Legion (Patch 7.3.5 build 25600) and is still in use today (as of Patch 7.3.5 Build 25632).<br />
<br />
WDC1 is an expanded version of WDB6. If you've been following along with the format changes in Legion thus far, you will notice that the theme has been saving space. Most recently, in WDB6, a 'common_data_block' was added in order to handle columns that are usually always one value but rarely are some other value.<br />
<br />
WDC1 takes this idea and adds in even more types of space-saving techniques. It adds in bitpacking (for fields who would be best suited to a field size that is not a multiple of 8), indexed values (for fields with values that are commonly always one of a small number of values but who do not have a common 'default' amongst those values), and indexed array values (for array fields where all fields in an array are commonly linked together, for example, with common spell flag values). It also keeps the 'common_data_block' technique from WDB6, but it streamlines the implementation of it, removing the need for a separate header block for 'common_data'. This is done by adding in a new block we call 'field_storage_info'.<br />
<br />
'field_storage_info' contains the size in bits and the offset in bits of every column, along with the size of the column's data in the different data blocks (currently the data blocks are 'pallet_data' and 'common_data'). The structure below explains most of this. Compression type '0' is normal in-field data. Compression types '1', '3', and '4' begin as bitpacked data in the record. For compression type '1', you can stop there, as that is the proper value. For compression types '3' and '4', you need to take the bitpacked value you obtained from the record and use it as an index into 'pallet_data'. For compression type '3', you pull a value from 'pallet_data' using the formula 'additional_data_offset + (index * 4)', where 'additional_data_offset' is the sum of 'additional_data_size' for every column before the current one. Compression type '4' is very similar, except that it is always an array. Array values are bundled together for compression type '4' - that single index will give you values for every single piece of the array at once. You can retrieve these values by using the formula 'additional_data_offset + (index * 4 * array_count) + (iter * 4)', where 'iter' is the 0-based array piece you wish to retrieve (you should iterate over all of them to retrieve all of the values for the array). Lastly, compression type '2' is just WDB6's old 'common_data_block' system; it is the only compression type where no data at all is read from the record. Just apply the 'default_value' (also provided by 'field_storage_info') when the ID for the record is not present in 'common_data'.<br />
<br />
The last major quirk of WDC1 is the 'relationship_map' system. In WDC1, any DB2 that had a primary key which was also a foreign key (in Blizzard's database system, I guess) was removed from the record and placed into this new structure. This is probably for increased lookup speed by the game executable. So, for example, almost all Spell*.db2 tables (besides Spell.db2 because it wouldn't be a foreign key there) had their SpellID columns removed and this new block added in their place. The easiest way to handle this block is just to add a new 'fake' column to the end of the DB2 in question and populate it with the values from the relationship map.<br />
<br />
==Structure==<br />
struct wdc1_db2_header<br />
{<br />
uint32_t magic; // 'WDC1'<br />
uint32_t record_count;<br />
uint32_t field_count;<br />
uint32_t record_size;<br />
uint32_t string_table_size;<br />
uint32_t table_hash; // hash of the table name<br />
uint32_t layout_hash; // this is a hash field that changes only when the structure of the data changes<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
uint32_t locale; // as seen in TextWowEnum<br />
uint32_t copy_table_size;<br />
uint16_t flags; // possible values are listed in Known Flag Meanings<br />
uint16_t id_index; // this is the index of the field containing ID values; this is ignored if flags & 0x04 != 0<br />
uint32_t total_field_count; // in WDC1 this value seems to always be the same as the 'field_count' value<br />
uint32_t bitpacked_data_offset; // relative position in record where bitpacked data begins; not important for parsing the file<br />
uint32_t lookup_column_count;<br />
uint32_t offset_map_offset; // Offset to array of struct {uint32_t offset; uint16_t size;}[max_id - min_id + 1];<br />
uint32_t id_list_size; // List of ids present in the DB file<br />
uint32_t field_storage_info_size;<br />
uint32_t common_data_size;<br />
uint32_t pallet_data_size;<br />
uint32_t relationship_data_size;<br />
};<br />
<br />
struct field_structure<br />
{<br />
uint16_t size;<br />
uint16_t offset;<br />
};<br />
<br />
wdc1_db2_header header;<br />
field_structure fields[header.total_field_count];<br />
if ((header.flags & 1) == 0) {<br />
// Normal records<br />
struct record_data<br />
{<br />
char data[header.record_size];<br />
};<br />
record_data records[header.record_count];<br />
char string_data[header.string_table_size];<br />
} else {<br />
// Offset map records -- these records have null-terminated strings inlined, and<br />
// since they are variable-length, they are pointed to by an array of 6-byte<br />
// offset+size pairs.<br />
char variable_record_data[header.offset_map_offset - sizeof(wdc1_db2_header) - (sizeof(field_structure) * header.total_field_count)];<br />
struct offset_map_entry<br />
{<br />
uint32_t offset;<br />
uint16_t size;<br />
};<br />
offset_map_entry offset_map[header.max_id - header.min_id + 1];<br />
}<br />
<br />
enum field_compression<br />
{<br />
// None -- the field is a 8-, 16-, 32-, or 64-bit integer in the record data<br />
field_compression_none,<br />
// Bitpacked -- the field is a bitpacked integer in the record data. It<br />
// is field_size_bits long and starts at field_offset_bits.<br />
// A bitpacked value occupies<br />
// (field_size_bits + (field_offset_bits & 7) + 7) / 8<br />
// bytes starting at byte<br />
// field_offset_bits / 8<br />
// in the record data. These bytes should be read as a little-endian value,<br />
// then the value is shifted to the right by (field_offset_bits & 7) and<br />
// masked with ((1ull << field_size_bits) - 1).<br />
field_compression_bitpacked,<br />
// Common data -- the field is assumed to be a default value, and exceptions<br />
// from that default value are stored in the corresponding section in<br />
// common_data as pairs of { uint32_t record_id; uint32_t value; }.<br />
field_compression_common_data,<br />
// Bitpacked indexed -- the field has a bitpacked index in the record data.<br />
// This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t, so the index<br />
// should be multiplied by 4 to obtain a byte offset.<br />
field_compression_bitpacked_indexed,<br />
// Bitpacked indexed array -- the field has a bitpacked index in the record<br />
// data. This index is used as an index into the corresponding section in<br />
// pallet_data. The pallet_data section is an array of uint32_t[array_count],<br />
//<br />
field_compression_bitpacked_indexed_array,<br />
};<br />
<br />
struct field_storage_info<br />
{<br />
uint16_t field_offset_bits;<br />
uint16_t field_size_bits;<br />
// additional_data_size is the size in bytes of the corresponding section in<br />
// common_data or pallet_data. These sections are in the same order as the<br />
// field_info, so to find the offset, add up the additional_data_size of any<br />
// previous fields which are stored in the same block (common_data or<br />
// pallet_data).<br />
uint32_t additional_data_size;<br />
field_compression storage_type;<br />
switch (storage_type)<br />
{<br />
case field_compression.field_compression_bitpacked:<br />
uint32_t bitpacking_offset_bits;<br />
uint32_t bitpacking_size_bits;<br />
uint32_t flags; // known values - 0x01: sign-extend (signed)<br />
break;<br />
case field_compression.field_compression_common_data:<br />
uint32_t default_value;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed:<br />
uint32_t bitpacking_offset_bits;<br />
uint32_t bitpacking_size_bits;<br />
uint32_t unk_or_unused3;<br />
break;<br />
case field_compression.field_compression_bitpacked_indexed_array:<br />
uint32_t bitpacking_offset_bits;<br />
uint32_t bitpacking_size_bits;<br />
uint32_t array_count;<br />
break;<br />
default:<br />
uint32_t unk_or_unused1;<br />
uint32_t unk_or_unused2;<br />
uint32_t unk_or_unused3;<br />
break;<br />
}<br />
};<br />
<br />
uint32_t id_list[header.id_list_size / 4];<br />
if (header.copy_table_size > 0) {<br />
struct copy_table_entry<br />
{<br />
uint32_t id_of_new_row;<br />
uint32_t id_of_copied_row;<br />
};<br />
copy_table_entry copy_table[header.copy_table_size / sizeof(copy_table_entry)];<br />
}<br />
field_storage_info field_info[header.field_storage_info_size / sizeof(field_storage_info)];<br />
char pallet_data[header.pallet_data_size];<br />
char common_data[header.common_data_size];<br />
if (header.relationship_data_size > 0) {<br />
// In some tables, this relationship mapping replaced columns that were used<br />
// only as a lookup, such as the SpellID in SpellX* tables.<br />
struct relationship_entry<br />
{<br />
// This is the id of the foreign key for the record, e.g. SpellID in<br />
// SpellX* tables.<br />
uint32_t foreign_id;<br />
// This is the index of the record in record_data. Note that this is<br />
// *not* the record's own ID.<br />
uint32_t record_index;<br />
};<br />
struct relationship_mapping<br />
{<br />
uint32_t num_entries;<br />
uint32_t min_id;<br />
uint32_t max_id;<br />
relationship_entry entries[num_entries];<br />
};<br />
relationship_mapping relationship_map;<br />
}<br />
<br />
==Further Quirks==<br />
The 'field structure' section introduced in WDB5 still exists here, but 'field storage info' has all of the same information and more. You may wonder - why should I even bother reading field structure now if I only care about WDC1 support? There is one good reason - array sizes. In WDC1, the field structure section reports the size of the individual field piece in the file (for compression type '0') in bytes. However, the field storage info section reports the size of the entire field in bits. The difference in these values is important if an array is the field in question. For example, in field structure, an array of uint32[3] would be reported as size '4', while in field storage info, it would be reported as '96'. This technicality greatly benefits us as a trivial bit of math allows you to instantly deduce array sizes when reading the file: 'ArrayLength = (SizeReportedByFieldStorageInfo / 8) / SizeReportedByFieldStructure'.<br />
<br />
On the note of relationship maps, the aspect that makes them tricky (the parsing is actually very simple) is that sometimes the column in question remains in DB2 data in addition to being in a relationship map. For example, Achievement.db2's CriteriaTreeID field was not removed, but it had a relationship map added anyway. Cases like this are difficult to handle properly. You could just allow the duplicate data to be added as a fake column to the end, as there may not be any harm, especially if you are expecting it. Alternatively, you could parse the game executable's DBCMeta - the fields removed by relationship maps are still present there. If there is a mismatch between the number of fields reported by DBCMeta and the number of fields reported by the DB2, then you know that the relationship map data is unique and needs to be restored. If the number of fields match, then the data in the relationship map already exists in the record and is superfluous.<br />
<br />
=Known Flag Meanings for WDB4+=<br />
flags & 0x01 = 'Has offset map'<br />
flags & 0x02 = 'Has secondary key'<br />
flags & 0x04 = 'Has non-inline IDs'<br />
flags & 0x10 = ? // WDC1 specific. used on all WDC1 files<br />
<br />
[[Category:Format]]</div>Simca