ADB

From wowdev
Jump to: navigation, search
This section only applies to versions ≥ Cata.

The structures described here are used in ADB files, which are a cache of dynamically streamed database entries, typically used for hotfixes and content that Blizzard wants to hide from dataminers until the last minute.

For more information, please check out the page on DB2 files. This article assumes you already have a basic understanding of how they function.

Table content structures

This page describes the structure of ADB files. For a list of existing DB2 files (and therefore ADB files) and their contents, see the categories DBC, Vanilla, Burning Crusade, Wrath of the Lich King, Cataclysm, Mists of Pandaria and Warlords of Draenor. If you add documentation for a file, please add the correct categories (also the build number) as well.

WCH3

This section only applies to versions Cata … < Legion (7.0.1.20740).

This file format was introduced in Cataclysm. It was phased out in favor of WCH4 in Legion (Patch 7.0.1 build 20740).

Structure

See DB2#WDB2 for how to adapt this structure for its .DB2 counterpart.

struct adb_header
{
  uint32_t magic;                                               // 'WCH3'
  uint32_t record_count;
  uint32_t field_count;                                         // array fields count only count as '1' field
  uint32_t record_size;
  uint32_t string_table_size;                                   // string block almost always contains at least one zero-byte
  uint32_t table_hash;
  uint32_t build;
  uint32_t timestamp_last_written;                              // set to time(0); when writing in WowClientDB2_Base::Save()
  uint32_t min_id;
  uint32_t max_id;
  uint32_t locale;                                              // as seen in TextWowEnum
  uint32_t copy_table_size;                                     // always zero for ADBs
};

template<typename record_type>
struct adb_file
{
  adb_header header;
  // static_assert (header.record_size == sizeof (record_type));

  if (header.max_id != 0)
  {
    int indices[header.max_id - header.min_id + 1];             // maps from id to row index in records[] below
    short string_lengths[header.max_id - header.min_id + 1];    // sum of lengths of all strings in row
  }
 
  record_type records[header.record_count];*
  char string_table[header.string_table_size];
};
  • Note: Each record is aligned to the length of the longest field type in the record. If you have a record with three fields: Int32, Int8, and Int8 - then there will be 2 bytes of padding at the end of every record, including the last one. 'record_size' should account for this (it would be '8' in the example given in the previous sentence).

WCH4

This section only applies to versions Legion (7.0.1.20740) … < Legion (7.0.1.20810).

This file format was introduced in Legion (Patch 7.0.1 build 20740). It was phased out in favor of WCH5 just a few builds later, also in Legion (Patch 7.0.1 build 20810). As such, support for WCH4 is likely important for almost nobody.

The major changes are the addition of an optional offset map which forces records to have inline strings and an optional block after the string block that contains non-inline IDs.

It is worth noting that min_id and max_id fields now see different use, despite the header remaining the same. min_id and max_id are always non-zero, even when the offset_map structure is present.

Note that, in most cases, the non-inline IDs structure still exists, even if it is made completely redundant by the existence of the offset map. After all, why do work once when you could do it twice?

Structure

See DB2#WDB3 for how to adapt this structure for its .DB2 counterpart.

template<typename record_type>
struct wch4_file
{
  adb_header header;
  struct offset_map_entry
  {
    uint32_t ID;
    uint32_t offset;                                             // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on
    uint16_t length;                                             // this is the length of the record located at the specified offset
  };
  offset_map_entry offset_map[header.max_id - header.min_id + 1];*
  record_type records[header.record_count]; 
  char string_table[header.string_table_size];
  uint32_t IDs[header.record_count];*
};

Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string block usually still exists, just as size 2 with two blank entries.

  • This part of the structure is optional.

WCH5

This section only applies to versions Legion (7.0.1.20810) … < Legion (7.0.3.22345).

This file format was introduced in Legion (Patch 7.0.1 build 20810) and was phased out in favor of WCH6 in Legion (Patch 7.0.3 Build 21414). As such, support for WCH5 is likely important for almost nobody.

While not a format change, the string table is now always 0 bytes if the offset map is present. In those cases, the 'string_table_size' header value changes its meaning to "offset to the start of the offset map". This is a minor optimization since parsing the file must begin with the offset map.

It's worth noting that WCH5 is missing two header fields compared to its .DB2 counterpart. Since the deduplication technology we call 'copy_table' is not employed in any of the .ADB formats, copy_table_size variable was always zero anyway. As for flags, Blizzard always loads the .DB2 file before loading the .ADB file, so the flags field being in both files is just redundant (for them); any .ADB file will be read using the flags field from the .DB2 file of the same name. This is inconvenient for us, as most (if not all) of us load .ADB files standalone at the moment, but there are workarounds (such as having a configuration file to remember the last used flags value for each named .DB2 file).

After Legion build 21737, an additional header field was added (right before 'build') called 'layout_hash'. This field's appearance is similar to the change to the WDB5 header made in the same build, except in that case, the 'build' field was removed when the 'layout_hash' field was added. For the post-21737 WCH5 header, both fields are present.

Structure

See DB2#WDB4 or DB2#WDB5 for how to adapt this structure for its .DB2 counterpart.

struct wch5_adb_pre_21737_header
{
  uint32_t magic;                                               // 'WCH5'
  uint32_t record_count;
  uint32_t field_count;                                         // array fields count only count as '1' field
  uint32_t record_size;
  uint32_t string_table_size;                                   // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map
  uint32_t table_hash;
  uint32_t build;
  uint32_t timestamp_last_written;                              // set to time(0); when writing in WowClientDB2_Base::Save()
  uint32_t min_id;
  uint32_t max_id;
  uint32_t locale;                                              // as seen in TextWowEnum
};

struct wch5_adb_post_21737_header
{
  uint32_t magic;                                               // 'WCH5' for .adb (cache)
  uint32_t record_count;
  uint32_t field_count;                                         // array fields count as the size of array for WDB4 but the entire array only counts as '1' field for WCH5
  uint32_t record_size;
  uint32_t string_table_size;                                   // if flags & 0x01 != 0, this field takes on a new meaning - it becomes an absolute offset to the beginning of the offset_map
  uint32_t table_hash;
  uint32_t layout_hash;                                         // from build 21737 onward, this field is present
  uint32_t build;
  uint32_t timestamp_last_written;                              // set to time(0); when writing in WowClientDB2_Base::Save()
  uint32_t min_id;
  uint32_t max_id;
  uint32_t locale;                                              // as seen in TextWowEnum
};

template<typename record_type>
struct wch5_file
{
  wch5_adb_post_21737_header header;
  struct offset_map_entry                                        // Optional - can be detected by pulling 'flags' from relevant DB2
  {
    uint32_t ID;
    uint32_t offset;                                             // this offset is absolute, not relative to another structure; this can (and often will) be zero, in which case you should ignore that entry and move on
    uint16_t length;                                             // this is the length of the record located at the specified offset
  };
  offset_map_entry offset_map[header.max_id - header.min_id + 1];*
  record_type records[header.record_count]; 
  char string_table[header.string_table_size];
  uint32_t IDs[header.record_count];                             // Optional - can be detected by pulling 'flags' from relevant DB2
};

Note: If offset_map exists, all the strings will be embedded inline in the records (as null-terminated c-strings). The string_table will not exist.

WCH6

This section only applies to versions Legion (7.0.3.22345) … < Legion (7.0.3.22451).

This file format was introduced in Legion (Patch 7.0.3 build 22345) and was phased out in favor of WCH7 in Legion (Patch 7.0.3 Build 22451).

Literally nothing changed in WCH6. The introduction of this incremented signature was probably a temporary countermeasure to combat a problem Blizzard was having with 'build' not being updated successfully. This would leave ADBs around basically forever (since they are only wiped when current build is greater than the one in the ADBs being loaded), which could cause conflicts if you downloaded an updated client with new DB2s.

WCH7

This section only applies to versions Legion (7.0.3.22451) … < Legion (7.0.3.22484).

This file format was introduced in Legion (Patch 7.0.3 build 22451) and was phased out in favor of WCH8 in Legion in the next build (Patch 7.0.3 Build 22484).

A new header field has been added between 'record_count' and 'field_count'. This field designates the size of a new table which is located between the string table and the non-inline IDs table. All records are padded to 'record_size'.

WCH8

This section only applies to versions Legion (7.0.3.22484) … < Legion (7.2.0.23436).

This file format was introduced in Legion (Patch 7.0.3 build 22484) and is still in use today. It was used alongside WDB5.

The only difference to WCH7 is the padding of files with an offset map. These are padded as per the WCH5 specification again; presumably as this was unnecessarily changed in WCH7.

DBCache.bin

This section only applies to versions ≥ Legion.

WDB5 and WCH8 were retired simultaneously in Patch 7.2.0 Build 23436. All files in the 'ADB' folder were consolidated into a single new file, DBCache.bin. It contains server-provided new and changed records for all client-side database tables. It even sometimes contains server-side database changes (entries with an unknown 'table_hash'). It is theorized that the file is a sort of 'packet replayer' for the WoW executable, used for updating client-side DB2s to latest hotfixes on boot up, before confirming with the server about the newest changes.

The file uses the same record structures as the WoW binary requires but with none of the mechanics found in newer DB2 formats. There is no offset map, field compression of any kind, and no deduplication. Strings are inline, and IDs are always provided before the record, regardless of whether or not there is an ID value in the record data additionally.

Version 1

This section only applies to versions Legion (7.2.0.23436) … < Legion (7.2.0.23780).
struct dbcache_file
{
  struct dbcache_file_header
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    uint32_t version;
    uint32_t build_id;
  };
  
  struct dbcache_entry
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    int32_t index;
    uint32_t data_size;
    uint32_t table_hash;
    uint32_t record_id;
    uint8_t is_valid;                                             // Some kind of boolean probably - it seems to be always 0 or 1
    uint8_t padd[3];
    if (data_size) uint8_t record[data_size];
  };
  
  dbcache_file_header header;
  while (!FEof())                                                 // This is code for '010 Editor' that means 'keep reading until the end of the file'
  {
    dbcache_entry entry;
  }
};

Version 2 through Version 4

This section only applies to versions Legion (7.2.0.23780) … < Legion (7.2.0.23436).

Version 2 added a new field directly after 'magic' in 'dbcache_entry'. At first, it was thought that the new field was just an expansion of the signature, as the field looked like 'XFTH2' for a long time. However, it appears to be something different.

There were no structure changes in Version 3 or Version 4.

struct dbcache_file_v2
{
  struct dbcache_file_header
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    uint32_t version;
    uint32_t build_id;
  };
  
  struct dbcache_entry_v2
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    uint32_t region_ish;                                          // 3 for eu, 4 for ptr, apparently 1 for us
    int32_t index;
    uint32_t data_size;
    uint32_t table_hash;
    uint32_t record_id;
    uint8_t is_valid;                                             // Some kind of boolean probably - it seems to be always 0 or 1
    uint8_t padd[3];
    if (data_size) uint8_t record[data_size];
  };
  
  dbcache_file_header header;
  while (!FEof())                                                 // This is code for '010 Editor' that means 'keep reading until the end of the file'
  {
    dbcache_entry_v2 entry;
  }
};

Version 5

This section only applies to versions ≥ Legion (7.2.0.23436).

Version 5 added a new field directly after 'build_id' in 'dbcache_file_header'. It is a simple SHA256 hash for the file's contents (minus the header or with the header having 00s in that area, not sure which).

struct dbcache_file_v5
{
  struct dbcache_file_header_v5
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    uint32_t version;
    uint32_t build_id;
    uint8_t sha256[32];
  };
  
  struct dbcache_entry_v2
  {
    uint32_t magic;                                               // 'XFTH' ('HoTFiX')
    uint32_t region_ish;                                          // 3 for eu, 4 for ptr, apparently 1 for us
    int32_t index;
    uint32_t data_size;
    uint32_t table_hash;
    uint32_t record_id;
    uint8_t is_valid;                                             // Some kind of boolean probably - it seems to be always 0 or 1
    uint8_t padd[3];
    if (data_size) uint8_t record[data_size];
  };
  
  dbcache_file_header_v5 header;
  while (!FEof())                                                 // This is code for '010 Editor' that means 'keep reading until the end of the file'
  {
    dbcache_entry_v2 entry;
  }
};