CASC

CASC is the name of the new file system that Blizzard has created to replace the outdated format of MPQ.

CASC v1

The CASC file system made its first debut in the Heroes of the Storm Technical Alpha, which was hosted on Blizzard's servers in late January. The form of CASC that Heroes of the Storm uses is designated by Blizzard as "CASC". In contrast, World of Warcraft's "build-playbuild-installer" config line clearly states it is generated by "ngdptool_casc2" (NGDP stands for Next Generation Download Procotol). These are the two most substantial changes between CASC v1 and CASC v2:

• Sections of CASC v1 data files are grouped together in collections of files we call "packages". These packages all have the same root folder, and if all of the files are not properly added with the package's base directory, the extraction process will produce an incredibly mangled directory output. This system is completely removed in CASC v2.
• CASC v1's Root file relates content hashes to file names. CASC v2's Root file relates content hashes to name hashes. Translating name hashes to file names requires use of the Jenkins Hash function [1], which in turn requires a listfile to generate the hashes. Essentially CASC v1 has its own listfile (in root). CASC v2 does not, and requires the user to provide names.

The remainder of this article will refer exclusively to the system called CASC v2 as 'CASC'. While many parts of the file system are identical between v1 and v2, there are enough changes to make explaining both formats at once inadvisable.

NGDP

CASC was introduced simultaneously with a new system for managing configuration, blob, and installation files called NGDP, or Next Generation Download Protocol. When the acronym 'NGDP' is used in conjunction with the term CASC, it is typically referring to the hosted components of the CASC file system, and its ability to stream data on the fly.

NGDP URLs

As of October 14th, 2014, the following generic NGDP URLs are known:

• http://us.patch.battle.net:1119/(ProgramCode)/cdns - a list domains available with game data per region
• http://us.patch.battle.net:1119/(ProgramCode)/versions - a list of the current game version, build config, and cdn config per region
• http://us.patch.battle.net:1119/(ProgramCode)/bgdl - similar to versions, but tailored for use by the Battle.net App background downloader process
• http://us.patch.battle.net:1119/(ProgramCode)/blob/game - a blob file that regulates game functionality for the Battle.net App
• http://us.patch.battle.net:1119/(ProgramCode)/blob/install - a blob file that regulates installer functionality for the game in the Battle.net App

NGDP Program Codes

As of October 14th, 2014, the following program codes are known to support NGDP:

• heroes
• storm - Note: Deprecated
• wow
• wowt
• wow_beta

CASC Online

Standard URL Hash Format

URL Format: http://(cdnsHost)/(cdnsPath)/(pathType)/(FirstTwoHexOfHash)/(SecondTwoHexOfHash)/(FullHash)

For WoW, cdnsHost of dist.blizzard.com.edgesuite.net should always be acceptable, and currently the cdnsPath of "tpr/wow" has never changed. If you have any doubts, check the NGDP URL for 'cdns', which contains both pieces of information.

Known path types are:

• config - contains the three types of config files: Build configs, CDN configs, and Patch configs
• data - contains archives, indexes, and unarchived standalone files (typically binaries, mp3s, and movies)
• patch - contains patch files

Example URL: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/5b/27/5b277d732299a79a935bc5a3f6ed3240

Config Files

Build Config

Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/4d/8b/4d8bb3b6fd0416aa9371a80aaefd2e93

CDN Config

Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/5b/27/5b277d732299a79a935bc5a3f6ed3240

Patch Config

This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 around build 19000 (approximately October 1st, 2014). The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material. The structure and purpose of all of the fields of this file are unknown at this time.

Data Files

Example index: Example archive:

File References

Files are referred to by many different pieces of data in CASC. A quick summary of them:

• Filename: The file's real name. Note that one file can have many names - essentially, one header hash can map to many different name hashes.
• Locale Flag:
• Content Flag:
• Name Hash: The file's name, after being hashed with the Jenkins Hash.
• Header Hash: The MD5 of the BLTE header of the compressed file.
• Content Hash: The MD5 of the entire file in its uncompressed state; the purest representation of the data.

States of CASC Data

CASC data comes in all forms and sizes.

Key CASC Files

Root

File signature: None The purpose of Root is to translate Content Hashes into file names

Encoding

File signature: "EN"

Encoding Header Structure

• The beginning of the file is compromised of this structure of 0x16 bytes. Structure names were invented by the author of this page.

Offset (Hex)	Type	Name	Description
0x00	char[2]	FileSignature	"EN"
0x02	uint8_t	UNK	???
0x03	uint8_t	checksumSizeA	The length of the checksums in table A.
0x04	uint8_t	checksumSizeB	The length of the checksums in table B.
0x05	uint16_t	flagsA	Flags for table A.
0x07	uint16_t	flagsB	Flags for table B.
0x09	uint32_t [BE]	numEntriesA	The number of entries in table A.
0x0D	uint32_t [BE]	numEntriesB	The number of entries in table B.
0x11	uint8_t	UNK	???
0x12	uint32_t [BE]	stringBlockSize	The size of the string block.

Install

File signature: "IN"

Install Header Structure

• The beginning of the file is compromised of this structure of 0x0A bytes. Structure names were invented by the author of this page.

Offset (Hex)	Type	Name	Description
0x00	char[2]	FileSignature	"IN"
0x02	uint32	UNK	???
0x06	uint32	numEntries	The number of entries in the body of the file

Install Header Entry Structure

• The remainder of the header is populated by these header entries, each a variable size (due to the strings). Structure names were invented by the author of this page.

Type	Name	Description
char[]	FlagName	The name of the optional flag for the entry
uint16	FlagType	A number shared amongst specific flags. For example, languages are '3'. Regions are '5'. Architecture type is '0'.
byte[28]	FileFlags	This appears to be a bit array represented in hex form. Each bit appears to represent an entry of this file; if the bit is enabled, then the flag named by FlagName is active for that file.

Install Entry Structure

• The rest of the file is populated by these normal entries, each a variable size (due to the strings). Structure names were invented by the author of this page.

Type	Name	Description
char[]	FileName	The name of the file.
char[16]	MD5	The MD5 of the uncompressed (?) file.
byte[28]	Size	The size of the file.

Download

File signature: "DL"

Patch

File signature: "PA" The structure and purpose of all of the fields of this file are unknown at this time.

Blizzard-Created Archives

In its natural state, the vast majority of the data for any CASC-based game exists in the archives.

Archives

Archives are extensionless 256 MB files that are usually only stored on the Blizzard CDNs. Their naming follows the standard URL hash format using the '/data/' path type.

The structure of the archives is presumably just file fragment after file fragment. You will never need to parse it because you can just look up offset + size of your file fragment in the index files and then take the piece directly out of the archive.

Archive Indexes (.index)

These '.index' files reveal to the user where the compressed game files are located within the archives. All indexes (except the Archive-Group index, see below) are named after their archive (only difference is these have an extension). '.index' files are stored on the CDN using the standard hash naming scheme (remember they have an extension though). They are also located in the directory 'INSTALL_DIR/Data/indices/' for a WoW install.

Normal Index Entry Structure

• The entire file is populated by these standard index entries of 0x18 (hex) bytes. Structure names were invented by the author of this page.
• NOTE: This structure uses big endian numbers.

Offset (Hex)	Type	Name	Description
0x00	char[16]	HeaderHash	The MD5 of the BLTE header for the compressed fragment that this index entry represents
0x10	uint32	Offset	Position of the fragment in the archive
0x14	uint32	Size	Size of the fragment

Archive-Group Index (.index)

Archive-group is actually a very special '.index' file. While virtually all '.index' files are under 2 MB, the archive-group '.index' file is always over 15 MB. It is essentially a merger of all .index files, with a structure change. There is a new uint16 field that serves as an index for the array of archives from this build's CDN config.

Therefore, it is critical that you identify this outlier - if you try to parse it as a regular '.index' purely because of its extension, your program will undoubtedly fail. You can identify it because it will be named the same as the 'archive-group' hash listed in the CDN config. Additionally, it will not be listed as an archive hash in the CDN config. As discussed before, the different file structure and irregular file size are also viable methods to avoid parsing this file (or to avoid parsing the other '.index' files).

Merged Index Entry Structure

• The entire file is populated by these 'merged' index entries of 0x1A (hex) bytes. Structure names were invented by the author of this page.
• NOTE: This structure uses big endian numbers.

Offset (Hex)	Type	Name	Description
0x00	char[16]	HeaderHash	The MD5 of the BLTE header for the compressed fragment that this index entry represents
0x10	uint16	ArchiveIndex	If you placed the hashes of the 'archives = ' line of the CDN config in an array, this number would be the index for that array
0x12	uint32	Offset	Position of the fragment in the archive
0x16	uint32	Size	Size of the fragment

Journal-based Data Files

During the installation process for a Blizzard game, the program will download the required files as requested by root, encoding, download, and install. It stores the downloaded data fragments in data files in "INSTALL_DIR\Data\data\". The program will record the content hash (BLTE-compressed hash), size, and position of the file as well as the number of the data file that it is in. It places those four parameters into journal files with the extension '.idx'.

.IDX Journals

Example file path: INSTALL_DIR\Data\data\0e00000054.idx

.IDX journals contain references. There used to be one .IDX file per journal, and the naming scheme used to have two separate meanings. The '0e' part of the file name used to designate which archive the .IDX file was associated with. This changed halfway through the Warlords Beta, and the current .IDX names are just iteration numbers.

.IDX Header Structure

???

.IDX Entry Structure

• The rest of the file is populated by these normal entries, each 0x10 bytes in size. Structure names were invented by the author of this section because official names were not available.
• Note: .IDX files are chunked into groups of 0x1000 bytes. If a chunk is not filled to exactly 0x1000 bytes, the gap will be filled with '00's.

Offset (Hex)	Type	Name	Description
0x00	char[9]	HeaderHash	The MD5 of the BLTE header of the compressed file
0x09	uint10*	DataNumber	The number of the data file to read from
0x10.25	uint30*	Offset	The position to begin reading from in the data file
0x14	uint32	Size	The amount to read from the data file

• * designates unusual data types. It is probably easiest to read the DataNumber as a Byte (and put it into a UInt16) and the Offset as a UInt32. Then use bit-shifting and a mask on Offset to update DataNumber and apply a mask to update Offset.

.XXX Data Files

Example file path: INSTALL_DIR\Data\data\data.015