CASC: Difference between revisions

From wowdev
Jump to navigation Jump to search
m (Download Entry)
(47 intermediate revisions by 8 users not shown)
Line 24: Line 24:
==NGDP Program Codes==
==NGDP Program Codes==


As of September 14th, 2015, the following program codes are known to support NGDP:
As of May 16th, 2017, the following NGDP program codes are known:
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="90" | Program
! width="90" | Program
! width="250" | Description
! width="250" | Description
! width="120" | Status
|-
|-
| agent || Battle.net Agent
| agent || Battle.net Agent
| style="color: green;" | Active
|-
|-
| bna || Battle.net App
| bna || Battle.net App
| style="color: green;" | Active
|-
|-
| bnt || Heroes of the Storm Alpha (Deprecated)
| bnt || Heroes of the Storm Alpha
| style="color: red;" | Deprecated
|-
|-
| clnt || Client (?)
| catalogs || Catalog
| style="color: green;" | Active
|-
| clnt || Client
| style="color: red;" | Deprecated
|-
|-
| d3 || Diablo 3 Retail
| d3 || Diablo 3 Retail
| style="color: green;" | Active
|-
|-
| d3cn || Diablo 3 China
| d3cn || Diablo 3 China
| style="color: green;" | Active
|-
|-
| d3t || Diablo 3 Test
| d3t || Diablo 3 Test
| style="color: green;" | Active
|-
|-
| demo || ? (Partial)
| demo ||
| style="color: orange;" | Partial
|-
|-
| hero || Heroes of the Storm Retail
| hero || Heroes of the Storm Retail
| style="color: green;" | Active
|-
|-
| herot || Heroes of the Storm Test
| herot || Heroes of the Storm Test
| style="color: green;" | Active
|-
| heroc || Heroes of the Storm Tournament
| style="color: green;" | Active
|-
|-
| hsb || Hearthstone Retail
| hsb || Hearthstone Retail
| style="color: green;" | Active
|-
|-
| hst || Hearthstone Test (Partial)
| hst || Hearthstone Test
| style="color: orange;" | Partial
|-
|-
| pro || Overwatch Retail
| pro || Overwatch Retail
| style="color: green;" | Active
|-
|-
| prot || Overwatch Test
| prot || Overwatch Test
| style="color: green;" | Active
|-
| proc || Overwatch Tournament
| style="color: green;" | Active
|-
|-
| prodev || Overwatch Dev
| prodev || Overwatch Dev
| style="color: blue;" | Active (encrypted)
|-
|-
| sc2 || StarCraft II (Deprecated)
| s1 || StarCraft 1
| style="color: green;" | Active
|-
| s1t || StarCraft 1 Test
| style="color: green;" | Active
|-
| sc2 || StarCraft II
| style="color: red;" | Deprecated
|-
|-
| s2 || StarCraft II Retail
| s2 || StarCraft II Retail
| style="color: green;" | Active
|-
|-
| s2t || StarCraft II Test
| s2t || StarCraft II Test
| style="color: red;" | Deprecated
|-
|-
| s2b || StarCraft II Beta
| s2b || StarCraft II Beta
| style="color: red;" | Deprecated
|-
|-
| test || ? (Partial)
| test ||
| style="color: red;" | Deprecated
|-
|-
| storm || Heroes of the Storm (Deprecated)
| storm || Heroes of the Storm
| style="color: red;" | Deprecated
|-
|-
| war3 || Warcraft III (Partial)
| war3 || Warcraft III
| style="color: orange;"| Partial
|-
|-
| wow || World of Warcraft Retail
| wow || World of Warcraft Retail
| style="color: green; | Active
|-
|-
| wowt || World of Warcraft Test
| wowt || World of Warcraft Test
| style="color: green; | Active
|-
|-
| wow_beta || World of Warcraft Beta
| wow_beta || World of Warcraft Beta
| style="color: green; | Active
|}
|}


Line 161: Line 202:
This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 around build 19000 (approximately October 1st, 2014).
This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 around build 19000 (approximately October 1st, 2014).
The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material.
The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material.
The structure and purpose of all of the fields of this file are unknown at this time.
The structure and purpose of all of the fields of this file requires further research.
 
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="140" | Value name
! width="650" | Description
|-
| patch-entry || Repeats 3 times with patch entries for install, download and encoding file
|-
| patch || Patch file itself (needs research)
|-
| patch-size || Size of patch file (optional?)
|}


Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/b6/c8/b6c844d423c0c3e8620a171828080b06
Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/b6/c8/b6c844d423c0c3e8620a171828080b06
====patch-entry====
The format of these strings is:
<pre>patch-entry = <type> <content hash> <content size> <BLTE-header hash> <BLTE-encoded size> <encoding string></pre>
followed by sets of
<pre><old BLTE-header hash> <old content size> <patch hash> <patch size></pre>
This serves to pair old files to the patch needed to bring them up to date, as well as providing the information required to detect if the file is already up to date.


==Data Files==
==Data Files==
Line 282: Line 346:


== Overwatch ==
== Overwatch ==
Overwatch has keys included in the client, but they can also be streamed from the server.  
Overwatch has keys included in the client/keyring, but they can also be streamed from the server.
  key_name          key                              type     seen in
  key_name          key                              type     seen in         method    used for
  FB680CB6A8BF81F3  62D90EFA7F36D71C398AE2F1FE37BDB9  salsa20   0.8.0.24919 (hardcoded)
  FB680CB6A8BF81F3  62D90EFA7F36D71C398AE2F1FE37BDB9  salsa20 0.8.0.24919     keyring
  402CD9D8D6BFED98  AEB0EADEA47612FE6C041A03958DF241  salsa20   0.8.0.24919 (hardcoded)
  402CD9D8D6BFED98  AEB0EADEA47612FE6C041A03958DF241  salsa20 0.8.0.24919     keyring
  DBD3371554F60306  34E397ACE6DD30EEFDC98A2AB093CD3C  salsa20   0.8.0.24919 (streamed from server)
  DBD3371554F60306  34E397ACE6DD30EEFDC98A2AB093CD3C  salsa20 0.8.0.24919     keyring
  11A9203C9881710A  2E2CB8C397C2F24ED0B5E452F18DC267  salsa20   0.8.0.24919 (streamed from server)
  11A9203C9881710A  2E2CB8C397C2F24ED0B5E452F18DC267  salsa20 0.8.0.24919     keyring
  A19C4F859F6EFA54  0196CB6F5ECBAD7CB5283891B9712B4B  salsa20   0.8.0.24919 (streamed from server)
  A19C4F859F6EFA54  0196CB6F5ECBAD7CB5283891B9712B4B  salsa20 0.8.0.24919     keyring
  87AEBBC9C4E6B601  685E86C6063DFDA6C9E85298076B3D42  salsa20   0.8.0.24919 (streamed from server)
  87AEBBC9C4E6B601  685E86C6063DFDA6C9E85298076B3D42  salsa20 0.8.0.24919     keyring
  DEE3A0521EFF6F03  AD740CE3FFFF9231468126985708E1B9  salsa20   0.8.0.24919 (streamed from server)
  DEE3A0521EFF6F03  AD740CE3FFFF9231468126985708E1B9  salsa20 0.8.0.24919     keyring
  8C9106108AA84F07  53D859DDA2635A38DC32E72B11B32F29  salsa20   0.8.0.24919 (streamed from server)
  8C9106108AA84F07  53D859DDA2635A38DC32E72B11B32F29  salsa20 0.8.0.24919     keyring
  49166D358A34D815  667868CD94EA0135B9B16C93B1124ABA  salsa20   0.8.0.24919 (streamed from server)
  49166D358A34D815  667868CD94EA0135B9B16C93B1124ABA  salsa20 0.8.0.24919     keyring
  1463A87356778D14  69BD2A78D05C503E93994959B30E5AEC  salsa20   (streamed from server)
  1463A87356778D14  69BD2A78D05C503E93994959B30E5AEC  salsa20 ≤ 1.0.3.0      keyring
  5E152DE44DFBEE01  E45A1793B37EE31A8EB85CEE0EEE1B68  salsa20   (streamed from server)
  5E152DE44DFBEE01  E45A1793B37EE31A8EB85CEE0EEE1B68  salsa20 ≤ 1.0.3.0      keyring
  9B1F39EE592CA415  54A99F081CAD0D08F7E336F4368E894C  salsa20   (streamed from server)
  9B1F39EE592CA415  54A99F081CAD0D08F7E336F4368E894C  salsa20 ≤ 1.0.3.0      keyring
  24C8B75890AD5917  31100C00FDE0CE18BBB33F3AC15B309F  salsa20   (included in game)
  24C8B75890AD5917  31100C00FDE0CE18BBB33F3AC15B309F  salsa20 ≤ 1.0.3.0      keyring
  EA658B75FDD4890F  DEC7A4E721F425D133039895C36036F8  salsa20   (included in game)
  EA658B75FDD4890F  DEC7A4E721F425D133039895C36036F8  salsa20 ≤ 1.0.3.0      keyring
  026FDCDF8C5C7105  8F41809DA55366AD416D3C337459EEE3  salsa20   (included in game)
  026FDCDF8C5C7105  8F41809DA55366AD416D3C337459EEE3  salsa20                 keyring
  CAE3FAC925F20402  98B78E8774BF275093CB1B5FC714511B  salsa20   (included in game)
  CAE3FAC925F20402  98B78E8774BF275093CB1B5FC714511B  salsa20                 keyring
  061581CA8496C80C  DA2EF5052DB917380B8AA6EF7A5F8E6A  salsa20
  061581CA8496C80C  DA2EF5052DB917380B8AA6EF7A5F8E6A  salsa20                 keyring
  BE2CB0FAD3698123  902A1285836CE6DA5895020DD603B065  salsa20
  BE2CB0FAD3698123  902A1285836CE6DA5895020DD603B065  salsa20                 keyring
  57A5A33B226B8E0A  FDFC35C99B9DB11A326260CA246ACB41  salsa20   1.1.0.0.30200 (Ana)
  57A5A33B226B8E0A  FDFC35C99B9DB11A326260CA246ACB41  salsa20 1.1.0.0.30200   keyring  Ana
  42B9AB1AF5015920  C68778823C964C6F247ACC0F4A2584F8  salsa20   1.2.0.1.30684 (Summer Games)
  42B9AB1AF5015920  C68778823C964C6F247ACC0F4A2584F8  salsa20 1.2.0.1.30684   keyring  Summer Games
  4F0FE18E9FA1AC1A  89381C748F6531BBFCD97753D06CC3CD  salsa20   1.2.0.1.30684
  4F0FE18E9FA1AC1A  89381C748F6531BBFCD97753D06CC3CD  salsa20 1.2.0.1.30684   keyring
  7758B2CF1E4E3E1B  3DE60D37C664723595F27C5CDBF08BFA  salsa20   1.2.0.1.30684
  7758B2CF1E4E3E1B  3DE60D37C664723595F27C5CDBF08BFA  salsa20 1.2.0.1.30684   keyring
  E5317801B3561125  7DD051199F8401F95E4C03C884DCEA33  salsa20   1.4.0.2.32143 (Halloween Terror)
  E5317801B3561125  7DD051199F8401F95E4C03C884DCEA33  salsa20 1.4.0.2.32143   keyring  Halloween Terror
  16B866D7BA3A8036  1395E882BF25B481F61A4D621141DA6E  salsa20   1.4.1.0.31804 (Bastion Blizzcon 2016 skin)
  16B866D7BA3A8036  1395E882BF25B481F61A4D621141DA6E  salsa20 1.4.1.0.31804   keyring  Bastion Blizzcon 2016 skin
  11131FFDA0D18D30  C32AD1B82528E0A456897B3CE1C2D27E  salsa20   1.5.0.1.32795 (Sombra)
  11131FFDA0D18D30  C32AD1B82528E0A456897B3CE1C2D27E  salsa20 1.5.0.1.32795   keyring  Sombra
  CAC6B95B2724144A  73E4BEA145DF2B89B65AEF02F83FA260  salsa20   1.5.0.1.32795 (Ecopoint: Antarctica)
  CAC6B95B2724144A  73E4BEA145DF2B89B65AEF02F83FA260  salsa20 1.5.0.1.32795   keyring  Ecopoint: Antarctica
  B7DBC693758A5C36  BC3A92BFE302518D91CC30790671BF10  salsa20   1.5.0.1.32795 (Genji Oni skin)
  B7DBC693758A5C36  BC3A92BFE302518D91CC30790671BF10  salsa20 1.5.0.1.32795   keyring  Genji Oni skin
  90CA73B2CDE3164B  5CBFF11F22720BACC2AE6AAD8FE53317  salsa20   1.6.1.0.33236 (Oasis map)
  90CA73B2CDE3164B  5CBFF11F22720BACC2AE6AAD8FE53317  salsa20 1.6.1.0.33236   keyring  Oasis map
  6DD3212FB942714A  E02C1643602EC16C3AE2A4D254A08FD9  salsa20   1.6.1.0.33236  
  6DD3212FB942714A  E02C1643602EC16C3AE2A4D254A08FD9  salsa20 1.6.1.0.33236   keyring
  11DDB470ABCBA130  66198766B1C4AF7589EFD13AD4DD667A  salsa20   1.6.1.0.33236 (Winter Wonderland)
  11DDB470ABCBA130  66198766B1C4AF7589EFD13AD4DD667A  salsa20 1.6.1.0.33236   keyring  Winter Wonderland
5BEF27EEE95E0B4B  36BCD2B551FF1C84AA3A3994CCEB033E  salsa20                  keyring
9359B46E49D2DA42  173D65E7FCAE298A9363BD6AA189F200  salsa20                  keyring  Diablo's 20th anniversary
1A46302EF8896F34  8029AD5451D4BC18E9D0F5AC449DC055  salsa20  1.7.0.2.34156  keyring  Year of the Rooster
693529F7D40A064C  CE54873C62DAA48EFF27FCC032BD07E3  salsa20  1.8.0.0.34470  keyring
388B85AEEDCB685D  D926E659D04A096B24C19151076D379A  salsa20  1.8.0.0.34470  keyring
E218F69AAC6C104D  F43D12C94A9A528497971F1CBE41AD4D  salsa20  1.9.0.0.34986  keyring
F432F0425363F250  BA69F2B33C2768F5F29BFE78A5A1FAD5  salsa20  1.10.0.0.35455  cmf
061D52F86830B35D  D779F9C6CC9A4BE103A4E90A7338F793  salsa20  1.10.0.0.35455  cmf
1275C84CF113EF65  CF58B6933EAF98AF53E76F8426CC7E6C  salsa20                  keyring
D9C7C7AC0F14C868  3AFDF68E3A5D63BABA1E6821883F067D  salsa20                  keyring
BD4E42661A432951  ????????????????????????????????  salsa20  1.11.0.0.36376            Unknown event


== World of Warcraft ==
== World of Warcraft ==
WoW's dbfilesclient/tactkey.db2 and dbfilesclient/tactkeylookup.db2 contain keys and key_names (called lookups there) respectively. Either can be streamed from server.
WoW's [[DB/TactKey|TactKey.db2]] and [[DB/TactKeyLookup|TactKeyLookup.db2]] contain keys and key_names (called lookups there) respectively. Either can be streamed from server.
These files cannot be bruteforced by requesting hotfix data (Blizzard has guards in place)
 
The id field in the table below corresponds to the id field in the db2's.
The id field in the table below corresponds to the id field in the db2's.
  key_name          key                              type      id  seen in                  used for
  key_name          key                              type      id  seen in                  used for
Line 325: Line 402:
  FFB9469FF16E6BF8  D514BD1909A9E5DC8703F4B8BB1DFD9A  salsa20  41  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
  FFB9469FF16E6BF8  D514BD1909A9E5DC8703F4B8BB1DFD9A  salsa20  41  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
  23C5B5DF837A226C  1406E2D873B6FC99217A180881DA8D62  salsa20  42  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
  23C5B5DF837A226C  1406E2D873B6FC99217A180881DA8D62  salsa20  42  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
  3AE403EF40AC3037  ????????????????????????????????  salsa20  51
  3AE403EF40AC3037  ????????????????????????????????  salsa20  51 WOW-21249patch7.0.3_Beta
  E2854509C471C554  433265F0CDEB2F4E65C0EE7008714D9E  salsa20  52                           Warcraft movie items
  E2854509C471C554  433265F0CDEB2F4E65C0EE7008714D9E  salsa20  52 WOW-21249patch7.0.3_Beta  Warcraft movie items
  8EE2CB82178C995A  DA6AFC989ED6CAD279885992C037A8EE  salsa20  55                           BlizzCon 2016 Murlocs
  8EE2CB82178C995A  DA6AFC989ED6CAD279885992C037A8EE  salsa20  55 WOW-21531patch7.0.3_Beta  BlizzCon 2016 Murlocs
  5813810F4EC9B005  01BE8B43142DD99A9E690FAD288B6082  salsa20  56                           Fel Kitten
  5813810F4EC9B005  01BE8B43142DD99A9E690FAD288B6082  salsa20  56 WOW-21531patch7.0.3_Beta  Fel Kitten
  7F9E217166ED43EA  05FC927B9F4F5B05568142912A052B0F  salsa20  57  WOW-21531patch7.0.3_Beta
  7F9E217166ED43EA  05FC927B9F4F5B05568142912A052B0F  salsa20  57  WOW-21531patch7.0.3_Beta
  C4A8D364D23793F7  D1AC20FD14957FABC27196E9F6E7024A  salsa20  58  WOW-21691patch7.0.3_Beta  Demon Hunter #1 cinematic (legion_dh1)
  C4A8D364D23793F7  D1AC20FD14957FABC27196E9F6E7024A  salsa20  58  WOW-21691patch7.0.3_Beta  Demon Hunter #1 cinematic (legion_dh1)
Line 348: Line 425:
  460C92C372B2A166  946D5659F2FAF327C0B7EC828B748ADB  salsa20  74  WOW-21952patch7.0.3_Beta  Stormheim Alliance cinematic (legion_g_a_sth)
  460C92C372B2A166  946D5659F2FAF327C0B7EC828B748ADB  salsa20  74  WOW-21952patch7.0.3_Beta  Stormheim Alliance cinematic (legion_g_a_sth)
  8165D801CCA11962  CD0C0FFAAD9363EC14DD25ECDD2A5B62  salsa20  75  WOW-21952patch7.0.3_Beta  Stormheim Horde cinematic (legion_g_h_sth)
  8165D801CCA11962  CD0C0FFAAD9363EC14DD25ECDD2A5B62  salsa20  75  WOW-21952patch7.0.3_Beta  Stormheim Horde cinematic (legion_g_h_sth)
  A3F1C999090ADAC9  ???????????????????????????????? salsa20  81  WOW-22578patch7.1.0_PTR  Firecat Mount
  A3F1C999090ADAC9  B72FEF4A01488A88FF02280AA07A92BB salsa20  81  WOW-22578patch7.1.0_PTR  Firecat Mount
  18AFDF5191923610  ????????????????????????????????  salsa20  82  WOW-22578patch7.1.0_PTR
  18AFDF5191923610  ????????????????????????????????  salsa20  82  WOW-22578patch7.1.0_PTR
3C258426058FBD93  ????????????????????????????????  salsa20  91  WOW-23436patch7.2.0_PTR
094E9A0474876B98  ????????????????????????????????  salsa20  92  WOW-23910patch7.2.5_PTR
3DB25CB86A40335E  02990B12260C1E9FDD73FE47CBAB7024  salsa20  93  WOW-23789patch7.2.0_PTR
0DCD81945F4B4686  1B789B87FB3C9238D528997BFAB44186  salsa20  94  WOW-23789patch7.2.0_PTR
486A2A3A2803BE89  ????????????????????????????????  salsa20  95  WOW-23789patch7.2.0_PTR


=States of CASC Data=
=States of CASC Data=
Line 455: Line 537:
====Encoding Layout Table Entry Block Structure====
====Encoding Layout Table Entry Block Structure====
*'''Each of the tables have numEntries entries of 4096 bytes which contains these structures, followed by padding.'''
*'''Each of the tables have numEntries entries of 4096 bytes which contains these structures, followed by padding.'''
*'''Note:''' Between the data and padding of the last table is an empty hash (0x10 0 bytes) followed by 0xFFFFFFFF.
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="80" | Offset (Hex)
! width="80" | Offset (Hex)
Line 469: Line 552:
| 0x15 || uint32_t [BE] || fileSize || The compressed size of the file.
| 0x15 || uint32_t [BE] || fileSize || The compressed size of the file.
|}
|}


====String blocks====
====String blocks====
Line 536: Line 618:
File signature: "IN"
File signature: "IN"


====Install Header Structure====
The install file lists files installed on disk. Since the install file is shared by architectures and OSses, there are also tags to select a subset of files. When using multiple tags, a binary combination of the bitfields of files to be installed can be created.
*'''The beginning of the file is compromised of this structure of 0x0A bytes. Structure names were invented by the author of this page.'''
 
====Header Structure====
The file begins with a 10 byte header describing the number of tags and files listed. '''Structure names were invented by the author of this page.'''
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="80" | Offset (Hex)
! width="80" | Offset (Hex)
Line 565: Line 649:
| char[] || name ||  
| char[] || name ||  
|-  
|-  
| uint16_BE_t || type || A number shared amongst specific flags. For example, OS = 1, Arch = 2, locale = 3, region = 4, category = 5, alternate = 0x4000
| uint16_BE_t || type || A number shared amongst specific flags. Can change per game/version.
==== World of Warcraft====
 
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18125|max_expansionlevel=6|max_build=6.0.1.18761}}
 
Arch = 1, Locale = 2, OS = 3
 
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.0.1.18764|max_expansionlevel=6|max_build=6.2.2.20426}}
 
Arch = 1, Category = 2, Locale = 3, OS = 4, Region = 5
 
{{Template:SectionBox/VersionRange|min_expansionlevel=6|min_build=6.2.2.20438|max_expansionlevel=7}}
 
OS = 1, Arch = 2, Locale = 3, Region = 4, Category = 5
|-  
|-  
| char[divru (header.entries, CHAR_BIT)] || files || A bitfield that lists which files are installed when the specified tag is installed.
| char[divru (header.entries, CHAR_BIT)] || files || A bitfield that lists which files are installed when the specified tag is installed.
Line 580: Line 677:
| char[] || FileName ||  The name of the file.
| char[] || FileName ||  The name of the file.
|-  
|-  
| char[header.hash_size] || hash || The hash of the uncompressed (?) file. Usually MD5.
| char[header.hash_size] || hash || The hash of the uncompressed file. Usually MD5.
|-  
|-  
| uint32_BE_t || Size || The size of the file.
| uint32_BE_t || Size || The size of the file.
|}
|}
====C-like structure====
char I; char N;
uint8_BE_t _unk3;
uint8_BE_t hash_size;
uint16_BE_t num_tags;
uint32_BE_t num_files;
struct {
  string name;
  uint16_BE_t type;
  char flags[divru (num_files, CHAR_BIT)];
} tags[num_tags];
struct {
  string name;
  char hash[hash_size];
  uint32_BE_t size;
} files[num_files];


===Download===
===Download===
This file has this structure:
This file has this structure:
- Header
 
- Entries[Header.EntryCount]
* Header
- Tags[Header.TagsCount]
* Entries[Header.EntryCount]
* Tags[Header.TagsCount]


====Download Header====
====Download Header====
Line 614: Line 731:
| char[16] || Hash || This hash is found in every node of the encoding file. (Reverse lookup)
| char[16] || Hash || This hash is found in every node of the encoding file. (Reverse lookup)
|-
|-
| char[10] || unk || ???
| char || Unk || ??? (Always 0 7.2.0.24015)
|-
| uint32_t [BE] || FileSize || The compressed size of the file.
|-
| char[5] || Unk || ???
|}
|}


Line 669: Line 790:
  {
  {
   uint16_t_BE magic; // 'PA'
   uint16_t_BE magic; // 'PA'
   uint8_t unk1; // 1
   uint8_t version; // 1 or 2
   uint8_t key_size_a; // <= 0x10
   uint8_t key_size_a; // <= 0x10
   uint8_t size_b; // <= 0x10
   uint8_t size_b; // <= 0x10
Line 680: Line 801:
  #if encoding_information_apparently_added_after_18179
  #if encoding_information_apparently_added_after_18179
  uint8_t encoding_key[16];
  uint8_t encoding_key[16];
// uint8_t encoding_decoded_key[16]; // probably since PA2
  uint32_t_BE decoded_size;
  uint32_t_BE decoded_size;
  uint32_t_BE encoded_size;
  uint32_t_BE encoded_size;
Line 731: Line 853:


====Normal Index Entry Structure====
====Normal Index Entry Structure====
*'''The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null HeaderHash fields. The last chunk is a table-of-contents, listing the LAST HeaderHash in each chunk, as well as some unknown footer data. Structure names were invented by the author of this page.'''
*'''The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null HeaderHash fields. The last chunk is a table-of-contents, listing the LAST HeaderHash in each chunk. If there is more than one block then the table-of-contents also includes an array of the lower md5 of each block excluding the final one. Structure names were invented by the author of this page.'''
*'''NOTE: This structure uses big endian numbers.'''
*'''NOTE: This structure uses big endian numbers.'''
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
Line 748: Line 870:
====Structure====
====Structure====
  struct index_entry {
  struct index_entry {
   char blte_header_hash[0x10];
   char blte_header_hash[footer.keySizeInBytes]; // sizes are encoded in footer
   uint32_BE_t blte_encoded_size;
   uint32_BE_t blte_encoded_size;
   uint32_BE_t offset_to_blte_encoded_data_in_archive;
   uint32_BE_t offset_to_blte_encoded_data_in_archive;
  };
  };
  struct index_block {
  struct index_block {
   static constexpr const block_size = 0x1000;
   static constexpr const block_size = 0x1000;
Line 757: Line 880:
   char padding[block_size - sizeof (entries)];
   char padding[block_size - sizeof (entries)];
  } blocks[];
  } blocks[];
struct {
  struct {
    char last_hash[footer.keySizeInBytes]; // last hash of a block
  } entries[num_blocks];
  struct {
    char lower_part_of_md5_of_block[footer.checksumSize];
  } blocks_hash[num_blocks - 1]; // last block is not included
} toc;
struct {
  char index_blocks_hash[checksumSize]; // client tries to read with 0x10, then backs off when smaller
  char toc_hash[checksumSize];
  char _10; // always 1
  char _11;
  char _12;
  char _13;
  char offsetBytes;
  char sizeBytes;
  char keySizeInBytes;
  char checksumSize; // <= 0x10
  uint32_t numElements; // BigEndian in _old_ versions (e.g. 18179)
  char lower_part_of_md5_of_footer[checksumSize];
} footer;
* x = _lower_part_of_md5_of_footer; _lower_part_of_md5_of_footer = 0, qword (md5 {&_10, &_10+0x14}) == x
* md5 {last 0x14+checksumSize=0x1c bytes of index file} == archive_name
* qword (md5 {&blocks, num_blocks * block_size}) == index_blocks_hash (i.e. without footer and toc)
* qword (md5 {&toc + &index_blocks_hash}) == toc_hash (lower part of md5 of toc data concatenated with the index_blocks_hash)


===Archive-Group Index (.index)===
===Archive-Group Index (.index)===
Line 882: Line 1,035:
Example file path: INSTALL_DIR\Data\data\0e00000054.idx
Example file path: INSTALL_DIR\Data\data\0e00000054.idx


.IDX journals contain references. There used to be one .IDX file per journal, and the naming scheme used to have two separate meanings. The '0e' part of the file name used to designate which archive the .IDX file was associated with. This changed halfway through the Warlords Beta, and the current .IDX names are just iteration numbers.
.IDX journals contain a mapping from keys to the location of their data in the local CASC archives. There used to be one .IDX file per journal, and the naming scheme used to have two separate meanings. The '0e' part of the file name used to designate which archive the .IDX file was associated with. This changed halfway through the Warlords Beta.  Now there are 16 indices total, and the first byte of the hex filename says which of the 16 indices it is, while the remainder of the hex filename is just a version number that increments when a new set of files is added to the local archives.
 
To determine which of the 16 indices a key is bucketed in, the key is hashed by xoring together each 4-bit nibble in the first 9 bytes of the key:
 
  uint8_t cascGetBucketIndex(const uint8_t k[16]) {
    uint8_t i = k[0] ^ k[1] ^ k[2] ^ k[3] ^ k[4] ^ k[5] ^ k[6] ^ k[7] ^ k[8];
    return (i & 0xf) ^ (i >> 4);
  }
 


====.IDX Header Structure====
====.IDX Header Structure====
???
 
The header is little-endian:
 
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="80" | Offset (Hex)
! width="70" | Type
! width="90" | Name
! width="900" | Description
|-
| 0x00 || uint32 || HeaderHashSize || The number of bytes to use for the hash at +04; usually 0x10.
|-
| 0x04 || uint32 || HeaderHash || This should equal the value of pc after calling hashlittle2 on the following HeaderHashSize bytes of the file with an initial value of 0 for pb and pc.
|-
| 0x08 || uint16 || Unk0 || Must be 7
|-
| 0x0a || uint8 || BucketIndex || The bucket index of this file; should be the same as the first byte of the hex filename.
|-
| 0x0b || uint8 || Unk1 || Must be 0
|-
| 0x0c || uint8 || EntrySizeBytes || Must be 4
|-
| 0x0d || uint8 || EntryOffsetBytes || Must be 5
|-
| 0x0e || uint8 || EntryKeyBytes || Must be 9
|-
| 0x0f || uint8 || ArchiveFileHeaderBytes || Must be 30
|-
| 0x10 || uint64 || ArchiveTotalSizeMaximum || The maximum size of a casc installation; 0x4000000000, or 256GiB.
|-
| 0x18 || char[8] || padding || The header is padded with zeroes to the next 0x10-byte boundary.
|-
| 0x20 || uint32 || EntriesSize || This is the length in bytes of the entries in the index file.
|-
| 0x24 || uint32 || EntriesHash || This should equal the value of pc after calling hashlittle2 on the following EntriesSize bytes of the file with an initial value of 0 for pb and pc.
|}


====.IDX Entry Structure====
====.IDX Entry Structure====
*'''The rest of the file is populated by these normal entries, each 0x10 bytes in size. Structure names were invented by the author of this section because official names were not available.'''
*'''The rest of the file is populated by these normal entries, each 0x12 bytes in size. Structure names were invented by the author of this section because official names were not available.'''
*'''Note: .IDX files are chunked into groups of 0x1000 bytes. If a chunk is not filled to exactly 0x1000 bytes, the gap will be filled with '00's.'''
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
{| border="1" cellpadding="2" style="background:#FCFCFC; color:black"
! width="80" | Offset (Dec)
! width="80" | Offset (Dec)
Line 896: Line 1,090:
! width="900" | Description
! width="900" | Description
|-
|-
| 00 || char[9] || HeaderHash || The MD5 of the BLTE header of the compressed file
| 00 || char[9] || Key || The first 9 bytes of the key for this entry.
|-
| 09 || uint10* || DataNumber || The number of the data file to read from
|-  
|-  
| 10.25 || uint30* || Offset || The position to begin reading from in the data file
| 09 || uint40* || Offset || Unlike the other little-endian integers in this file, this is a big-endian 5-byte integer.  The top 10 bits are the number of the archive (data.%03d), and the bottom 30 bits are the offset in that archive to the file data.
|-  
|-  
| 14 || uint32 || Size || The amount to read from the data file
| 14 || uint32 || Size || The length of the file in bytes.
|}
|}
*'''* designates unusual data types. It is probably easiest to read the DataNumber as a Byte (and put it into a UInt16) and the Offset as a UInt32. Then use bit-shifting and a mask on Offset to update DataNumber and apply a mask to update Offset.
*'''* designates unusual data types. In C#, you can read the Offset by reading a Byte, reading a big-endian UInt32, shifting the byte left 32 bits, and ORing them together.  Use a 30-bit mask (0x3fffffff) to get the file offset, and right shift the value 30 bits to get the archive number.


===.XXX Data Files===
===.XXX Data Files===

Revision as of 17:18, 22 May 2017

CASC is the name of the new file system that Blizzard has created to replace the outdated format of MPQ.

CASC v1

The CASC file system made its first debut in the Heroes of the Storm Technical Alpha, which was hosted on Blizzard's servers in late January. The form of CASC that Heroes of the Storm uses is designated by Blizzard as "CASC". In contrast, World of Warcraft's "build-playbuild-installer" config line clearly states it is generated by "ngdptool_casc2" (NGDP stands for Next Generation Download Procotol). These are the two most substantial changes between CASC v1 and CASC v2:

  • Sections of CASC v1 data files are grouped together in collections of files we call "packages". These packages all have the same root folder, and if all of the files are not properly added with the package's base directory, the extraction process will produce an incredibly mangled directory output. This system is completely removed in CASC v2.
  • CASC v1's Root file relates content hashes to file names. CASC v2's Root file relates content hashes to name hashes. Translating name hashes to file names requires use of the Jenkins Hash function [1], which in turn requires a listfile to generate the hashes. Essentially CASC v1 has its own listfile (in root). CASC v2 does not, and requires the user to provide names.

The remainder of this article will refer exclusively to the system called CASC v2 as 'CASC'. While many parts of the file system are identical between v1 and v2, there are enough changes to make explaining both formats at once inadvisable.

NGDP

CASC was introduced simultaneously with a new system for managing configuration, blob, and installation files called NGDP, or Next Generation Download Protocol. When the acronym 'NGDP' is used in conjunction with the term CASC, it is typically referring to the hosted components of the CASC file system, and its ability to stream data on the fly.

NGDP URLs

As of October 14th, 2014, the following generic NGDP URLs are known:

Keep in mind Blizzard's CDN is pretty shit at caching sometimes so you after an update to the above files it might switch back and forth between the old and new version of the files for a few hours.

NGDP Program Codes

As of May 16th, 2017, the following NGDP program codes are known:

Program Description Status
agent Battle.net Agent Active
bna Battle.net App Active
bnt Heroes of the Storm Alpha Deprecated
catalogs Catalog Active
clnt Client Deprecated
d3 Diablo 3 Retail Active
d3cn Diablo 3 China Active
d3t Diablo 3 Test Active
demo Partial
hero Heroes of the Storm Retail Active
herot Heroes of the Storm Test Active
heroc Heroes of the Storm Tournament Active
hsb Hearthstone Retail Active
hst Hearthstone Test Partial
pro Overwatch Retail Active
prot Overwatch Test Active
proc Overwatch Tournament Active
prodev Overwatch Dev Active (encrypted)
s1 StarCraft 1 Active
s1t StarCraft 1 Test Active
sc2 StarCraft II Deprecated
s2 StarCraft II Retail Active
s2t StarCraft II Test Deprecated
s2b StarCraft II Beta Deprecated
test Deprecated
storm Heroes of the Storm Deprecated
war3 Warcraft III Partial
wow World of Warcraft Retail Active
wowt World of Warcraft Test Active
wow_beta World of Warcraft Beta Active

CASC Online

Standard URL Hash Format

URL Format: http://(cdnsHost)/(cdnsPath)/(pathType)/(FirstTwoHexOfHash)/(SecondTwoHexOfHash)/(FullHash)

Current CDNs for a specific program can be found in the http://us.patch.battle.net:1119/(ProgramCode)/cdns file. Blizzard regularly shuffles around CDNs and sometimes even adds/removes CDNs, so be sure to parse the CDNs file to stay up to date. For WoW, the cdnsPath of "tpr/wow" has never changed.

Known path types are:

  • config - contains the three types of config files: Build configs, CDN configs, and Patch configs
  • data - contains archives, indexes, and unarchived standalone files (typically binaries, mp3s, and movies and files mentioned in buildconfig like root, install and download)
  • patch - contains patch archives and indexes

Blizzard regularly cleans old builds from the CDN so any example files mentioned in this article might be unavailable at the time of reading (feel free to update them to something more recent if this is the case).

Example URL: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/8e/37/8e374bc49d4f3314a2a4497b065441e3

Config Files

Build Config

Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/8e/37/8e374bc49d4f3314a2a4497b065441e3

Some of the files listed in this file are explained later on in this article.

Value name Description
root Content hash of the decoded root file, look this up in encoding to get the CDN encoded CDN key/Content hash
install Content hash of the decoded install file, look this up in encoding to get the encoded CDN key/Content hash
install-size Size of install file
download Content hash of the decoded download file, look this up in encoding to get the encoded CDN key/Content hash
download-size Size of download file
partial-priority Unknown
partial-priority-size Unknown
encoding First key is the content hash of the decoded encoding file, second one is the CDN key
encoding-size Encoding sizes
patch Unknown
patch-size Unknown
patch-config Patch config file location (see Patch Config)
build-name Name of the build
build-playbuild-installer Type of installer for the Battle.net app to use
build-product Product name
build-uid Program code (see NGDP Program Codes)

CDN Config

Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/c6/38/c638428d66fb3c1fec6b7eb567e2e615

Value name Description
archives CDN keys of all archives (and by appending .index to the hash their indexes)
archive-group CDN key of the the combined index file (see Archive-Group Index)
patch-archives CDN keys of patch archives (needs research)
patch-archive-group CDN key of probably the combined patch index file (needs research)
builds List of build configs this config supports (optional)

Patch Config

This configuration file was added after all of the others. It first appeared in CASC v1 for Heroes of the Storm in August 2014. It then appeared in WoW for CASC v2 around build 19000 (approximately October 1st, 2014). The purpose of this file is to reduce redundant downloads. It achieves this by directing the system to download patch files to apply and update previously downloaded material. The structure and purpose of all of the fields of this file requires further research.

Value name Description
patch-entry Repeats 3 times with patch entries for install, download and encoding file
patch Patch file itself (needs research)
patch-size Size of patch file (optional?)

Example file: http://dist.blizzard.com.edgesuite.net/tpr/wow/config/b6/c8/b6c844d423c0c3e8620a171828080b06

patch-entry

The format of these strings is:

patch-entry = <type> <content hash> <content size> <BLTE-header hash> <BLTE-encoded size> <encoding string>

followed by sets of

<old BLTE-header hash> <old content size> <patch hash> <patch size>

This serves to pair old files to the patch needed to bring them up to date, as well as providing the information required to detect if the file is already up to date.

Data Files

Example index: http://dist.blizzard.com.edgesuite.net/tpr/wow/data/00/72/0072651343c29797b9da4aad2d0c93fa.index

Example archive: http://dist.blizzard.com.edgesuite.net/tpr/wow/data/00/72/0072651343c29797b9da4aad2d0c93fa

Patch Files

File References

Files are referred to by many different pieces of data in CASC. A quick summary of them:

  • Filename: The file's real name. Note that one file can have many names - essentially, one header hash can map to many different name hashes.
  • Locale Flag:
  • Content Flag:
  • Name Hash: The file's name, after being hashed with the Jenkins Hash.
  • Header Hash: The MD5 of the BLTE header of the compressed file.
  • Content Hash: The MD5 of the entire file in its uncompressed state; the purest representation of the data.

BLTE encoded files

Any files stored inside the data files are BLTE encoded, which means before reading anything in the file, first you have to decode it. The documentation below refers to decoded files!

It consists of these chunks in the following order:

  • Header
  • ChunkInfo (only if Header.headerSize > 0)
  • Data


To read a BLTE encoded file:

  1. Read the Header chunk
  2. Read the ChunkInfo chunk if Header.headerSize > 0
  3. Read each of the Data chunks and combine them to create the complete file

Note: If there is no ChunkInfo struct, there is just one Data chunk.


  • Header
Offset (Hex) Type Name Description
0x00 char[4] FileSignature "BLTE"
0x04 uint32_t [BE] headerSize Size of the BLTE header (BLTE header = Header + ChunkInfo).


  • ChunkInfoEntry
Offset (Hex) Type Name Description
0x00 uint32_t [BE] compressedSize Compressed size of the chunk (the compression mode byte is included).
0x04 uint32_t [BE] decompressedSize Decompressed chunk of the size.
0x08 char[16] checksum The checksum of the compressed chunk (the compression mode byte is included).


  • ChunkInfo
Offset (Hex) Type Name Description
0x00 uint8_t [BE] flags Flags of some sort.
0x02 uint24_t [BE] chunkCount The number of chunks.
0x04 ChunkInfoEntry[chunkCount] chunks The chunk info for the chunks in the file.


  • Data
Offset (Hex) Type Name Description
0x00 char encodingMode Available values: N, Z, F, E
0x01 char[ChunkInfo.compressedSize] data The encoded data.


Example implementation as Binary Template can be found here: BLTE-Template


Encoding modes:

  • N: Plain data.
  • Z: Zlib encoded data.
  • F: Recursively encoded BLTE data.
  • E: encrypted: one of salsa20, arc4, rc4.
struct 
{
  unsigned char key_name_length;              // 0x8
  unsigned char key_name[key_name_length];
  unsigned char IV_length;                    // 0x4
  unsigned char IV[IV_length];
  char type; // 'S': salsa20, 'A': arc4
} E_chunk;

key_name is resolved by client to the actual key. keys are distributed via keyrings and some keys are hardcoded.

Encryption keys

Battle.net app

key_name          key                               type      seen in
2C547F26A2613E01  37C50C102D4C9E3A5AC069F072B1417D  salsa20   Battle.net App Alpha 1.5.0

Overwatch

Overwatch has keys included in the client/keyring, but they can also be streamed from the server.

key_name          key                               type     seen in         method    used for
FB680CB6A8BF81F3  62D90EFA7F36D71C398AE2F1FE37BDB9  salsa20  0.8.0.24919     keyring
402CD9D8D6BFED98  AEB0EADEA47612FE6C041A03958DF241  salsa20  0.8.0.24919     keyring
DBD3371554F60306  34E397ACE6DD30EEFDC98A2AB093CD3C  salsa20  0.8.0.24919     keyring
11A9203C9881710A  2E2CB8C397C2F24ED0B5E452F18DC267  salsa20  0.8.0.24919     keyring
A19C4F859F6EFA54  0196CB6F5ECBAD7CB5283891B9712B4B  salsa20  0.8.0.24919     keyring
87AEBBC9C4E6B601  685E86C6063DFDA6C9E85298076B3D42  salsa20  0.8.0.24919     keyring
DEE3A0521EFF6F03  AD740CE3FFFF9231468126985708E1B9  salsa20  0.8.0.24919     keyring
8C9106108AA84F07  53D859DDA2635A38DC32E72B11B32F29  salsa20  0.8.0.24919     keyring
49166D358A34D815  667868CD94EA0135B9B16C93B1124ABA  salsa20  0.8.0.24919     keyring
1463A87356778D14  69BD2A78D05C503E93994959B30E5AEC  salsa20  ≤ 1.0.3.0       keyring
5E152DE44DFBEE01  E45A1793B37EE31A8EB85CEE0EEE1B68  salsa20  ≤ 1.0.3.0       keyring
9B1F39EE592CA415  54A99F081CAD0D08F7E336F4368E894C  salsa20  ≤ 1.0.3.0       keyring
24C8B75890AD5917  31100C00FDE0CE18BBB33F3AC15B309F  salsa20  ≤ 1.0.3.0       keyring
EA658B75FDD4890F  DEC7A4E721F425D133039895C36036F8  salsa20  ≤ 1.0.3.0       keyring
026FDCDF8C5C7105  8F41809DA55366AD416D3C337459EEE3  salsa20                  keyring
CAE3FAC925F20402  98B78E8774BF275093CB1B5FC714511B  salsa20                  keyring
061581CA8496C80C  DA2EF5052DB917380B8AA6EF7A5F8E6A  salsa20                  keyring
BE2CB0FAD3698123  902A1285836CE6DA5895020DD603B065  salsa20                  keyring
57A5A33B226B8E0A  FDFC35C99B9DB11A326260CA246ACB41  salsa20  1.1.0.0.30200   keyring   Ana
42B9AB1AF5015920  C68778823C964C6F247ACC0F4A2584F8  salsa20  1.2.0.1.30684   keyring   Summer Games
4F0FE18E9FA1AC1A  89381C748F6531BBFCD97753D06CC3CD  salsa20  1.2.0.1.30684   keyring
7758B2CF1E4E3E1B  3DE60D37C664723595F27C5CDBF08BFA  salsa20  1.2.0.1.30684   keyring
E5317801B3561125  7DD051199F8401F95E4C03C884DCEA33  salsa20  1.4.0.2.32143   keyring   Halloween Terror
16B866D7BA3A8036  1395E882BF25B481F61A4D621141DA6E  salsa20  1.4.1.0.31804   keyring   Bastion Blizzcon 2016 skin
11131FFDA0D18D30  C32AD1B82528E0A456897B3CE1C2D27E  salsa20  1.5.0.1.32795   keyring   Sombra
CAC6B95B2724144A  73E4BEA145DF2B89B65AEF02F83FA260  salsa20  1.5.0.1.32795   keyring   Ecopoint: Antarctica
B7DBC693758A5C36  BC3A92BFE302518D91CC30790671BF10  salsa20  1.5.0.1.32795   keyring   Genji Oni skin
90CA73B2CDE3164B  5CBFF11F22720BACC2AE6AAD8FE53317  salsa20  1.6.1.0.33236   keyring   Oasis map
6DD3212FB942714A  E02C1643602EC16C3AE2A4D254A08FD9  salsa20  1.6.1.0.33236   keyring
11DDB470ABCBA130  66198766B1C4AF7589EFD13AD4DD667A  salsa20  1.6.1.0.33236   keyring   Winter Wonderland
5BEF27EEE95E0B4B  36BCD2B551FF1C84AA3A3994CCEB033E  salsa20                  keyring
9359B46E49D2DA42  173D65E7FCAE298A9363BD6AA189F200  salsa20                  keyring   Diablo's 20th anniversary
1A46302EF8896F34  8029AD5451D4BC18E9D0F5AC449DC055  salsa20  1.7.0.2.34156   keyring   Year of the Rooster
693529F7D40A064C  CE54873C62DAA48EFF27FCC032BD07E3  salsa20  1.8.0.0.34470   keyring
388B85AEEDCB685D  D926E659D04A096B24C19151076D379A  salsa20  1.8.0.0.34470   keyring
E218F69AAC6C104D  F43D12C94A9A528497971F1CBE41AD4D  salsa20  1.9.0.0.34986   keyring
F432F0425363F250  BA69F2B33C2768F5F29BFE78A5A1FAD5  salsa20  1.10.0.0.35455  cmf
061D52F86830B35D  D779F9C6CC9A4BE103A4E90A7338F793  salsa20  1.10.0.0.35455  cmf
1275C84CF113EF65  CF58B6933EAF98AF53E76F8426CC7E6C  salsa20                  keyring
D9C7C7AC0F14C868  3AFDF68E3A5D63BABA1E6821883F067D  salsa20                  keyring
BD4E42661A432951  ????????????????????????????????  salsa20  1.11.0.0.36376            Unknown event

World of Warcraft

WoW's TactKey.db2 and TactKeyLookup.db2 contain keys and key_names (called lookups there) respectively. Either can be streamed from server. These files cannot be bruteforced by requesting hotfix data (Blizzard has guards in place)

The id field in the table below corresponds to the id field in the db2's.

key_name          key                               type      id  seen in                   used for
FA505078126ACB3E  BDC51862ABED79B2DE48C8E7E66C6200  salsa20   15  WOW-20740patch7.0.1_Beta  (lookup in db)
FF813F7D062AC0BC  AA0B5C77F088CCC2D39049BD267F066D  salsa20   25  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
D1E9B5EDF9283668  8E4A2579894E38B4AB9058BA5C7328EE  salsa20   39  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
B76729641141CB34  9849D1AA7B1FD09819C5C66283A326EC  salsa20   40  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
FFB9469FF16E6BF8  D514BD1909A9E5DC8703F4B8BB1DFD9A  salsa20   41  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
23C5B5DF837A226C  1406E2D873B6FC99217A180881DA8D62  salsa20   42  WOW-20740patch7.0.1_Beta  (lookup streamed from server)
3AE403EF40AC3037  ????????????????????????????????  salsa20   51  WOW-21249patch7.0.3_Beta
E2854509C471C554  433265F0CDEB2F4E65C0EE7008714D9E  salsa20   52  WOW-21249patch7.0.3_Beta  Warcraft movie items
8EE2CB82178C995A  DA6AFC989ED6CAD279885992C037A8EE  salsa20   55  WOW-21531patch7.0.3_Beta  BlizzCon 2016 Murlocs
5813810F4EC9B005  01BE8B43142DD99A9E690FAD288B6082  salsa20   56  WOW-21531patch7.0.3_Beta  Fel Kitten
7F9E217166ED43EA  05FC927B9F4F5B05568142912A052B0F  salsa20   57  WOW-21531patch7.0.3_Beta
C4A8D364D23793F7  D1AC20FD14957FABC27196E9F6E7024A  salsa20   58  WOW-21691patch7.0.3_Beta  Demon Hunter #1 cinematic (legion_dh1)
40A234AEBCF2C6E5  C6C5F6C7F735D7D94C87267FA4994D45  salsa20   59  WOW-21691patch7.0.3_Beta  Demon Hunter #2 cinematic (legion_dh2)
9CF7DFCFCBCE4AE5  72A97A24A998E3A5500F3871F37628C0  salsa20   60  WOW-21691patch7.0.3_Beta  Val'sharah #1 cinematic (legion_val_yd)
4E4BDECAB8485B4F  3832D7C42AAC9268F00BE7B6B48EC9AF  salsa20   61  WOW-21691patch7.0.3_Beta  Val'sharah #2 cinematic (legion_val_yx)
94A50AC54EFF70E4  C2501A72654B96F86350C5A927962F7A  salsa20   62  WOW-21691patch7.0.3_Beta  Sylvanas warchief cinematic (legion_org_vs)
BA973B0E01DE1C2C  D83BBCB46CC438B17A48E76C4F5654A3  salsa20   63  WOW-21691patch7.0.3_Beta  Stormheim Sylvanas vs Greymane cinematic (legion_sth)
494A6F8E8E108BEF  F0FDE1D29B274F6E7DBDB7FF815FE910  salsa20   64  WOW-21691patch7.0.3_Beta  Harbingers Gul'dan video (legion_hrb_g)
918D6DD0C3849002  857090D926BB28AEDA4BF028CACC4BA3  salsa20   65  WOW-21691patch7.0.3_Beta  Harbingers Khadgar video (legion_hrb_k)
0B5F6957915ADDCA  4DD0DC82B101C80ABAC0A4D57E67F859  salsa20   66  WOW-21691patch7.0.3_Beta  Harbingers Illidan video (legion_hrb_i)
794F25C6CD8AB62B  76583BDACD5257A3F73D1598A2CA2D99  salsa20   67  WOW-21846patch7.0.3_Beta  Suramar cinematic (legion_su_i)
A9633A54C1673D21  1F8D467F5D6D411F8A548B6329A5087E  salsa20   68  WOW-21846patch7.0.3_Beta  legion_su_r cinematic
5E5D896B3E163DEA  8ACE8DB169E2F98AC36AD52C088E77C1  salsa20   69  WOW-21846patch7.0.3_Beta  Broken Shore intro cinematic (legion_bs_i)
0EBE36B5010DFD7F  9A89CC7E3ACB29CF14C60BC13B1E4616  salsa20   70  WOW-21846patch7.0.3_Beta  Alliance Broken Shore cinematic (legion_bs_a)
01E828CFFA450C0F  972B6E74420EC519E6F9D97D594AA37C  salsa20   71  WOW-21846patch7.0.3_Beta  Horde Broken Shore cinematic (legion_bs_h)
4A7BD170FE18E6AE  AB55AE1BF0C7C519AFF028C15610A45B  salsa20   72  WOW-21846patch7.0.3_Beta  Khadgar & Light's Heart cinematic (legion_iq_lv)
69549CB975E87C4F  7B6FA382E1FAD1465C851E3F4734A1B3  salsa20   73  WOW-21846patch7.0.3_Beta  legion_iq_id cinematic
460C92C372B2A166  946D5659F2FAF327C0B7EC828B748ADB  salsa20   74  WOW-21952patch7.0.3_Beta  Stormheim Alliance cinematic (legion_g_a_sth)
8165D801CCA11962  CD0C0FFAAD9363EC14DD25ECDD2A5B62  salsa20   75  WOW-21952patch7.0.3_Beta  Stormheim Horde cinematic (legion_g_h_sth)
A3F1C999090ADAC9  B72FEF4A01488A88FF02280AA07A92BB  salsa20   81  WOW-22578patch7.1.0_PTR   Firecat Mount
18AFDF5191923610  ????????????????????????????????  salsa20   82  WOW-22578patch7.1.0_PTR
3C258426058FBD93  ????????????????????????????????  salsa20   91  WOW-23436patch7.2.0_PTR
094E9A0474876B98  ????????????????????????????????  salsa20   92  WOW-23910patch7.2.5_PTR
3DB25CB86A40335E  02990B12260C1E9FDD73FE47CBAB7024  salsa20   93  WOW-23789patch7.2.0_PTR
0DCD81945F4B4686  1B789B87FB3C9238D528997BFAB44186  salsa20   94  WOW-23789patch7.2.0_PTR
486A2A3A2803BE89  ????????????????????????????????  salsa20   95  WOW-23789patch7.2.0_PTR

States of CASC Data

CASC data comes in all forms and sizes.

Key CASC Files

Root

File signature: None The purpose of Root is to translate Content Hashes into file names


Encoding

File signature: "EN"

The encoding file contains data which is used to map content hash to file key.

The file contains the following in order:

  • File header
  • String block #1
  • Table A header
  • Table A entries
  • Table B header
  • Table B entries
  • String block #2


Encoding Header Structure

  • The beginning of the file is compromised of this structure of 0x16 bytes. Structure names were invented by the author of this page.
Offset (Hex) Type Name Description
0x00 char[2] FileSignature "EN"
0x02 uint8_t UNK ???
0x03 uint8_t checksumSizeA The length of the checksums in table A.
0x04 uint8_t checksumSizeB The length of the checksums in table B.
0x05 uint16_t flagsA Flags for table A.
0x07 uint16_t flagsB Flags for table B.
0x09 uint32_t [BE] numEntriesA The number of entries in table A.
0x0D uint32_t [BE] numEntriesB The number of entries in table B.
0x11 uint8_t UNK ???
0x12 uint32_t [BE] stringBlockSize The size of string block #1.


Encoding Table Header Block Structure

  • Each of the tables have numEntries entries of this structure of 0x20 bytes. They are used to locate what entry in the next part of the table contains a hash and to verify the integrity of that entry once it is read.
Offset (Hex) Type Name Description
0x00 char[checksumSizeA] firstHash The hash of the first file in the entry.
0x10 char[checksumSizeA] blockHash The checksum of the entry.


Encoding Table Entry Block Structure

  • Each of the tables have numEntries entries of 4096 bytes which contains these structures, followed by padding.
Offset (Hex) Type Name Description
0x00 uint16_t keyCount The number of keys.
0x02 uint32_t [BE] fileSize The decompressed size of the file.
0x06 char[checksumSizeA] hash The hash of the file content.
0x16 char[checksumSizeA*keyCount] keys The hash of the BLTE encoded file's BLTE header as referenced in index (.IDX) files. == headerHash

Encoding Layout Table Header Block Structure

  • Each of the tables have numEntries entries of this structure of 0x20 bytes. They are used to locate what entry in the next part of the table contains a hash and to verify the integrity of that entry once it is read.
Offset (Hex) Type Name Description
0x00 char[checksumSizeB] firstKey The key of the first file in the entry.
0x10 char[checksumSizeB] blockHash The checksum of the entry.


Encoding Layout Table Entry Block Structure

  • Each of the tables have numEntries entries of 4096 bytes which contains these structures, followed by padding.
  • Note: Between the data and padding of the last table is an empty hash (0x10 0 bytes) followed by 0xFFFFFFFF.
Offset (Hex) Type Name Description
0x00 char[checksumSizeB] key The key of the file.
0x10 uint32_t [BE] stringIndex The index into string block #1.
0x14 char UNK ???
0x15 uint32_t [BE] fileSize The compressed size of the file.

String blocks

The two string blocks contain descriptions of file layouts, providing information about the sections and compression mode of the files.

  • Block #1 is referenced by the layout table (see above).
  • Block #2 is the description of the encoding file itself.


The string uses the following format:

<encoding_mode>:{<comma-separated subchunks>}
Note: Usually <encoding_mode> is b for BLTE in the top chunk.


It specifies each subchunk in this form:

<size>=<encoding_mode>


<size>:

Value refers to the number of bytes that chunk (at a minimum, see below) contains.
The value might contain K, M or *.
* If K is present, multiply the number with 1024.
* If M is present, multiply the number with 1048576.
* If * is present, the chunk is "greedy" and it contains the rest of the bytes in the file in addition to any number specified.


<encoding_mode>:

Values will be either n, z, f, or e.
It can also include a specifier (ex: =z:{6,mpq}) for encoder parameters (ex: z:{6, mpq} means level == 6 and windowBits == 0).


n None
z Zlib
Parameters:
level - default value: 9
windowBits - default value: 15 (note: the value mpq means windowBits == 0)
f Frame
c Crypt


Example:

b:{64=n,256K*=z}


010 Template:

https://gist.github.com/heksesang/fdda3e4f8a5ed53b71ed

Function for parsing encoder profiles:

https://gist.github.com/heksesang/b15057fe3f093eebee3a


Install

File signature: "IN"

The install file lists files installed on disk. Since the install file is shared by architectures and OSses, there are also tags to select a subset of files. When using multiple tags, a binary combination of the bitfields of files to be installed can be created.

Header Structure

The file begins with a 10 byte header describing the number of tags and files listed. Structure names were invented by the author of this page.

Offset (Hex) Type Name Description
0x00 char[2] FileSignature "IN"
0x02 uint8_t UNK ???
0x03 uint8_t hash_size size of hashes used for files (usually md5 -> 16)
0x04 uint16_BE_t num_tags number of tags in header of file
0x06 uint32_BE_t num_entries The number of entries in the body of the file

Tags Structure

After the header, an array with information about available tags follows. Each tag has a bitfield listing the files installed when the given tag is chosen.

Type Name Description
char[] name
uint16_BE_t type A number shared amongst specific flags. Can change per game/version.

World of Warcraft

This section only applies to versions WoD (6.0.1.18125) … WoD (6.0.1.18761).

Arch = 1, Locale = 2, OS = 3

This section only applies to versions WoD (6.0.1.18764) … WoD (6.2.2.20426).

Arch = 1, Category = 2, Locale = 3, OS = 4, Region = 5

This section only applies to versions WoD (6.2.2.20438) … Legion.

OS = 1, Arch = 2, Locale = 3, Region = 4, Category = 5

char[divru (header.entries, CHAR_BIT)] files A bitfield that lists which files are installed when the specified tag is installed.

Files Structure

The remainder of the file is populated by a list of files with their content hash, each a variable size (due to the strings). Structure names were invented by the author of this page.

Type Name Description
char[] FileName The name of the file.
char[header.hash_size] hash The hash of the uncompressed file. Usually MD5.
uint32_BE_t Size The size of the file.

C-like structure

char I; char N;
uint8_BE_t _unk3;
uint8_BE_t hash_size;
uint16_BE_t num_tags;
uint32_BE_t num_files;

struct {
  string name;
  uint16_BE_t type;
  char flags[divru (num_files, CHAR_BIT)];
} tags[num_tags];

struct {
  string name;
  char hash[hash_size];
  uint32_BE_t size;
} files[num_files];

Download

This file has this structure:

  • Header
  • Entries[Header.EntryCount]
  • Tags[Header.TagsCount]

Download Header

Type Name Description
char[2] Signature The signature for this file (always "DL")
char[3] unk ???
int [BE] EntryCount The amount of file entries in this file
short [BE] TagCount The amount of tag entries in this file

Download Entry

Type Name Description
char[16] Hash This hash is found in every node of the encoding file. (Reverse lookup)
char Unk ??? (Always 0 7.2.0.24015)
uint32_t [BE] FileSize The compressed size of the file.
char[5] Unk ???

Download Tag

Type Name Description
string Name A C-String indicating this tag's Name.
short [BE] Type Hash type
char[N] Bits an array of size N = Header.EntryCount / 8 + (Header.EntryCount % 8 > 0 ? 1 : 0); that is basically a massive bit mess. Use Schroeppel's 8 bits reverse function on it to have bits.

Patch

Type Name Description
char[2] Signature The signature for this file (always "PA")
char 1
char size_a <= 0x10
char size_b <= 0x10
char size_c <= 0x10
char size_d <= 0x18
char[19] unk ???
char[16] Encoding file The hash for encoding file (same as second string in build config file)
int Uncompressed Uncompressed encoding file size in bytes
int Compressed Compressed encoding file size in bytes
char EncodingFormatLength Length of the following string
char[EncodingFormatLength] EncodingFmt Encoding string (same format as string blocks in encoding file)
char[] ??? byte array until the end of the file


header+entries needs to be less than 0x10000 bytes (at least in wow-18179). md5sum is only checked for header+entries, file might be larger thus.

struct PatchManifest_Header
{
  uint16_t_BE magic; // 'PA'
  uint8_t version; // 1 or 2
  uint8_t key_size_a; // <= 0x10
  uint8_t size_b; // <= 0x10
  uint8_t size_c; // <= 0x10
  uint8_t size_d; // (size_d - 0xc) <= 0x12.
  uint16_t_BE entry_count; // (key_size_a + 20) * entry_count + sizeof (PatchManifest_Header) < 0x10000
  uint8_t unk2; // flags
} header;

#if encoding_information_apparently_added_after_18179
uint8_t encoding_key[16];
// uint8_t encoding_decoded_key[16]; // probably since PA2
uint32_t_BE decoded_size;
uint32_t_BE encoded_size;
uint8_t encoding_format_length;
char encoding_format[encoding_format_length];
#endif

struct PatchManifest_Entry
{
  uint8_t key[header.key_size_a];
  uint8_t md5_of_entry_data[0x10];
  uint32_t_BE offset_entry_data; // in this file
} entries[header.entry_count]; // sorted ascending by key

// at positions given in PatchManifest_Entry
struct entry_data // maximum size: 2^header.size_d!
{
  struct
  {
    uint8_t num_entries; // <= 0x10.
    uint8_t key[header.key_size_a];
    uint40_t_BE unk; // yes, 5 bytes!
    struct
    {
      uint8_t key[header.key_size_a];
      uint40_t_BE unk1;
      uint8_t key[header.key_size_c];
      uint40_t_BE unk2;
    } sub_entries[num_entries];
  } entries[]; // count unspecified: read until the next sub_entries[].num_entries would be 0 
               // OR entry_data would be bigger than 2^header.size_d
};

// in my example file (bd260d7f3a9008620a90033b561a6289), after the last
// entries_data which ended with num_entries == 0, there was further data. 
// something above is thus not correct, or incomplete.

Blizzard-Created Archives

In its natural state, the vast majority of the data for any CASC-based game exists in the archives.

Archives

Archives are extensionless 256 MB files that are usually only stored on the Blizzard CDNs. Their naming follows the standard URL hash format using the '/data/' path type.

The structure of the archives is presumably just file fragment after file fragment. You will never need to parse it because you can just look up offset + size of your file fragment in the index files and then take the piece directly out of the archive.

The fragments are all BLTE encoded.

Archive Indexes (.index)

These '.index' files reveal to the user where the compressed game files are located within the archives. All indexes (except the Archive-Group index, see below) are named after their archive (only difference is these have an extension). '.index' files are stored on the CDN using the standard hash naming scheme (remember they have an extension though). They are also located in the directory 'INSTALL_DIR/Data/indices/' for a WoW install. Note that the index files are not complete -- some HeaderHash entries obtained from the encoding file will not appear in the index. These can be fetched directly from the CDN using the HeaderHash in the standard hash naming scheme.

Normal Index Entry Structure

  • The file is divided into 4kb chunks populated by these standard index entries of 0x18 (hex) bytes. Each chunk is zero-padded to a full 4kb, though there may be more than 0x18 bytes of padding at the end of a chunk -- be sure the check for all-null HeaderHash fields. The last chunk is a table-of-contents, listing the LAST HeaderHash in each chunk. If there is more than one block then the table-of-contents also includes an array of the lower md5 of each block excluding the final one. Structure names were invented by the author of this page.
  • NOTE: This structure uses big endian numbers.
Offset (Hex) Type Name Description
0x00 char[16] HeaderHash The MD5 of the BLTE header for the compressed fragment that this index entry represents
0x10 uint32 Size Size of the fragment
0x14 uint32 Offset Position of the fragment in the archive

Structure

struct index_entry {
  char blte_header_hash[footer.keySizeInBytes]; // sizes are encoded in footer
  uint32_BE_t blte_encoded_size;
  uint32_BE_t offset_to_blte_encoded_data_in_archive;
};

struct index_block {
  static constexpr const block_size = 0x1000;
  index_entry entries[block_size / sizeof (index_entry)];
  char padding[block_size - sizeof (entries)];
} blocks[];

struct {
  struct {
    char last_hash[footer.keySizeInBytes]; // last hash of a block
  } entries[num_blocks];

  struct {
    char lower_part_of_md5_of_block[footer.checksumSize]; 
  } blocks_hash[num_blocks - 1]; // last block is not included
} toc;

struct {
  char index_blocks_hash[checksumSize]; // client tries to read with 0x10, then backs off when smaller
  char toc_hash[checksumSize];
  char _10; // always 1
  char _11;
  char _12;
  char _13;
  char offsetBytes;
  char sizeBytes;
  char keySizeInBytes;
  char checksumSize; // <= 0x10
  uint32_t numElements; // BigEndian in _old_ versions (e.g. 18179)
  char lower_part_of_md5_of_footer[checksumSize];
} footer;
  • x = _lower_part_of_md5_of_footer; _lower_part_of_md5_of_footer = 0, qword (md5 {&_10, &_10+0x14}) == x
  • md5 {last 0x14+checksumSize=0x1c bytes of index file} == archive_name
  • qword (md5 {&blocks, num_blocks * block_size}) == index_blocks_hash (i.e. without footer and toc)
  • qword (md5 {&toc + &index_blocks_hash}) == toc_hash (lower part of md5 of toc data concatenated with the index_blocks_hash)

Archive-Group Index (.index)

Archive-group is actually a very special '.index' file. While virtually all '.index' files are under 2 MB, the archive-group '.index' file is always over 15 MB. It is essentially a merger of all .index files, with a structure change. There is a new uint16 field that serves as an index for the array of archives from this build's CDN config.

Therefore, it is critical that you identify this outlier - if you try to parse it as a regular '.index' purely because of its extension, your program will undoubtedly fail. You can identify it because it will be named the same as the 'archive-group' hash listed in the CDN config. Additionally, it will not be listed as an archive hash in the CDN config. As discussed before, the different file structure and irregular file size are also viable methods to avoid parsing this file (or to avoid parsing the other '.index' files).

Merged Index Entry Structure

  • The entire file is populated by these 'merged' index entries of 0x1A (hex) bytes. Structure names were invented by the author of this page.
  • NOTE: This structure uses big endian numbers.
Offset (Hex) Type Name Description
0x00 char[16] HeaderHash The MD5 of the BLTE header for the compressed fragment that this index entry represents
0x10 uint16 ArchiveIndex If you placed the hashes of the 'archives = ' line of the CDN config in an array, this number would be the index for that array
0x12 uint32 Offset Position of the fragment in the archive
0x16 uint32 Size Size of the fragment

patch-archives

Like archives, these are binary blobs of fragments indexed by an accompanying .index file with the same name.

These are not BLTE encoded but are ZBSDIFF1 blobs. Again, the index is a hash, size, offset tuple.

Journal-based Data Files

During the installation process for a Blizzard game, the program will download the required files as requested by root, encoding, download, and install. It stores the downloaded data fragments in data files in "INSTALL_DIR\Data\data\". The program will record the content hash (BLTE-compressed hash), size, and position of the file as well as the number of the data file that it is in. It places those four parameters into journal files with the extension '.idx'.

Shared Memory

The shared memory file is called 'shmem' and is usually located in the same folder as the data and .IDX journals. This file contains the path where the data files are stored, which is the current version of each of the .IDX files, and which areas of the data files have unused space. The file is recreated every time a client is started.

Shared Memory Header Structure

  • The first part of the header.
Offset (Hex) Type Name Description
0x00 uint32_t BlockType A value indicating what type of block this is. For this block, the value is 4.
0x04 uint32_t NextBlock The offset of the next block.
0x08 char[0x100] DataPath The path of the data files. This is prefixed with "Global\" if the path is an absolute path.


  • Followed by a number of these entries. The count can be calculated like this: (NextBlock - 264 - idxFileCount * 4) / 8
Offset (Hex) Type Name Description
0x00 uint32_t Size The size of the block.
0x04 uint32_t Offset The offset of the block.


  • Followed by a number of these entries. The count is equal to number of .IDX files (usually 16).
Offset (Hex) Type Name Description
0x00 uint32_t Version The version number. Used to identify the .IDX filename.


Shared Memory Free Space Structure

After a small header, this structure is split up into two equal parts. The first part contains entries with the number of unused bytes. The second part contains entries with the position of the unused bytes.

There can be up to 1090 entries. Each of the two parts will always be 5450 bytes, so if there are fewer than 1090 entries, the rest of the bytes will be padded with '\0'.

  • The header part of the structure.
Offset (Hex) Type Name Description
0x00 uint32_t BlockType A value indicating what type of block this is. For this block, the value is 1.
0x04 uint32_t NextBlock The offset of the next block.
0x08 char[0x18] Padding Padding at the end of the header.


  • This is the number of unused bytes. There can be up to 1090 entries of these. If there are fewer, the rest of the area is padded.
Offset (Hex) Type Name Description
0x00 uint10* DataNumber This is always set to 0 in this part of the block.
0x01 uint30* Count The number of unused bytes.


  • This is the position of the unused bytes. There can be up to 1090 entries of these. If there are fewer, the rest of the area is padded.
Offset (Hex) Type Name Description
0x00 uint10* DataNumber The number of the data file where the unused bytes are located.
0x01 uint30* Offset The position within the data file where the unused bytes are located.

.IDX Journals

Example file path: INSTALL_DIR\Data\data\0e00000054.idx

.IDX journals contain a mapping from keys to the location of their data in the local CASC archives. There used to be one .IDX file per journal, and the naming scheme used to have two separate meanings. The '0e' part of the file name used to designate which archive the .IDX file was associated with. This changed halfway through the Warlords Beta. Now there are 16 indices total, and the first byte of the hex filename says which of the 16 indices it is, while the remainder of the hex filename is just a version number that increments when a new set of files is added to the local archives.

To determine which of the 16 indices a key is bucketed in, the key is hashed by xoring together each 4-bit nibble in the first 9 bytes of the key:

 uint8_t cascGetBucketIndex(const uint8_t k[16]) {
   uint8_t i = k[0] ^ k[1] ^ k[2] ^ k[3] ^ k[4] ^ k[5] ^ k[6] ^ k[7] ^ k[8];
   return (i & 0xf) ^ (i >> 4);
 }


.IDX Header Structure

The header is little-endian:

Offset (Hex) Type Name Description
0x00 uint32 HeaderHashSize The number of bytes to use for the hash at +04; usually 0x10.
0x04 uint32 HeaderHash This should equal the value of pc after calling hashlittle2 on the following HeaderHashSize bytes of the file with an initial value of 0 for pb and pc.
0x08 uint16 Unk0 Must be 7
0x0a uint8 BucketIndex The bucket index of this file; should be the same as the first byte of the hex filename.
0x0b uint8 Unk1 Must be 0
0x0c uint8 EntrySizeBytes Must be 4
0x0d uint8 EntryOffsetBytes Must be 5
0x0e uint8 EntryKeyBytes Must be 9
0x0f uint8 ArchiveFileHeaderBytes Must be 30
0x10 uint64 ArchiveTotalSizeMaximum The maximum size of a casc installation; 0x4000000000, or 256GiB.
0x18 char[8] padding The header is padded with zeroes to the next 0x10-byte boundary.
0x20 uint32 EntriesSize This is the length in bytes of the entries in the index file.
0x24 uint32 EntriesHash This should equal the value of pc after calling hashlittle2 on the following EntriesSize bytes of the file with an initial value of 0 for pb and pc.

.IDX Entry Structure

  • The rest of the file is populated by these normal entries, each 0x12 bytes in size. Structure names were invented by the author of this section because official names were not available.
Offset (Dec) Type Name Description
00 char[9] Key The first 9 bytes of the key for this entry.
09 uint40* Offset Unlike the other little-endian integers in this file, this is a big-endian 5-byte integer. The top 10 bits are the number of the archive (data.%03d), and the bottom 30 bits are the offset in that archive to the file data.
14 uint32 Size The length of the file in bytes.
  • * designates unusual data types. In C#, you can read the Offset by reading a Byte, reading a big-endian UInt32, shifting the byte left 32 bits, and ORing them together. Use a 30-bit mask (0x3fffffff) to get the file offset, and right shift the value 30 bits to get the archive number.

.XXX Data Files

Example file path: INSTALL_DIR\Data\data\data.015

These files consist of a sequence of headers with corresponding BLTE data.


  • The data header.
Offset (Hex) Type Name Description
0x00 char[0x10] BlteHash The hash of the BLTE header (see the section above). If the BLTE header doesn't include a block table, this is a hash of the complete BLTE file. This hash is in reverse, so reverse it before you use it. This is also the same hash that is used as file key on CDN and in the .idx files.
0x10 uint32_t Size The size of this header + the following data.
0x14 char[0x0A] UNK Unknown bytes. Most likely not needed by the games.


  • The BLTE data.
Offset (Hex) Type Name Description
0x00 char[Header.Size - 30] Data The BLTE file data. See the BLTE section above.

hashpath

hashpath (string path) → uint32_t
{
  string normalized = toupper (path).replace (from: '/', to: '\\')
  uint32_t pc = 0, pb = 0;
  hashlittle2 (normalized, strlen (normalized), &pc, &pb);
  return pc;
}