Save file structure

[Necrolis]

The field isn't terminated, its a fixed size(1 DWORD), and spaces are not discarded(not by the game), thus an item code is between 1 and 4 BYTEs long, with the remaining BYTEs being padded with spaces. This what what Nefarius' code demonstrates, thus to pull out a reversed value you can just do:

Code: Select all

DWORD dwCode = *(DWORD*)(&pData[nPosition]);

or

Code: Select all

char szCode[5] = {0}; (DWORD)szCode = *(DWORD*)(&pData[nPosition]);

[SVR]

A strange thing I have just run into is that item types are noted here http://home.roadrunner.com/~svr/formats/extend.htm as ASC8, that's to say null(\0)-terminated strings, while in fact it seems to be space (0x20)-terminated. I don't recall reading this anywhere.

Just to clarify a bit. ASC8 means each 8 bits can be interpreted as an ascii character. Not necessarily null terminated. The bits column is the number of ASC8 'entities' there are.

All defs are from the point of view of the binary stream and not any fixed position type you can relate to memory position. Refer to binary stream discussions throughout this thread for more info.

[Gamall]

Just to clarify a bit. ASC8 means each 8 bits can be interpreted as an ascii character. Not necessarily null terminated. The bits column is the number of ASC8 'entities' there are.

I made that remark on this topic precisely because this is not what your formats page says:

On: http://home.roadrunner.com/~svr/formats/formats.htm

ASC8 8 bit per character null terminated string. The bits field is max number of characters. Displays as text.

All defs are from the point of view of the binary stream and not any fixed position type you can relate to memory position. Refer to binary stream discussions throughout this thread for more info.

I know; I read the topic before I started coding, and I didn't have any problem with binary streams thanks to those discussions.



The reason behind my post was that rather than keep implementing the item reader "by hand", which works but is tedious, I wanted to determine whether it would be possible to generate the code directly from your VSR format specifications. Computers being the dumb things they are, the specification needs to be exact and complete for this method to work; the discrepancy between the specification of ASC8 (explicitely null terminated) and its use (not null terminated) is the first (minor) hurdle I ran into.

Unfortunately my free time went from "scant" to "null" in the meanwhile. I'll go back to it probably sometime after February.


PS: by the way, many thanks for Udie and your file format pages. Both are extremely useful

[Necrolis]

seeing as everyone is updating, might as well update the structs:

Code: Select all

#pragma pack(1)
struct D2SkillKeyStrc
{
	WORD nSkill;		//+00
	union
	{
		struct
		{
			BYTE nKey;			//+02
			BYTE nKeyEx;		//+03
		};

		WORD wItemSlot;			//+02
	};
};

enum eD2MercSaveFlags
{
	MERCSAVE_DEAD  = 0x100000,
};

struct D2MercSaveStrc
{
	DWORD fFlags;							// +0AF
	DWORD dwSeed;							// +0B3
	WORD wName;								// +0B7
	WORD wType;								// +0B9
	DWORD dwExperience;						// +0BB
};

struct D2SaveHeaderStrc						//sizeof 0x14F
{
	DWORD dwMagic;							// +000 = 0xAA55AA55
	DWORD dwVersion;						// +004
	DWORD dwSize;							// +008
	DWORD dwSum;							// +00C
	DWORD dwWeaponSet;						// +010
	char szName[16];						// +014
	DWORD fFlags;							// +024
	BYTE nClass;							// +028
	BYTE nStats;							// +029 - set to a static 16 stats...
	BYTE nSkills;							// +02A - set to amozon skill count
	BYTE nLevel;							// +02B
	DWORD dwCreateTime;						// +02C
	DWORD dwLastTime;						// +030
	DWORD dwPlayTime;						// +034 = 0xFFFFFFFF, only on realms
	D2SkillKeyStrc pSkillKeys[16];			// +038
	D2SkillKeyStrc pButtonSkills[4];		// +078 / +07C / +080 / + 084
	BYTE nComp[16];							// +088
	BYTE nColor[16];						// +098
	BYTE nTown[3];							// +0A8
	DWORD dwMapSeed;						// +0AB
	D2MercSaveStrc pMercData;				// +0AF
	BYTE nRealmData[16];					// +0BF
	BYTE nSaveField;						// +0CF - used by client, realm only I'd say
	DWORD dwLastLevel;						// +0D0 - realm as well
	DWORD dwLastTown;						// +0D4 - ditto
	BYTE nLastDiff;							// +0D8 - ditto
	BYTE nRealmDataEx[118];					// +0D9
};

struct D2QuestSaveStrc									//sizeof 0x12A
{
	DWORD dwSection;									//+00
	DWORD dwVersion;									//+04
	WORD wSize;											//+08
	BYTE nQuestData[MAX_DIFFICULTY][MAX_QUESTFLAGS];	//+0A / +6A / +CA
};

struct D2WaypointSaveStrc								//sizeof 0x50
{
	WORD wSection;										//+00
	WORD wSize;											//+02
	DWORD dwVersion;									//+04		
	D2WaypointStrc nWaypointData[MAX_DIFFICULTY];		//+08 / +20 / +38
};

struct D2NPCIntroSaveStrc								//sizeof 0x34
{
	WORD wSection;										//+00
	WORD wSize;											//+02
	BYTE nIntroData[2][MAX_DIFFICULTY][MAX_NPC_INTRO];	//+04 / +14 / +24
};

struct D2CorpseSaveStrc									//sizeof 0x10
{
	WORD wSection;										//+00
	WORD bItems;										//+02
	DWORD dwUnused;										//+04 - this is totally {filtered} by the setting code...
	D2CoordStrc pPosition;								//+08
};

[ciuly]

Hello again

I switched over to SVRs VSD way of parsing the items after I bumped into an item containing JM in it's binary data (I still don't know which item is that since I've got over 120 items in the save file and I'm trying to get those out.) Just to update peoples memory, I was parsing the items using JM as a delimiter and validating the number of extracted items against the actual item count.

I'm having a bit of trouble with the bStack variable which isn't mentioned anywhere. I tried quite a few combinations but it just keeps failing.
any hints on this?

Thanks.

[ciuly]

I think I nailed down the bStack issue.
So I changed the way I play in order to improve my program. I am now playing for a little bit and then checking that all extracted items are what they supposed to be. For this I wrote some code to get the name of the items and I'm using atma and udietoo to make sure I get the same name for the item (I am assuming that if atma shows me the same name as my program, than I have extracted the item correctly. That, plus some asserts in code, plus that I know how many items I have to extract should guarantee that I have extracted all items correctly).

Now, I had a few bumpers here to, managed to track them down by peeking into various txt files but now I'm lost.
Both atma and udietoo show a name of "Leather Gloves" for an item type of lgl (that's LGL) (that's normal item, no magical, set, whatever, simple white item). I've searched all txt files in the mpq's and there is no normal leather glove item in there and all lgl's are simple "Gloves". But they do have a (L) appended to the name. I never knew what the "(letter here)" means so I just stripped it out. It appears to be code for Leather. I didn't understand why the code was needed since other items have the Leather in their name, like Leather Boots. Which brings me to issue #2: my code extracts LBT as Cracked Leather Boots, but game and atma and udietoo shows it as Cracked Boots. I have no clue how to generally handle this kind of scenario.

Is there some info in txt's or somewhere to which is which (I mean the letter from parenthesis)
I'll be constructing a table for these for now, but I'd probably need something more generic than a hardcoded table.
Also, any info on when to "cut" the "Leather" and any other such attributes from a name?

Also, I have more naming problems:
- Khalim's Heart in txt appears as KhalimHeart. Same goes for other quest items. Do I need to hard code this as well or is there some other file with correct info? Or maybe some transformation that will always get me correct quest item name?
- health and mana potions also have other names in txt files than what appears in game. what's the thing with these?
- grand charm appears in txt as large charm or charm large. again, another hardcoding?

right now, to keep me moving, I've created a txt file with code <-> correct name and I'm using this to override the names I'm getting from txt files. But this file is starting to grow a bit too much.

I would appreciate any good hints on item naming.

Thanks.

[Nefarius]

Names in txt files are irrelevant, the game uses string tables for localization, you should just load the vanilla txt files and replicate the item save/load routine in D2Common.dll. The code the game uses is really straightforward.

Code: Select all

	D2ItemDataStrc* pItemData = GetItemData(pItem);

	BOOL bError = 0;

	ITEMSAVE_WriteLocationToStream( hStream, pItem );

	ItemsTXTStrc* pItemsTxt = D2GetItemRecord( pItem->nIndex );
	
	ASSERT( pItemsTxt );

	if ( bGamble )
	{
		DWORD dwCode = pItemsTxt->NormCode;

		if ( dwCode == CODE_BLANK || !dwCode )
		{
			dwCode = pItemsTxt->Code;
		}

		hStream->write_s( dwCode, 32 );

		return;
	}

	hStream->write_s( pItemsTxt->Code, 32 );
	hStream->write_s( ITEMS_GetFilledSockets( pItem ), 3 );

etc

Some of the call locations in vanilla:

Code: Select all

	{0x6FC438E4,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FC44A74,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FC44E80,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FCC1C3E,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FCC4790,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FCF043D,(DWORD)ITEMSAVE_WriteItemToStreamEx,4},
	{0x6FC45DB5,(DWORD)ITEMSAVE_ReadInfoFromStream,4},
	{0x6FC45E6D,(DWORD)ITEMSAVE_ReadInfoFromStream,4},
	{0x6FC49C92,(DWORD)ITEMSAVE_ReadInfoFromStream,4},
	{0x6FC49CF5,(DWORD)ITEMSAVE_ReadInfoFromStream,4},

[ciuly]

it's been over 10 years since I last done dissassembly and over 5 since I last wrote an asm line of code. I'm a bit "old" for that
I'd rather write things nicely from scratch, if possible.

So, thanks for pointing me in the right direction. Obviously I wrote a tbl parser, VSD-based since I already wrote the VSD parser. had to make some adjustments to it and improve the VSD itself, but that's another story.

So now I am getting 112 out of 114 items out correctly.
The ones that are still a trouble are 2 runes, part of runewords: Io is read as Po (in Delimirum) and Jah is read as Jo (in Enigma). All other runewords I have amongst those 114 read up ok.

What am I doing wrong in this case? I am using expansionstring.tbl and string.tbl in case there are more needed. The order I use is search in expansion first and if not found, search in string.

thanks.

[kingpin]

What am I doing wrong in this case? I am using expansionstring.tbl and string.tbl in case there are more needed. The order I use is search in expansion first and if not found, search in string.

You should always search in this order: Patchstring.tbl / expansionstring.tbl / string.tbl

[ciuly]

hah, that was it dunno how I missed the obvious. Thanks a bunch

[ciuly]

weird item I just found. attached.
atma (5.05, latest if I recall correctly. the site is erroring for some reason) shows it as having
- defense : 13
- durability: 19 of 20

udietoo (1.10.011, latest according to svr's site) shows it as having
- defense : 12 (when viewing the item in the save file) OR 22 (when editing the item)
- durability: 19 of 18

and finally, the game (1.0.10.39, with expansion pack and a couple of txt mods) shows it as having
- defense : 13
- durability: 19 of 19

I don't know about this but till now I figured that (amongst others) the defense, durability and maxdurability do not depend on what mods are installed in the game, so what is in the save file is what it should be read and thus displayed.
but there is something obviously wrong here. I thus admitted the possibility that mods do have some influence and ran udietoo with -direct cmd line option as suggested in readme and pointed working dir to diablo install dir. same results.
I checked and itemstatscost is not modded. don't know what other files might be involved in modifying save data.

I only noticed this because I just added an auto-repair function to my muling app and was testing to see if it behaves correctly. this is the first set of 37 items and only this one is weird.
another "first" is the high quality code: 7. this could constitute a clue in the items weirdness as it's the first one with it for the past few hundred items since I started saving the name as well; and I'm picking up everything the game drops and cross test with atma and udietoo just to be sure I cover all bases.

to be noted that with SVRs VSD I read the defense as 22 ...

here's the XML with all the info, based on VSD

Code: Select all

<item>
  <majic>JM</majic>
  <bQuest>False</bQuest>
  <unk21>0</unk21>
  <bId>True</bId>
  <unk25>0</unk25>
  <bIllegal1>False</bIllegal1>
  <unk31>0</unk31>
  <bDup>False</bDup>
  <bSocketed>False</bSocketed>
  <unk34>0</unk34>
  <bIllegal>False</bIllegal>
  <unk37>0</unk37>
  <bEar>0</bEar>
  <bStarter>False</bStarter>
  <unk42>0</unk42>
  <unk44>0</unk44>
  <bSimple>False</bSimple>
  <bEthereal>False</bEthereal>
  <unk47>1</unk47>
  <bInscribed>False</bInscribed>
  <unk51>0</unk51>
  <bRuneWord>False</bRuneWord>
  <unk53>0</unk53>
  <unk60>101</unk60>
  <iLoc>0</iLoc>
  <iPos>0</iPos>
  <iCol>0</iCol>
  <iRow>0</iRow>
  <iStoredIn>1</iStoredIn>
  <dwType>skp </dwType>
  <nGems>0</nGems>
  <dwGUID>3256562386</dwGUID>
  <iLevel>5</iLevel>
  <iQual>3</iQual>
  <bVarGfx>False</bVarGfx>
  <bClass>False</bClass>
  <hiQual>7</hiQual>
  <bHasRand>0</bHasRand>
  <iDef>22</iDef>
  <iMaxDur>18</iMaxDur>
  <iCurDur>19</iCurDur>
  <prop1>((16, 9), (75, 31))</prop1>
</item>

any ideas or explanations?

[Necrolis]

1) please edit your posts, instead of double posting, this will bump the topic
2) you might have missed a bool in a optional section, which could account for the 1 point discrepancy(udietoo's 22 is from the saveadd not being applied iirc)
3) 1.0.10.39? you sure you don't mean 1.10f?
4) what txt mods are you running?

[ciuly]

1. ok. I only posted new since it was over a day from my last post. figured it's natural that way I was mostly active on another forum for over 6 years, moderating and all. different policies, harder to adapt but I will correct myself in time.

2. I don't think I missed anything. well, not unless udietoo is also missing it since I'm using its vsd files. in the end, because of this, my app should behave the same as udietoo (only in respect to reading items and their properties). but considering that there's a difference inside udietoo itself, between displaying item info in save file and editing that item, the issue might just be there.
just seeing saveadd, made me search this topic and saw it's connected to itemstatcost, not itemstatScost that I searched for, so I searched again and indeed in the modded dir there is an itemstatcost.bin file. being .bin doesn't help much in comparing contents against original txt files to see what's changed. unless I'm missing some tools.

I am now reading into itemstatcost.txt now. hopefully I'll figure out what's wrong as I'm not using it at all.

3. not sure where to look. I read that from the exe's file version information.

4. this will be hard to track down as all are put there a long time ago. searching through the archives and comparing file contents and the only one I could identify is ExpandedStashLoD110.zip
But, from what I remember, I added mods for increasing private stash, cube and inventory space; then one mod for setting character level and stats to 99 (so I can get items from everywhere at any level, be able to transmute in peace, etc - helps in my testings) and that's about it. I might have added other too, but I don't remember.
I can upload the zipped data dir if you think it would help. it's about 400k (4.5 mb unzipped)

thanks.

le:
upon further reading on bin files and learning I won't be able to use those to get the modded values, I decided to not support such mods.
in consequence, I removed all mods and placed back just the one with increased inv/stash/cube sizes.
I then dropped the weird item and went on playing/testing. things look good.

if the issue re-appears, I'll post back. then it won't be a mod-issue and maybe tracking it down will be easier.

thanks for your time

[ciuly]

Kind of stuck again.

I have an etheral, socketed, simple antlers, with required strength 14 (req level 12, item level 19), +1 to Cyclone Armor (druid only)
code is dr3

I read the required strength from armors.txt which reads 24. instead of 14. So I looked in itemstatcost.txt at strength but that's the +x to strength stat.
So I then looked in string.tbl to see the stat code by text (searched for required strength) which returns ItemStats1e
I searched all txt's for it but none refer it.
So I went with checking multiple items with required strength and noticed that indeed there is a -10 added to the value read from the item.

So question: is this a hardcoded thing or am I missing something?

later edit: going forward I found an item where the required strength is exactly what's in the txt file...

[k0r3l1k]

Ethereal items reduce the requirement of an item. That's why the value in armor.txt was different from te eth item in game

[ciuly]

right. but there should be either a static formula or some txt rules. I searched all txts for ethereal but didn't find anything that would jump in front. there was a function index (23) in properties.txt but that's about it. file description from here doesn't say what the function does just names it "ethereal". not very helpful.

any suggestions?

[k0r3l1k]

the formula that reduces str and dex requirements of ethereal items is hard coded, though i havent looked for/located it.

[ciuly]

ok, thanks. I'll try to pickup a lot of different ethereal's, maybe I can deduce it; or somebody who knows it shares it in the meanwhile

[Necrolis]

ok, thanks. I'll try to pickup a lot of different ethereal's, maybe I can deduce it; or somebody who knows it shares it in the meanwhile

its calculated at runtime in the itemreq evaulator, iirc when I replaced this the reduction was a fixed 40%

[ciuly]

indeed it appears the reduction is 40%. thanks

I have some more issues I can't figure out. I figured out the weapons speed. And until right now I thought I figured out the weapons class as well (reading "itemtype" from itemtypes.txt based on item code) and I just found a balanced knife and boom, I'm off-track.

So, what's the right way of getting the weapon class? (mace class, sword class, dagger class in this last instance)

The second problem is with auto/default/whatever properties.
I have the issue with "Arcadian Mace of Quality". It has 3 properties in the item file:
+2 to Maximum Damage
46% Damage to Daemons
+70 to Attack Rating against Daemons

but ATMA also shows a: 150% Damage to Undead.
So I started looking through AutoMagic.txt but there is nothing there that would match this. Not to mention that the mace has null in the "auto prefix" column.
I then searched for the property index (122) in the txt files maybe something would jump out in front but nothing did.

What am I missing here?

Thanks.

[Necrolis]

the itemtype indices need to be adjusted down by 2(due to the line mess in the txt)

maces have a hardcoded +50% damage to undead

[ciuly]

I'm searching by code in itemtypes, so I don't have indices there. but now that you mention it I feel like I'm missing something: where would I need indices into itemtypes.txt?

maces have a hardcoded +50% damage to undead

that accounts for 50%, but I'm still missing a 100 :|
I looked again over the stuff with 122 maybe I can see something with 100 there that might just be it, but nothing.

I now looked over all the maces I have and found that all of the maces (the entire class) I have, have 150% damage to undead (including non-magical ones) (I have over 50 of different types and stats)

So I would assume that actually the hardcoded value is 150%.
I noticed the same thing on wands.

I think I'll assume that stats/properties I can't find/link to txt files are hardcoded and go with that.

Thanks for the tip

[Hans]

It shows 150% since it does not show the +50%. If it stated 50% on the mace then it would do half damage to Undead.

[ciuly]

just a note for future readers so I don't forget:
- magical ethereal items seem to have a 30% drop and not 40% in requirements. (I only found one such item until now)

there's another thing I can't figure out with this specific item. it is
- axe of venom
- ethereal
- one hand damage 6-16 (I decode 4-11)
- required strength 22 (reduced 30% from 32)
- axe class
- item level 21
- +21 poison damage over 4 seconds

I can't figure out how 4-11 grew to 6-16
I have another axe, magical non-ethereal, with +7 poison damage over 3 seconds, that one remained nicely at 4-11. So I figure that this isn't the property influencing the outcome.

So I figured the ethereal flag did some stuff but I can't find a "nice" percentage to grow it with.
to grow 4 to 6 you need a percentage between: 37.5025% and 62.5%
to grow 11 to 16 you need one between: 40.91% and 50%

the only value that looks "nice" and usable for both min and max is 50%
but I'll need another few such items to be able to see if there is indeed such a rule and what's the value.

any suggestions in this direction?

Thanks.

[Necrolis]

You must remember that percentages are truncated, so are the values they produce, tomorrow i'll check up the exact eth boni and post em(I have recoded all of them, however, some are stubs and other don't use the blizzard methods). As for the damage to undead, its possible that heroeditor etc isn't rebaseing the stats to 0 - saveadd and instead using 0(thus 50 would become 50 + saveadd)