Save file structure

[jeff_childers]

I understand at least I assume I understand.

here are the bits for the small red potion

01001010 01001101 000100000000000010100000000000000110010100000000000000001000 00100000011000010111000000110000 001

the first 16 is JM
followed by 60 bits to the start of the type field:
00100000011000010111000000110000
now if I convert 'poti' to it binary form:
1110000 1101111 1110100 1101001
my item has 9 'on' bits
what the field should be needs 17 'on' bits. Is there a differant way to convert the field to ascii.

[SVR]

where did you get "poti" from ?
The code for small red potion is rps_
I suspect you have the bits swapped around in your list there.
What order are they in ?
I assume you are listing them in byte order because the JM are Hi bit first. you can't look at it this way.
the actual order is Lo bit first...

Code: Select all

your order
01001010 01001101 00010000 00000000 10100000 00000000 01100101 
00000000 00000000 10000010 00000110 00010111 00000011 0000001?

correct order
01010010 10110010 00001000 00000000 00000101 00000000 10100110 
00000000 00000000 0100[color=#ff0000]0001 0110[/color][color=#0000ff]0000 1110[/color][color=#ff0000]1000 1100[/color][color=#0000ff]0000 ?100[/color]0000

[color=red]0001 0110[/color] -> 0110 1000 = 68h = h
[color=blue]0000 1110[/color] -> 0111 0000 = 70h = p
[color=red]1000 1100[/color] -> 0011 0001 = 31h = 1
[color=blue]0000 ?100[/color] -> 001? 0000 = 20h = _

hp1_ = Lesser Healing Potion

You have to remember it's a bit stream, read left to right, Lo bit first.
We are used to reading numbers right to left, takes some getting used to;)

[jeff_childers]

I believe some of my problems resides in the fact that I am on a big endian machine (OSX). For the most part I have been decoding by changing how I decode the bits to match what the expected result should be. Thus using HI bit on my machine JM characters are decoded correctly and are not when I use lo bit first as many of the other fields are not yielding the expected result.

I had assumed that the item type was from ItemTypes.txt. Which is where I found 'poti'. After looking a bit I can't seem to find what file contains the item type abbreviations. I take that back is it the cost column in misc.txt?

Any suggestions? BTW thanks for your help it has been appricated.

[SVR]

Big/little endian makes no difference with bytes. Thats why they use a bitstream.

Yes the item codes are the ones in misc,armor & weapons.txt

When you say JM decoded correctly, it shouldn't ;-)
if you are reading right it should come out backwards.

Study the order of bits I listed and you will see *why* you read Lo bit first.
It is so the fields are contiguous. If you reverse the bits, the fields become split.

If you read the whole item into a byte array you can read the bits one at a time from there into your variable(s), lo bit first.

psuedo code ...

set variable to 0.
set outPtr to 0
bitPtr and bytePtr are where the were from last read.

get the Bit "bitPtr" from buffer[bytePtr],
bump the bitPtr,
if its 8 then bump the byte ptr and set bit ptr to 0
if Bit = 1 then Variable = Variable + (2^outPtr)
bump outPtr
if outPtr < bitsToRead then goto nextBit
done

for bit position 76, bytePtr would be 9 and bitPtr would be 4 assuming 0 as starting point. (basic usually counts from 1)

It doesnt matter how many bits Variable is or if its big/little endian it will come out the correct value if you read the correct number of bits.

Variable will contain a correct 'J' after reading 8 bits starting at position 0.
Variable will contain 'J' in the lo byte and 'M' in the high byte if you read 16 bits, regardless of endingness.

Hope that clears it up a bit.

[arcnon]

SVR, I looked at your web site and it has helped me very much... thanks. on the types.htm page you note a comparision with bIsCharm where is this defined? Also SType is compared to book scroll and body where is this defined?

[SVR]

they are added in code. Stype is the string in itemTypes.txt, bIsCharm is the charm column.
You'll also find an iType (I think) and it is what file the Item is defined in (misc,armor,weapons).
I never figured out the logic the game uses to determine which is which.

Also, bNoDur. Tells if the item has durability or not. It's a composite of the NoDurability flag and the iType I think. misc don't have durability and NoDurability means it's not used/display (but still there ?).

I think the logic is in a post a while back.

[jeff_childers]

I guess I am not understanding what you are saying.

To get the value of sType I have to read it from the structure somewhere so I can check the index and see if sType == book,scro to include this field. Same with the bIsCharm and iType. Where/how do I read these fields in the structure or logically extrapulate these values so I can do my comparision?

[SVR]

You missed the key phrase "in itemtypes.txt". They are not in the structure. You have to lookup the itemcode in the text files and extract the information.

Use itemcode to get the misc,armor or weapon record.
Use this record to get the type, etc.
Use type to lookup additional info in itemtypes.txt

[Yohann]

+00ab from [ptGame+78], should be initial seed of game

0x00AB is the ID of the MAP (the starting seed value) not of the game.

[-=Wraith-Lunati=-]

#include <iostream>

using namespace std;

struct goof
{
char *question[];
};

goof newbie;

newbie.question = "k so I've read the whole thing on getting the checksum and tried to write my own C++ version of it with absolutely no success...anyone know what the code would be in C++? and yes I realise this isn't a coding forum or a code helper site (hence why I didn't post the code I came up with) but I'm running out of options, cigarettes and energy. thx in advance. P.S. I saw the perl example and have a verion in C but I don't know much C even tho they are supposedly pretty much the same. /hangsheadinshame"

int main()
{
cout << newbie.question << endl;
}

-EOF

BTW: yes I realise that you can't compile this into an actual working program...just trying to have a lilttle C++ fun

[SVR]

Hmm, I thought this was posted somewhere but I can't find it.

C version, kinda odd because there is no rotate operator so you have to test the high bit and add.

Code: Select all

// pucData - pointer to the byte stream of the .d2s file 
// iSize - number of bytes in the stream ( filesize ) 
DWORD Checksum( unsigned char *pucData, int iSize ) 
{ 
    // delete old checksum at offset 0x0C 
    *((unsigned int*)(pucData+12)) = 0; 

    // init new checksum with 0 
    unsigned int uiCS = 0; 

    // this is the whole checksum calculation 
    for ( int i = 0; i < iSize; ++i ) 
        uiCS = (uiCS<<1) + pucData[i] + ( uiCS & 0x80000000 ? 1 : 0 ); 

    // write new checksum to stream 
    *((unsigned int*)(pucData+12)) = uiCS; 
    return uiCS;
} 

inline asm version...
(mucho faster,nicer ;-)

Code: Select all

DWORD asmChecksum( unsigned char *data, int cnt )
{ 
__asm {
    mov eax,0           // temp sum in eax
    mov edi,data        // data ptr in edi
    mov ecx,cnt         // count in ecx
    mov [edi+12],eax    // zero crc dword in data
    xor ebx,ebx
    }

// this is the whole checksum calculation 
LOOP1:
__asm {
    rol eax,1           // rotate sum
    mov bl,[edi]        // add data byte
    add eax,ebx
    inc edi             // move data ptr
    loop LOOP1          // repeat 'cnt' times
    }

// write new checksum to stream 
__asm {
    mov edi,data        // save checksum
    mov [edi + 12],eax  // back to data record
    }
    return *(DWORD *) (&data[12]);
}

[Paul Siramy]

Hmmm, I was about to write that this line was wrong

Code: Select all

for ( int i = 0; i < iSize; ++i )

and that it should be

Code: Select all

for ( int i = 0; i < iSize; [color=red][b]i++[/b][/color] )

But... I just tried with VC6, and the result is the same : the 1st time 'i' will be used it will be 0 (and not 1 like I first tough).

So, this post is mostly un-necessary, except for clarifications

[SVR]

I never even noticed that !

The C code is from Stoned2000 I believe, I use my asm version in UdieToo.
Not sure of the rule for pre-increment in for loops. All these years I've only used post

[Havvoric]

The for loop expands to the following while:
int i=0;
while(i<iSize)
{
...
++i;
}

Therefore, the order of the increment does not matter, as the result is not pre- or post- assigned to anything. Hope this clarifies, on this technical point!

[Wraithan]

Edit: Nevermind think I can figure it out from the C version above that I seem to have missed...

[Nameless]

Does anybody know if there are any bits in the header (or somewhere else) that I can set in a program that modifies the save file which get reset when D2 saves the file?

What I want to do is to make a backup file of the save file only when the save file comes directly from D2 (and so is "known-good") but not if my program has already modified the file (e.g. you run the program on the save file twice with different options).

edit:
could I use the timestamp?
e.g. set it to 0 when I modify the file.

[Jarulf]

[quote=Nameless";p="278280"]Does anybody know if there are any bits in the header (or somewhere else) that I can set in a program that modifies the save file which get reset when D2 saves the file?

What I want to do is to make a backup file of the save file only when the save file comes directly from D2 (and so is "known-good") but not if my program has already modified the file (e.g. you run the program on the save file twice with different options).

edit:
could I use the timestamp?
e.g. set it to 0 when I modify the file.[/quote]

Check out the few flag entries at the start of the file. I think some are unused. There are also some values set to specific values as noted, I don't remember if/how the game check them upon load, but it might exist some that is not checked and thus unused and hence could also be used. Experiment It should all be in ithe initial main header part though.

[Nameless]

The timestamp seems to work fine, so I used that.

[Careface]

how do you play a mod with plugy? on the readme it doesnt say anything about playing with additional mod that is in a folder

[Nefarius]

This topic is about the savefile architecture.
Post in the PlugY / D2Mod forum about PlugY questions.

[linuxcat]

I started to write a character decoder in php. There is still a bit left to do and several parts that are untested, but it works well if you want to get the general idea of what the character looks like. I think it would work well linked from a ladder. If a modder wanted to do that for their mod, this would be a nice start.
The only line you should have to change is the path to the savefiles. This is the 3rd line of index.php You may want to also change the header line. If someone types in something invalid I use header() to push them back to the ladder. I commented that out of each of the php files before zipping them up. The entire download is 45k.
charinfo.zip
I use the code via a link like this:
http://your.ladder.page/charinfo/?name=charname

What is complete:
Character class, ladder, expansion, has died, hardcore, title, level, skill hotkeys, selected attacks, town, merc type (class, name, exp, special ability), some quest info (have you used: imbue, cow level, socket, personalize), waypoints, stats (strength, energy, dexterity, vitality, life, mana, stamina, level, exp, gold), skills, item list, merc item list, golem item.

What is left to do:
I think the offsets for the quests are correct. I just started new characters and have not made it to the end yet. all the way through nightmare socket is correct.
I have not tested the player greetings.
I got the unique names from UniqueItems.txt and not the language file. Some of them need to be rephrased to match the game. I will just go down the list at battle.net/diablo2exp/ and change them sometime soon(ish).
I do not have any code to decode the corpse item list. I will need to die and not collect my body to fix that...
iron golem item is untested...I do not have a character with one to test.
item decode is not complete. The magic prefixes and suffixes and item types decode well. Most of the stats decode, but they have changed enough from the specs I found that they do not decode completely.

I basically got my information from 3 places:
http://www.xmission.com/~trevin/DiabloI ... rmat.shtml
http://home.stx.rr.com/svr/formats/d2s.htm
This forum.

Enjoy,

Jack

EDIT:
The item for the iron golem (if any) shows up, the items on your body (if any) show up, and the unique names should now be correct.

[natnit]

Ok, so I've read through this thread about 10 times through (literally), I'm sure I'm missing something though. Sorry to dig up an old thread, but I'm desperate!

I've managed to implement the checksum nicely, and parse all the data leading up to the stats interface, and that's where I'm having major trouble.

I'm using the read_bits definition that people have suggested, as well as Adrian's breakdown (i.e. read 9 bit key, store 10-bit strength if its 0x0, etc).
Strength comes out right, but the second key is 0x4 (=stat points) and that gives me 40? I'm almost certain I'm doing something wonky. Here's what I have starting from the "gf":

Code: Select all

002820C003081881010F402003050A0C00F8820300BE000180164800A005140044820500918081A1C1E10A00C07F

I am working with 1.12...so that may be the issue. Further, the fact that I'm running MXL could be a problem too. Don't worry, I'm not trying to cheat or anything: I have Laz's blessing (trying to write up a skill reallocator, and will only release it to him). Any and all help is greatly appreciated.

Edit: Uninstalled MXL and it works mostly ok now. Contacted BrotherLaz for ItemStatCost.txt, so hopefully I can work on that later. I played with a fresh cLOD barbarian, and got the following output from my parser:

Code: Select all

i = 0, cursor = 19, strength = 30
i = 1, cursor = 38, energy = 10
i = 2, cursor = 57, dexterity = 20
i = 3, cursor = 76, vitality = 25
i = 6, cursor = 106, cur_life = 14080
i = 7, cursor = 136, max_life = 14080
i = 8, cursor = 166, cur_mana = 2560
i = 9, cursor = 196, max_mana = 2560
i = 10, cursor = 226, cur_stam = 23552
i = 11, cursor = 256, max_stam = 23552
i = 12, cursor = 272, [acronym="Character Level"]clvl[/acronym] = 1

Looks like the fields that are 21-bits wide are acting up, and that is both trying the #define read_bits and the _SL_ReadBits solutions.

Thanks again!

[Necrolis]

They aren't acting up, they are just unshifted, hp, mana and stamina use 8 bit precision, so you need to >> 8 or /256 to get the in game value

[natnit]

Ahh, silly me, should've noticed that!

Thanks for the quick reply Necrolis. Made the necessary changes and now that part is working perfectly.

[ciuly]

Hello,

I am trying to get a hold of the fileformats for d2s and d2i. I am mainly interested in d2i (I'm working on an automated muling application (basically an item dumper). I can dump all items with one problem: socketed items loose their fillings (and I get the fillings separately). So I am trying to find a way around this. Currently I'm working for v1.10 but all the info I have found on the net is ... apparently wrong.
for example the 1.10 format from SVR
he says bSocketed is bit number 28 (offset 27, calculated based on his table)
Trevin, in his 1.09 format also says it's at offset 27.
BUT, I did the following. found a normal shield in the game. saved it with atma as d2i. used the socket formula to add some sockets (all this to have as little changes in the item itself as possible). I compared the binary data in a hex editor and I found that the socket bit is at offset 28 (bit number 29). counting from left to right, the socketed bit is in the 4th byte (counting from 1), the 5th bit if counting bits from 1 and the 4th bit if counting them from 0 (again, left to right). so we have byte 4: 00001000 for a socketed item. but both formats mentioned expect it to be 00010000.
Same problem I have with ethereal bit. In my tests it's bit 34 (offset 33). in both formats above however, the placement is way off (not 1 bit as above, but more). I didn't test anything else since I'm not particulary interested in anything else, but I suspect most of the stuff from SVR is wrong OR for another version and not for 1.10 as advertised.
I made a small util to convert the binary d2i to xml format and that's how I noticed the issues. Otehr values are also wrong (my code is based on SVR format),
I tried contacting tenshi but his mailbox over hotmail is no longer available. I'll give SVR a PM with this post in case he's still around.

Anybody has soemthing I can use? I could waste a few days to figure things out but I'd rather not
thanks.