Sphere emulator part 4
Pouch
First of all, the client must find out that the loot bag has fallen out. To do this, the server sends a special packet of size 29 (0x1D) byte with object ID and server coordinates:
{
0x1D, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, MinorByte(localId), MajorByte(localId), 0x5C, 0x86, (byte) x_1,
(byte) x_2, (byte) x_3, (byte) x_4, (byte) y_1, (byte) y_2, (byte) y_3, (byte) y_4, (byte) z_1,
(byte) z_2, (byte) z_3, (byte) z_4, (byte) z_5, 0x20, 0x91, 0x45, 0x06, 0x00
};When opening the bag, the client sends a package with a length of 26 (0x1A) byte, which we already saw when picking up an object from the ground. Most likely, similar packages are used for actions with the game world. This time, however, the “package type” is different (0x5C 0x46 0xE1), and the target ID lies in 11 and 12 bytes (0x41 0x00):
1A 00 A8 42 70 27 2C 01 00 AC D3 41 00 5C 46 E1 06 9E DE 00 0A 3A F0 F4 06 00In response, the server gives information about the occupied slots in the bag: the ID of the item and its weight for each slot. Examples of packages for 1-4 occupied slots under the spoiler.
Examples
case 1:
itemList = new byte[]
{
0x19, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, MinorByte(localId), MajorByte(localId), 0x5C,
0x46, 0x61, 0x02, 0x00, 0x0A, 0x82, 0x00, item0_1, item0_2, item0_3, 0x70, 0x0D, 0x00, 0x00, 0x00
};
break;
case 2:
itemList = new byte[]
{
0x23, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, MinorByte(localId), MajorByte(localId), 0x5C,
0x46, 0x61, 0x02, 0x00, 0x0A, 0x82, 0x00, item0_1, item0_2, item0_3, /*weight*/ 0xC0, 0x00, 0x00,
0x00, 0x50, 0x10, 0x84, item1_1, item1_2, item1_3, /*weight*/ 0x00, 0x4B, 0x00, 0x00, 0x00
};
break;
case 3:
itemList = new byte[]
{
0x2E, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, MinorByte(localId), MajorByte(localId), 0x5C,
0x46, 0x61, 0x02, 0x00, 0x0A, 0x82, 0x00, item0_1, item0_2, item0_3, 0x30, 0x00, 0x00, 0x00, 0x50,
0x10, 0x84, item1_1, item1_2, item1_3, 0x00, 0x08, 0x00, 0x00, 0x80, 0x82, 0x20, 0x08, item2_1,
item2_2, item2_3, 0x2C, 0x00, 0x00, 0x00, 0x00
};
break;
case 4:
itemList = new byte[]
{
0x38, 0x00, 0x2C, 0x01, 0x00, 0x00, 0x00, MinorByte(localId), MajorByte(localId), 0x5C,
0x46, 0x61, 0x02, 0x00, 0x0A, 0x82, 0x00, item0_1, item0_2, item0_3, 0x30, 0x00, 0x00, 0x00, 0x50,
0x10, 0x84, item1_1, item1_2, item1_3, 0x00, 0x08, 0x00, 0x00, 0x80, 0x82, 0x20, 0x08, item2_1,
item2_2, item2_3, 0x2C, 0x00, 0x00, 0x00, 0x14, 0x04, 0x61, item3_1, item3_2, item3_3, 0x80, 0x19,
0x00, 0x00, 0x00
};Packages are similar in appearance, and by substituting obviously incorrect IDs in the remaining cells, we can always send the last one (4 items) regardless of the number of items.
Following the information about the occupied slots, the server must send the items themselves. I have not yet learned how to assemble item packages in code without crutches, so welcome to the rabbit hole.
Items
Let’s use the gift of foresight and decide in advance that the same items in the package are always encoded the same way. This is (absolutely) not so, but reason is still more expensive. A few percent of incorrect items, at best, will simply not be displayed, and at worst, they will lead to a client crash. We’ll deal with them later.
Depending on the type of item, packages are divided into several standard groups:
Mantras
Weapons, armor
Powders, alchemical ingredients, monster parts
Rings
Unique (wood, food, tokens, bags, scrolls, formulas, books, etc.)
Package examples:
Header sync id type "static" game_id suffix bag_id count,etc
Мантра “Три звездочки"
28002C0100 029E AE87 A48F 0F80842E090000000000000000409145 E65912 1560 A0F510 A0C0020100
Шлем IX на 90 выносливости
2B002C0100 FC0A 06FB D48B 0F80842E090000000000000000409145 A61613 1560 C05D1F A0900500FFFFFFFF
Усики сколопендры
30002C0100 08C5 87C0 148B 0F80842E090000000000000000409145 A62310 1560 C01018 A0900500FFFFFFFF0516080000
Кольцо воздуха на 34 титул
3C002C0100 E035 A451 E08B 0F80842E090000000000000000409145 26FCA3 4701 0644A3 000A5900F0FFFFFF5F7807B5BB2FB2B4B036B934B7B3 981A 00
Хлебная лепешка
2E002C0100 A485 659E 348A 0F80842E090000000000000000409145 068002 0C9C C00314 B200E0FFFFFFBFC0020100
Малая книга мантр
2D002C0100 763B 4571 6486 0FF0502E09308023198095F7F85F9145 068002 0CC0 D60114 2620A0900500FFFFFFFF
Формула
A8002C0100 0A03 CCF1 908C 0F80842E090000000000000000409145 068002 0C80 A90314 582000A0F00D046919000005FFFFFFFF000200854F00708B0000B08B0000283C43F4000040EB00000000000040E107281400If you look closely, some segments of the packages are similar to each other. In general, the package will include:
Object ID (should match one of the previously sent occupied slots)
Item type (e.g. E08B for regular rings). Stored offset, so the correct value for rings is 760.
Static sequence like 0F80842E090000000000000000409145
Item ID in the game database (files params/group_*.cfg)
Suffix (for example, 4701 for an air ring). For non-unique items without a suffix – always 1560
Container ID (for example, the ID of the bag containing the item)
Additional possible fields:
Quantity (0516080000 for one centipede tendril). For Powders, Ingredients, and Monster Parts, it’s always listed, even if 1. For Mantras, it’s only listed if greater than 1.
Premium level flag (050F08). Indicated only for premium items.
The flag of the item created by the player. It differs depending on the level of specialization.
There is no entry in the game database for unique items. Instead, each individual item has its own type (for example, 551 for a ruby ring and 552 for a diamond ring) and most often, its package. The list of types that I managed to find is under the spoiler.
Types
Token = 8,
Mutator = 30,
SeedCastle = 40,
XpPillDegree = 47,
TokenMultiuse = 66,
TradeLicense = 68,
ScrollLegend = 90,
ScrollRecipe = 91,
TokenIsland = 104,
TokenIslandGuest = 105,
Bead = 236,
BackpackLarge = 400,
BackpackSmall = 401,
Sack = 405,
Chest = 406,
MantraBookSmall = 409,
RecipeBook = 410,
MantraBookLarge = 411,
MantraBookGreat = 412,
MapBook = 413,
KeyBarn = 418,
PowderFinale = 451,
PowderTarget = 453,
PowderAmilus = 454,
PowderAoE = 455,
ElixirCastle = 471,
ElixirTrap = 472,
WeaponSword = 500,
WeaponAxe = 501,
WeaponCrossbow = 502,
Arrows = 503,
RingDiamond = 551,
RingRuby = 552,
Ruby = 553,
RingGold = 555,
AlchemyMineral = 600,
AlchemyPlant = 601,
AlchemyMetal = 602,
FoodApple = 650,
FoodPear = 651,
FoodMeat = 652,
FoodBread = 653,
FoodFish = 655,
AlchemyBrushwood = 700,
Key = 701,
Map = 703,
Inkpot = 704,
Firecracker = 705,
Ear = 706,
EarString = 708,
MonsterPart = 709,
Firework = 712,
ArmorChest = 750,
ArmorAmulet = 751,
ArmorBoots = 752,
ArmorGloves = 754,
ArmorBelt = 755,
ArmorShield = 756,
ArmorHelmet = 757,
ArmorPants = 758,
ArmorBracelet = 759,
Ring = 760,
ArmorRobe = 761,
RingGolem = 762,
AlchemyPot = 800,
Blueprint = 804,
QuestArmorChest = 949,
QuestArmorAmulet = 950, // unused?
QuestArmorBoots = 952,
QuestArmorGloves = 953,
QuestArmorBelt = 954,
QuestArmorShield = 955,
QuestArmorHelmet = 956,
QuestArmorPants = 957,
QuestArmorBracelet = 958, // unused?
QuestArmorRing = 959, // unused?
QuestArmorRobe = 960,
QuestWeaponSword = 961,
QuestWeaponAxe = 962,
QuestWeaponCrossbow = 963,
SpecialGuild = 976, // sometimes different
SpecialAbility = 977, // same type for specialization itself
ArmorHelmetPremium = 990,
MantraWhite = 1000,
MantraBlack = 1001,I don’t really want to understand such a zoo of options, and correctly separating the fields in each package is entertainment for at least a couple of months (the same data in them can lie in different places with different shifts, I tried). Fortunately, we can take ready-made data from the live server, replace the minimum required set of fields in them and give it to the client, right?
If you answered yes to this question, you unfortunately didn’t read my previous articles, because the Sphere likes to hurt differently.
Okay, I lied a little – the ready-made data from the live server is enough for situations where the client receives exactly one item. In the next section, we will see how to assemble a set of packages for all occasions, but for now, under the spoiler, an example of encoding. Standard streams in C# do not support non-integer byte shifts, so here and below I use my fork bitstream with a few additional helper methods.
Encoding
stream.WriteUInt16(Id);
stream.WriteBits(_skip1);
stream.WriteUInt16(Type, 10);
stream.WriteBits(_skip2);
stream.WriteBytes(X, 4, true);
stream.WriteBytes(Y, 4, true);
stream.WriteBytes(Z, 4, true);
stream.WriteBytes(T, 4, true);
stream.WriteUInt16(GameId, 14);
if (FourBitShiftedSuffix)
{
stream.WriteUInt16(SuffixMod, 12);
}
else
{
stream.WriteByte((byte) SuffixMod);
}
stream.WriteBits(_skip3);
stream.WriteUInt16(BagId);
stream.WriteBits(_skip4);
if (ObjectType is ObjectType.Arrows or ObjectType.Bead or ObjectType.Ruby or ObjectType.Token
or ObjectType.AlchemyBrushwood or ObjectType.AlchemyMetal or ObjectType.AlchemyMineral
or ObjectType.AlchemyPlant or ObjectType.ElixirCastle or ObjectType.ElixirTrap or ObjectType.FoodApple
or ObjectType.FoodBread or ObjectType.FoodFish or ObjectType.FoodMeat or ObjectType.FoodPear
or ObjectType.MantraBlack or ObjectType.MantraWhite or ObjectType.MonsterPart or ObjectType.PowderAmilus
or ObjectType.PowderFinale or ObjectType.PowderTarget or ObjectType.RingDiamond or ObjectType.RingRuby
or ObjectType.SeedCastle or ObjectType.TokenIsland or ObjectType.PowderAoE)
{
stream.WriteUInt16(Count);
}Separators and bit shifts
Items in the package are written linearly ([предмет1] [разделитель] [предмет2] […]). Theoretically, we could divide it into sequences of known length, and then parse each of them as a separate item. This approach will not always work:
An item can take a non-integer number of bytes, the final shift in the stream depends on the type of item
Before some suffixes, you need to add an additional 4 bits, otherwise the client will not recognize them
Player crafted item marker is different for each rank of each specialty
For premium items, fixed 3 bytes and a marker are added (different sequences for 1 and 2 premium levels)
Some items just need a longer sequence
The biggest problem is in the last paragraph. Let’s take, for example, 5 different helmets without suffixes, which differ only in the ID in the game database (the client determines the characteristics, requirements, etc. by this ID). Writing four of them will take a conditional 30 bytes, and writing the fifth one will take a conditional 35 bytes. In this case, the packages for the first four helmets will be the same with ID accuracy, that is, we can store one package and replace the ID in it with the desired one before sending. For the fifth helmet, such an operation will cause the client to crash. I couldn’t find any pattern.
Let’s go back to the list of item packages (above) one more time. Subsequence 0F80842E090000000000000000409145 in the middle of the packet always starts with 11 bytes and occupies 16 bytes, but according to the small book of mantras (0FF0502E09308023198095F7F85F9145) you can see that its contents can change. After looking at a couple dozen more thousand packets, we will notice that the first byte is always 0x0F and the last 4 bytes must satisfy a set of conditions:
Byte 0 – LSB is 8, 9, or 0
Byte 1 – MSB is 4 or 5
Byte 2 – odd number
Byte 3 is equal to 0x44 or 0x45
It remains to make a filter out of these conditions (any packets that contain the desired byte sequence will do) and divide each packet into separate items. The shift in the stream is not known in advance, so you have to search bit by bit. Code example:
try
{
while (containerStream.ValidPosition)
{
var test = containerStream.ReadBytes(4, true);
if (!containerStream.ValidPosition) break;
containerStream.SeekBack(32);
if (IsObjectPacket(test))
{
var pos = (containerStream.Offset - 16) * 8 + containerStream.Bit;
var currentOffset = containerStream.Offset;
var currentBit = containerStream.Bit;
containerStream.Seek(currentOffset - 14, currentBit);
containerStream.ReadBits(2);
var typeCheck = containerStream.ReadUInt16(10);
if (Enum.IsDefined(typeof(ObjectType), typeCheck))
{
offsets.Add(pos);
}
else
{
Console.WriteLine($"Unknown: {typeCheck} at {currentOffset} {currentBit}");
}
containerStream.Seek(currentOffset, currentBit);
}
containerStream.ReadBit();
}
}
catch (IOException)
{
}
finally
{
if (offsets.Count > 0)
{
containerStream.Seek(offsets[0] / 8, (int) (offsets[0] % 8));
}
}
offsets.Add(containerStream.Length * 8);The code can definitely be optimized, for example, to move forward along the stream by the minimum packet size each time the sequence with the item is found. The protocol, however, does not guarantee that a ping or something else will not get between parts of one long packet (for an inventory whose length is guaranteed to be more than 1400 bytes, a structure is possible [инвентарь1][пинг][инвентарь2]). Instead of handling exceptions, you can also take something decent.
It is better to save the resulting packages, they will be useful to us the next time we finally put on the dropped items. As a free bonus, we get object separators in the package. The main thing is not to round the length of records of individual items to an integer number of bytes, otherwise the client will not cope with them. The storage structure is something like this:
[BsonId]
public int DbId { get; set; }
public ushort Id { get; set; }
public Bit[] _skip1 { get; set; }
public ushort Type { get; set; }
public Bit[] _skip2 { get; set; }
public byte[] X { get; set; }
public byte[] Y { get; set; }
public byte[] Z { get; set; }
public byte[] T { get; set; }
public ushort GameId { get; set; }
public ushort SuffixMod { get; set; }
public Bit[] _skip3 { get; set; }
public ushort BagId { get; set; }
public Bit[] _skip4 { get; set; }
public ushort Count { get; set; }
public bool IsPremium { get; set; }
public Bit[] _premiumSkip { get; set; }
public Bit[] _strangeSkip { get; set; }
public string FriendlyName { get; set; }
public SphGameObject? GameObject { get; set; }
public byte[] Packet { get; set; }
public long BitsRead { get; set; }
public bool FourBitShiftedSuffix { get; set; }
public bool IsStrangeSuffix { get; set; }
public ObjectPacketEncodingGroup EncodingGroup { get; set; }
public ObjectType ObjectType { get; set; }Any database will do, in my case – LiteDB. For early beta versions of Godot (4.0 beta 2 and around), it was necessary to change the target version of .NET to 4.7/4.8 in order for the project to build, with new ones there is no such problem.
By the way, for experiments you do not need to get loot from monsters, just open the trade window with any NPC – they store items in exactly the same form, and the package will immediately contain a whole set of necessary data.
Now it remains only to get the necessary records from the database, replace the set of fields in them, write them into a package in a row and give them to the client. If there is no mistake anywhere, the client must correctly recognize the objects.
Code example
ID is the number of a specific package from the collected base, in your case they will be different.
var weaponArmorNotShiftedId = 243;
var weaponArmorShiftedId = 153;
var ringNotShiftedId = 666;
var ringShiftedId = 637;
var mantraId = 354;
var alchemyId = 374;
var powderId = 147;
var foodAppleId = 2161;
// var keyId = 296;
// var mantraBookId = 317;
// var tokenId = 330;
// var diamondRingId = 569;
ObjectPacket result;
var objectType = item.ObjectType.GetPacketObjectType();
var suffixMod = item.Suffix == ItemSuffix.None
? (ushort)81
: (ushort)GameObjectDataHelper.ObjectTypeToSuffixLocaleMap[item.ObjectType][item.Suffix].value;
var dbId = -1;
if (GameObjectDataHelper.WeaponsAndArmor.Contains(item.ObjectType))
{
dbId = suffixMod > 1000 ? weaponArmorShiftedId : weaponArmorNotShiftedId;
}
else if (GameObjectDataHelper.Mantras.Contains(item.ObjectType))
{
dbId = mantraId;
}
else if (GameObjectDataHelper.Powders.Contains(item.ObjectType))
{
dbId = powderId;
}
else if (GameObjectDataHelper.AlchemyMaterials.Contains(item.ObjectType))
{
dbId = alchemyId;
}
else if (item.ObjectType is GameObjectType.Ring)
{
dbId = suffixMod > 1000 ? ringShiftedId : ringNotShiftedId;
}
else if (item.ObjectType is GameObjectType.FoodApple)
{
dbId = foodAppleId;
}
if (dbId == -1)
{
Console.WriteLine(
$"NOT FOUND: Type: {Enum.GetName(item.ObjectType)} Suffix: {suffixMod} {Enum.GetName(item.Suffix)}");
dbId = 4;
}
result = MainServer.LiveServerObjectPacketCollection.FindOne(x => x.DbId == dbId);
var client = MainServer.ActiveClients!.GetValueOrDefault(clientId, null);
if (client is null)
{
return null;
}
result.Id = client.GetLocalObjectId(item.Id);
if (objectType is not ObjectType.FoodApple)
{
result.GameId = (ushort)item.GameId;
result.SuffixMod = suffixMod;
}
var bagLocalId = Client.GetLocalObjectId(clientId, bagId);
result.BagId = bagLocalId;
result.Count = (ushort)item.ItemCount;
result.GameObject = MainServer.GameObjectCollection.FindById(item.GameObjectDbId);
if (objectType is not ObjectType.FoodApple)
{
result.FriendlyName =
MainServer.GameObjectCollection.FindById((int)result.GameId)!.Localisation[Locale.Russian];
var type = result.GameObject.ObjectType;
result.Type = (ushort)objectType;
if (GameObjectDataHelper.ObjectTypeToSuffixLocaleMap.ContainsKey(type))
{
result.GameObject.Suffix = GameObjectDataHelper.ObjectTypeToSuffixLocaleMap[type]
.GetSuffixById(result.SuffixMod);
}
}
return result;That’s all, next time we’ll look at trading with vendors, try on freshly picked things and finally go out into the outside world.
See you!
