A C++ beautiful syntax evolution with packet serialization

After the first version of Helbreath Empire, I worked on Helbreath Olympia. There is no better place to watch a language change than a game's network code. Every feature a player touches crosses the wire, so the server has hundreds of packets, and whatever it costs to write one, you pay hundreds of times.

1. Bytes by hand

The original Helbreath server, as it was handed down, writes a packet the way you would in C. There is a char buffer and a cursor cp. For every field you cast the cursor to a pointer of the right width, store the value, and move the cursor forward by hand. This is from RequestInitDataHandler, which sends a character's stats when they log in:

dwp  = (DWORD *)(pBuffer + DEF_INDEX4_MSGID);
*dwp = MSGID_PLAYERCHARACTERCONTENTS;
wp   = (WORD *) (pBuffer + DEF_INDEX2_MSGTYPE);
*wp  = DEF_MSGTYPE_CONFIRM;

cp = (char *)(pBuffer + DEF_INDEX2_MSGTYPE + 2);

ip   = (int *)cp;
*ip  = m_pClientList[iClientH]->m_iHP;
cp  += 4;

ip   = (int *)cp;
*ip  = m_pClientList[iClientH]->m_iMP;
cp  += 4;

ip   = (int *)cp;
*ip  = m_pClientList[iClientH]->m_iSP;
cp  += 4;

Three lines per field, and that function goes on for dozens of fields. The bank does the same with single bytes, *cp = cBankItemIndex; cp++;, and reads them back with cBankItemIndex = *cp;. Nothing here is wrong, but nothing helps you either. The 4 must match the type, the reader must walk the same offsets in the same order, and the message length passed to the send call is a number typed by hand. When one side changes and the other does not, the bytes still arrive. They just mean something else.

The only language feature at work is the C cast. It is fast and it is honest about what happens in memory, which is why it stayed so long.

2. A helper: Push

The first step away from that is obvious once you have typed the same three lines a thousand times: put them in a function. Push(cp, value) writes the value at the cursor and advances it by sizeof the value. Only a commented-out line of it is left in the code today (//Push(cp, npc->m_cName, 5);), so this is a sketch of the style rather than a quote:

template <typename T>
void Push(char *& cp, T value)
{
    *(T *)cp = value;
    cp += sizeof(T);
}

Push(cp, int(hp));
Push(cp, int(mp));
Push(cp, int(sp));

A function template and a reference to the pointer are enough. One line per field instead of three, and the width now comes from the type, so the cast and the += 4 can no longer disagree. The cursor is still yours to carry around, though, and the buffer still has no idea how large it is.

3. A packet object with a variadic Write

Helbreath Empire's first commits, in May 2017, already had a packet class over ENet. It owns the buffer, checks for overflow before every write, and writes any number of values in one call. Its variadic template peels off one value and recurses on the rest:

void write() {}

template<typename T, typename ...Args>
void write(T t, Args && ...args) {
    w(t);
    write(args...);
}

template <typename T>
void w(T v)
{
    if(HasOverflowed(sizeof(T)))
        return;
    *GetPointer<T>() = (T)v;
    move(sizeof(T));
}

The bank handler of that version reads a whole request in one line and answers in two:

i8 index, section;
u32 amount;
in->read(index, section, amount);

OutPacket out(112);
out.write(GameMsg::Bank, BankMsg::Store);
out.write(false, index);

Variadic templates came with C++11, and this is where they paid for themselves. The message ids became an enum class instead of a #define, strings got their own overload with a length in front, and an overflow became something the packet notices instead of something memory notices later.

4. Stream operators

The same idea can be written as a stream, each operator returning the packet so the calls chain:

template <typename T>
OutPacket & operator<<(OutPacket & out, const T & v)
{
    out.w(v);
    return out;
}

out << GameMsg::Bank << BankMsg::Store << false << index;

This style is not in the repositories anymore, so again this is the shape, not a quote. It reads very well for writing: the whole packet is one sentence, left to right, in wire order. Operator overloading has been in C++ since the beginning, and std::ostream taught everyone this idiom. Reading is where it gets awkward. in >> index >> section still needs every variable declared beforehand, uninitialized, a few lines above. The variadic call won in the end because it does both directions with the same spelling.

5. Typed reads: values come back

The next step fixed exactly that. Instead of passing variables in to be filled, you name the types and the packet hands the values back. A single value first, already in the August 2017 rewrite:

template<typename T>
T ReadAs()
{
    T value;
    read(value);
    return value;
}

Then several, in August 2025, as a tuple. Today it lives in kazilib's net/packet.h, shared by both games:

// Braced: the reads run left to right
template <typename... Args>
tuple<Args...> As() { return tuple<Args...>{ ReadAs<Args>()... }; }

And the bank request from step 3 becomes this, in Helbreath Millennium's server:

void Bank::NetOp_Store(IPkt * const in)
{
    auto [index, section, amount] = in->As<i8, i8, u32>();
    // ...
}

Three features meet in that one line. A pack expansion (C++11) calls ReadAs once per type. The braces matter: inside a braced initializer the elements are evaluated in order, so the fields come off the wire left to right, which a plain function call would not promise. And structured bindings (C++17) unpack the tuple into named, initialized locals. No uninitialized variables waiting to be filled, and the wire layout is written once, in the angle brackets, where you can read it.

6. ESerializable: objects that send themselves

Single values are only half of it. A game sends items, quests, options, login records: objects with several fields that travel together and appear in many packets. The August 2017 rewrite added an interface for them:

class ESerializable
{
public:
    virtual void write(OutPacket & out) = 0;
    virtual void read(InPacket * in) = 0;
    virtual EPacketSize GetPacketSize() = 0;
};

In the end: two values, then and now

The whole story fits in one small job: read two values out of a packet, a bank slot and an amount. This is how it was done at the start:

char  cIndex;
int * ip;
int   iAmount;
char * cp = (char *)(pData + DEF_INDEX2_MSGTYPE + 2);

cIndex = *cp;
cp++;

ip      = (int *)cp;
iAmount = *ip;
cp     += 4;

And this is how it is done now:

auto [index, amount] = in->As<i8, u32>();

Nine lines became one. The types are written once, in the order they travel, the values come back already named, and no cursor is moved by hand.