Understanding Variable Length Encoding

While reading the MQTT specification, I stumbled upon the Variable Length Encoding section. I’ve seen this technique many times before but never took the time to understand it deeply. This time, I decided to dig in — both for my own clarity and for my work on niniKIT’s VariableLengthEncoder.

The Spec Says…

The Remaining Length is encoded using a variable length encoding scheme which uses a single byte for values up to 127. Larger values are handled as follows: the least significant seven bits of each byte encode the data, and the most significant bit is used to indicate that there are following bytes in the representation. Thus each byte encodes 128 values and a “continuation bit”. The maximum number of bytes in the Remaining Length field is four.

(Source: MQTT 3.1.1 Specification)

Figuring It Out

If we want to send the number 0x4 (0b0100) over a network, we typically reserve 4 bytes for an integer. But in this case, we only need 1 byte.
Why send more than needed?

Example 2:
321 decimal (0x141) → 0b0001 0100 0001
With fixed-length encoding, we’d send 4 bytes. With variable-length encoding, we only need 2 bytes.

How Variable Length Encoding Works

• Bits 0–6 in each byte are data bits.
• Bit 7 is the continuation bit — if it’s 1, another byte follows; if it’s 0, this is the last byte.

The encoding process:

1. Take the lower 7 bits of the number.
2. If more bits remain, set the continuation bit to 1 and continue.
3. Repeat until no bits remain, with the last byte having continuation bit 0.

Example: 321

Below is a step-by-step breakdown from my notes:

The process efficiently splits the number into chunks of 7 bits, with each chunk prefixed by a continuation flag.

Get Debadarshan Parida’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Here is the implemenation in Go.
https://github.com/debadarshana/niniKIT/blob/main/VariableLengthEncoder/VLE.go

Why This Helps

Instead of always sending 4 bytes, we send only as many bytes as needed:

• Small numbers → 1 byte
• Medium numbers → 2 or 3 bytes
• Larger numbers → up to 4 bytes

Benefits

1. Space efficiency — fewer bytes for small numbers.
2. Self-delimiting — no need to send a length field.
3. Protocol-friendly — used in MQTT, in other serialization protocol
4. Simple to implement.

Limitations

• For very large numbers (e.g., 0xFFFFFFFF), this can require more than 4 bytes.
• Not optimal if numbers are uniformly large.