MQTT Payload Design for IIoT: JSON vs Sparkplug B

An MQTT topic answers where a message is routed; its payload must answer what the bytes mean, when the measurement applied, and whether a consumer may use it. MQTT 5.0 deliberately leaves that application contract open. It provides PUBLISH metadata such as Payload Format Indicator, Content Type, and User Properties, but it does not define an IIoT JSON object, a unit system, or a quality model. A durable payload design therefore starts with semantics before serialization.

This article owns the payload boundary: envelopes, scalar and structured values, timestamps, quality, units, validation, and change control. It compares JSON with a pinned Sparkplug/Protobuf vector without claiming a universal byte, throughput, security, or interoperability result. Namespace, birth/death, retained lifecycle state, and spBv1.0 message rules belong to the companion Sparkplug lifecycle article; use it when the system needs that application protocol rather than merely a well-defined JSON contract.

Start with an envelope a receiver can interpret

An envelope separates transport context from the measurement itself. It should let a receiver identify the schema, identify the producing source, determine the measurement time, and decide whether the value is usable. The smallest useful shape depends on the system, but the fields need unambiguous definitions. For example:

{
  "schema": "com.example.iiot.telemetry/1",
  "source": "plant-a/line-2/press-07",
  "observedAt": "2023-11-14T22:13:20.000Z",
  "sequence": 7,
  "metrics": {
    "temperature": { "value": 24.5, "type": "number", "unit": "Cel", "quality": "good" }
  }
}

schema is an application identifier, not an MQTT feature. source identifies the producing contract and must not be assumed trustworthy merely because it appears in the body; deployment authorization should bind publisher identity and permitted topic/source combinations. observedAt is the time the source observed the value, whereas broker arrival time is a separate operational fact. sequence can help detect gaps or duplicates within one producer epoch, but it is not a globally ordered event ID.

Keep fields orthogonal. Do not overload timestamp to mean both sampled time and upload time, or status to mean communication health, engineering validity, and alarm severity. If a gateway needs both source and gateway timing, publish named fields such as observedAt and ingestedAt, each with a documented clock and precision. ISO 8601 UTC strings are readable in JSON; an integer epoch can be compact and avoids timezone parsing, but its unit and range must be explicit. Either choice needs a clock-synchronization and out-of-order policy.

Preserve type, unit, and quality as data

A scalar telemetry value is one measurement with one value: temperature, valve position, or a counter. A structured value represents a coherent object: a vibration spectrum, three-axis acceleration, a batch result, or a device configuration snapshot. Do not flatten structured telemetry into invented names such as axis_1_42 when the shape matters to consumers. Conversely, do not wrap every scalar in a large object if a documented scalar contract is enough.

JSON has only number, string, boolean, null, array, and object types. A JSON number does not distinguish a 32-bit integer from a 64-bit counter, a decimal quantity from a floating-point approximation, or a code that only looks numeric. A JavaScript consumer can lose integer precision above its safe-integer range; another decoder may retain a wider integer. For identifiers, sequence values with a defined range, money-like values, and values requiring exact decimals, state the representation explicitly and test the actual producer and consumer libraries. A quoted decimal may be safer for exact transfer, but then consumers need validation and conversion rules.

The type field in the example makes the intended meaning visible, but a schema can also make it implicit. Choose one approach and document it. A numeric engineering value needs a unit such as Cel, kPa, rpm, or a locally governed code. A bare key named temperature is not enough: different components can legitimately use Celsius, Fahrenheit, or a raw sensor count. Adopt a naming convention that describes the quantity rather than the display label, publish the unit beside the value or in an immutable schema, and define conversion ownership. Do not silently convert at multiple hops.

Quality answers a different question from type. good, uncertain, and bad are useful only when their meanings and transitions are documented. A quality value can convey sensor failure, substituted data, a communication timeout, or an out-of-range check, but those are not interchangeable. Include an optional bounded reason code where operations need diagnosis; do not expose stack traces, credentials, personal data, or unrestricted device errors in a telemetry payload. Consumers should make a conscious decision about whether a non-good value updates a display, historian, calculation, or control workflow.

Make absent, null, and stale distinct states

Missing, null, and stale values are three different signals. A missing metric can mean “this producer version does not support it,” “this update does not change it,” or “the producer omitted it incorrectly.” null can mean “known but unavailable,” but only if the schema explicitly reserves that meaning. Stale usually means the last value was valid when observed but is now older than the receiving application’s freshness budget. It should be derived from observedAt, arrival time, and a documented timeout rather than inferred from an arbitrary zero or empty string.

For a partial-update topic, omission may correctly mean “unchanged.” For a snapshot topic, omission may mean “not present in this snapshot.” Those contracts cannot share one decoder rule. Name the message mode or schema accordingly, and keep retained operational state separate from an event stream. A retained snapshot may help a late subscriber learn the last published representation; it is not evidence that the measurement is still fresh. An event records something that happened once and should carry an event identifier, occurrence time, and retention policy appropriate to that fact. A state message represents the latest known condition and needs expiry or freshness handling.

This distinction prevents a common failure: treating a broker-delivered retained JSON document as live process truth. A receiver must apply its own freshness rule even when QoS delivered the message successfully. MQTT QoS governs a client-to-broker delivery interaction; it does not certify sensor validity or create exactly-once business processing. Design duplicate handling and stale-data behavior at the application boundary.

Evolve schemas without making old consumers guess

Version the contract from the first message. A major version in schema, such as com.example.iiot.telemetry/1, gives consumers a clear compatibility boundary. Additive optional fields can be compatible only if old consumers ignore unknown fields and new consumers have defaults for absent fields. Renaming a key, changing its unit, changing scalar to object, changing a number to string, or assigning new meaning to null is a semantic change even when the JSON still parses. Publish a new major schema or topic and support a deliberate migration window.

Canonicalization is equally important when a payload is signed, hashed, deduplicated, or compared in tests. JSON object member order is not semantic, whitespace is not a reliable identity, and number formatting can vary between encoders. Define the exact canonicalization algorithm before using a JSON byte sequence as an identifier. Otherwise hash a stable application representation rather than pretending arbitrary JSON bytes are equivalent. This illustration uses sorted keys and compact separators; it does not establish a production-wide canonicalization standard.

Keep a schema registry or versioned repository where producers and consumers can inspect required fields, allowed types, ranges, unit rules, nullability, examples, and deprecation dates. A human-readable schema document is useful, but a machine-enforced validator at the trust boundary is what rejects malformed input before it enters a historian, dashboard, or control-adjacent workflow.

Validate in stages. First reject a message whose MQTT topic, authenticated publisher, Content Type, or declared schema is not permitted for that intake. Then apply cheap byte and depth limits before JSON parsing, validate the parsed structure against the selected schema, and finally apply semantic checks such as permitted units, plausible ranges, monotonic counters, and source-specific freshness. Validation failures should produce a bounded reason code and metric, not an exception dump containing the payload. A rejected message must not overwrite the last known good state; whether it is quarantined, dropped, or retried is an explicit operational policy.

Measure one vector; do not generalize it

JSON repeats names and punctuation, while Protobuf encodes fields by numeric tags and typed wire values. That difference can matter on constrained links, but the result depends on field names, values, optional metadata, batching, compression, and the actual encoder. The following self-contained example compares exactly one compact JSON object with one hand-constructed vector based on the pinned Eclipse Tahu schema. It is intentionally an offline byte calculation, not a broker or device test.

python3 - <<'PY'
import json,struct
def v(n):
 b=bytearray()
 while n>127:b.append((n&127)|128);n>>=7
 b.append(n);return bytes(b)
def out(n,b,h=False):print(f'{n} bytes={len(b)}'+(f' hex={b.hex()}' if h else ''))
j=json.dumps({'metrics':[{'name':'temp_c','type':'Double','value':24.5}],'seq':7,'timestamp':1700000000000},sort_keys=True,separators=(',',':')).encode()
m=b'\x0a'+v(6)+b'temp_c'+b'\x20'+v(10)+b'\x69'+struct.pack('<d',24.5)
s=b'\x08'+v(1700000000000)+b'\x12'+v(len(m))+m+b'\x18'+v(7)
out('json',j);out('sparkplug',s,True)
PY

json bytes=94
sparkplug bytes=30 hex=0880d095ffbc3112130a0674656d705f63200a6900000000008038401807

Batch deliberately and preserve message boundaries

Batching several measurements can amortize MQTT topic and framing overhead, but it increases the loss and retry unit, peak memory, decoding latency, and ambiguity when one record is invalid. A batch should state whether every item has its own observedAt, whether ordering is meaningful, how duplicates are identified, and whether the consumer may process a valid subset. Limit item count, decoded bytes, nesting depth, string length, and processing time before parsing untrusted data. Reject or quarantine oversized and malformed input with bounded diagnostics; do not log entire payloads if they can contain sensitive operational or personal information. Privacy review should classify sources and fields before publication, then set retention and access rules for diagnostics as well as telemetry storage.

Compression is a separate decision from serialization. It can help when the payload is large and repetitive enough to justify it, but small messages may not benefit after headers and implementation cost. It also makes inspection harder and can expose resource-exhaustion risks if a receiver decompresses unbounded input. Enforce compressed and decompressed size ceilings, authenticate and authorize publishers, use TLS for broker connections, and review topic permissions. Payload validation does not replace broker authorization; broker authorization does not replace payload validation.

Choose a protocol, then keep the boundary observable

JSON is often a good choice when teams need inspectable messages, broadly available tooling, and a contract they can validate clearly. A structured binary protocol can be appropriate when its schema, generated implementations, and lifecycle semantics match the participating system. Sparkplug adds its own agreed namespace and lifecycle model; this article does not repeat those rules. The useful choice is not “JSON versus binary” in the abstract, but a documented contract whose consumers can reject safely, evolve predictably, and diagnose in production.

At each receiver, record bounded metadata: schema version, source identity, payload size, validation result, freshness decision, and a correlation or sequence value where permitted. Avoid treating raw payload logs as a default diagnostic store. Create migration adapters that read the old schema, validate it, emit the new schema, and expose a count of rejected or ambiguous records. Run both paths against representative examples before retiring the old one; do not relabel a topic or mutate a unit in place.

References and verification

• OASIS, MQTT Version 5.0, published 2019-03-07.
• Eclipse Tahu, pinned sparkplug_b.proto, commit 5736e404889d4b95910613040a99ba79589ffb13.
• MQTT payload-envelope boundary derived from the MQTT 5.0 PUBLISH contract above; JSON names, units, quality, and evolution remain application contracts.
• The byte-count example above does not establish performance, compatibility, security, broker behavior, or a universal size.
• Eclipse Sparkplug, Specification 3.0, ratification ballot concluded 2022-10-21.