cross_platform_compatibility.md

Cross-Platform Compatibility Guide

Because MessageFrame serializes data using the standard, widely adopted MessagePack wire format, you do not need to compile this C++ library on your backend or gateway side.

Any language with a MessagePack library (Python, Go, Node.js, Rust, etc.) can natively unpack and read messages generated by MessageFrame out of the box.

---

Wire Format Layout

When MessageFrame::serialize() is called, it packs everything into a top-level MessagePack Array of exactly 3 elements:

[[Header Data],         // Element 0: 8-element array — timestamp, msg_cnt, source, target, msg_id, msg_type, version, flags
{Parameter Map},        // Element 1: FLAT map, see below
[Binary Attachments]    // Element 2: Array of pairs [ [Name, Raw Binary], ... ]
]

This predictable layout allows non-C++ readers to skip parts of the message they don't need or route heavy binary payloads efficiently without full parsing overhead.

Parameter map is flat, not nested

The device/parameter split you see in the C++ API (msg.add("sdr", "gain", ...)) only exists on the C++ side. On the wire, HybridMessageMap::pack() writes a single flat MessagePack map. Each key is one string combining device and parameter, joined by 0x1F (ASCII Unit Separator, not a dot):

{
  "sdr\x1Fgain":        [20, 10.0],
  "device_core\x1Ffw_version": [30, "v3.2.1"]
}

So parameters["sdr"]["gain"] is wrong on every platform — there is no nested "sdr" object. You need to split each key on 0x1F yourself.

Values are tagged, not raw

Each value is packed as a 2-element array [type_tag, value], not a bare scalar:

type_tag	C++ type
10	Int64
20	Double
30	String
40	Bool

So 10.0 on the wire actually looks like [20, 10.0], and you need value[1] (optionally checking value[0] if you care about the type) to get the real number.

---

Decoding in Python

Python handles MessagePack data effortlessly, unpacking binary strings into native bytes types with zero-copy execution speed.

Prerequisites

pip install msgpack

Reader Example

import msgpack

SEPARATOR = "\x1f"  # ASCII Unit Separator — the real device/parameter delimiter

TYPE_INT64, TYPE_DOUBLE, TYPE_STRING, TYPE_BOOL = 10, 20, 30, 40

def parse_message_frame(raw_bytes: bytes):
    frame = msgpack.unpackb(raw_bytes, use_list=True, raw=False)

    header      = frame[0]
    parameters  = frame[1]   # FLAT map: "device\x1Fparameter" -> [type_tag, value]
    attachments = frame[2]

    # 1. Work with parameters — split the flat key, unwrap the tagged value
    for flat_key, tagged_value in parameters.items():
        device, param = flat_key.split(SEPARATOR, 1)
        type_tag, value = tagged_value

        if device == "sdr" and param == "gain":
            print(f"SDR Gain: {value} (type_tag={type_tag})")

    # 2. Handle Zero-Copy Binary Attachments
    for attach in attachments:
        name = attach[0]
        raw_data = attach[1]  # native Python 'bytes' object
        print(f"Attachment Received: '{name}' | Size: {len(raw_data)} bytes")

# Usage Example (Assuming raw_bytes came from a socket or MQTT broker)
# parse_message_frame(raw_bytes)

---

Decoding in Go (Golang)

Go is widely used in high-performance cloud gateways and IoT brokers. You can easily decode the C++ generated stream using the popular vmihailenco/msgpack package.

Code Example

package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/vmihailenco/msgpack/v5"
)

const separator = "\x1f" // ASCII Unit Separator

const (
	TypeInt64  = 10
	TypeDouble = 20
	TypeString = 30
	TypeBool   = 40
)

func parseMessageFrame(payload []byte) {
	var frame []interface{}
	if err := msgpack.Unmarshal(payload, &frame); err != nil {
		log.Fatalf("Failed to unpack frame: %v", err)
	}

	// Parameters: a FLAT map, keys are "device\x1Fparameter"
	parameters := frame[1].(map[string]interface{})
	attachments := frame[2].([]interface{})

	// 1. Read Parameters — split key, unwrap [type_tag, value]
	for flatKey, tagged := range parameters {
		parts := strings.SplitN(flatKey, separator, 2)
		if len(parts) != 2 {
			continue
		}
		device, param := parts[0], parts[1]

		pair := tagged.([]interface{})
		typeTag, value := pair[0], pair[1]

		if device == "sdr" && param == "gain" {
			fmt.Printf("SDR Gain: %v (type_tag=%v)\n", value, typeTag)
		}
	}

	// 2. Process Binary Attachments
	for _, att := range attachments {
		pair := att.([]interface{})
		name := pair[0].(string)
		rawData := pair[1].([]byte) // Unpacked directly as raw byte array
		fmt.Printf("Attachment: %s, Size: %d bytes\n", name, len(rawData))
	}
}

---

Decoding in Node.js / JavaScript (TypeScript)

Node.js is ideal for streaming telemetry directly to web dashboards via WebSockets. Using the official @msgpack/msgpack library, MessageFrame buffers are automatically unpacked into JavaScript objects and native Uint8Array binary blobs.

Prerequisites

npm install @msgpack/msgpack

Reader Example

const { decode } = require("@msgpack/msgpack");

const SEPARATOR = "\x1f"; // ASCII Unit Separator — the real device/parameter delimiter

function parseMessageFrame(buffer) {
    const frame = decode(buffer);

    const header      = frame[0];
    const parameters  = frame[1]; // FLAT object: "device\x1Fparameter" -> [type_tag, value]
    const attachments = frame[2];

    // 1. Read Parameters — split each flat key, unwrap the tagged value
    for (const [flatKey, tagged] of Object.entries(parameters)) {
        const [device, param] = flatKey.split(SEPARATOR);
        const [typeTag, value] = tagged;

        if (device === "sdr" && param === "gain") {
            console.log(`SDR Gain: ${value} (type_tag=${typeTag})`);
        }
    }

    // 2. Process Zero-Copy Binary Attachments
    attachments.forEach(attach => {
        const name = attach[0];
        const rawData = attach[1]; // native Uint8Array

        console.log(`Attachment: '${name}' | Size: ${rawData.byteLength} bytes`);
    });
}

---

Decoding in Rust

Rust provides excellent safety and blazing-fast performance for decoding network streams. You can easily unpack the MessageFrame byte arrays using the standard rmp-serde crate.

Prerequisites

Add this to your Cargo.toml:

[dependencies]
rmp-serde = "1.3"
rmpv = { version = "1", features = ["with-serde"] }
serde = { version = "1.0", features = ["derive"] }

Reader Example

use std::collections::HashMap;

const TYPE_INT64: u8 = 10;
const TYPE_DOUBLE: u8 = 20;
const TYPE_STRING: u8 = 30;
const TYPE_BOOL: u8 = 40;
const SEPARATOR: char = '\u{1F}'; // ASCII Unit Separator

fn parse_message_frame(payload: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    // 0: Header (8-element array, kept generic here)
    // 1: Parameters — a FLAT map: "device\x1Fparameter" -> [type_tag, value]
    // 2: Attachments — array of [name, raw_bytes] pairs
    type FrameLayout = (
        rmpv::Value,
        HashMap<String, (u8, rmpv::Value)>,
        Vec<(String, Vec<u8>)>,
    );

    let (_header, parameters, attachments): FrameLayout = rmp_serde::from_slice(payload)?;

    // 1. Read Parameters — split the flat key, unwrap the tagged value
    for (flat_key, (type_tag, value)) in &parameters {
        if let Some((device, param)) = flat_key.split_once(SEPARATOR) {
            if device == "sdr" && param == "gain" {
                println!("SDR Gain: {:?} (type_tag={})", value, type_tag);
            }
        }
    }

    // 2. Process Binary Attachments
    for (name, raw_data) in attachments {
        println!("Attachment: '{}' | Size: {} bytes", name, raw_data.len());
    }

    Ok(())
}

---

🎯 Advantages for Distributed Teams

• Zero Boilerplate: No .proto or .fbs structural schemas to sync between front-end, backend, and embedded firmware teams.
• Agile Prototyping: Add a new telemetry parameter in your C++ firmware, and your Python analytics server receives it instantly without modifications or code re-generation.
• Native Raw Binary Performance: Zero CPU cycles spent on converting binary waveforms or video captures into Base64 formats.