point_cloud.h File Reference

Zero-copy, schema-aware 3-D point cloud container with per-field type protocol. More...

#include <cstdint>
#include <iostream>
#include <string>
#include <string_view>
#include <type_traits>
#include <unordered_map>
#include <vector>
#include "../base/bytes.h"
#include "../base/quantize.h"
#include "./header.h"

Include dependency graph for point_cloud.h:

Go to the source code of this file.

Namespaces
	vlink
	vlink::zerocopy

Detailed Description

Zero-copy, schema-aware 3-D point cloud container with per-field type protocol.

PointCloud carries a contiguous array of fixed-size point records together with a compact, self-describing schema (the embedded Protocol). Schemas are encoded as two uint64_t nibble-packed integers plus a comma-separated 152-byte name string, so a receiver can introspect every field without sharing an external IDL. The 40-byte Header prefix carries sequencing and dual-timestamp metadata.

Point format diagram

Per-point layout (tightly packed, no padding):
 
    XYZ float        :  [ x: float32 | y: float32 | z: float32 ]                 (12 B)
    XYZI float       :  [ x | y | z | intensity: float32 ]                       (16 B)
    XYZRGB           :  [ x | y | z | r: u8 | g: u8 | b: u8 | pad: u8 ]          (16 B)
    XYZ + features   :  [ x | y | z | t: u32 | label: u8 | conf: f32 | ... ]     (variable)
    XYZ double       :  [ x: float64 | y: float64 | z: float64 ]                 (24 B)
 
Schema is encoded into two uint64_t integers, one nibble per field
(high nibble = first field):
 
    size_num: 0x0408...  (each nibble is the field byte-size)
    type_num: 0x0A0B...  (each nibble is a PointCloud::Type enum value)
    names:    "x,y,z,intensity,label"  (stored in 152 bytes)

Supported field types

Enum	C++ type	Bytes
kBoolType	bool	1
kInt8Type	int8_t	1
kUint8Type	uint8_t	1
kInt16Type	int16_t	2
kUint16Type	uint16_t	2
kInt32Type	int32_t	4
kUint32Type	uint32_t	4
kInt64Type	int64_t	8
kUint64Type	uint64_t	8
kFloatType	float	4
kDoubleType	double	8

Protocol nested struct

Member	Type	Purpose
size_num	uint64_t	Per-field byte sizes packed in nibbles
type_num	uint64_t	Per-field Type tags packed in nibbles
names[152]	char[]	Comma-separated field names (3..16 fields)

Wire format

PointCloud is POD; memcpy of the struct snapshot plus the packed point buffer forms the wire payload. The sizeof value is locked by static_assert and forms a permanent contract: vlink::zerocopy::* containers offer NO forward and NO backward binary compatibility – every field, including reserved bytes and the Protocol descriptor, is part of the wire contract.

static_assert(sizeof(PointCloud) == 256, "Sizeof must be 256 bytes.");

Memory layout

Offset  Size  Field
------  ----  ----------------------------------------------------------------
     0    40  Header   header
    40   168  Protocol protocol_  (size_num + type_num + names[152])
   208     8  uint8_t* data_
   216     8  size_t   capacity_
   224     8  size_t   size_
   232     4  uint32_t reserved_buf_
   236     4  uint32_t reserved_buf2_
   240     2  uint16_t pack_size_
   242     2  uint16_t extent_
   244     1  uint8_t  downsample_
   245     1  uint8_t  reserved_buf3_
   246     1  bool     vertical_
   247     1  bool     is_owner_
   248     8  uint64_t index_
------  ----  ----------------------------------------------------------------
 Total   256  bytes (alignas 8)
 
Wire envelope:
[ magic_begin (4) | version (4) | PointCloud (256) | point data (size_ * pack_size_) | magic_end (4) ]

Reserved bytes

reserved_buf_ (offset 232), reserved_buf2_ (offset 236) and reserved_buf3_ (offset 245) are wire-locked padding bytes. They travel through the wire format unchanged but MUST NOT be repurposed by application code because future library revisions may bind them to real fields. The uint8_t downsample_ at offset 244 records the voxel-grid downsampling level applied to the payload (see downsample()): 0 disables downsampling, 255 is the most aggressive.

Optional storage compression

The extent and vertical create options trade fidelity for footprint along two independent axes. When extent_ > 0 the three leading XYZ coordinates are linearly quantised with Quantize::encode<int16_t>(extent, value) and restored with Quantize::decode<float/double>(extent, stored), halving (or better) their in-memory footprint; the embedded schema is rewritten so the XYZ fields advertise kInt16Type and get_value_v3f() / get_value_v3d() transparently dequantise back to floating point. The generic get_value<int16_t>() returns the raw int16_t storage, whereas get_value<float>() / get_value<double>() also dequantise (matching v3f / v3d). Points whose XYZ fall outside (-extent, +extent) cannot be represented and are discarded on insert rather than saturated. When vertical_ is true the serialised wire payload is emitted in structure-of-arrays order (xx..x yy..y zz..z, one field column after another) which packs better under a downstream entropy coder; the in-memory buffer stays interleaved (row-major) and operator<< restores that layout, so the field accessors are unaffected.

Optional voxel-grid downsampling

downsample() collapses spatially-close points by mapping each point's quantised XYZ onto a voxel grid and keeping the first point seen in each voxel, compacting the survivors to the front of the buffer and shrinking size(). It applies only to quantised clouds (extent_ > 0) and records the requested level into the downsample_ marker, which get_downsample() reports. The marker is producer-set, not an automatically maintained invariant: freshly created clouds record 0, and clear() resets it to 0; it is NOT recomputed when points are subsequently added or modified.

Example – typical fill and publish

vlink::zerocopy::PointCloud pc;
pc.create_v3f<float>(1000, {"intensity"});   // schema: x, y, z, intensity (all f32)
 
pc.push_value_v3f(1.0F, 2.0F, 3.0F, 0.8F);
pc.push_value_v3f(4.0F, 5.0F, 6.0F, 0.5F);
 
auto key_map = pc.get_key_map();
float intensity = pc.get_value<float>(0, key_map, "intensity");
 
vlink::Bytes wire;
pc >> wire;
 
vlink::zerocopy::PointCloud rx;
rx << wire;