Skip to content
ossl-devPublic

About

Universal Shared Memory

Resources

Contributing

Stars

90 stars

Watchers

0 watching

Forks

Repository files navigation

Zinc logo

Zinc - Universal Shared Memory Library

CI License: MIT Rust Python Go Node.js Bun Deno C++ Java C#

Zinc gives processes in any language access to the same memory pages, across Rust, Python, Go, Node.js, Bun, Deno, C++, Java, C#, and more. The data path uses mmap with zero-copy views. Notifications use Linux futex calls or a macOS polling fallback.

Windows is not supported. Zinc is a Linux/macOS library built on POSIX shared memory (shm_open + mmap).

Think SharedArrayBuffer, but cross-language and cross-process.


Why

SharedArrayBuffer lets worker threads share memory within a process. Sharing video frames, model outputs, or game state across processes often means serializing data through a socket or writing platform-specific shared-memory code. Zinc provides shared mappings through one core and a set of language adapters.

Zinc maps the same physical RAM pages into both processes via POSIX shared memory. Write a float in Python, read it in Go without serializing it or copying it through a socket. Mapping shared pages avoids an extra transfer step. Reads, writes, and synchronization still have a cost.


Quick start

Use the adapters from a source checkout; registry publishing is still on the roadmap. Keep the creator running while readers open the region. These snippets use 16384 bytes, valid for common Linux and macOS page sizes; query the system page size for other targets.

git clone https://github.com/ossl-dev/zinc
cd zinc
cargo build --release --manifest-path core/Cargo.toml

Rust

use zinc_core::SharedRegion;

let region = SharedRegion::create("my-data", 16384)?;
unsafe { std::ptr::write(region.as_ptr() as *mut f32, 42.0) };
region.notify();
std::io::stdin().read_line(&mut String::new())?;

Python

from zinc import SharedRegion
import numpy as np

r = SharedRegion.open("my-data")
if not r.wait(1000):
    raise TimeoutError("writer did not publish data")
arr = r.as_numpy(dtype=np.float32)
print(arr[0])  # 42.0, same physical memory

Go

import (
    "zinc"
    "unsafe"
)

r, err := zinc.Open("my-data")
if err != nil { panic(err) }
defer r.Close()
if !r.Wait(1000) { panic("writer did not publish data") }
data := r.Bytes()
val := *(*float32)(unsafe.Pointer(&data[0]))

TypeScript (Bun)

import { SharedRegion } from "./adapters/bun/src/index.ts";

const r = SharedRegion.open("my-data");
if (!r.wait(1000)) throw new Error("writer did not publish data");
const buffer = r.buffer();
const view = new Float32Array(buffer.buffer, buffer.byteOffset, buffer.byteLength / 4);
console.log(view[0]); // 42.0

That's it. Every language sees the same bytes. Use atomic data or an application protocol to coordinate concurrent access. Notify/wait publishes updates but does not provide mutual exclusion.


How it works

┌──────────────────────────────────────┐
│  Python  │  Go  │  C++  │ Java │ C#  │  ← Adapters (thin FFI wrappers)
│  (cffi)  │(cgo) │(hpp)  │(JNA) │(P/Invoke)
├──────────────────────────────────────┤
│        include/zinc.h                │  ← C ABI (opaque void* handles)
│   zinc_create / zinc_open / ...      │
├──────────────────────────────────────┤
│        Rust core (cdylib)            │  ← Single source of truth
│   SharedRegion / Ring / Sync         │
├──────────────────────────────────────┤
│   POSIX shm_open + mmap + futex      │  ← Platform layer
│   (Linux, macOS)                     │
└──────────────────────────────────────┘

The C ABI exposes nine functions. FFI adapters call the core through that ABI; Rust and Node use the crate directly. The Rust core compiles to libzinc_core.{so,dylib}. Adapters handle native types, error translation, and view lifetimes.

The 9 C functions

Function Purpose
zinc_create Create owned region, returns opaque handle
zinc_open Open existing region
zinc_ptr Raw pointer to data area (after the first header page)
zinc_capacity Usable bytes
zinc_close Drop handle, unmap, maybe unlink
zinc_notify Signal waiters (futex on Linux, polling on macOS)
zinc_wait Block until notified or timeout
zinc_try_wait Consume a pending notification without blocking
zinc_version Major/minor version for compatibility checks

Ownership model

  • Creator owns the segment, others open it
  • Ref-counted via atomic in the 64-byte header
  • Closing the creator unlinks the name
  • Existing mappings remain valid until their handles close
  • Crash recovery remains on the roadmap

Performance

Shared mappings avoid a separate data transfer. Reads and writes still use memory bandwidth and cache coherence, and notifications add synchronization costs. See the benchmark documentation for measured workloads and their limits.

Target: notify/wait roundtrip < 5µs on Linux.


Language adapters

Language Mechanism Status Path README
Rust Direct crate Core ready core/ API
Python cffi + numpy Tests passing adapters/python/ README
Go cgo Tests passing adapters/go/ README
Node.js napi-rs Tests passing adapters/node/ README
Bun bun:ffi Tests passing adapters/bun/ README
Deno Deno.dlopen Tests passing adapters/deno/ README
C++ Header-only RAII Tests passing adapters/cpp/ README
Java JNA Tests passing adapters/java/ README
C# P/Invoke Tests passing adapters/csharp/ README

Building

# Build the Rust core
cargo build --release --manifest-path core/Cargo.toml

Output: target/release/libzinc_core.{so,dylib}. The C header is checked in. After changing C exports, regenerate it with cargo run -p zinc-core --example generate_header --features generate-header --locked.


Prerequisites

  • Rust 1.99.0 for development (pinned via rustup); the core library supports Rust 1.85 or later, rustup.rs
  • Optional: Moon ≥ 2.0, moonrepo.dev (monorepo tool)
  • Language runtimes as needed

Development

See DEVELOPMENT_GUIDE.md for full build instructions, test suite, and architecture notes.


License

MIT (declared in the Cargo package metadata).

About

Universal Shared Memory

Resources

Contributing

Stars

90 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages