A media service splits frame processing across two processes. The decoder produces a 16 MB frame, and an encoder consumes it. Sending each frame through a byte-stream IPC mechanism may require the producer to copy bytes into a kernel buffer and the consumer to copy them out again.

Shared memory changes the data path. The operating system maps the same physical memory into both processes. After setup, the decoder can write the frame into that region and the encoder can read those same bytes with ordinary memory instructions.

This removes per-message data transfer through the kernel, but it also removes services that a stream or queue would otherwise provide. Shared memory has no built-in message boundaries, no automatic notification when data arrives, and no protection against two participants updating the same fields at once. The application must define those rules.

Processes normally have isolated virtual address spaces. An address such as 0x70000000 in Process A has no necessary relationship to the same numeric address in Process B.

A shared-memory object lets both processes create page-table entries that refer to the same physical page frames:

The virtual addresses differ, but both mappings reach the same underlying pages. A store performed through Process A's mapping changes the memory that Process B can read through its mapping.

The kernel participates when a process creates, maps, resizes, or removes the region. It may also handle page faults as pages become resident. Once valid mappings exist, an ordinary load or store does not require a system call for each access.

This is the main performance property of shared memory:

Data stays in a common memory region, so processes communicate through loads and stores instead of asking the kernel to carry each message between them.

The kernel still enforces the mapping permissions. A process with a read-only mapping cannot write through it. Processes without permission to open or inherit the object do not gain access to the region.

A mapping answers one question: where can both processes access the same bytes?

It leaves several questions to the application:

Without answers, the region is a shared array of bytes rather than a usable IPC channel.

A small layout might contain a header followed by a fixed-capacity payload:

The format_version identifies the layout understood by both programs. payload_length frames the message inside a buffer whose physical size is fixed. The producer must reject payloads larger than PAYLOAD_CAPACITY, and the consumer must validate the length before using it.

These fields still need a publication rule. If the producer updates payload_length before it finishes writing payload, the consumer can observe a plausible length and incomplete bytes. A semaphore, process-shared mutex, or another defined synchronization mechanism must order the handoff.

The two writes must both complete before ready is posted. That ordering is the entire contract, and shared memory alone does not provide it.

The synchronization operation creates the required ordering between payload writes and payload reads. Sleeping for a guessed interval or repeatedly checking a plain integer does not provide that guarantee.

Linux applications commonly obtain shared memory through one of three interfaces. They all produce memory that multiple processes can access, but their creation and discovery rules differ.

Anonymous shared mappings

A process can create a mapping with MAP_SHARED | MAP_ANONYMOUS and then call fork(). The child inherits the mapping, and both processes continue to refer to the same pages.

This form works well when a parent establishes shared state before creating workers. It has no name that an unrelated process can open later.

Ordinary heap memory inherited through fork() does not behave this way. Writable heap pages use copy-on-write behavior, so a write by one process normally gives that process a private page. An explicitly shared mapping is required for continuing write visibility between the processes.

POSIX shared-memory objects

shm_open() creates or opens a named POSIX shared-memory object. The returned value is a file descriptor, so existing descriptor operations apply:

On Linux, these objects are normally backed by a tmpfs mounted at /dev/shm. The POSIX name /frame-buffer is an object name, not a regular pathname supplied to open().

Named objects let independently started processes find the same region. File permissions and the process umask control who can open it.

System V shared memory

The older System V API uses shmget(), shmat(), shmdt(), and shmctl(). Existing databases, runtimes, and older Unix software still use it.

System V objects live in a kernel-managed IPC namespace and have identifier and cleanup rules that differ from file descriptors. POSIX shared memory usually integrates more naturally with descriptor-based code, so it is a common choice for new Unix applications. Compatibility requirements may dictate the System V interface.

The remainder of this chapter uses POSIX shared memory because its lifecycle maps cleanly to familiar file-descriptor operations.

The creator should use O_CREAT | O_EXCL when it expects a new object:

O_EXCL prevents the program from opening a stale object under the assumption that it created a fresh one. If the call fails with EEXIST, the program can report the collision or follow an explicit recovery policy.

A new shared-memory object has length zero. The creator must size it before mapping:

Accessing pages beyond the current object length can cause SIGBUS, so size is part of the object's safety contract.

The creator maps the object:

MAP_SHARED makes updates visible through other mappings of the same object. MAP_PRIVATE would create a private copy-on-write view for modifications, which defeats the communication goal.

The first argument is NULL, allowing the kernel to choose a suitable virtual address. Participants do not need matching virtual addresses. Code should be designed with the assumption that their mapping addresses will differ.

After mmap() succeeds, the descriptor and mapping have separate lifetimes. Closing the descriptor does not remove the mapping:

The process continues to use message until it calls munmap() or terminates.

The following Linux program can run as either a writer or a reader. Its shared region contains two unnamed semaphores. ready tells the reader that the payload is complete, while consumed tells the writer that the reader has finished.

The second argument to sem_init() is 1, which requests a process-shared semaphore. A semaphore stored in shared memory but initialized with the default thread-only setting would not provide the required process-to-process contract.

Compile the program:

This program targets Linux. macOS does not support unnamed process-shared POSIX semaphores through sem_init(), so a macOS implementation needs a supported alternative such as named POSIX semaphores.

Start the writer in one terminal:

After it reports that the object is ready, run the reader in another terminal:

The reader prints:

The writer waits until the reader posts consumed. It then destroys the semaphores, unmaps the region, and removes the shared-memory name.

This program transfers only one message. A repeated producer-consumer channel needs a layout with multiple states or slots and carefully defined rules for reuse. The same core obligations remain: publish only complete data, prevent premature overwrite, validate every length, and define cleanup after failures.

The manual start order also ensures that the reader opens the object after both semaphores have been initialized. Independently supervised processes need an initialization state or a launcher that starts consumers only after creation is complete. Service code should also unwind each completed setup step if a later system call fails; the example exits immediately to keep those cleanup branches from obscuring the data path.

POSIX shared memory separates an object's name from its storage lifetime, much like a Unix file.

shm_unlink("/algomaster-shm-demo") removes the name. Processes that already hold a descriptor or mapping can continue using the object. The kernel reclaims its storage after the name is gone and the final open reference and mapping have been released.

Removing the name does not free the memory. Step 6 is what actually reclaims it, which is why a leaked mapping keeps the storage alive long after the name is gone.

Removing the name early is useful when all intended participants have already inherited or opened the object. It prevents new processes from opening it and ensures that storage is reclaimed after the current participants exit.

Named objects can remain after a process crashes before calling shm_unlink(). A later run using O_EXCL then receives EEXIST. Reliable programs define ownership and recovery instead of deleting every colliding object blindly. The existing object may belong to a live process or contain data needed for diagnosis.

On Linux, these commands help inspect named objects:

The first shows objects visible through the Linux tmpfs implementation. The second shows the capacity available to that filesystem. A mapping can be larger than the physical memory currently committed to its pages, and later writes can fail if the backing store cannot supply space.

The two processes in the first diagram mapped the same pages at different virtual addresses. A raw pointer stored by one participant can therefore be meaningless to another.

Consider this shared structure:

If Process A stores next = 0x7f100020, that value refers to an address in A's virtual address space. Process B may have mapped the region at 0x6a400000, so dereferencing A's pointer can access unrelated memory or fault.

Shared-memory structures usually store an offset from the mapping base:

Every participant reconstructs a local pointer using its own mapping base. An index into a fixed array can serve the same purpose.

Offsets require validation. Before converting an offset, the process must confirm that the target object fits within the mapped region and satisfies its alignment requirement. Data read from shared memory should receive the same bounds checks as data read from a socket or file.

Forcing every process to map at a fixed virtual address is fragile. The requested range may already be occupied, address-space layout randomization changes placement, and mappings can conflict with libraries or thread stacks. Relative references make the data format independent of placement.

Two processes can race on shared memory in the same way two threads can. An increment remains a read-modify-write operation, and a consumer can observe a structure while a producer is changing it.

The synchronization object must support use across processes. POSIX provides relevant configuration for several primitives:

Placing a normal process-private mutex in a shared region does not convert it into a process-shared mutex. Its attributes must be set before initialization, and all participants must agree on the object's initialized layout.

Crash behavior also matters. If a process exits while holding an ordinary process-shared mutex, the remaining processes may wait forever. POSIX robust mutexes can report EOWNERDEAD to the next owner, but the application must then inspect and repair the protected state before marking the mutex consistent. The operating system can report that an owner died; it cannot infer which multi-field update was partially completed.

For lock-free fields, the atomic type must be suitable for process-shared use on the target platform. If a language atomic implementation uses hidden process-private locks, placing the value in shared memory does not make those hidden locks shared. Linux software that requires this property commonly restricts itself to known lock-free widths and verifies its platform assumptions.

Shared memory couples processes to a binary data format. Both participants must agree on field widths, alignment, byte order, and state transitions.

Native C structures can contain compiler-inserted padding. Their layout can also change when a field is reordered, a type changes width, or a program moves to a different ABI. A stable protocol should use fixed-width integer types, explicit capacity values, and a format version.

A practical header may include:

The creator initializes the entire region before publishing that initialization is complete. A joining process validates magic, format_version, header_size, and region_size before following any offset or reading any payload.

generation can distinguish a fresh instance from stale state left under a reused name. It does not solve recovery by itself, but it gives participants a way to reject references created for an earlier instance.

Layout upgrades need an explicit policy. Common options include running only matching versions, creating a new object name for each incompatible format, or supporting a small set of versioned headers. Changing a shared structure while older processes remain attached can corrupt data even when both binaries are individually correct.

For a kernel-buffered IPC path, a payload often follows this simplified route:

With shared memory, both processes operate on one mapped payload area. There is no required kernel-buffer copy for each handoff and no required data-transfer system call per message.

That description has limits. The producer may still copy data from a private working buffer into the shared region, and the consumer may copy it into another private representation. Synchronization can require system calls when a process must sleep or wake. Initial page faults, cache misses, and cache-coherence traffic still consume time.

Shared memory tends to help when payloads are large, transfers are frequent, or the consumer can operate directly on the shared representation. For small or infrequent messages, simpler IPC can perform well enough while providing framing, buffering, and easier failure handling.

Contention can erase much of the expected gain. If producers and consumers update one shared counter on every operation, the corresponding cache line must move between cores. Separating frequently written control fields and assigning clear ownership to data slots reduces this traffic.

NUMA placement can also affect a large region. Access to memory attached to another CPU socket has higher latency than local access. Binding processes and allocating pages without considering where consumers run can turn shared memory into remote-memory traffic. This is a measurement and placement issue, not a change to shared-memory semantics.

Shared memory preserves process address-space isolation only for memory outside the mapped region. Every process with a writable mapping can corrupt any byte inside it, including synchronization objects and metadata.

Permissions should therefore be narrow. A creator can use mode 0600 for same-user communication and open with O_EXCL to avoid accidental reuse. Programs should account for umask, avoid predictable names when untrusted same-user processes are a concern, and drop write permission for participants that only need to read.

The data format still needs validation even among cooperating processes. A crashed writer can leave a stale length, an invalid offset, or a half-written record. The reader must reject values that lead outside the mapped region.

Resizing an active object is dangerous. If one process shrinks it while another retains a larger mapping, access to pages beyond the new end can raise SIGBUS. A stable design fixes the region size for its active lifetime or coordinates a versioned replacement.

The mapping also has no built-in peer-liveness signal. A process waiting for a publication that will never arrive needs a timeout, supervisor, or other failure policy. Shared memory keeps bytes accessible after a peer exits; it does not decide whether those bytes represent a committed message.

Shared memory is well-suited to communication on one host when the data volume makes repeated copying expensive and the participating processes can share a tightly specified protocol. Media frames, telemetry buffers, database buffer pools, and high-rate worker pipelines are common examples.

It is a weaker fit when messages are small, participants change independently, failure isolation matters more than copy cost, or the communication may later cross a machine boundary. A byte-stream or message-oriented interface often has a clearer ownership model in those cases.

The decision should include engineering cost. A working shared-memory channel needs synchronization, framing, bounds checks, lifecycle management, version compatibility, crash recovery, and performance measurement. Faster loads and stores are valuable only when the complete protocol remains correct.

Shared memory maps the same physical pages into multiple process address spaces. After setup, participants exchange data through ordinary loads and stores without a kernel-buffer copy for every message.

The mapping supplies shared bytes, while the application supplies the protocol. Correct designs define framing, ownership, process-shared synchronization, bounds validation, layout versioning, cleanup, and crash recovery. POSIX shared memory uses shm_open(), ftruncate(), mmap() with MAP_SHARED, and shm_unlink() to manage the object and its name.

Shared memory can reduce copy and system-call overhead for high-volume local communication. Its performance depends on access patterns, cache behavior, synchronization, and whether consumers can work directly on the shared representation.