AlgoMaster Logo

Message Queues and Signals as IPC

15 min readUpdated August 7, 2026
Listen to this chapter
Unlock Audio

A background service submits image-processing jobs to a pool of worker processes. Each job has a clear boundary: job ID 8421, operation generate-thumbnail, priority 5. Treating these jobs as an undifferentiated byte stream would require the workers to reconstruct every boundary themselves.

A message queue preserves those boundaries. The producer submits one message, the kernel stores it as one message, and a consumer receives it as one message. The queue can also retain several jobs while every worker is busy.

Signals solve a narrower IPC problem. A process can notify another that an event occurred, such as a shutdown request or child state change. Standard signals cannot preserve an arbitrary sequence of events or carry application records.

Both mechanisms are kernel-managed and asynchronous, but they provide different contracts. A queue transports discrete data. A signal primarily changes what work a process should perform.

Message Queues as Stores of Discrete Messages

A kernel message queue sits between producers and consumers:

Each successful send adds one record. Each successful receive removes one complete record. Two messages are not merged into one receive, and one message is not divided across several receives.

This is the defining difference from a pipe. A pipe preserves byte order but leaves framing to the application. A message queue preserves both order and message boundaries according to the queue's delivery policy.

The queue does not interpret the payload. It stores an opaque byte sequence plus mechanism-specific metadata, such as a POSIX priority or a System V message type. Both processes must agree on the payload format.

What the Kernel Tracks

A queue needs more state than a byte stream:

  • The current set of messages
  • The length of each message
  • Ordering metadata
  • Maximum message size
  • Maximum number of queued messages or bytes
  • Processes waiting to send
  • Processes waiting to receive
  • Ownership and permission information

The limits keep kernel memory use bounded. When the queue is full, a blocking sender waits until a consumer removes a message. When the queue is empty, a blocking receiver waits until a producer adds one.

Nonblocking operation replaces waiting with an immediate error, commonly EAGAIN. Timed operations add a deadline. These choices affect how a process waits, while message boundaries and queue capacity remain unchanged.

POSIX Message Queues

POSIX message queues use names such as /image-jobs. The main operations are:

  • mq_open() creates or opens a queue.
  • mq_send() adds one message with a priority.
  • mq_receive() removes one message and returns its priority.
  • mq_getattr() reports limits and current state.
  • mq_close() releases one process's descriptor.
  • mq_unlink() removes the queue's name.

mq_open() returns an mqd_t, a message-queue descriptor. POSIX does not require it to behave like an ordinary file descriptor in every API, even though Linux implements it using a file descriptor internally.

The creator can request queue limits with struct mq_attr:

mq_maxmsg limits the number of waiting messages. mq_msgsize limits the size of each one. The kernel may reject values above system limits, so a program should query the resulting attributes instead of assuming every request was accepted.

On Linux, POSIX queues are normally exposed through the mqueue filesystem at /dev/mqueue. Their limits are controlled through files under /proc/sys/fs/mqueue/.

Priority-Based Receive Order

Every POSIX message carries an unsigned priority. The queue returns a higher-priority message before a lower-priority message. Messages with equal priority are returned in first-in, first-out order.

MessagePriority
Job A9
Job B5
Job C5
Job D1

Receives return them in the order A, B, C, D. Higher priority comes first, and B and C tie, so they are returned in the order they were sent.

Priority is useful for urgent control work, but continuous high-priority traffic can delay low-priority messages indefinitely. A queue does not provide fairness across priorities.

Priority also changes the meaning of “ordered.” The queue is FIFO only among messages with the same priority. A consumer that requires global submission order should use one priority or place an explicit sequence number in every message.

A Complete POSIX Queue Example

The following Linux program acts as either a receiver or a sender. The receiver creates a queue that can hold eight job records. The sender opens that queue and submits one complete record with priority 5.

The example sends a native C structure because both processes run the same binary. Independently versioned programs should use an explicitly serialized format rather than relying on compiler padding and one machine's byte order.

Compile it on Linux:

Current glibc versions provide these functions through the main C library. The -lrt option also supports older glibc versions where POSIX message queues were exposed through librt.

Start the receiver:

After it reports that the queue is ready, send a job from another terminal:

The receiver prints:

The receiver closes and unlinks the queue after consuming the message. If it is terminated before cleanup, the name may remain and a later receiver using O_EXCL will fail with EEXIST. Remove that known demo queue with:

The cleanup mode is suitable only when no live instance owns the demo queue. Service code needs an ownership policy that distinguishes stale state from an active queue.

Message Size in the Queue Interface

mq_send() fails with EMSGSIZE when a message exceeds the queue's configured maximum. The kernel does not split an oversized record into several messages.

For mq_receive(), the supplied buffer must be at least as large as the queue's configured mq_msgsize. Providing a smaller buffer fails with EMSGSIZE, even if the next queued message happens to be shorter. Programs can call mq_getattr() and allocate or validate the receive buffer before reading.

A fixed maximum simplifies allocation and bounds kernel memory, but it creates an API limit. Large payloads can be placed in another storage area while the queue carries a small descriptor containing an identifier, length, and ownership information. That design must ensure the referenced data remains valid until the consumer has finished with it.

Message payloads also need validation. A successful receive proves that bytes came through the queue as one record. It does not prove that a length, enum value, string terminator, or embedded identifier is valid.

Process-Independent Queue Lifetime

A POSIX queue name and the queue object have separate lifetimes. mq_close() releases one descriptor. It does not remove the queue or discard its messages.

mq_unlink() removes the name. Processes that already opened the queue can continue using it until they close their descriptors. The kernel destroys the object after the name has been removed and the final open reference is released.

This separation allows a sender to enqueue work and exit before a receiver consumes it. The queue outlives the sending process.

The queue is still a kernel IPC object rather than a durable job broker. Its messages do not form a disk-backed log, and they do not provide acknowledgments or automatic redelivery. A receiver removes a message when mq_receive() succeeds. If it crashes before processing the job, the kernel does not put that message back.

Applications that require confirmed processing need a separate acknowledgment and retry protocol. A successful mq_send() means the queue accepted the message, not that a consumer completed the work.

System V Message Queues

System V message queues predate the POSIX interface and remain present in Unix software. Their main operations are:

  • msgget() creates or locates a queue using a numeric key.
  • msgsnd() sends a message.
  • msgrcv() receives a message.
  • msgctl() inspects, configures, or removes the queue.

Every System V message begins with a positive long value called mtype. A receiver can request the first message, the first message of one type, or a message selected through System V's type rules.

This differs from POSIX priority. A POSIX receiver always gets the highest-priority waiting message. A System V receiver supplies a type-selection argument on each receive.

System V queues use numeric identifiers and kernel IPC permissions rather than POSIX names. On Linux, administrators can inspect them with:

They remain until explicitly removed with msgctl(..., IPC_RMID, ...), an administrative command, or a system restart. A crashed owner can therefore leave a stale queue that consumes kernel resources.

New applications often prefer POSIX names, descriptors, and priority semantics. Existing software, compatibility requirements, or a need for System V type selection can make the older interface appropriate.

Contention, Throughput, and Backpressure

Every send and receive enters the kernel and copies the message across the user-kernel boundary. The queue also updates metadata, selects the next message, and may wake a waiting process.

This cost is often reasonable for small control records and jobs because the kernel supplies framing, synchronization, capacity limits, and blocking. Large messages amplify copy cost and consume the queue's bounded kernel memory.

Several producers can send concurrently, and several consumers can receive concurrently. The kernel protects the queue itself, so application code does not need a separate mutex around mq_send() or mq_receive(). Shared state referenced by a message may still require its own synchronization.

Queue capacity applies backpressure to producers. A blocking sender sleeps when the queue is full; a nonblocking sender receives EAGAIN; a timed sender can abandon or redirect the operation after a deadline. Dropping, retrying, or sending elsewhere remains an application policy.

Priorities can affect both latency and throughput. Finding the highest-priority message and repeatedly favoring urgent work can delay lower-priority traffic. Priority should represent a bounded service policy rather than an unlimited escape from queue order.

Signals as IPC Notifications

A signal communicates an event by number. The sender can use kill() for a process-directed signal:

Despite its name, kill() can send signals whose default action is not termination. The kernel checks whether the target exists and whether the sender has permission.

Successful return means the kernel accepted the signal request. It does not mean a handler ran, the target reached a safe shutdown point, or any application work completed.

Signals are useful when the event itself is the message:

  • Stop or continue execution
  • Request graceful termination
  • Report that a child changed state
  • Notify a process to reload or inspect ordinary state

They are a poor transport for application records. A signal number carries very little information, delivery interrupts normal execution, and standard signal instances can coalesce.

Coalescing of Standard Signals

If the same standard signal is generated several times while it is blocked or already pending, the process may retain only one pending instance:

Three generation attempts therefore cannot represent a reliable count of three jobs. The signal means that at least one relevant event is pending.

A useful design keeps detailed state elsewhere. The sender updates that state through an appropriate IPC mechanism, then sends a signal as a wakeup hint. The receiver checks the authoritative state and processes everything currently available.

A handler must remain restricted to async-signal-safe operations. Common designs set a volatile sig_atomic_t flag, accept the signal synchronously with sigwait(), or write a byte to a pre-created pipe so ordinary control flow can handle the event.

Queued Real-Time Signals and Their Limits

POSIX real-time signals, from SIGRTMIN through SIGRTMAX, queue multiple instances instead of coalescing them under normal operation. sigqueue() can attach a union sigval containing an integer or pointer-sized value.

Queued delivery makes real-time signals suitable for small notifications that must be counted. They remain bounded by system resource limits. When the pending-signal limit is reached, sigqueue() can fail with EAGAIN.

A pointer value sent to another process is not a usable shared pointer. Each process has its own virtual address space. Integer identifiers or compact values are safer payloads.

Real-time signals still provide a signal-delivery environment, including masking and handler-safety constraints. They do not offer variable-length records, queue inspection, application acknowledgments, or durable storage. A POSIX message queue is a clearer interface when the payload itself matters.

Message Queues vs. Signal Contracts

The mechanisms overlap only for small notifications:

PropertyMessage queueSignal
Main purposeTransport discrete recordsNotify a process of an event
PayloadBounded byte sequenceSignal number, optionally a small value
Repeated eventsOne queued message per successful sendStandard signals may coalesce
ConsumptionOne receiver removes one messageDelivery changes process control flow
BackpressureFull queue blocks or reports an errorPending-signal limits can reject queued real-time signals
Processing confirmationRequires an application acknowledgmentRequires an application acknowledgment

A shutdown request fits a signal because the event has a conventional meaning and carries little data. A job description belongs in a message queue because every job needs a preserved boundary and payload.

Neither mechanism proves that application work completed. If completion matters, the protocol needs an explicit response, acknowledgment, or observable state transition.

Loading simulation...

Security and Failure Boundaries

POSIX queue mode bits and ownership control which processes can open a queue. The process umask can further restrict the requested creation mode. System V queues use their own IPC permission fields.

Queue names should identify one owned instance. O_CREAT | O_EXCL prevents a creator from treating an existing queue as fresh state. A collision needs investigation or an ownership-aware cleanup policy.

Received messages should be treated as untrusted input whenever more than one process can write to the queue. Check the exact message length, version, enum ranges, string termination, identifiers, and any reference to external data.

Signal delivery also has permission checks, but a target should not treat receipt alone as authentication for a sensitive operation. Conventional control signals usually request that the process re-check its own configuration or state rather than trusting data supplied through the signal.

Process exit does not provide transactional cleanup. Queued messages can remain after a sender exits, and a receiver can disappear after removing a message. Queue ownership, retry rules, and stale-object removal must be designed explicitly.

Summary

Message queues preserve record boundaries and hold a bounded set of messages inside the kernel. POSIX queues add numeric priorities, while System V queues use message types and receiver-side selection. Full and empty queues provide blocking, nonblocking, or timed behavior.

Queue lifecycle is separate from process lifetime. A successful send confirms admission to the queue, and a successful receive removes one message. Acknowledgment, retry, redelivery, payload validation, stale-object cleanup, and durable storage remain application concerns.

Signals provide asynchronous event notification. Standard signals can coalesce, while real-time signals queue bounded instances with small values. Signals fit control events and wakeups; message queues fit discrete application records.

Quiz

Message Queues and Signals as IPC Quiz

5 quizzes