An order service needs the payment service to capture a payment. It publishes a Kafka record called PaymentCaptured, expecting the payment service to do the work. Reporting sees the same record and immediately counts the payment as revenue. The payment service then rejects the request because the authorization has expired.

The problem began before either consumer ran: the message announced an outcome that had not happened.

Commands and events express different things. A command requests an action; an event reports a fact. In this chapter, we’ll follow a payment workflow to see how that difference affects message contracts, service ownership, failure handling, and replay.

A command expresses something the sender wants a responsible service to do. CapturePayment asks the payment service to capture a previously authorized amount. The service must decide whether that operation is allowed and whether it can complete it.

An event expresses something that has happened. PaymentCaptured says the payment service has confirmed the capture. A consumer may react to that fact, but its reaction does not determine whether the original capture happened.

The distinction is about meaning, not transport. Applications can serialize both messages as JSON and store them in Kafka. Kafka does not interpret a type field and enforce command or event semantics.

One event does not have to come from one command. A service might publish PaymentAuthorizationExpired after detecting expiration without receiving a command. One command can also lead to several events over time. The contract should define which outcomes matter rather than assume a one-to-one relationship.

Names make the distinction easier to see, but they cannot replace a definition. For example, OrderAccepted establishes acceptance, not payment or shipment. Define precisely what must be true before the producer publishes each event.

Suppose order ord-1042 has a payment authorization for $1,299.00. The order workflow is ready to request capture. In this example, the payment service owns payment pay-501, permits one full capture for it, and knows the associated authorization. Partial captures are outside this contract.

The workflow writes a CapturePayment command to payments.commands, using pay-501 as the Kafka record key:

The command identifies the requested operation, the payment it concerns, and the amount. The command expresses 129900 in USD minor units, so it means $1,299.00. It contains no payment credentials; the payment service resolves its own authorized payment reference.

The payment service checks that the caller has permission to request this operation, that the amount agrees with its payment state, and that capture is still valid. The source field is descriptive metadata, not proof of authorization. Trusted producer access and application checks must support that trust.

If capture succeeds and the service durably records the result, it publishes this event to payments.events, also keyed by pay-501:

Here, correlationId connects messages belonging to the same checkout workflow. causationId identifies the command that directly caused this outcome. The command and event retain separate identities because the request and the confirmed capture are different occurrences. These envelope fields are application conventions, not Kafka requirements.

The diagram shows the successful path. Each arrow represents a message or an application operation; storing a command is only the beginning.

The payment service is authoritative about the capture. The order workflow requests it and reacts to its outcome. The payment service must reliably coordinate recording that outcome with publishing its event; a crash between a database update and publication must not leave the workflow waiting forever.

A command has one logical execution owner. That owner can run many instances. For CapturePayment, payment-service instances share the payment-execution consumer group and divide the command topic’s partitions among themselves.

An event can have several independent reactions. The order workflow updates checkout status, reporting records the payment, and notifications sends a receipt. Those applications use separate consumer groups when each needs the full event stream.

The following diagram contrasts shared command execution with independent event consumption. It assumes regular Kafka consumer groups.

Creating a second execution group does not add workers to the first. Both groups can read the same commands independently, potentially causing both to attempt capture. Additional execution instances should join the designated group rather than invent their own group IDs.

An audit application can read commands in a separate group if it only observes them. “One owner” does not mean nobody else may inspect the request; it means responsibility for executing it is explicit.

Commands and events use separate topics here because their audiences, access permissions, and replay expectations differ. The names payments.commands and payments.events communicate those purposes, but the names themselves enforce nothing. A topic containing both is possible, though every consumer must then distinguish requests from facts and follow the appropriate processing rules.

The same key in both topics does not create ordering across them. The workflow connects a result to its request through the message contract, not by comparing offsets from two different logs.

An asynchronous command passes through several distinct milestones. The sender publishes the request, the responsible service receives and evaluates it, and the business operation eventually reaches an outcome.

A successful produce acknowledgment means Kafka has met the producer’s configured acks requirement. It does not mean the payment service has accepted the command or that the payment provider has captured funds.

The workflow should therefore represent the payment as pending after publishing CapturePayment. It transitions to captured only after learning the confirmed result. If the contract includes an intermediate CapturePaymentAccepted response, that means the service accepted responsibility for processing; it still does not mean capture has completed.

The payment service can also legitimately reject a command. An expired authorization or an amount mismatch is a business decision, not necessarily a broken consumer. The service can record the rejection and publish a PaymentCaptureRejected outcome with a stable reason code and the original command’s identity. Once the service reliably records that outcome for publication, repeatedly attempting the unchanged request usually achieves nothing.

Define how requesters discover outcomes. They might consume a result topic or query a durable operation-status API using commandId. Do not assume that every asynchronous command has an automatic reply channel that Kafka supplies.

Also distinguish a requester’s waiting limit from the operation’s outcome. If the workflow waits ten seconds and sees no result, it knows only that it has not observed completion. The request might still be waiting, executing, or completed with its result delayed. Reporting failure solely because a timer expired can contradict what the payment system actually did.

Consumers can process both commands and events more than once. What matters is which action they repeat.

For a command, duplicate processing may repeat the requested operation. For an event, it may repeat a reaction such as sending a receipt. Neither becomes safe merely because its message type is correct.

Consider a payment service that calls the provider successfully and then crashes before saving its Kafka progress. After recovery or a rebalance, another instance may read the command again. Sharing one consumer group does not guarantee one execution of the external operation.

The command ID should remain stable when retrying the same request. The payment service needs durable handling records and rules that prevent repeated requests from causing repeated capture. The payment identity also matters: if the caller accidentally creates a new command ID for the same full capture, the service must still enforce its one-capture business rule.

An external provider introduces another failure boundary. A timeout after sending the capture request can mean either that capture never happened or that the response never reached the service. The following diagram shows how the service should treat that uncertainty conceptually.

Safe retry depends on the provider’s actual contract. If it supports an idempotency key, the service can associate a stable key with this capture operation and reuse it according to the provider’s rules, including any retry time limit. Otherwise, reconciliation may require a status lookup or manual resolution. The diagram does not imply unlimited retries or guaranteed automatic recovery.

Do not publish PaymentCaptureRejected while the outcome is merely unknown. That event would assert more than the service knows. Persist enough information to resume resolution, and define when an unresolved operation needs attention.

Kafka transactions can coordinate supported Kafka reads and writes, but they do not make an external payment API call part of that transaction. The payment operation needs its own duplicate-handling and recovery strategy.

A command can become inappropriate while waiting for the service to process it. The sample’s expiresAt means the payment service must not start a new capture attempt at or after that deadline. This is a rule the application checks, not a per-record expiration mechanism Kafka applies to the JSON field.

The contract must define the boundary precisely. Here, an operation already started before the deadline still needs a confirmed outcome afterward. If a retry arrives after expiration for a command that already succeeded, the service reports the recorded success. It does not reinterpret the completed operation as expired. An unresolved attempt still needs reconciliation.

Topic retention serves a different purpose. It controls how long records remain available, not whether a business request remains valid. Kafka can retain a command after its deadline or remove it before an unavailable service reads it. Retention and command-status tracking must support the expected backlog and recovery period.

Replay makes the distinction especially visible. Replaying PaymentCaptured into a new reporting table can rebuild knowledge of past payments. Replaying CapturePayment into a live execution handler can attempt financial operations again unless its durable state prevents them.

Replaying events also requires care. A notification consumer can resend every historical receipt if its replay invokes live side effects. Separate rebuilding state from performing those effects, and make the intended replay behavior explicit for each consumer.

Keep command execution history for as long as callers can legitimately retry or replay requests under the contract. A duplicate check that forgets commands after a day cannot protect a handler that blindly executes a month of retained requests. Expiration rules and durable payment state provide additional boundaries, but design them together.

Finally, cancellation is a new business request. Removing a command from an application’s view or timing out its caller does not undo a capture already underway. A later RefundPayment command may be appropriate if capture succeeded, subject to the payment service’s rules. The original PaymentCaptured event remains a true historical fact.

Consider a message named PaymentRequested. It sounds like an event, but its name leaves two possible contracts.

It can be a genuine fact: the payment service has durably registered a payment request and is announcing that registration. Consumers may use it to show pending work or measure demand.

It can also be a disguised instruction: the order workflow publishes it specifically to make the payment service capture funds and waits for that action. In that design, you still need to define the execution owner, validation rules, deadlines, and result handling. Calling the message an event does not remove those command responsibilities.

There is also a legitimate event-driven alternative. The order service publishes OrderPlaced because it accepted an order. The payment service subscribes and, according to its own business policy, decides whether to begin payment work. The event remains a fact even though a consumer takes action because of it.

The important decision is who owns the policy. With CapturePayment, the workflow explicitly requests a particular step and the payment service validates and performs it. With a reaction to OrderPlaced, the payment service owns the rule that maps order acceptance to payment work. Either can be appropriate, but the responsibility should be clear enough to locate a missing or duplicated step.

Use an explicit command when a caller needs a designated service to attempt a specific operation. Publish an event when a system has established a fact that other applications can interpret under their own responsibilities. Many workflows naturally use both: request an action, record its outcome, then publish the resulting fact.

Transport remains a separate choice. A command can use a direct API call when the caller needs an immediate answer, or Kafka when durable asynchronous delivery suits the workflow. Using Kafka does not eliminate the need to track pending work, deadlines, and outcomes.

Commands request actions from a responsible service; events report established facts. That difference determines who validates a request, who may announce an outcome, and how consumers should react.

Keep publication separate from business completion. Preserve identities through retries, represent uncertain outcomes honestly, and define expiration and replay behavior around real side effects. Clear command and event contracts make a workflow easier to understand and safer to recover when processing fails.