An online store uses Kafka to share order events with reporting and fulfillment services. The reporting service needs to read orders, but it should never publish a fake order or delete the topic. A separate payments service handles data that reporting should not be able to access at all.

Putting these applications on a private network limits who can reach Kafka. It still leaves a question: what should Kafka allow each application to do once it connects?

Kafka's security model combines protected connections, verified identities, and permissions on Kafka resources. This chapter explains how those controls work together, follows a request through them, and identifies the responsibilities that remain with your infrastructure and applications. The examples assume an Apache Kafka 4.x cluster running in KRaft mode with the built-in ACL authorizer.

Start with the access the store actually needs. The order service publishes to orders.placed. The reporting service reads that topic using the consumer group order-reporting. A platform administrator creates topics and manages permissions.

Those are different responsibilities, so they should use different credentials. If reporting accidentally runs a topic deletion command, Kafka should reject it. If someone steals reporting's credentials, the permissions should limit the damage to the access reporting already has.

Several controls contribute to this design:

Each control has a specific scope. An encrypted connection can belong to an application with excessive permissions. A correctly authenticated application can still send incorrect business data. A firewall can restrict reachability without distinguishing two services running in the same network.

Kafka supports deployments with security disabled, and different endpoints can use different security settings. A working connection is not evidence that the cluster has the intended security controls. You need to know how the deployment protects that connection and which permissions Kafka enforces.

A listener is a named endpoint on which a Kafka server accepts connections. Its configuration determines the security protocol that endpoint uses. A broker can expose several listeners, such as one for applications and another for communication between brokers.

Kafka provides four security protocol names:

Kafka uses SSL in configuration names, although modern deployments use TLS, or Transport Layer Security. TLS protects data while it travels over a connection. The client also checks the server's certificate to establish that it is connecting to the expected server. Certificate trust and hostname verification are part of that protection.

SASL, or Simple Authentication and Security Layer, provides a framework for authentication mechanisms. Kafka supports mechanisms such as SCRAM, Kerberos, and OAuth bearer tokens. Selecting a SASL mechanism does not by itself encrypt Kafka records; the SASL_SSL protocol combines SASL authentication with TLS transport protection.

For our store, assume application connections use SASL_SSL. The order service first establishes TLS and verifies the broker, then authenticates with SASL. Kafka uses the resulting identity when deciding whether to accept its requests.

Protecting the application connection is only part of the work. An order record also travels between brokers during replication. KRaft brokers and controllers exchange cluster metadata and control requests over their own connections.

The diagram shows the communication paths that need explicit security decisions. It groups several machines together to keep the boundaries visible.

Enabling TLS on the application listener does not automatically secure the other paths. Restrict internal endpoints at the network level and configure their authentication and encryption deliberately. Listener names such as INTERNAL describe intent; they do not enforce network isolation.

Clients also connect to brokers that metadata identifies after the initial bootstrap connection. The same protection must hold when a producer reaches a partition leader on another broker or a consumer reconnects after a failure.

After authentication, Kafka represents the identity as a principal. For example, the order service might authenticate as the principal User:order-service, while reporting uses User:reporting-service. Here, User is a principal type; it does not mean a person must use the identity.

The authentication mechanism determines how the client proves its identity. With mutual TLS, both sides present certificates, and the server derives the client identity from its certificate, subject to configured mapping rules. With SASL, the configured mechanism establishes the identity using credentials such as a username and password or a token.

The exact principal matters because permission rules must match it. A certificate's full distinguished name and a shortened service name are different identities unless the server's mapping rules connect them.

Several familiar Kafka fields serve other purposes:

Suppose reporting changes its client.id to order-service. It should still authenticate as User:reporting-service, and Kafka should still apply reporting's permissions. Otherwise, a label the application chooses would let it impersonate another service.

Separate service credentials also make changes easier to contain. If reporting's credential leaks, the team can replace it without replacing the order producer's credential. Sharing one administrator credential across both applications would remove that separation and give each application unnecessary authority.

Authorization evaluates an operation against the authenticated principal and the resource involved. Kafka's built-in rules are access control lists, or ACLs. They express allow or deny decisions using a principal, operation, source host, and resource pattern.

Resources include topics, consumer groups, transactional IDs, and the cluster itself. A topic permission does not automatically grant access to a consumer group.

For the store, the intended policy is:

This table describes business intent, not a complete set of ACL commands. The exact operations required depend on the client features in use, including transactions and administrative calls.

Consider reporting's two resources. Permission to read orders.placed controls access to order records. Permission to use order-reporting controls its group operations, including normal offset commits. If you allow the topic but omit the group permission, the service can authenticate successfully and still fail to run as that group.

This is an example of least privilege: give an application the access its job requires. Reporting has no reason to change topic retention. A bad deployment should not be able to shorten the store's replay history simply because it already has permission to read orders.

Authorization Enforcement

An authorizer is the server component that makes permission decisions. For KRaft clusters, Kafka supplies StandardAuthorizer, which stores ACLs in cluster metadata. Configure it on the relevant server nodes, including brokers and controllers. Authentication alone does not enable ACL enforcement.

With StandardAuthorizer and allow.everyone.if.no.acl.found=false, Kafka denies protected operations when ordinary principals lack permission. Configured superusers bypass ACL checks. Keep that distinction explicit when testing: success with an administrator credential tells you little about the application's policy.

Assume orders.placed already exists and the order service has the permissions its producer requires. It wants to publish an order event with key ord-1042.

The sequence below separates connection setup from authorization of the produce request. The connection can carry many requests after authentication succeeds, and the authorizer checks each one.

An authorization success lets the broker continue handling the operation. The write can still fail for operational reasons, such as insufficient in-sync replicas. Security checks do not replace Kafka's storage, replication, or acknowledgment behavior.

Now suppose the deployment accidentally gives the producer reporting's credentials. TLS can succeed. SASL can succeed. The broker should then reject the write because the connection belongs to User:reporting-service, which lacks permission to publish orders.

That distinction helps diagnose failures:

Repeatedly retrying the same write with reporting's credentials cannot repair that policy mismatch. The deployment needs the correct identity. Granting reporting write access just to silence the error would make the mistaken deployment succeed while weakening the intended separation.

Kafka ACLs control access to Kafka resources. They do not interpret each order and decide which customer the reader represents.

Suppose orders.placed contains records for two tenants, where a tenant is a customer or team sharing the platform. Adding tenant-id to each record does not create a Kafka permission boundary. An application with topic read access can request records across that topic's partitions; Kafka's standard ACLs do not filter them by tenant header, record key, or JSON field.

The same issue applies when an application needs only part of an event. If reporting should see order totals but not delivery addresses, filtering out addresses after receiving the records still gives reporting access to the original data.

One possible design uses a trusted processor to produce a restricted reporting stream. The processor can read the original topic, while reporting receives access only to the derived topic.

This design gives the derived topic its own access permissions. Its correctness depends on the processor removing the intended fields and on reporting having no access to the original topic. Separate topics also add storage and operational work, so choose boundaries that match real access requirements.

TLS has a different boundary: it protects connections. The broker terminates TLS, and TLS does not encrypt the partition files on disk. Protect stored data through the storage platform, host permissions, and key management. Include replicas, backups, and any remote storage in that decision.

If brokers must never see particular fields in plaintext, the application needs a separate encryption design before publishing them. That introduces key distribution and processing constraints for consumers.

Finally, Kafka cannot withdraw a record that a consumer has already copied into a database or log file. Revoking read permission restricts subsequent access through Kafka; the receiving systems need their own retention and access controls.

A shared cluster needs both access separation and resource management. Reporting might have exactly the right permissions and still overload the cluster by replaying months of orders at once. Quotas can limit supported categories of client resource use, but they do not provide complete isolation of CPU, disk, or failures between tenants.

For the store, that means reviewing two separate questions: whether reporting may read the order stream, and how much shared capacity its replay may consume. Workloads requiring stronger isolation may need separate infrastructure.

Operational access deserves the same attention as application access. Protect the machines running Kafka, the credentials they hold, and the tools that can alter cluster configuration. Kafka protocol permissions cannot contain an attacker who already controls a broker process or its host.

Connected services introduce additional boundaries. A Kafka Connect worker has Kafka credentials, but its management endpoint and destination database require their own controls. A schema service or monitoring endpoint also needs separate protection. A managed Kafka provider may offer identity management, roles, or audit features beyond Apache Kafka's built-in model; verify what the provider actually enforces.

Before using the store's policy in production, test with the actual service identities. Confirm that reporting can read orders and commit progress, then confirm that it cannot publish orders, read payment data, or delete a topic. Repeat relevant checks when permissions, credentials, or listener settings change.

These checks make the policy observable. A successful permitted operation shows that the application can work. A rejection at the authorization stage confirms that Kafka blocks that operation for the tested identity.

Kafka security combines protected connections, authenticated principals, and authorization on Kafka resources. Configure each communication path deliberately, give services separate identities, and verify permissions using those identities.

The boundaries matter as much as the controls. Topic access does not provide record-level filtering, TLS does not protect stored files, and ACLs do not isolate shared capacity. A complete design also accounts for hosts, storage, connected services, and the data consumers retain after reading it.