When teams reach for a message broker, the instinct is often to pick whatever the team already knows, or whatever the first Stack Overflow answer suggests. After building on both Kafka and SQS at scale, I've developed a clearer mental model for when each one earns its place.
The core difference isn't throughput
Most comparisons lead with throughput. Kafka handles millions of events per second; SQS is capped at 3,000 messages per second per queue (though this is often a non-issue in practice). But throughput is rarely the deciding factor.
The real distinction is philosophy.
-
SQS is a job queue. A message goes in, a consumer picks it up, processes it, deletes it. The message is gone. This is the work-queue pattern and it is extremely well-suited to task distribution.
-
Kafka is a distributed log. Events are appended to a topic and retained. Multiple consumers can read the same event independently. You can replay history. The log is the truth.
When I'd choose SQS
Decoupling services where the event has one clear consumer. If you're triggering an image resize job after an upload, SQS is perfect. One message, one job, one consumer group. You get built-in visibility timeouts, DLQs, and FIFO semantics if you need them — all managed for you on AWS.
Serverless Lambda triggers. SQS integrates with Lambda out of the box and scales the concurrency for you. There's very little operational overhead, which matters when your team is small.
Short retention windows. If you don't need event history beyond a few days, SQS is cheaper and simpler than standing up a Kafka cluster.
When I'd choose Kafka
Multiple independent consumers of the same event. If an order.created
event needs to trigger inventory, billing, and a notification service, Kafka's
consumer-group model shines. Each service maintains its own offset. They can
fall behind and catch up without worrying about the event being deleted.
Event sourcing and audit trails. Kafka's log retention makes it a natural fit for event sourcing. You can rebuild state by replaying events, audit what happened and when, and add new services that backfill from the beginning of history.
High-volume, time-series data. Kafka was built for this. Log aggregation, metrics pipelines, clickstream data — if you're dealing with continuous, high-frequency streams, Kafka's partitioned architecture handles this elegantly.
The question I always ask
"Does anything, other than the primary consumer, need to read this event?"
If the answer is no — now or in the foreseeable future — SQS is almost always the right call. It's cheaper, simpler, and requires no operational burden on AWS.
If the answer is yes, or if the event represents something that happened to your system (rather than a task to be done), lean towards Kafka. The ability to replay, branch, and add consumers without changing the producer is worth the additional operational complexity.
A word on managed Kafka
If you do go Kafka, use a managed service. AWS MSK or Confluent Cloud removes the single biggest pain point — broker management — and lets you focus on the schemas and consumer logic. Running Kafka yourself on EC2 is rarely worth it in 2025.
The choice between Kafka and SQS isn't about which tool is better. It's about understanding the nature of the data flow you're modelling. Job distribution wants queues. Event history wants a log. Knowing the difference will save you significant architectural rework down the line.