Amazon SQS #

When your application is already running on AWS infrastructure, choosing a message broker fully managed by the cloud provider is a very sensible decision. Amazon Simple Queue Service (SQS) is a managed queue service that requires no server provisioning, no cluster to maintain, and scales automatically from one message to billions of messages per day. Unlike RabbitMQ, which requires you to understand exchanges and bindings, or Kafka, which requires managing partitions and consumer groups, SQS is designed to be as simple as possible: send a message to a queue, retrieve a message from a queue, delete the message after processing. This simplicity is both its strength and its limitation — and understanding that tradeoff is the key to using SQS effectively.

SQS Basics #

SQS uses a model different from traditional brokers. There are several unique concepts you won’t find in RabbitMQ or Kafka.

Visibility Timeout #

This is the most important concept in SQS and the most commonly misunderstood. When a consumer retrieves (receives) a message from SQS, the message is not immediately deleted — it becomes invisible to other consumers for the duration of the visibility timeout. The consumer has the visibility timeout period to process and delete the message. If the message isn’t deleted within that time, SQS assumes processing failed and makes the message visible again.

sequenceDiagram
    participant C as Consumer
    participant SQS

    C->>SQS: ReceiveMessage
    SQS-->>C: message (visibility timeout starts: 30s)
    Note over SQS: message invisible\nto other consumers

    alt Processing succeeds
        C->>SQS: DeleteMessage
        Note over SQS: message permanently deleted
    else Timeout expires (consumer crash)
        Note over SQS: message becomes visible again
        SQS-->>C: message redelivered to another consumer
    end

Standard Queue vs FIFO Queue #

SQS has two queue types with very different characteristics:

Standard QueueFIFO Queue
ThroughputNearly unlimited300 msg/s (3,000 with batching)
OrderingBest-effort (not guaranteed)Strict — first-in, first-out
DuplicationPossible (at-least-once)Exactly once
Queue nameFree-formMust end with .fifo
PriceCheaperMore expensive
Use caseHigh throughput, duplicate-tolerantFinancial transactions, critical ordering
flowchart TD
    A{"Message ordering\nmust be guaranteed?"} -- Yes --> B{"Throughput\n> 300 msg/s?"}
    A -- No --> C{"Can duplicate messages\nbe tolerated?"}

    B -- Yes --> D["Consider Kafka\nor a rearchitecture"]
    B -- No --> E[FIFO Queue]

    C -- Yes --> F["Standard Queue\ncheaper and faster"]
    C -- No --> G{"Need a 5-minute\ndeduplication window?"}

    G -- Yes --> E
    G -- No --> F

Message Groups (FIFO Queues) #

FIFO queues support the message group concept — each group is processed sequentially, but different groups can be processed in parallel. This provides scalability while still guaranteeing per-entity ordering.

FIFO Queue — order-processing.fifo

MessageGroupId = "order-123"  → [event-1] [event-2] [event-3]  ← ordered
MessageGroupId = "order-456"  → [event-1] [event-2]             ← ordered, parallel with order-123
MessageGroupId = "order-789"  → [event-1]                        ← ordered, parallel with both groups

Setting Up Dependencies #

Add the AWS SDK v2 for SQS to your project. AWS SDK v2 is the modern version that uses a builder pattern and natively supports async.

For Maven:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>bom</artifactId>
            <version>2.25.60</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- SQS client -->
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>sqs</artifactId>
    </dependency>
    <!-- URL Connection HTTP client (lightweight, no Netty needed) -->
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>url-connection-client</artifactId>
    </dependency>
</dependencies>

For Gradle:

dependencies {
    implementation platform('software.amazon.awssdk:bom:2.25.60')
    implementation 'software.amazon.awssdk:sqs'
    implementation 'software.amazon.awssdk:url-connection-client'
}

For local development without an AWS account, use LocalStack:

# Run LocalStack — a local AWS services emulator
docker run -d \
  --name localstack \
  -p 4566:4566 \
  -e SERVICES=sqs \
  localstack/localstack

# Create a queue via the AWS CLI (pointing to LocalStack)
aws --endpoint-url=http://localhost:4566 sqs create-queue \
  --queue-name order-queue \
  --region ap-southeast-1

Creating the SQS Client #

Client configuration differs between production environments (AWS) and development (LocalStack):

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.sqs.SqsClient;

import java.net.URI;

public class SqsClientFactory {

    // ✓ CORRECT: production — use DefaultCredentialsProvider
    // Automatically finds credentials from: env vars → ~/.aws/credentials → IAM role
    public static SqsClient createProductionClient() {
        return SqsClient.builder()
            .region(Region.AP_SOUTHEAST_1)
            .credentialsProvider(DefaultCredentialsProvider.create())
            .httpClient(UrlConnectionHttpClient.builder().build())
            .build();
    }

    // ✓ CORRECT: development with LocalStack
    public static SqsClient createLocalStackClient() {
        return SqsClient.builder()
            .region(Region.AP_SOUTHEAST_1)
            .endpointOverride(URI.create("http://localhost:4566"))
            .credentialsProvider(StaticCredentialsProvider.create(
                AwsBasicCredentials.create("test", "test") // LocalStack doesn't validate credentials
            ))
            .httpClient(UrlConnectionHttpClient.builder().build())
            .build();
    }

    // ✗ ANTI-PATTERN: hardcoding credentials in code
    // Credentials can leak into version control
    public static SqsClient createInsecureClient() {
        return SqsClient.builder()
            .credentialsProvider(StaticCredentialsProvider.create(
                AwsBasicCredentials.create("«redacted:AKIA…»", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY")
            ))
            .build();
    }
}
Never hardcode AWS credentials (Access Key ID and Secret Access Key) in source code. Use DefaultCredentialsProvider in production, which automatically reads from the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, the ~/.aws/credentials file, or an IAM role when running on EC2/ECS/Lambda.

Queue Management #

Before sending or receiving messages, the queue must exist. You can create and manage queues programmatically.

import software.amazon.awssdk.services.sqs.model.*;

import java.util.Map;

public class SqsQueueManager {

    private final SqsClient sqsClient;

    public SqsQueueManager(SqsClient sqsClient) {
        this.sqsClient = sqsClient;
    }

    // Create a Standard Queue
    public String createStandardQueue(String queueName) {
        CreateQueueRequest request = CreateQueueRequest.builder()
            .queueName(queueName)
            .attributes(Map.of(
                QueueAttributeName.VISIBILITY_TIMEOUT, "30",          // seconds
                QueueAttributeName.MESSAGE_RETENTION_PERIOD, "86400", // 1 day (seconds)
                QueueAttributeName.RECEIVE_MESSAGE_WAIT_TIME_SECONDS, "20", // long polling
                QueueAttributeName.MAX_MESSAGE_SIZE, "262144"         // 256 KB (SQS max)
            ))
            .build();

        CreateQueueResponse response = sqsClient.createQueue(request);
        System.out.println("Queue created: " + response.queueUrl());
        return response.queueUrl();
    }

    // Create a FIFO Queue
    public String createFifoQueue(String queueName) {
        // FIFO queue names must end with .fifo
        if (!queueName.endsWith(".fifo")) {
            queueName = queueName + ".fifo";
        }

        CreateQueueRequest request = CreateQueueRequest.builder()
            .queueName(queueName)
            .attributes(Map.of(
                QueueAttributeName.FIFO_QUEUE, "true",
                QueueAttributeName.CONTENT_BASED_DEDUPLICATION, "false", // use explicit deduplication IDs
                QueueAttributeName.VISIBILITY_TIMEOUT, "30",
                QueueAttributeName.MESSAGE_RETENTION_PERIOD, "86400"
            ))
            .build();

        CreateQueueResponse response = sqsClient.createQueue(request);
        return response.queueUrl();
    }

    // Create a Standard Queue with a Dead Letter Queue
    public String createQueueWithDLQ(String mainQueueName, String dlqName) {
        // 1. Create the DLQ first
        String dlqUrl = createStandardQueue(dlqName);

        // 2. Get the ARN of the DLQ
        GetQueueAttributesResponse dlqAttrs = sqsClient.getQueueAttributes(
            GetQueueAttributesRequest.builder()
                .queueUrl(dlqUrl)
                .attributeNames(QueueAttributeName.QUEUE_ARN)
                .build()
        );
        String dlqArn = dlqAttrs.attributes().get(QueueAttributeName.QUEUE_ARN);

        // 3. Create the main queue with a redrive policy pointing to the DLQ
        String redrivePolicy = String.format(
            "{\"deadLetterTargetArn\":\"%s\",\"maxReceiveCount\":\"3\"}",
            dlqArn
        );
        // maxReceiveCount=3 → messages go to the DLQ after failing to process 3 times

        CreateQueueRequest request = CreateQueueRequest.builder()
            .queueName(mainQueueName)
            .attributes(Map.of(
                QueueAttributeName.VISIBILITY_TIMEOUT, "30",
                QueueAttributeName.RECEIVE_MESSAGE_WAIT_TIME_SECONDS, "20",
                QueueAttributeName.REDRIVE_POLICY, redrivePolicy
            ))
            .build();

        CreateQueueResponse response = sqsClient.createQueue(request);
        System.out.println("Main queue: " + response.queueUrl());
        System.out.println("DLQ: " + dlqUrl);
        return response.queueUrl();
    }

    // Get a queue URL by name
    public String getQueueUrl(String queueName) {
        return sqsClient.getQueueUrl(
            GetQueueUrlRequest.builder().queueName(queueName).build()
        ).queueUrl();
    }
}

Sending Messages (Producer) #

Sending a Single Message #

import software.amazon.awssdk.services.sqs.model.MessageAttributeValue;
import software.amazon.awssdk.services.sqs.model.SendMessageRequest;
import software.amazon.awssdk.services.sqs.model.SendMessageResponse;

public class SqsProducer {

    private final SqsClient sqsClient;
    private final String queueUrl;

    public SqsProducer(SqsClient sqsClient, String queueUrl) {
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    // Send a simple message to a Standard Queue
    public String send(String messageBody) {
        SendMessageRequest request = SendMessageRequest.builder()
            .queueUrl(queueUrl)
            .messageBody(messageBody)
            .delaySeconds(0) // delivery delay (0-900 seconds), default 0
            .messageAttributes(Map.of(
                "source-service", MessageAttributeValue.builder()
                    .dataType("String")
                    .stringValue("order-service")
                    .build(),
                "event-type", MessageAttributeValue.builder()
                    .dataType("String")
                    .stringValue("order.created")
                    .build(),
                "retry-count", MessageAttributeValue.builder()
                    .dataType("Number")
                    .stringValue("0")
                    .build()
            ))
            .build();

        SendMessageResponse response = sqsClient.sendMessage(request);
        System.out.println("Message sent — MessageId: " + response.messageId());
        return response.messageId();
    }

    // Send to a FIFO Queue — requires MessageGroupId and MessageDeduplicationId
    public String sendToFifo(String messageBody, String messageGroupId, String deduplicationId) {
        SendMessageRequest request = SendMessageRequest.builder()
            .queueUrl(queueUrl)
            .messageBody(messageBody)
            // MessageGroupId — all messages in the same group are processed in order
            .messageGroupId(messageGroupId)
            // MessageDeduplicationId — SQS rejects duplicate messages within a 5-minute window
            // Use a value unique per message (e.g. UUID or a content hash)
            .messageDeduplicationId(deduplicationId)
            .build();

        SendMessageResponse response = sqsClient.sendMessage(request);
        return response.messageId();
    }
}

Batch Sends — Saving Money #

SQS bills per API request, not per message. Sending 10 messages in one batch request costs the same as sending 1 message. Always use batches when possible.

import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequest;
import software.amazon.awssdk.services.sqs.model.SendMessageBatchRequestEntry;
import software.amazon.awssdk.services.sqs.model.SendMessageBatchResponse;

import java.util.ArrayList;
import java.util.List;
import java.util.UUID;

public class SqsBatchProducer {

    private final SqsClient sqsClient;
    private final String queueUrl;

    public SqsBatchProducer(SqsClient sqsClient, String queueUrl) {
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    // ✗ ANTI-PATTERN: sending one by one in a loop
    // 100 messages = 100 API calls = 100x the cost
    public void sendOneByOne(List<String> messages) {
        for (String msg : messages) {
            sqsClient.sendMessage(SendMessageRequest.builder()
                .queueUrl(queueUrl)
                .messageBody(msg)
                .build());
        }
    }

    // ✓ CORRECT: batch send — max 10 messages per batch request
    // 100 messages = 10 API calls = 10x cheaper
    public void sendBatch(List<String> messages) {
        // SQS limits batches to 10 messages
        int batchSize = 10;

        for (int i = 0; i < messages.size(); i += batchSize) {
            List<String> batch = messages.subList(i, Math.min(i + batchSize, messages.size()));
            sendSingleBatch(batch);
        }
    }

    private void sendSingleBatch(List<String> batch) {
        List<SendMessageBatchRequestEntry> entries = new ArrayList<>();

        for (int i = 0; i < batch.size(); i++) {
            entries.add(SendMessageBatchRequestEntry.builder()
                .id(String.valueOf(i))   // unique ID within the batch (not a global message ID)
                .messageBody(batch.get(i))
                .build());
        }

        SendMessageBatchResponse response = sqsClient.sendMessageBatch(
            SendMessageBatchRequest.builder()
                .queueUrl(queueUrl)
                .entries(entries)
                .build()
        );

        // Check for messages that failed to send within the batch
        if (!response.failed().isEmpty()) {
            response.failed().forEach(failure -> {
                System.err.printf(
                    "Failed to send message ID=%s: %s — %s%n",
                    failure.id(), failure.code(), failure.message()
                );
                // Here: retry the failed messages or send to a fallback
            });
        }

        System.out.printf("Batch sent: %d succeeded, %d failed%n",
            response.successful().size(), response.failed().size());
    }
}

Receiving and Processing Messages (Consumer) #

Long Polling #

SQS supports two polling modes: short polling and long polling. Always use long polling.

SHORT POLLING (WaitTimeSeconds=0):
  Consumer → SQS: "Any messages?"
  SQS → Consumer: "None" (even though messages may exist)
  [repeat every second]
  Result: many empty responses, high cost, unpredictable latency

LONG POLLING (WaitTimeSeconds=1-20):
  Consumer → SQS: "Any messages? Wait up to 20 seconds"
  SQS: [waits until a message arrives or times out]
  SQS → Consumer: message (as soon as one is available)
  Result: fewer empty responses, lower cost, lower latency

import software.amazon.awssdk.services.sqs.model.DeleteMessageRequest;
import software.amazon.awssdk.services.sqs.model.Message;
import software.amazon.awssdk.services.sqs.model.ReceiveMessageRequest;
import software.amazon.awssdk.services.sqs.model.ReceiveMessageResponse;

public class SqsConsumer {

    private final SqsClient sqsClient;
    private final String queueUrl;
    private volatile boolean running = true;

    public SqsConsumer(SqsClient sqsClient, String queueUrl) {
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    public void start() {
        System.out.println("Consumer started...");

        while (running) {
            try {
                // ✓ Long polling — wait up to 20 seconds if there are no messages
                ReceiveMessageRequest receiveRequest = ReceiveMessageRequest.builder()
                    .queueUrl(queueUrl)
                    .maxNumberOfMessages(10)      // max 10 messages per receive (SQS max)
                    .waitTimeSeconds(20)           // long polling
                    .visibilityTimeout(30)         // the consumer has 30 seconds to process
                    .messageAttributeNames("All")  // fetch all message attributes
                    .attributeNames(              // fetch queue attributes (ApproximateReceiveCount, etc.)
                        software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.APPROXIMATE_RECEIVE_COUNT,
                        software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.SENT_TIMESTAMP
                    )
                    .build();

                ReceiveMessageResponse response = sqsClient.receiveMessage(receiveRequest);
                List<Message> messages = response.messages();

                if (messages.isEmpty()) {
                    continue; // no messages, poll again
                }

                for (Message message : messages) {
                    processAndDelete(message);
                }

            } catch (Exception e) {
                System.err.println("Error while polling: " + e.getMessage());
                // don't crash — wait a moment then try again
                sleep(5000);
            }
        }
    }

    private void processAndDelete(Message message) {
        try {
            System.out.printf("Processing MessageId=%s: %s%n",
                message.messageId(), message.body());

            // Check how many times this message has been received
            String receiveCount = message.attributes().get(
                software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.APPROXIMATE_RECEIVE_COUNT
            );
            System.out.println("Received " + receiveCount + " times");

            doProcess(message.body());

            // ✓ Delete the message after successful processing
            // Use the ReceiptHandle (not MessageId) for deletion
            sqsClient.deleteMessage(DeleteMessageRequest.builder()
                .queueUrl(queueUrl)
                .receiptHandle(message.receiptHandle())
                .build());

            System.out.println("Message processed and deleted successfully.");

        } catch (Exception e) {
            System.err.printf("Failed to process MessageId=%s: %s%n",
                message.messageId(), e.getMessage());
            // Don't delete the message — let the visibility timeout expire
            // SQS will redeliver this message to another consumer
            // After maxReceiveCount is reached, it automatically goes to the DLQ
        }
    }

    private void doProcess(String body) throws Exception {
        // business logic
    }

    public void stop() {
        running = false;
    }

    private void sleep(long ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
    }
}

Extending the Visibility Timeout #

If you know processing will take longer than the visibility timeout, extend it before it expires:

import software.amazon.awssdk.services.sqs.model.ChangeMessageVisibilityRequest;

public class LongRunningProcessor {

    private final SqsClient sqsClient;
    private final String queueUrl;

    public LongRunningProcessor(SqsClient sqsClient, String queueUrl) {
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    public void processHeavyTask(Message message) throws Exception {
        // Run a timer to extend the visibility timeout every 25 seconds
        // (initial visibility timeout = 30 seconds, extend 5 seconds before it expires)
        java.util.concurrent.ScheduledExecutorService scheduler =
            java.util.concurrent.Executors.newSingleThreadScheduledExecutor();

        scheduler.scheduleAtFixedRate(() -> {
            try {
                sqsClient.changeMessageVisibility(
                    ChangeMessageVisibilityRequest.builder()
                        .queueUrl(queueUrl)
                        .receiptHandle(message.receiptHandle())
                        .visibilityTimeout(30) // extend by another 30 seconds
                        .build()
                );
                System.out.println("Visibility timeout extended.");
            } catch (Exception e) {
                System.err.println("Failed to extend visibility timeout: " + e.getMessage());
            }
        }, 25, 25, java.util.concurrent.TimeUnit.SECONDS);

        try {
            // Do the heavy processing that takes a long time
            doHeavyWork(message.body());

            // Delete the message when finished
            sqsClient.deleteMessage(DeleteMessageRequest.builder()
                .queueUrl(queueUrl)
                .receiptHandle(message.receiptHandle())
                .build());

        } finally {
            scheduler.shutdown();
        }
    }

    private void doHeavyWork(String body) throws Exception {
        // simulate heavy processing — e.g. video encoding, large report generation
        Thread.sleep(60_000); // 1 minute
    }
}

Batch Deletes — Saving Money on Deletion #

Just like batch sends, deletes can also be batched:

import software.amazon.awssdk.services.sqs.model.DeleteMessageBatchRequest;
import software.amazon.awssdk.services.sqs.model.DeleteMessageBatchRequestEntry;

public class SqsBatchConsumer {

    private final SqsClient sqsClient;
    private final String queueUrl;

    public SqsBatchConsumer(SqsClient sqsClient, String queueUrl) {
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    public void processAndDeleteBatch() {
        ReceiveMessageRequest receiveRequest = ReceiveMessageRequest.builder()
            .queueUrl(queueUrl)
            .maxNumberOfMessages(10)
            .waitTimeSeconds(20)
            .build();

        List<Message> messages = sqsClient.receiveMessage(receiveRequest).messages();
        if (messages.isEmpty()) return;

        List<DeleteMessageBatchRequestEntry> toDelete = new ArrayList<>();
        List<String> failed = new ArrayList<>();

        for (Message message : messages) {
            try {
                doProcess(message.body());
                // Collect the receipt handles for batch deletion
                toDelete.add(DeleteMessageBatchRequestEntry.builder()
                    .id(message.messageId())
                    .receiptHandle(message.receiptHandle())
                    .build());
            } catch (Exception e) {
                failed.add(message.messageId());
                System.err.println("Failed to process: " + message.messageId());
            }
        }

        // ✓ Batch delete all successfully processed messages in one API call
        if (!toDelete.isEmpty()) {
            var deleteResponse = sqsClient.deleteMessageBatch(
                DeleteMessageBatchRequest.builder()
                    .queueUrl(queueUrl)
                    .entries(toDelete)
                    .build()
            );

            if (!deleteResponse.failed().isEmpty()) {
                deleteResponse.failed().forEach(f ->
                    System.err.println("Failed to delete MessageId=" + f.id() + ": " + f.message())
                );
            }
        }

        System.out.printf("Batch complete: %d processed, %d failed%n",
            toDelete.size(), failed.size());
    }

    private void doProcess(String body) throws Exception {
        // business logic
    }
}

Dead Letter Queues #

SQS has a built-in DLQ mechanism via the Redrive Policy. When a message has been received more than maxReceiveCount times without being successfully deleted, SQS automatically moves it to the DLQ.

flowchart TD
    P[Producer] --> Q["order-queue\nmaxReceiveCount=3"]
    Q --> C{"Consumer\nprocessed successfully?"}
    C -- Yes\nDeleteMessage --> DONE[Message deleted]
    C -- No\nTimeout / Error --> Q
    Q -->|Received 3 times| DLQ[dlq-order-queue]
    DLQ --> MON["DLQ Monitor\nAlert & Investigation"]
    MON -->|After the fix| REDRIVE["Redrive to the\nmain queue"]

Monitoring and reprocessing messages from the DLQ:

public class DlqMonitor {

    private final SqsClient sqsClient;
    private final String dlqUrl;
    private final String mainQueueUrl;

    public DlqMonitor(SqsClient sqsClient, String dlqUrl, String mainQueueUrl) {
        this.sqsClient = sqsClient;
        this.dlqUrl = dlqUrl;
        this.mainQueueUrl = mainQueueUrl;
    }

    // Check the number of messages in the DLQ
    public int getDlqMessageCount() {
        GetQueueAttributesResponse response = sqsClient.getQueueAttributes(
            GetQueueAttributesRequest.builder()
                .queueUrl(dlqUrl)
                .attributeNames(QueueAttributeName.APPROXIMATE_NUMBER_OF_MESSAGES)
                .build()
        );
        return Integer.parseInt(
            response.attributes().get(QueueAttributeName.APPROXIMATE_NUMBER_OF_MESSAGES)
        );
    }

    // Read messages from the DLQ for investigation
    public void inspectDlq() {
        ReceiveMessageRequest request = ReceiveMessageRequest.builder()
            .queueUrl(dlqUrl)
            .maxNumberOfMessages(10)
            .waitTimeSeconds(1)  // short poll for manual inspection
            .attributeNames(
                software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.APPROXIMATE_RECEIVE_COUNT,
                software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.SENT_TIMESTAMP
            )
            .build();

        List<Message> messages = sqsClient.receiveMessage(request).messages();
        System.out.println("Messages in DLQ: " + messages.size());

        for (Message msg : messages) {
            System.out.printf(
                "MessageId=%s | Received %s times | Body: %s%n",
                msg.messageId(),
                msg.attributes().get(
                    software.amazon.awssdk.services.sqs.model.MessageSystemAttributeName.APPROXIMATE_RECEIVE_COUNT
                ),
                msg.body()
            );
        }
    }

    // Redrive — move messages from the DLQ back to the main queue after a bug is fixed
    // The AWS Console provides a built-in "Start DLQ redrive" feature,
    // but this is a manual implementation if you need more control:
    public void redriveMessages() {
        List<Message> messages;
        int redrived = 0;

        do {
            messages = sqsClient.receiveMessage(
                ReceiveMessageRequest.builder()
                    .queueUrl(dlqUrl)
                    .maxNumberOfMessages(10)
                    .waitTimeSeconds(1)
                    .build()
            ).messages();

            for (Message msg : messages) {
                // Resend to the main queue
                sqsClient.sendMessage(SendMessageRequest.builder()
                    .queueUrl(mainQueueUrl)
                    .messageBody(msg.body())
                    .build());

                // Delete from the DLQ
                sqsClient.deleteMessage(DeleteMessageRequest.builder()
                    .queueUrl(dlqUrl)
                    .receiptHandle(msg.receiptHandle())
                    .build());

                redrived++;
            }
        } while (!messages.isEmpty());

        System.out.println("Total messages redrived: " + redrived);
    }
}

SNS + SQS Integration — The Fan-out Pattern #

SQS by itself doesn’t support fan-out (one message to many consumers). For that, combine it with Amazon SNS (Simple Notification Service). SNS acts as the publisher that distributes messages to all subscribed SQS queues.

flowchart TD
    P[Producer] --> SNS["SNS Topic\norder-events"]
    SNS --> Q1["SQS Queue\nanalytics-queue"]
    SNS --> Q2["SQS Queue\nnotification-queue"]
    SNS --> Q3["SQS Queue\naudit-queue"]
    Q1 --> C1[Analytics Service]
    Q2 --> C2[Notification Service]
    Q3 --> C3[Audit Service]
import software.amazon.awssdk.services.sns.SnsClient;
import software.amazon.awssdk.services.sns.model.*;

public class SnsSqsFanout {

    private final SnsClient snsClient;
    private final SqsClient sqsClient;

    public SnsSqsFanout(SnsClient snsClient, SqsClient sqsClient) {
        this.snsClient = snsClient;
        this.sqsClient = sqsClient;
    }

    public void setupFanout(String topicName, List<String> queueArns) {
        // 1. Create the SNS Topic
        CreateTopicResponse topicResponse = snsClient.createTopic(
            CreateTopicRequest.builder().name(topicName).build()
        );
        String topicArn = topicResponse.topicArn();

        // 2. Subscribe each SQS queue to the SNS topic
        for (String queueArn : queueArns) {
            snsClient.subscribe(SubscribeRequest.builder()
                .topicArn(topicArn)
                .protocol("sqs")
                .endpoint(queueArn)
                .build());
        }

        System.out.println("Fan-out setup complete. Topic ARN: " + topicArn);
    }

    // Publish to SNS — automatically delivered to all subscribing SQS queues
    public void publish(String topicArn, String message) {
        snsClient.publish(PublishRequest.builder()
            .topicArn(topicArn)
            .message(message)
            .subject("order-event")
            .build());
    }
}
When SQS receives a message from SNS, the message body is wrapped in an SNS JSON envelope. SQS consumers need to parse that envelope to get the original message. Use the Message attribute inside the SNS JSON notification, not the SQS message body directly.

Handling Duplicate Messages #

Standard Queues can deliver a message more than once (at-least-once delivery). Consumers must be idempotent — processing the same message twice must not produce different effects.

import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class IdempotentConsumer {

    // In production: use Redis or DynamoDB to store processed IDs
    // with a TTL matching the queue retention period
    private final Set<String> processedIds = ConcurrentHashMap.newKeySet();

    public void processIdempotent(Message message) throws Exception {
        String messageId = message.messageId();

        // ✗ ANTI-PATTERN: process directly without checking for duplicates
        // doProcess(message.body()); // could execute twice!

        // ✓ CORRECT: check whether it's already been processed
        if (processedIds.contains(messageId)) {
            System.out.println("Duplicate ignored: " + messageId);
            return; // return immediately, but still delete later
        }

        // Process the message
        doProcess(message.body());

        // Mark it as processed
        processedIds.add(messageId);
    }

    private void doProcess(String body) throws Exception {
        System.out.println("Processing: " + body);
        // business logic that produces effects (write to DB, send email, etc.)
    }
}

For production, use DynamoDB as the idempotency store:

import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.*;

import java.util.Map;

public class DynamoDbIdempotencyStore {

    private final DynamoDbClient dynamoDb;
    private static final String TABLE = "sqs-processed-messages";

    public DynamoDbIdempotencyStore(DynamoDbClient dynamoDb) {
        this.dynamoDb = dynamoDb;
    }

    // Try to mark a message as "processing"
    // Uses a conditional write — only succeeds if the messageId doesn't exist
    public boolean tryMarkAsProcessing(String messageId) {
        try {
            dynamoDb.putItem(PutItemRequest.builder()
                .tableName(TABLE)
                .item(Map.of(
                    "messageId", AttributeValue.fromS(messageId),
                    "status", AttributeValue.fromS("PROCESSING"),
                    "ttl", AttributeValue.fromN(            // auto-delete after 24 hours
                        String.valueOf(System.currentTimeMillis() / 1000 + 86400)
                    )
                ))
                .conditionExpression("attribute_not_exists(messageId)")
                .build());
            return true; // success — this message has never been processed

        } catch (software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException e) {
            return false; // already processed — this is a duplicate
        }
    }

    public void markAsCompleted(String messageId) {
        dynamoDb.updateItem(UpdateItemRequest.builder()
            .tableName(TABLE)
            .key(Map.of("messageId", AttributeValue.fromS(messageId)))
            .updateExpression("SET #s = :completed")
            .expressionAttributeNames(Map.of("#s", "status"))
            .expressionAttributeValues(Map.of(":completed", AttributeValue.fromS("COMPLETED")))
            .build());
    }
}

When to Use SQS and When Not To #

USE SQS WHEN:
  ✓ You're already in the AWS ecosystem and want a fully managed service
  ✓ You don't want the hassle of maintaining a broker — SQS is zero-ops
  ✓ Simple task queues without complex routing
  ✓ Highly variable volume — SQS auto-scales without configuration
  ✓ You need easy integration with Lambda, SNS, S3, and other AWS services
  ✓ Costs need to be controlled — pay per request, no idle cost
  ✓ You need FIFO with easily configurable exactly-once delivery

CONSIDER ALTERNATIVES WHEN:
  ✗ You need complex routing (exchange patterns) → RabbitMQ is more flexible
  ✗ Very high volume (millions of msg/s) with ultra-low latency → Kafka
  ✗ You need to replay processed messages → Kafka with log retention
  ✗ You're not on AWS and don't want vendor lock-in → RabbitMQ or Kafka
  ✗ You need streaming and real-time aggregation → Kafka Streams or Kinesis
  ✗ Messages larger than 256 KB → SQS doesn't support it, use S3 + SQS pointer

Summary #

  • The visibility timeout is SQS’s core concept — messages aren’t deleted when received, only made invisible. Delete messages explicitly with DeleteMessage after successful processing.
  • Standard Queues for high throughput with duplicate tolerance; FIFO Queues for strict ordering and exactly-once delivery — more expensive and more limited throughput.
  • Always use long polling (WaitTimeSeconds=20) to reduce empty responses, save API costs, and lower latency compared to short polling.
  • Batch sends and batch deletes save up to 10x the cost because SQS bills per API request, not per message. Maximum 10 messages per batch.
  • Dead Letter Queues are configured via a Redrive Policy at the queue level — messages automatically enter the DLQ after failing to process maxReceiveCount times.
  • Extend the visibility timeout (ChangeMessageVisibility) if processing takes longer than the initial timeout — this prevents messages from being redelivered to other consumers while still being processed.
  • Standard Queues are at-least-once — consumers must be idempotent. Use DynamoDB with conditional writes as the idempotency store in production.
  • SNS + SQS for the fan-out pattern — SNS distributes one message to many SQS queues in parallel, providing a capability SQS alone doesn’t have.
  • SQS fits best for the AWS ecosystem, zero-ops task queues, and highly variable volume — not for complex routing, replay, or real-time streaming.

← Previous: RabbitMQ   Next: Google Pub/Sub →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact