The problem with synchronous calls
You built an API that processes orders. It works perfectly in development. When a user clicks "Buy", the server saves the order to the database, charges the credit card, and sends a confirmation email. The whole flow takes 1.2 seconds.
Now you deploy to production. Traffic spikes. The email service slows down. Your API starts hanging for five seconds per request. Users click refresh. The database connection pool exhausts. The email service crashes, and suddenly your entire API returns 500 errors because one downstream dependency failed.
You need a buffer. You need a way to say "I received your request, I'll get to it" without making the user wait for every single step to finish. That is what Amazon SQS does. It sits between your services, absorbing bursts of traffic and letting workers process messages at their own pace.
SQS as a digital post office
SQS is a message queue. Think of it like a post office box. You drop a letter in the box and walk away. You don't wait for the mail carrier to deliver it. The carrier picks up the letter when they are ready and delivers it. If the carrier is busy, the letter waits safely in the box.
In Go, you interact with SQS using the AWS SDK for Go v2. The SDK handles the HTTP requests, JSON serialization, and credential signing. Your job is to load the configuration, create a client, and call the queue operations. The SDK is modular, so you only import the packages you need.
Sending your first message
Here is the simplest way to send a message. You load the default configuration, create an SQS client, and call SendMessage.
package main
import (
"context"
"log"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/sqs"
)
func main() {
// LoadDefaultConfig reads environment variables, shared config files, and EC2 metadata.
// It returns a config object that the SDK client uses for credentials and region.
cfg, err := config.LoadDefaultConfig(context.Background())
if err != nil {
log.Fatalf("unable to load SDK config: %v", err)
}
// NewFromConfig creates an SQS client bound to the loaded configuration.
// The client is immutable and safe for concurrent use by multiple goroutines.
client := sqs.NewFromConfig(cfg)
// SendMessage pushes a message to the queue.
// QueueUrl is the full ARN or URL of the target queue.
// MessageBody is the payload, limited to 256 KB per message.
result, err := client.SendMessage(context.Background(), &sqs.SendMessageInput{
QueueUrl: aws.String("https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue"),
MessageBody: aws.String("Hello from Go"),
})
if err != nil {
log.Fatalf("unable to send message: %v", err)
}
// MessageId is a unique identifier for the message in the queue.
// You can use it for tracking or debugging downstream processing.
log.Printf("Message sent: %s", *result.MessageId)
}
The context.Background() call provides a base context for the request. The SDK uses this context to handle timeouts and cancellation. If the context expires, the SDK aborts the HTTP request and returns an error. Always pass a context as the first argument to SDK methods. This is a hard convention in the AWS SDK for Go.
The aws.String helper wraps a plain string into a pointer. The SDK uses pointers for optional fields to distinguish between "not set" and "set to empty string". You don't need to define a variable and take its address; aws.String does that in one call.
How the SDK loads credentials
The config.LoadDefaultConfig function follows a specific chain to find credentials. It checks environment variables first, like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If those are missing, it looks for a shared config file at ~/.aws/config or ~/.aws/credentials. If you are running on an EC2 instance or ECS task, it queries the instance metadata service for temporary credentials.
This chain means you can write code that works locally with env vars and in production with IAM roles without changing a single line. The SDK handles the switching automatically.
If you run the code without credentials and without metadata access, the SDK returns an error like no EC2 IMDS role found, configure your credentials. The error message tells you exactly what is missing.
Receiving and processing messages
Sending is easy. Receiving is where the complexity lives. SQS doesn't push messages to your server. Your server must poll the queue. You ask "do you have messages?", and SQS replies with a batch of messages or an empty list.
You need a loop that polls, processes, and deletes messages. Deleting is crucial. SQS messages have a visibility timeout. When you receive a message, it becomes invisible to other workers for a set period, usually 30 seconds. If you don't delete the message before the timeout expires, it becomes visible again. Another worker picks it up, and you process the same message twice.
Here is a realistic worker loop.
package main
import (
"context"
"log"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/sqs"
)
func worker(ctx context.Context, client *sqs.Client, queueURL string) {
// WaitTimeSeconds enables long polling.
// The SDK holds the HTTP connection open for up to 20 seconds waiting for messages.
// This reduces empty responses and saves API costs compared to short polling.
input := &sqs.ReceiveMessageInput{
QueueUrl: aws.String(queueURL),
MaxNumberOfMessages: aws.Int32(10),
WaitTimeSeconds: aws.Int32(20),
}
for {
// Check context cancellation to allow graceful shutdown.
// If the parent context is cancelled, the loop exits cleanly.
if err := ctx.Err(); err != nil {
log.Printf("worker stopping: %v", err)
return
}
// ReceiveMessage fetches up to MaxNumberOfMessages.
// It returns immediately if messages are available, or waits up to WaitTimeSeconds.
result, err := client.ReceiveMessage(ctx, input)
if err != nil {
log.Printf("receive error: %v", err)
continue
}
// Messages can be nil if the queue is empty.
// Always check the length before iterating to avoid a nil pointer panic.
if len(result.Messages) == 0 {
continue
}
for _, msg := range result.Messages {
// ProcessMessage handles the business logic.
// It must complete before the visibility timeout expires.
if err := processMessage(ctx, *msg.Body); err != nil {
log.Printf("processing failed for %s: %v", *msg.MessageId, err)
// On failure, the message remains in the queue.
// It becomes visible again after the visibility timeout.
// SQS will redeliver it up to the queue's ReceiveCount limit.
continue
}
// DeleteMessage removes the message from the queue.
// This acts as an acknowledgment that processing succeeded.
_, err := client.DeleteMessage(ctx, &sqs.DeleteMessageInput{
QueueUrl: aws.String(queueURL),
ReceiptHandle: msg.ReceiptHandle,
})
if err != nil {
log.Printf("delete error for %s: %v", *msg.MessageId, err)
}
}
}
}
func processMessage(ctx context.Context, body string) error {
// Simulate work. In a real app, this might parse JSON and update a database.
// Ensure this function respects context cancellation if the work is long-running.
log.Printf("processing: %s", body)
return nil
}
The ReceiptHandle is a temporary token you get when you receive a message. You must pass this exact token to DeleteMessage. It expires along with the visibility timeout. If you try to delete a message after the timeout, the SDK returns message not found.
The loop checks ctx.Err() at the top. This allows you to stop the worker by cancelling the context. If you don't check the context, the goroutine will keep polling even after your application tries to shut down. Goroutine leaks happen when a goroutine waits on a channel or a network call that never completes. Always have a cancellation path.
Pitfalls and runtime traps
SQS is robust, but it has quirks that catch beginners.
Nil pointer panics on empty results
ReceiveMessage returns a struct where Messages is a slice. If the queue is empty, Messages is nil. If you write for _, msg := range result.Messages without checking the length, the code works. But if you try to access result.Messages[0] or call len(result.Messages) on a nil slice in certain contexts, you might hit a panic. The safe pattern is to check len(result.Messages) == 0 and continue.
Visibility timeout tuning
The visibility timeout is the grace period you have to process and delete a message. If your processing takes 45 seconds but the timeout is 30 seconds, the message reappears while you are still working on it. Another worker picks it up. You end up with duplicate processing.
Set the visibility timeout slightly higher than your 99th percentile processing time. If processing is highly variable, increase the timeout and use a dead-letter queue to catch messages that fail repeatedly.
Short polling burns money
If you set WaitTimeSeconds to 0 or omit it, the SDK uses short polling. The SDK sends a request, SQS checks the queue, and returns immediately. If the queue is empty, you get an empty response. You pay for the request, and you get nothing. You might send hundreds of empty requests per second.
Long polling holds the connection open. You only pay when a message arrives. It also reduces latency because you don't wait for the next round-trip. Always use long polling in production.
Message ordering on standard queues
Standard SQS queues guarantee at-least-once delivery, but not order. If you send message A then message B, the worker might receive B before A. If order matters, use a FIFO queue. FIFO queues require a MessageGroupId and limit throughput to 3000 messages per second. Standard queues scale much higher but sacrifice order.
Error handling with the SDK
The SDK returns standard Go errors. You can check for specific AWS errors using awserr.IsError or by checking the error code. If the queue URL is wrong, the error contains queue does not exist. If you exceed the rate limit, you get throttling. The error messages are plain text, not codes. Inline the error check and handle the specific case.
if err != nil {
// The compiler rejects this with an undefined-variable error if you forget to import the awserr package.
// Use the standard errors package or awserr to inspect the error code.
log.Printf("error: %v", err)
}
When to use SQS versus alternatives
Decoupling is powerful, but it adds complexity. You trade immediate feedback for resilience. Choose the right tool for the job.
Use SQS when you need to decouple services and handle bursts of traffic without dropping requests. Use SQS when downstream processing is slow or unreliable, and you want to protect your API from failures. Use SQS when you need to scale workers independently of the producer.
Use SNS when you need to fan out a single event to multiple subscribers simultaneously. SNS is a pub/sub service. You publish once, and SNS delivers to SQS queues, HTTP endpoints, and Lambda functions. Combine SNS and SQS for complex event routing.
Use direct HTTP calls when the downstream service is fast, reliable, and the caller needs an immediate response. If the user is waiting for a result and the operation takes milliseconds, a queue adds unnecessary latency.
Use a database queue when you need strong consistency and transactional guarantees. SQS is eventually consistent. If you need to atomically write to a database and enqueue a message, a database queue or a transactional outbox pattern is safer.
Where to go next
SQS handles the transport. Your application still needs to manage state, authentication, and data models.
- JWT authentication in Go
- How to Handle JSON Columns in PostgreSQL from Go
- Implement repository pattern in Go
SQS doesn't guarantee order on standard queues. Idempotency is your safety net. Build your workers to handle duplicates gracefully, and the queue will scale with you.