AWS field guide
Put backpressure between producers and workers
Let a durable buffer absorb uneven demand while worker concurrency protects downstream capacity.
Use this pattern when
Work can complete asynchronously and arrival rate may exceed processing capacity.
Reference architecture
Responsibilities and controls, not a deployment template.
Producer
Accepts work quickly
Amazon SQS
Durable pressure buffer
Concurrency limit
Protects dependencies
AWS Lambda
Idempotent worker
Downstream system
Finite capacity
Decisions that shape the pattern
- Choose standard or FIFO from ordering requirements.
- Size batches around downstream limits.
- Set visibility timeout beyond worst-case processing time.
Security boundaries
- Separate send and receive permissions.
- Encrypt sensitive messages.
- Avoid placing secrets or large payloads directly in messages.
Reliability posture
- Redrive poison messages to a DLQ.
- Make handlers idempotent.
- Alarm on queue age, not only queue depth.
Starter implementation
Start from deployable infrastructure
Review every permission, limit, Region, and cost assumption before production.
import { Duration, Stack, StackProps } from 'aws-cdk-lib';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import * as athena from 'aws-cdk-lib/aws-athena';
import * as bedrock from 'aws-cdk-lib/aws-bedrock';
import * as budgets from 'aws-cdk-lib/aws-budgets';
import * as cloudfront from 'aws-cdk-lib/aws-cloudfront';
import * as origins from 'aws-cdk-lib/aws-cloudfront-origins';
import * as cloudtrail from 'aws-cdk-lib/aws-cloudtrail';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as patterns from 'aws-cdk-lib/aws-ecs-patterns';
import * as events from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import * as glue from 'aws-cdk-lib/aws-glue';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as kms from 'aws-cdk-lib/aws-kms';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as sources from 'aws-cdk-lib/aws-lambda-event-sources';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as secretsmanager from 'aws-cdk-lib/aws-secretsmanager';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import { Construct } from 'constructs';
export class PatternStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const dlq = new sqs.Queue(this, 'DeadLetterQueue');
const queue = new sqs.Queue(this, 'WorkQueue', {
visibilityTimeout: Duration.seconds(90),
deadLetterQueue: { queue: dlq, maxReceiveCount: 5 },
});
const worker = new lambda.Function(this, 'Worker', {
runtime: lambda.Runtime.NODEJS_20_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('worker'),
reservedConcurrentExecutions: 10,
});
worker.addEventSource(new sources.SqsEventSource(queue, { batchSize: 10, reportBatchItemFailures: true }));
}
}
Before production
Adoption checklist
- 01Measure downstream capacity.
- 02Set a concurrency ceiling.
- 03Configure a DLQ and redrive policy.
- 04Test partial batch failures.
- 05Create an oldest-message alarm.
From the journal
Related field notes
Selected from service names and architecture signals used by this pattern.
AWS Health Version Catalog: Proactive Lifecycle Management?
AWS Health now offers a version catalog, aiming to shift software lifecycle management from reactive to proactive. But what's the real cost?
CloudWatch Omni: AI Observability, But What's the Catch?
AWS launches CloudWatch Omni for AI workloads. It promises trace, evaluate, and experiment, but the real limits are unstated.
CloudWatch Omni: AI Observability Arrives, With Caveats
CloudWatch gets an AI overlay for investigation, but the real-world impact hinges on agent adoption and data fidelity.
Was this playbook useful?
One click helps prioritize deeper examples and updates.