A decision framework for choosing AWS Lambda vs ECS/Fargate for an event-driven pipeline

Abstract

Many cloud applications use message queues to process background tasks without blocking the main application. AWS Lambda and AWS Fargate are two common AWS services for this type of workload. Lambda uses a serverless function model, while Fargate uses a managed container-based model. Both services can process messages from Amazon SQS. However, they differ in latency behaviour, cost structure, runtime control, debugging, observability, and operational effort. This thesis compares the performance of AWS Lambda and AWS Fargate for message-driven workloads based on controlled experiments, a cost model and supporting interview responses. Two comparable prototype systems were developed: one using SQS, Lambda, and DynamoDB, and another using SQS, Fargate, and DynamoDB. The systems were tested using steady, bursty, and heavier burst workload profiles. The main experiment yielded 2,040 result records across 30 runs in the AWS eu-north-1 region. All the workloads were successfully completed on both platforms without any failures. In the controlled environment, Lambda performed better than Fargate in terms of latency, particularly for bursty and heavier burst workload profiles. The cost model revealed that Lambda was more cost-effective for short, irregular workloads, while Fargate was more cost-effective for steadier, longer-running workloads or workloads that better leveraged the running container task. Responses from the interviews supported the practical analysis of these findings, which highlighted runtime control, debugging, observability, cost predictability, and team experience as important decision factors. The thesis concludes with a practical decision-making framework for choosing between Lambda and Fargate. The framework recommends that the decision should not rely only on latency or cost. In addition, teams should consider workload patterns, processing time, runtime control, observability, debugging needs, cost predictability, and team capacity.