Challenges in Serverless Computing

Lambda is cheap… until it runs 20 million times per hour. We analyzed 400+ serverless workloads. 60% had NO cost guardrails. Serverless is supposed to be cheap. But in reality it's the fastest way to burn cash invisibly (if you don’t set guardrails from day one.) Here’s what I saw: 🚩 No concurrency limits. One bad loop → 3,000 concurrent Lambdas → instant spike → ops team wakes up confused. 🚩 Event storms with zero throttling. SNS, SQS, DynamoDB Streams triggering like crazy—no retry logic, no backpressure. 🚩 No budget alerts. No anomaly detection. People trust the cloud bill after the fire. Not before. 🚩 Over-reliance on default memory configs. 2GB Lambdas running a 150MB function, 1000x a minute. Multiply that by 30 regions. 🚩 Huge cold start waste. Dev teams chasing speed. But paying for idle spin-ups in traffic patterns they never profiled. I think the worst part is that most teams had no visibility into which invocation patterns were driving 80% of cost. They just assumed “It’s serverless, it must be optimized.” Here’s the framework we now apply to every client before scaling serverless to prod. 𝗧𝗵𝗲 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗖𝗼𝘀𝘁 𝗚𝘂𝗮𝗿𝗱𝗿𝗮𝗶𝗹𝘀 𝗖𝗵𝗲𝗰𝗸𝗹𝗶𝘀𝘁: ✅ Set concurrency caps for all Lambdas ✅ Define sane retry/backoff policies on event sources ✅ Profile and right-size memory + duration (not guess) ✅ Use real-time cost anomaly detection (per function) ✅ Tag all workloads with ownership + purpose (for chargeback clarity) Serverless doesn’t need to be expensive. But it will be (if you treat it like a free lunch.)

Serverless has one fundamental problem: State. Platforms like Cloud Run and Lambda are fantastic at compute. But they don't manage execution state. Here's how companies waste huge $$$: 𝐒𝐭𝐚𝐫𝐭 𝐬𝐢𝐦𝐩𝐥𝐞 Lambdas, Cloud Run, etc. Stateless handlers. HTTP in, HTTP out. Life is simple. 𝐀𝐝𝐝 𝐪𝐮𝐞𝐮𝐞𝐬 "Some requests are getting lost." You add a queue (e.g., SQS) to buffer work. Now you have at-least-once delivery and basic flow control. But you also now have two systems to reason about: compute and messaging. -> You’ve introduced distributed coordination. 𝐀𝐝𝐝 𝐫𝐞𝐭𝐫𝐲 𝐥𝐨𝐠𝐢𝐜 "Sometimes the container disappears mid-request." Cold starts, OOM kills, rolling deployments, provider preemption. So you add retries at the application level, usually retrying the whole request. -> You are now duplicating work and entering the space of consistency anomalies. 𝐀𝐝𝐝 𝐚 𝐃𝐋𝐐 "Some retries never succeed." Flaky dependencies, bad inputs, partial failures. You can't drop the request, so you route it to a Dead Letter Queue and build tooling to monitor and reprocess it. -> More custom infrastructure, more operational surface area. 𝐀𝐝𝐝 𝐢𝐝𝐞𝐦𝐩𝐨𝐭𝐞𝐧𝐜𝐲 𝐤𝐞𝐲𝐬 "Some customers were charged twice." A queue lease expired. A function timed out. A client retried. So you implement idempotency keys, deduplication tables, and cleanup jobs. -> State now leaks into every boundary of the system. What started as "just put SQS in front" becomes weeks of careful distributed systems engineering. You realize you are re-implementing parts of reliability theory in application code. 𝐀𝐝𝐝 𝐚𝐧 𝐨𝐫𝐜𝐡𝐞𝐬𝐭𝐫𝐚𝐭𝐨𝐫 "This is getting hard to reason about." You introduce Step Functions, Airflow, Temporal, or a custom DAG engine. Now you have multiple sources of truth: the application, the workflow engine, and a very upset Head of Finances. Keeping them consistent becomes a permanent concern. The result? You didn't build "serverless", but: a message broker, a retry framework, a dedup system, a saga engine, a workflow scheduler, a recovery pipeline. All glued together :) Durable execution makes your program itself the state machine: every step is checkpointed transactionally in the database, so crashes, restarts, and redeploys resume execution exactly where it left off. Instead of rebuilding reliability with queues, retries, and deduplication, the runtime guarantees exactly-once progress and deterministic replay by construction. Don't take my word for it and try out this guide on deploying a DBOS app on Google Cloudrun.

Serverless doesn't shrink your attack surface. It relocates it. 𝘓𝘦𝘢𝘳𝘯𝘪𝘯𝘨 𝘚𝘦𝘳𝘷𝘦𝘳𝘭𝘦𝘴𝘴 𝘚𝘦𝘤𝘶𝘳𝘪𝘵𝘺 by Joshua Arvin Lat makes the core argument clearly: teams that move fast with serverless architectures often move fast past the security model those architectures actually require. 𝗧𝗵𝗲 𝗲𝘅𝗽𝗼𝘀𝘂𝗿𝗲 𝗹𝗮𝘆𝗲𝗿 When critical systems and sensitive data migrate to serverless infrastructure without a corresponding shift in security posture, the gap becomes an incident waiting 'on a timeline'. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝘄𝗼𝗿𝘁𝗵 𝗮𝗽𝗽𝗹𝘆𝗶𝗻𝗴 🔹 Audit your serverless functions for privilege escalation paths before attackers do. IAM misconfiguration in serverless is not a configuration smell; it is an exploitable vector.  🔹 Run offensive and defensive exercises against your own infrastructure. Understanding how attacks unfold against vulnerable serverless apps is the gap between theoretical compliance and operational security.  🔹 Track regression on function permissions and event trigger scope after every deployment. Silent scope creep in serverless is one of the least monitored failure modes in cloud security. 𝗥𝗶𝘀𝗸 𝘁𝗼 𝘄𝗮𝘁𝗰𝗵 🔹 Teams with strong cloud fluency often underestimate serverless-specific attack surfaces precisely because the infrastructure feels abstracted. That confidence is the vulnerability. 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗴𝗮𝘁𝗲 𝗳𝗼𝗿 𝘀𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗿𝗲𝗮𝗹𝗶𝘁𝘆 𝗔𝘂𝗱𝗶𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆: Can you reconstruct exactly what a function accessed, when, and under what permissions?  ⚙️ 𝗣𝗿𝗶𝘃𝗶𝗹𝗲𝗴𝗲 𝘀𝗰𝗼𝗽𝗲: Are function roles scoped to minimum required access, and is that verified post-deploy?  𝗗𝗲𝗳𝗲𝗻𝘀𝗲 𝘃𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻: Have you tested your defenses using the same offensive techniques an attacker would use? 🔍 𝗖𝗿𝗼𝘀𝘀-𝗰𝗹𝗼𝘂𝗱 𝗰𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝘆: Does your security posture hold equally across AWS, Azure, and GCP, or are there platform-specific gaps?  𝗜𝗻𝗰𝗶𝗱𝗲𝗻𝘁 𝗿𝗲𝗮𝗱𝗶𝗻𝗲𝘀𝘀: If a serverless function is compromised tonight, do you have detection and containment playbooks ready to execute? ⚠️ If your team runs serverless workloads on two or more cloud providers, where are the security assumptions you've carried over from one platform that don't actually apply to another? #ServerlessSecurity #CloudSecurity #AWSecurity #AzureSecurity #GoogleCloudSecurity #PenetrationTesting #CloudArchitecture #PrivilegeEscalation #SecurityEngineering #ResponsibleAI

A client spent over $5,000 a month on a single Lambda function - and most of it was wasted. Here's what happened. The function handled a CPU-heavy task, so the team cranked up the memory to 10GB, hoping to speed things up. The team knew that, with Lambda, more memory = more CPU. What they didn't realise was that the CPU scaling was not vertical - you unlock another vCPU for every 1.8GB of memory. At 10GB, the function had 6 vCPUs. But this was a Node.js function... Since Node.js is single-threaded, it'd only use one CPU unless you spawn child processes. That meant most of the CPU power they paid for were not utilized by their code. The function didn’t need that much memory either, so most of their spending was wasted. It was an honest mistake, but it had a big impact. The fix was simple, right-size the memory setting and cost went back to an acceptable level. But it was a bitter lesson for the team... I hope sharing their story saves you from making the same mistake! Lambda gives you a single lever to control performance and cost. I like that simplicity, but it also makes it easy to get things wrong by a huge margin! And sometimes, you just need extra CPU without all the extra memory as well. And before you blame serverless for high costs, remember that container workloads can run into the same issue. A misconfigured instance size or auto-scaling setting would lead to just as much waste. At least in this case, the cost was tied to actual usage - the function was fairly busy - and not idle servers doing nothing. I've worked in places that spent 10s of thousands dollars a month on EC2 clusters running at 5% CPU! (but no one bats an eyelid..) The good news is that, there are tools to help with this kind of problem. You can use the Lambda Power Tuning tool, although it requires a bit of effort to fine-tune each function. There's also the AWS Compute Optimizer - it just needs to be enabled, and it will give you recommendations over time. The main downside is that it only suggests changes when it's very confident in them, so it might take a while. And if you’re running a Lambdalith (e.g. a monolithic function that handles all the routes in an API), then these tools won’t be very effective. A function doing multiple things with very different performance needs is much harder to optimize. That's something to keep in mind if you decide to go down this path.

My thoughts around #serverless are ever evolving. After 8 years of writing code that is hosted in serverless compute and integrating with various serverless components, I've boiled things down to this. 1) I don't love Lambda hosting APIs if I have API complexity including east/west traffic 2) Serverless "solutions" are too easy to get started and too hard to manage at scale. Not traffic, at code changes. I tend to not use them anymore 3) Serverless "components" are the sweet spot. I wish there were more fundamental serverless building blocks. Queues, Storage, Streams, Databases, Caches. The things we build solutions on top of. Best of breed would be amazing 4) People still equate serverless to functions far too often. I hope #3's growth eases that 5) Observability is still too hard. But it's a lot better Bottom line is this. I think we pushed serverless always and even serverless first too far. I 100% know (not believe) that serverless fits in pretty much any scenario, but I think we do ourselves no favors as developers by not reaching for EC2 or Containers when it's appropriate. Lambda is too often a starting point even when it would be simpler to run a system process hosting a web framework. We convince ourselves that the pain of setting up and running EC2 vs Lambda in IaC is compared to the pain in managing a Lambda sprawl when an Express server would have been just fine. We modify code way more than we setup IaC And lastly, something like SQS and DynamoDB are blocks that remain regardless of the compute. You can even mix and match. Take the EC2 web server and pair it with DynamoDB. Serverless can be the backbone and key part, but the compute, well it doesn't always have to be Lambda. Love to hear your thoughts!

Azure Functions now supports serverless agentic AI runtimes. Every team building on Azure AI Foundry is asking whether they need this. Most of them are asking the wrong question. The question is not whether serverless is simpler. It is whether the trade-offs match the workload. When I evaluate the serverless-versus-container agent runtime decision, three fault lines matter: 1. State management under the serverless execution model. Azure Functions agents are stateless between invocations by design. If your agent needs persistent thread context between calls — multi-turn reasoning chains, accumulated tool call history, long-horizon task coordination — you are externalizing that state to Cosmos DB or Azure Storage on every round trip. That is a viable pattern, but it is not free — latency accumulates, and your agent's blast radius for state corruption expands every time you add an external dependency. AKS-hosted agents keep state in memory across a session — the trade-off is operational complexity, not architectural compromise. 2. Cold start behavior in latency-sensitive agent workflows. Consumption plan Azure Functions cold start is measured in seconds, not milliseconds. For human-in-the-loop agent interactions where a user is waiting for a response, that latency is visible and unacceptable — move to Premium plan or Container Apps with minimum replica counts. For background agentic pipelines where no human is watching the clock, consumption plan is operationally appropriate and cost-efficient. The workload decides the plan, not the preference. 3. Governance surface for multi-agent topologies. In a multi-agent setup, each Azure Functions agent has its own managed identity and its own VNet integration scope — or it doesn't, and you have a shared-identity problem. Container Apps sandboxes give you a more natural boundary for grouping agents that share an access scope. Design the identity and network topology of your agent graph before you pick the runtime, not after the agents are already deployed. Serverless agent runtimes genuinely lower the operational barrier for getting agentic AI into production. But lower barrier is not the same as lower risk. Match the runtime to the workload characteristics.

Just published: "Serverless MCP: Stateless Execution for Enterprise AI Tools" Most teams build MCP servers with persistent connections and session state. For enterprise workflows—where tools orchestrate across Salesforce, Stripe, and other systems of record—there's a better way. What serverless architecture eliminates: - Server affinity and connection limits - Session state synchronization - Cache staleness and stale reads - Complex failure recovery (no connection state to reconstruct) What stateless execution forces: - Backend systems as source of truth (your CRM, ERP, payments—not cached copies) - Idempotent operations by design (no duplicate charges, no duplicate records) - Self-contained requests (any worker handles any call) - Cleaner separation between protocol and execution layers The article explains: - The three architectural choices that define serverless MCP - When stateless execution matters (and when it doesn't) - Server architecture comparison (side-by-side) - How to decide which pattern fits your system Includes a complete open-source reference implementation (Dewy Resort sample app) demonstrating the patterns. Read it here: https://lnkd.in/gTKSDg6d Understanding the tradeoffs matters more than following trends.

Thrilled that mine and Qian Li's paper was just published in the Communications of the ACM! The big challenge we've been working on since grad school is how to make serverless computing work well for stateful applications. This is tricky because it's hard to make long-lived stateful workflows reliable in ephemeral serverless execution environments. The solution, which we talk about in this paper and are building at DBOS, Inc., is using database transactions to make programs durable. The key idea is to store the execution state of a serverless workflow in a database. That way, if the workflow is interrupted, it can automatically recover from where it left off by looking up its execution state in the database. If you know that stateful programs are durable, it becomes a lot easier to make them serverless. You can deploy them to ephemeral executors and leverage their durability to autoscale them and move them between executors as needed. That way, you can build incredibly reliable stateful programs that also benefit from the scalability and convenience of serverless.

⚡ 𝗦𝘁𝗼𝗽 𝗼𝘃𝗲𝗿𝘀𝗽𝗲𝗻𝗱𝗶𝗻𝗴 𝗼𝗻 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 — 𝗮𝗻𝗱 𝘂𝗻𝗹𝗼𝗰𝗸 𝗵𝗶𝗴𝗵 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗯𝗿𝗲𝗮𝗸𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝗻𝗸! Last week, I had the privilege of running a 𝗵𝗮𝗻𝗱𝘀‐𝗼𝗻 AWS workshop for one of our strategic customers focused on 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗮𝗻𝗱 𝗰𝗼𝘀𝘁 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗔𝗪𝗦 𝗟𝗮𝗺𝗯𝗱𝗮. 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗶𝘀 𝗮𝗺𝗮𝘇𝗶𝗻𝗴 — it scales effortlessly and eliminates infrastructure management. But without the right optimization approaches, performance costs can quietly creep up. That’s exactly what we tackled together. Here’s what we covered and why it matters: 🔍 𝗞𝗲𝘆 𝗪𝗼𝗿𝗸𝘀𝗵𝗼𝗽 𝗛𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 • 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 𝗟𝗮𝗺𝗯𝗱𝗮 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝘁𝗶𝗲𝗿𝘀 - how memory & timeout settings impact throughput and latency.  • 𝗖𝗼𝘀𝘁 𝘃𝘀. 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝘁𝗿𝗮𝗱𝗲‐𝗼𝗳𝗳𝘀 - finding that sweet spot where faster doesn’t mean pricier.  • 𝗥𝗲𝗮𝗹‐𝘄𝗼𝗿𝗹𝗱 𝘁𝘂𝗻𝗶𝗻𝗴 𝘁𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲𝘀 - including provisioned concurrency strategies and cold start reduction.  • 𝗠𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴 & 𝗼𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 - using CloudWatch and X‑Ray to pinpoint where optimization delivers the best ROI.  • 𝗛𝗮𝗻𝗱𝘀‐𝗼𝗻 𝗹𝗮𝗯𝘀 - customers experimented with real Lambda functions, measured results, and saw cost drops in real time. Each participant walked away with actionable insights - not just theory - 𝘁𝗼 𝗿𝗲𝗱𝘂𝗰𝗲 𝘁𝗵𝗲𝗶𝗿 𝗟𝗮𝗺𝗯𝗱𝗮 𝗯𝗶𝗹𝗹𝘀 𝘄𝗵𝗶𝗹𝗲 𝗸𝗲𝗲𝗽𝗶𝗻𝗴 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗵𝗶𝗴𝗵. That’s practical cloud engineering in action. 🤵 𝗪𝗵𝗮𝘁 𝗢𝘂𝗿 𝗖𝘂𝘀𝘁𝗼𝗺𝗲𝗿 𝗔𝗰𝗵𝗶𝗲𝘃𝗲𝗱 Before the workshop:  • Lambda costs were unpredictable  • Functions experienced occasional latency spikes  • No formal performance benchmarking After the session:  ✔️ Clear performance baselines established  ✔️ Cost‑effective configurations implemented  ✔️ Team empowered with tooling and metrics to iterate independently The difference was noticeable - not just on the dashboard, but in team confidence. 📶 𝗪𝗵𝘆 𝗧𝗵𝗶𝘀 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 𝗙𝗼𝗿 𝗬𝗼𝘂 Serverless is a powerful paradigm - but only when optimized with intent. In every environment I see:  • Over‑allocated memory  • Lack of observability  • Minimal performance profiling Even small tweaks can result in 𝗺𝗲𝗮𝘀𝘂𝗿𝗮𝗯𝗹𝗲 𝗰𝗼𝘀𝘁 𝘀𝗮𝘃𝗶𝗻𝗴𝘀 𝗮𝗻𝗱 𝘀𝗺𝗼𝗼𝘁𝗵𝗲𝗿 𝗲𝗻𝗱‐𝘂𝘀𝗲𝗿 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲𝘀. 🏗️ 𝗪𝗼𝗿𝗸𝘀𝗵𝗼𝗽 𝗹𝗶𝗻𝗸 : https://lnkd.in/exZQmbhw 🥡 𝗛𝗲𝗿𝗲’𝘀 𝗮 𝘀𝗶𝗺𝗽𝗹𝗲 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆:  If you’re using AWS Lambda in production, treat 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗮𝗻𝗱 𝗰𝗼𝘀𝘁 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 as a first‑class concern - not an afterthought. What’s the biggest cost challenge you’re seeing with serverless workloads right now? #𝗔𝗪𝗦 #𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 #𝗟𝗮𝗺𝗯𝗱𝗮 #𝗖𝗹𝗼𝘂𝗱𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 #𝗖𝗼𝘀𝘁𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆

Where does the industry stand on LLM cold starts? Most “serverless” GPU inference isn’t actually serverless. It’s prewarmed instances with a serverless wrapper. Here’s where things stand today: The baseline (no optimization) Loading a 32B model from scratch: 60–120 seconds. Unusable for any interactive application. Current industry approach: snapshots Container + CPU memory snapshots bring it down to 35-45 seconds Better. Still not serverless. And still one snapshot = one instance. No horizontal scaling. The benchmarking problem nobody talks about: You’ve seen the claims. 40x faster cold starts. 10x faster. Faster than what exactly? Most of these numbers compare against an arbitrary baseline — often just container spin-up time, not true request → first token latency. Some providers measure “model loaded” and call it a cold start. That’s not a cold start. That’s half the problem. The only concrete true cold start number we could find from a major cloud provider: Google Cloud measured TTFT for a Gemma 4B model at 19 seconds. That’s a 4B model. Not 32B. The problem nobody talks about: Even with GPU snapshots, framework initialization . CUDA graph capture, kernel compilation, memory pool setup , adds seconds regardless of how fast your hardware is. On vLLM alone: 65 seconds of initialization for a 32B model. Even if weights are already in memory. What true serverless actually requires: Cold start = request → first token. Nothing less. For interactive use: under 1–2 seconds. The industry is optimizing around the edges. The core problem is still unsolved. Next: how we approached it differently at InferX .