Category Archives: DevOps

Cloud,Architecture,DevOps,Lambda,Serverless

Lambda's 90-Minute Timeout — Lambda Is Slowly Becoming EC2

Leave a comment

I have a favorite AWS service, and it’s not a close race. It’s Lambda.

I’ve said this out loud in enough architecture reviews that people roll their eyes at me. But I mean it, and the reason is embarrassingly simple: Lambda is where my ideas go to become real. When I have a half formed thought at 11pm — “what if I wired this webhook to that API and dropped the result in DynamoDB?” — Lambda is the surface where that thought turns into running code before I lose the plot. No instance to launch. No AMI to pick. No security group to reason about. No patching schedule looming in the back of my head. I write the handler, I deploy, it runs. If it’s a bad idea, I delete it and pay nothing for the privilege of having been wrong.

That frictionlessness is worth more than it sounds, and I say that as someone with scar tissue. I’ve been running EC2 since 2009, back in the pre-VPC days when “the cloud” meant EC2-Classic, elastic IPs you had to babysit, and a security model that felt like leaving your front door propped open with a brick. Standing up a prototype in 2009 meant provisioning an instance, SSHing in, installing your runtime, configuring a service, and then — the part everyone forgets — owning that box forever. Patching it. Watching its disk fill up. Wondering if it was still running three months later, quietly costing you money. Lambda erased all of that. For POCs and prototypes, it is the single best tool I have ever used, because it lets me test options fast and throw the losers away without ceremony.

So this post is a little bittersweet. Because the thing I love about Lambda — that it hides the infrastructure — is exactly the thing that’s slowly eroding.

The news: 90 minutes on Managed Instances

On September 9, 2026, AWS announced that Lambda Managed Instances now support a 90-minute function timeout — six times the classic 15-minute ceiling that has defined Lambda’s mental model for years.

A couple of important qualifiers, because the headline oversimplifies. Lambda Managed Instances are a newer execution mode where AWS provisions and manages longer lived compute behind your function, letting you choose capacity providers — think C9G (compute optimized) versus M9G (general purpose) — rather than only tuning a memory slider. The 90-minute timeout applies to asynchronous invocations and event-source-mapping (ESM) flows — queues, streams, event driven fan in. It does not apply to synchronous request/response invocations, which is the right call: no sane API gateway should hold a connection open for an hour and a half. Pair this with durable functions and the existing 1-year ceiling on async event retention, and a picture emerges. Lambda is quietly absorbing workloads that used to be EC2’s birthright, one feature at a time. The AWS Compute Blog deep dive lays out the mechanics if you want the full spec.

When 90 minutes actually matters

To be fair — and I want to be fair, because I love this service — there are real workloads that hit the 15-minute wall hard and hurt:

  • Large scale data processing. ETL jobs that chew through a few million rows, backfills, nightly aggregations. The kind of thing you’d previously chop into artificial subbatches purely to fit the timeout.
  • Media transcoding. Encoding a long video is not something you can meaningfully checkpoint at minute 14 and resume cleanly.
  • Long running AI inference. Batch inference, embedding generation over a large corpus, or agentic workflows that make many sequential model calls. These routinely blow past 15 minutes and don’t decompose neatly.

For these, 90 minutes isn’t a luxury — it’s the difference between “one clean function” and “an elaborate orchestration you built only to dodge a limit.”

The thesis: Lambda is becoming EC2

Here’s where the wry part lives. Trace the feature creep with me:

  • Timeouts went from 5 minutes, to 15, and now to 90 on Managed Instances.
  • You now pick a capacity provider — C9G vs M9G — which is, let’s be honest, choosing an instance family with a friendlier name.
  • Durable functions give you long lived, resumable state.
  • Async event retention stretches out to a full year.

Squint at that list. Longer running compute, instance family selection, durable state, extended lifecycles. That’s not a list of serverless features. That’s a list of EC2 features wearing a serverless hoodie. The Screaming in the Cloud crowd put it perfectly: Lambda slowly becomes EC2, one feature at a time.

At what point does “serverless” stop being serverless? I don’t think there’s a clean line — it’s a gradient, and we’re sliding down it. And I feel this one personally, because the entire reason Lambda earned my affection is that it hid these knobs from me. Now the knobs are growing back. It’s like watching a friend who moved to the city for the simplicity slowly acquire a lawn, a garage, and opinions about mulch.

The architectural rethink

If you’re a team that’s been fanning long work across Step Functions purely to escape the 15-minute limit, this genuinely warrants a rethink. Some of those state machines exist not because your problem is a workflow, but because the timeout forced you to pretend it was.

So: could you collapse a 40-minute, artificially chunked Step Functions saga into a single 90-minute function? Sometimes, yes. But weigh the tradeoffs honestly:

  • Cost. Lambda bills per millisecond of allocated memory. A single function grinding for 80 minutes at high memory can cost more than a right sized EC2 or Fargate task doing the same work. Scale-to-zero is a gift; long steady state compute is where it stops being one.
  • Observability. A Step Functions graph shows you exactly which step failed. A monolithic 90-minute function is a black box you have to instrument yourself.
  • Retry semantics. If a function fails at minute 85, you rerun the whole thing. Step Functions lets you retry the one step that broke. That granularity is not free to give up.
  • Cold starts. Larger, longer functions with heavier dependencies mean heavier cold starts. For batch work this rarely matters, but know it’s there.

My rule of thumb: if your long job is genuinely one atomic thing (transcode this file, process this dataset), a single 90-minute function is now the cleaner design. If it’s several distinct steps with independent failure modes, keep the orchestrator. Don’t collapse a workflow just because you finally can.

The verdict

Here’s my opinionated take, and I won’t fence-sit: the 90-minute timeout is a genuinely good addition, and it does not change where Lambda actually wins.

Lambda still beats EC2 decisively on the things that made me love it — scale-to-zero, zero patching, per-millisecond billing, and being the best prototyping surface on the planet. Nothing about a longer timeout erodes that. If anything, it removes one of the last “well, actually, you’ll hit the timeout” objections I used to hear in reviews.

But let’s be clear eyed about the trajectory. Lambda is accreting EC2’s shape, and every knob it grows is a small tax on the simplicity that was its whole point. That’s not a criticism so much as a maturation — the service is meeting real workloads where they are. I just hope, selfishly, that the frictionless idea to code path I fell for in the first place stays a first class citizen and doesn’t get buried under capacity providers and instance families.

For now, it’s still the first place my 11pm ideas go. Long may that last.

Where do you draw the serverless line? If you’ve collapsed a Step Functions saga into a single long function — or refused to — I’d love to hear how it went.

I have a favorite AWS service, and it’s not a close race. It’s Lambda.

I’ve said this out loud in enough architecture reviews that people roll their eyes at me. But I mean it, and the reason is embarrassingly simple: Lambda is where my ideas go to become real. When...

DevOps,Cloud Architecture

The Silicon Under Your Self-Hosted LLMs — Graviton5, R9g, and the Real TCO of Open Weights

Last week we argued the models are ready. This week: the hardware just caught up too.

In Open Weight AI Models vs. Frontier APIs — The 2026 Cost Performance Tipping Point, we made the case that self-hosting open-weight models had finally crossed the economic line for a large slice of production workloads. But that post treated “self-hosted” as an abstraction — GPU rental math, per-token pricing curves, fine-tuning economics. It never asked the more grounded question: what silicon do you actually run these things on, and what does that silicon cost you per token?

On August 31, 2026, AWS made Amazon EC2 R9g and R9gd instances generally available, powered by Graviton5. These are memory-optimized Arm instances, and for a specific and growing class of LLM inference, they change the substrate calculus. This post goes one layer below last week’s argument — down to the memory controllers, the L3 cache, and the watts.

Inference is a memory bandwidth problem

Here is the counterintuitive thing that trips up teams sizing LLM infrastructure: for autoregressive token generation, you are almost never compute bound. Generating one token requires streaming the entire set of active model weights from memory through the compute units, then doing it again for the next token. At batch size one, arithmetic intensity is brutally low. The GPU or CPU spends most of its cycles waiting on memory.

That means the single most important spec for inference throughput is not FLOPS. It is memory bandwidth. This is why the Graviton5 memory subsystem matters more than the headline “25% better compute per vCPU” figure.

Graviton5 moves to DDR5-8800 MT/s memory, up from 5600 MT/s in Graviton4 — AWS calls it the fastest memory available in the cloud, and for a bandwidth-bound workload that is the number that moves tokens per second. Pair that with a 5x larger L3 cache, and more of a quantized model’s hot working set — attention KV cache, frequently touched layers — stays close to the cores instead of round-tripping to DRAM. The 25% per-vCPU compute uplift is real and welcome, but for inference it is the supporting act. Bandwidth and cache locality are the headliner.

Where CPU inference is “good enough” — and where it isn’t

Let me be precise, because Arm CPU inference gets oversold in both directions. R9g is not a GPU replacement. It is a GPU avoider for the right workloads.

CPU inference on R9g-class hardware is genuinely good enough when:

  • The model is small and quantized. A 7B–13B model at 4-bit (GGUF Q4, AWQ, or similar) has a weight footprint of roughly 4–8 GiB. That streams comfortably from DDR5-8800, and the whole model fits in memory many times over.
  • You are serving batch or async workloads. Document enrichment, classification pipelines, overnight summarization, embedding generation — anything where p99 latency is measured in seconds, not milliseconds, and where you care about cost per million tokens more than time to first token.
  • Your traffic is spiky or cost sensitive. CPU instances scale horizontally and cleanly on Spot, and you are not paying for an idle accelerator between bursts.

GPUs remain necessary when you need low single-request latency at interactive chat speeds, when you are serving large dense models (70B+ at high precision), or when you need very high concurrent batch throughput per node. The honest architecture is a split fleet: GPUs for the interactive tier, R9g for the batch and cost-sensitive tier. Last week’s post argued most tasks fit in the 7B–70B range; a meaningful fraction of those tasks also fit on a CPU, and that fraction is where R9g earns its place.

The real TCO at the hardware layer

R9g scales to 192 vCPU and 1,536 GiB of memory across 11 sizes, from r9g.medium up to r9g.metal-48xl. That memory ceiling is the point. A single r9g.48xlarge with 1,536 GiB holds a small library of quantized models resident in RAM simultaneously — no swapping, no cold-load penalty on model switch. For a multi-tenant inference gateway routing across a dozen fine-tuned variants, that is a real operational simplification.

A rough sizing intuition for capacity planning:

tokens/sec (batch=1)  ~=  memory_bandwidth / model_weight_bytes

# 4-bit 13B model, ~7 GiB active weights
# Graviton5 sustained BW is materially higher than Graviton4's,
# so per-node token throughput rises without adding a GPU line item.

The other half of TCO is energy. AWS describes Graviton5 as the most energy efficient processor it has ever built. For inference fleets that run continuously, the watts-per-token line eventually dominates the bill — and it is the line that most FinOps dashboards under-count because it hides inside the instance price. Fewer watts per token at the same throughput is a compounding advantage across a 24/7 fleet.

For the storage-hungry variants, r9gd adds local NVMe SSD — useful for staging model weights, vector index shards, or KV-cache spillover without hammering EBS. And on the largest sizes, R9g doubles network and EBS bandwidth versus R8g (up to 100 Gbps network and 72 Gbps EBS on the 48xlarge), with up to 3x higher packet-processing performance — which matters when your inference node is also fronting a high-QPS retrieval layer. Instance Bandwidth Configuration (IBC) lets you shift the EBS-versus-VPC allocation by 25% to match whichever side your pipeline leans on.

Underneath it all, R9g runs on the AWS Nitro System with the Nitro Isolation Engine — the first formally verified cloud hypervisor, with isolation guarantees established by mathematical proof rather than test coverage. For teams running customer data through self-hosted models, that isolation assurance is a compliance story you can actually put in writing.

Migration is a non-event

The best thing about R9g for anyone already on Arm: R8g to R9g is a drop-in. For most applications there are no code changes — you select the equivalent R9g size and your workload runs faster. It supports Amazon Linux 2023 and 2, Ubuntu 22.04+, RHEL 8.4+, SLES 15 SP3+, and Debian 12+. Containerized inference on EKS, ECS, or vanilla Kubernetes works as-is, and multi-arch Arm64 images run unchanged. Track the delta with the Graviton Savings Dashboard so the savings show up as a number your finance team believes.

R9g and R9gd launched in US East (N. Virginia, Ohio), US West (Oregon), and Europe (Frankfurt), available across Savings Plans, On-Demand, Spot, Dedicated Instances, and Dedicated Hosts.

Practical takeaways

  1. Size for bandwidth, not FLOPS. For token generation, memory bandwidth and cache locality set your throughput ceiling. Graviton5’s DDR5-8800 and 5x L3 cache target exactly that bottleneck.
  2. Run a split fleet. GPUs for the interactive tier; R9g for batch, async, and cost-sensitive inference on quantized 7B–13B models.
  3. Consolidate models in memory. Use the 1,536 GiB ceiling on large R9g sizes to keep many quantized variants resident and eliminate cold-load latency.
  4. Count the watts. Energy per token compounds on a 24/7 fleet — bake it into your TCO model, not just the sticker instance price.
  5. Migrate first, optimize later. If you are on R8g, move to R9g as a no-code-change swap and measure the delta on the Graviton Savings Dashboard before you re-architect anything.

The models were ready last week. The substrate is ready this week. The interesting question for the rest of 2026 is no longer whether to self-host open weights, but how much of your inference fleet quietly moves off accelerators and onto CPUs you were already paying for. Where does your split land?

Last week we argued the models are ready. This week: the hardware just caught up too.

In Open Weight AI Models vs. Frontier APIs — The 2026 Cost Performance Tipping Point, we made the case that self-hosting open-weight models had finally crossed the economic line for a large slice...

DevOps,Serverless,Lambda,GenAI

Killing the Cold-Start Tax on Serverless AI — Lambda SnapStart for Container Images

Leave a comment

Lambda has been my favorite service for years. When I need to stand up a POC or a pilot, nothing beats it — I can deploy code in minutes, wire it to an event, and pay only for what actually runs. So it always stung a little that the standard architecture review for a real time inference endpoint ended the same way: “Lambda would be perfect for this… except cold starts.” Bursty traffic, event driven triggers, pay per use economics — serverless was the obvious fit on paper, and then someone would pull up a P99 latency chart and the conversation moved to a container platform or a provisioned endpoint instead.

That objection just lost most of its teeth — and the timing could not be better. With frontier AI taking off, the workloads I most want to prototype on Lambda are exactly the ones cold starts punished hardest. With Lambda SnapStart now supporting container image functions, the single biggest reason architects steered AI workloads away from Lambda is largely gone — and the design conversation shifts from “how do we survive cold starts” to “how do we design our init phase to be snapshotted.”

Why cold starts are especially brutal for AI

I learned this the hard way. A cold start is Lambda running your initialization code — loading the function, starting the runtime, and executing everything outside the handler — before it can serve the first request. For the plain CRUD functions I’d been happily shipping for years, that’s tens of milliseconds and nobody notices. The first time I dropped a model into a function, I found out that an AI workload is a different universe.

Three things stack up. First, the container image is large. A model runtime plus its transitive dependency tree — think a framework, a tokenizer library, numeric packages, and a CUDA adjacent stack — routinely produces images in the multi gigabyte range. Second, importing those dependencies is expensive: pulling a large ML framework into memory and resolving its native extensions can burn several seconds on its own, before you’ve touched a model. Third, and worst, you load weights at init. Reading a few hundred megabytes of parameters off disk or out of S3 and deserializing them into memory is the dominant cost, and it happens on every cold start.

Add those together and a “warm” invocation that returns in 80 ms is sitting behind a cold start path that takes five to ten seconds. I’ve watched a demo that flew on my laptop fall apart the moment a few concurrent requests forced Lambda to scale out into fresh environments. For a real time inference or agentic tool call, that is not a tail latency nuisance — it is timeouts, blown SLAs, and a genuinely bad user experience the moment traffic spikes. The cold start tax is levied precisely when you can least afford it.

What SnapStart for container images actually is

SnapStart attacks the problem at the mechanism level rather than asking you to shrink your dependencies. When you publish a function version, Lambda runs your function through the entire Init phase once, then takes a Firecracker microVM snapshot of the memory and disk state of that fully initialized environment, encrypts it, and caches it. On a subsequent cold start, Lambda does not re-run your initialization — it restores the microVM from that snapshot and jumps straight to handling the request.

The strategic point for AI workloads: your model load, your framework imports, your client construction all happen once, at publish time, and get frozen into the snapshot. Every future scale-out event restores from that frozen, ready to serve state instead of paying the multi second init bill again.

SnapStart itself is not new — it has been available for zip packaged functions across Java, Python, and .NET. What changed is that it now works for container image functions, and for me that’s the whole story. Containers are how I actually package these workloads. The 250 MB unzipped limit on zip functions and layers was always a poor fit for the gigabyte scale ML stacks I was building; the 10 GB container image support was the natural home. Until now I had to choose: container packaging or snapshot acceleration. I can finally have both.

How to make it work well

Here’s the mental model I’ve settled on. SnapStart rewards a specific architectural discipline: make the init phase do the expensive work, because init is what gets snapshotted.

Concretely, move model loading, dependency imports, and client/session setup to module scope — outside the handler — so they execute during Init and land in the snapshot:

# Runs at Init -> captured in the snapshot
import torch
from my_runtime import load_model

MODEL = load_model("/opt/ml/model")   # heavy: happens once, at publish
MODEL.eval()

def handler(event, context):
    # Warm path only: no model load here
    return MODEL.infer(event["input"])

The catch is uniqueness, and it’s the one that bit me. When Lambda restores many environments from one snapshot, any state created during init is shared across all of them — seeded random number generators, unique IDs, cached credentials, open connections. Freeze a database connection into the snapshot and you will hand every restored environment a stale, possibly closed socket. Seed a PRNG at init and every environment produces the same “random” sequence. I’ve debugged exactly that, and staring at duplicate “random” IDs across invocations is a humbling afternoon.

The fix is runtime hooks. For container images you implement before-snapshot and after-restore hooks so that the runtime coordinates the lifecycle. Use the before-checkpoint hook to gracefully close connections you don’t want frozen, and the after-restore hook to regenerate anything that must be unique: re-seed entropy, refresh temporary credentials, and re-establish network connections. Note the budget — after-restore work counts against a 10-second restore timeout, so keep it lean.

A few more practical levers:

  • Right-size memory. Memory scales CPU on Lambda, and restore plus after-restore work is CPU sensitive. For model serving functions, more memory usually pays for itself in lower restore and inference time.
  • Mind the image and layers. Snapshotting doesn’t excuse a sloppy image. Put model weights and heavy dependencies in stable lower layers, keep your handler code in a thin top layer, and you improve both build hygiene and load behavior.
  • Verify against real deployments. Restore latency depends on snapshot size, so benchmark P99 with your actual model rather than trusting a hello world number.

When it’s the right call — and when it isn’t

I want to be honest about the boundaries, because I’ve talked myself into using Lambda where I shouldn’t have. SnapStart on containers is a strong fit for bursty, spiky inference: workloads that idle then spike, where you’d otherwise overpay for always on capacity or eat cold starts on every scale-out. It’s excellent for agentic tool calls, where an orchestrator fans out to many short lived, independently scaling functions, and for RAG retrieval endpoints that load an embedding model and query a vector store. These are exactly the prototypes I keep reaching for right now.

It is a weaker fit in two cases. For sustained, high throughput inference that keeps environments warm anyway, provisioned concurrency or a dedicated container platform can still win on steady state cost and predictability — SnapStart’s advantage is amortizing init across intermittent cold starts, which matters less when you rarely have one. And for very large models needing GPU acceleration, Lambda is the wrong tool entirely; those belong on dedicated GPU endpoints such as SageMaker. SnapStart accelerates CPU bound init, not the physics of serving a 70B-parameter model.

The takeaway

Serverless AI just got materially more viable, and my favorite service is back on the table for the workloads I care about most. The reflexive “cold starts kill us” objection — the one that ended so many of my architecture reviews — no longer holds for a large and growing class of inference and agentic workloads. That doesn’t mean the work disappears; it moves. The new discipline is designing your init phase for snapshotting: front load the expensive work, handle uniqueness with runtime hooks, and right-size for restore.

  1. Audit your init phase. Everything expensive — model load, imports, client setup — should run outside the handler so it lands in the snapshot.
  2. Handle uniqueness explicitly. Use before-snapshot and after-restore hooks to close/reopen connections, refresh credentials, and re-seed entropy.
  3. Benchmark P99 with your real model, not a toy function, and right-size memory to the restore path.
  4. Pick the fit deliberately — bursty inference and agentic tool calls, yes; sustained high throughput or GPU bound serving, look elsewhere.
  5. Revisit dismissed designs. Endpoints you ruled out over cold starts deserve a second look.

The objection didn’t just weaken — it changed shape. I’m already reopening a couple of “we can’t use Lambda for that” calls I made last year. Which of yours are you ready to revisit?

Lambda has been my favorite service for years. When I need to stand up a POC or a pilot, nothing beats it — I can deploy code in minutes, wire it to an event, and pay only for what actually runs. So it always stung a little that the standard...

DevOps,Cloud,Architecture,Security

AWS Continuum — When Your Security Tool Talks to Your AI Coding Agent

Leave a comment

Over 60 percent of production code at Fortune 500 companies now contains blocks authored by an AI coding agent. That is not a projection — it is an industry estimate for 2026. The code works. It compiles, passes tests, ships. It also leaks credentials, trusts user input it should not, and pulls phantom dependencies with startling regularity. Your SAST scanner finds some of this — three days later, after the PR merged and the pattern propagated across four services.

AWS Continuum for code vulnerabilities is built to kill that delay. With its August 2026 announcement extending integrations into Claude Code, OpenAI Codex, and Kiro, the security feedback loop moved from “scan after commit” to “secure while writing.”

What Continuum Actually Does

Strip away the marketing and Continuum is an agent team loop — a harness that orchestrates multiple models, each selected for the task at hand, connected to your environment context. Four stages:

  1. Discovery. Continuum scans your code for vulnerabilities. Frontier models can now trace multi step attack paths that would take a human team weeks. Detection is no longer the bottleneck.

  2. Prioritization. Continuum reads your account configurations, IAM policies, network topology, and exposure surfaces before ranking a finding. A SQL injection in code that never reaches production ranks below one sitting on a public endpoint. Context kills noise.

  3. Validation. Continuum builds a working exploit in a sandbox. If it cannot actually weaponize the finding, the finding drops in priority. This is how it culls false positives — and false positives are what make security teams ignore scanner output.

  4. Remediation. It generates a fix, validated in the same sandbox, and returns it to the developer or coding agent. Not a Jira ticket pointing to a CWE page. An actual code patch. As Chet Kapoor put it: the harness is infrastructure, treated with the same rigor AWS applies to identity and policy enforcement.

The Claude Code / Codex / Kiro Integration: Why It Matters

Before August, Continuum operated on deployed code. Useful, but reactive. The Anthropic and OpenAI partnerships change the geometry.

Here is how it works. You prompt Claude Code (or Codex, or Kiro) for a function. The agent generates candidate code. Before you see it, the Agent Security Runtime (ASR) — a lightweight process inside the agent’s execution environment — intercepts the output and evaluates it against Continuum’s policy engine. The ASR returns one of four verdicts: ALLOW, ALLOW_WITH_WARNING, BLOCK_AND_REGENERATE, or BLOCK_AND_ESCALATE. On a block, the agent regenerates with remediation constraints. The developer sees the secure version first.

The latency tax is negligible. AWS reports a p99 overhead of 180 milliseconds — imperceptible when AI code generation itself takes one to three seconds for a medium complexity function. The evaluation loop runs up to three attempts before escalating, so the agent gets multiple chances to self correct before bothering a human.

Critically, both Anthropic and OpenAI confirmed the integration sits at the agent runtime layer, not as a post processing filter. Continuum sees and influences code before the developer does. That required each company to expose internal APIs to the Continuum SDK that third party developers cannot access. As Rivian CISO Mike Johnson noted: “This shortens what really matters: timeline to fix serious vulnerabilities.”

Why AI Generated Code Breaks Your Existing Security Stack

Three failure modes make traditional scanners insufficient for AI authored code:

Insecure defaults at scale. Ask a model to “write a function that authenticates users” and it will. It will not add rate limiting, constant time comparison, or JWT secret rotation unless you ask. Multiply that across thousands of functions and you get a codebase where security hardening is systematically absent. SAST catches individual patterns. It does not catch the organizational trend.

Library hallucination. AI agents sometimes suggest packages that do not exist in any registry. Attackers register these hallucinated names and publish malicious versions. At least 47 confirmed dependency confusion via hallucination incidents occurred in 2025, including two that led to production ransomware. Continuum’s ASR verifies every suggested dependency against your private registry, public registries, and a known malicious blocklist in real time.

Deprecated API patterns. Models trained on pre-2024 data still suggest hashlib.md5() for password hashing. It compiles. It runs. It is cryptographically catastrophic. Continuum maintains a Deprecated Security Patterns library covering over 4,800 API patterns across Python, JavaScript, Java, Go, C#, Ruby, and Rust, updated weekly and auto pushed to every active ASR instance.

What This Means for DevSecOps Teams

If your security architecture looks like “developer writes code → CI runs SAST/SCA → security triages findings → developer fixes three weeks later,” Continuum collapses that into a single step. The code suggestion is the remediation. AWS’s research found that developers who receive a secure suggestion as their first output are 84 percent more likely to use it as is, versus developers who get a standard suggestion followed by a separate alert.

That is not a workflow optimization. It is a behavioral change. Security teams have spent years trying to “shift left.” The reality has been shifting alerts left, not shifting secure defaults left. Continuum pushes security into the generative moment — before commit, before review, before the developer even reads the output.

For teams already running Continuum on existing code, the integration creates two modes with one outcome:

  • Existing code: Continuum discovers, prioritizes, validates, and remediates across your deployed environment.
  • Greenfield code: The Continuum plugin inside Codex, Claude Code, or Kiro delivers security validated suggestions in the development environment.

Both feed into Security Hub Extended — a dashboard that aggregates AI generated code findings across every developer, maps them to OWASP Top 10 and CWE identifiers, and pushes into your existing SIEM and ticketing workflows. One pane of glass, whether the code is legacy or was generated five seconds ago.

Practical Takeaways

  1. Request preview access now. Continuum is in gated preview. The Claude Code, Codex, and Kiro integrations are “coming soon.” Get in the queue — 1,200 enterprise accounts activated within 48 hours of the August announcement.

  2. Start with Context Profiles. Continuum lets you declare security posture, data classification, and trust boundaries per repository. A PCI scoped service gets stricter policy evaluation than an internal admin tool. Define these before turning on the ASR.

  3. Audit your dependency allow list. Continuum’s package verification is only as good as your organizational package inventory. If you do not have one, build it now. If you do, check it against what your AI agents have actually been suggesting.

  4. Instrument the feedback loop. Track ALLOW versus BLOCK_AND_REGENERATE ratios per team and per agent. Rising block rates on a specific agent or codebase tell you something about prompt quality, project complexity, or both. This is telemetry you have never had before.

  5. Do not rip out your pipeline scanners yet. Continuum addresses the generative layer. You still need SAST, SCA, and DAST for human authored code and runtime behavior. Layered defense, not replacement.

Looking Forward

CISA’s recent Guidance on AI-Assisted Software Development Security recommends real time security interception at the generation layer. Their research found that 34 percent of AI generated code passing all CI/CD checks still contained at least one exploitable vulnerability. That number should keep every security leader awake.

Continuum is the first production grade answer to that problem. It is not perfect — gated preview means rough edges, and the “coming soon” on agent integrations means your team cannot wire it up today. But the architectural bet is right: security has to live where the code is born, and in 2026, code is born inside AI agents. The sooner your security toolchain understands that, the better.

Over 60 percent of production code at Fortune 500 companies now contains blocks authored by an AI coding agent. That is not a projection — it is an industry estimate for 2026. The code works. It compiles, passes tests, ships. It also leaks credentials, trusts user input it should...

DevOps,PlatformEngineering,AI,Cloud,Kubernetes

Your Platform Engineering Team Is Now Your AI Infrastructure Team

Leave a comment

Your internal developer platform was designed for a world of stateless containers. A request arrives, a pod handles it, the pod dies. Scaling is horizontal. Failure recovery is a restart. Observability is structured logs and request traces. Your platform team got very good at this.

Now hand that team a fleet of autonomous AI agents that hold conversation state for hours, spike GPU consumption unpredictably, call external tools on their own initiative, and fail in ways that look nothing like an HTTP 500. Same team. Fundamentally different workload. The question is not whether platform engineering owns this — it is whether the team evolves fast enough to operate it.

The CNCF Has Already Made the Call

In July 2026, the Cloud Native Computing Foundation published a technical analysis arguing that agentic AI systems should be built on existing cloud native infrastructure, not bespoke ML stacks. The core thesis: agents are distributed systems with additional reasoning capabilities, and the operational problems they introduce — securing identities, coordinating long running workflows, managing state, ensuring observability, recovering from failures — are precisely the problems the cloud native ecosystem spent the last decade solving.

The paper walked through a Kubernetes based multi agent security platform combining Dapr, OpenTelemetry, SPIFFE, Falco, and Kafka. No custom orchestrator. No special purpose scheduler. Just the same primitives your platform team already operates, extended with agent aware abstractions.

This is a deliberate signal. The CNCF is not positioning agents as a research curiosity that lives in a data science silo. It is positioning them as the next class of production workload that runs on the same infrastructure your platform team already owns.

Kubernetes 1.36: The Scheduler Learns About GPUs

If the CNCF paper was the strategic argument, Kubernetes 1.36 (shipped May 2026) is the tactical proof. The release is best described by the ScaleOps team’s summary: “less about brand new mechanics and more about the defaults catching up to two years of accumulated AI workload scar tissue.”

Three Dynamic Resource Allocation (DRA) enhancements — Partitionable Devices, Consumable Capacity, and Device Taints and Tolerations — all moved to Beta and shipped enabled by default. Together they replace the old integer GPU device plugin model, where a single card was allocated wholesale regardless of actual utilization, with primitives that can express how modern accelerators are partitioned, shared, and recovered when they fail.

For platform teams, the headline feature is Workload Aware Preemption (alpha). Before 1.36, the scheduler would preempt individual pods to make room for higher priority work, which could leave a distributed agent fleet with seven of eight workers running but unable to make progress. The new behavior treats a PodGroup as a single preemption unit and only proceeds with eviction after verifying the high priority group can actually fit.

There is also Mutable Pod Resources for Suspended Jobs (now beta, enabled by default). A queue controller can suspend a running job, adjust its CPU, memory, or GPU requests to match available cluster capacity, and unsuspend it — without destroying and recreating pods. For agent workloads that hold in memory state, this is the difference between a graceful resource adjustment and a hard restart that loses hours of accumulated context.

The message is clear: the Kubernetes ecosystem is building first class primitives for exactly the workloads platform teams are about to inherit.

AWS ECS: Auto Recovery for Agent Connectivity Loss

Managed container platforms are adapting too. On August 31, AWS announced that Amazon ECS now automatically detects and recovers container instances that lose agent connectivity to the control plane. ECS surfaces a new AGENT_CONNECTIVITY health event across Fargate, Managed Instances, and EC2. On Fargate and Managed Instances, recovery is automatic — drain, replace, deregister. On EC2, you wire the event into your own workflow.

This matters because agentic workloads are particularly sensitive to control plane disconnection. A stateless web server that loses its orchestrator is an inconvenience — the load balancer routes around it. An autonomous agent that loses contact may continue executing stale instructions, burn resources on obsolete work, or silently drop state that cannot be reconstructed. Auto recovery at the platform level is a prerequisite, not a nice to have.

What Actually Changes for Platform Teams

The operational model shift from stateless containers to autonomous agents is not incremental. Here is where the differences bite:

Scheduling becomes resource aware in new dimensions. Stateless containers need CPU and memory. Agents need GPU shares, sometimes fractional, sometimes across multiple accelerators. Your IDP’s resource request templates need to understand DRA claims, not just resources.requests.cpu.

Failure recovery is no longer “just restart it.” An agent that has been running for six hours, maintaining conversation state and accumulated tool call context, cannot simply be killed and restarted. Your platform needs checkpointing primitives, graceful drain hooks that give agents time to persist state, and recovery paths that restore context rather than starting from zero. The Kubernetes 1.36 in place vertical scaling feature is relevant here — resizing resources without restarting the pod means you can adapt to changing demand without losing state.

Observability must explain decisions, not just measure latency. Traditional traces show you the path a request took through your microservices. Agent observability needs to capture reasoning paths, tool invocations, and the context that led to each autonomous decision. OpenTelemetry is being extended for this, but your IDP’s default dashboards and alerting rules were not built for it. Dynatrace’s 2026 State of SRE and Platform Engineering report found that monitoring AI systems is now SREs’ number one use case at 58%, ahead of automation and SLO management. Your platform’s observability stack needs to catch up.

Cost attribution gets harder. A stateless container’s cost is predictable: CPU hours times instance price. An agent’s cost is variable: model inference tokens, tool call API charges, GPU time that fluctuates with reasoning complexity. The InfoQ Cloud and DevOps Trends 2026 report captures this well — Shweta Vohra from the FinOps Foundation described the current state as “agents’ chaos at the moment is bigger than the microservices times we saw.” Your IDP needs cost attribution that tracks token consumption per agent per task, not just pod level compute.

Why Not a Separate “AI Infra” Team?

There is a tempting pattern: stand up a dedicated AI infrastructure team, give them their own cluster, let them figure it out. Resist this.

The InfoQ trends report found that platform teams are evolving from builders to enablers. Mark Silvester noted that platform teams at his clients are becoming “AI native enablers” — and when the central platform is not good enough, teams build shadow platforms that fragment governance. An isolated AI infra team creates exactly this fragmentation: two deployment pipelines, two observability stacks, two cost models, two incident response processes. The agents still need network policies, secrets management, identity federation, and CI/CD — all things your platform team already provides.

The better model: extend the existing IDP. The platform team already owns the paved road. Widen it for a new vehicle type. Do not build a separate highway.

The Platform Team Audit Checklist

If you are on a platform engineering team, here is what to evaluate in your IDP today:

  1. GPU and accelerator support in your resource model. Can developers request fractional GPUs or specific accelerator types through your self service catalog? If your IDP still only exposes CPU and memory, you are already behind.
  2. State preservation primitives. Do you offer checkpointing, persistent volumes with fast attach, or graceful drain hooks with configurable timeouts longer than 30 seconds? Agent workloads need them.
  3. Agent aware health checks. Your liveness and readiness probes were designed for HTTP endpoints. Add checks that verify agent control plane connectivity, reasoning loop health, and tool call availability.
  4. Observability for reasoning, not just requests. Extend your default telemetry to capture tool invocations, token consumption, and decision traces. OpenTelemetry semantic conventions for GenAI are your starting point.
  5. Cost attribution per agent task. Integrate token level cost tracking into your chargeback model. If your FinOps dashboards only show pod level compute, they will miss the majority of agent operating cost.

The Road Ahead

The CNCF made the architectural argument. Kubernetes 1.36 shipped the scheduling primitives. AWS is hardening its managed platforms for agent resilience. The ecosystem is converging on a clear answer: agentic AI runs on cloud native infrastructure, and the platform engineering team is the natural owner.

The platform teams that move now — extending their IDPs with GPU aware scheduling, stateful recovery, agent observability, and token cost attribution — will be the ones that keep the paved road paved. The ones that wait will find their developers building shadow AI platforms in the same way they once built shadow Kubernetes clusters: fast, fragmented, and ungovernable.

Your platform engineering team built the internal developer platform. They are about to build the internal agent platform. Same team. Bigger mandate. Start the audit today.

Your internal developer platform was designed for a world of stateless containers. A request arrives, a pod handles it, the pod dies. Scaling is horizontal. Failure recovery is a restart. Observability is structured logs and request traces. Your platform team got very good at this.

Now hand that team a...

GenerativeAI,Architecture,DevOps,Engineering

Scaling the Agentic Product Development Lifecycle

Leave a comment

Your AI coding agent just shipped a 400 line pull request across three microservices, updated the integration tests, and opened a draft PR — all while you were in a planning meeting. Now what? Who reviews it? How do you know it didn’t introduce a subtle security flaw or violate your team’s architectural conventions? And how do you do this reliably across forty engineers, not just one?

This is the scaling problem nobody warned us about. The individual productivity gains from agentic coding tools are real and well documented. But the organizational challenges of running AI agents as quasi team members — with governance, context management, and meaningful measurement — are where most engineering orgs are currently stumbling.

From Pair Programmer to Team Member

The mental model shift matters. When agents operated as autocomplete on steroids — suggesting a line or two in your editor — the human remained firmly in control. Every suggestion was evaluated in real time, accepted or rejected with a keystroke. The blast radius of a bad suggestion was a single line.

Today’s agentic workflows look fundamentally different. Tools like Kiro, Claude Code, and Amazon Q Developer can execute multi step plans: reading existing code, generating implementation across multiple files, running tests, and iterating on failures autonomously. The agent isn’t pair programming anymore. It’s operating as an independent contributor with a task assignment.

This changes three things simultaneously:

  1. Review surface area explodes. A human writing code produces artifacts shaped by their own mental model. An agent produces artifacts shaped by its context window and instructions — which may or may not align with tribal knowledge about why the codebase is structured a certain way.

  2. Accountability becomes ambiguous. If an agent generated the code and a human approved the PR, who owns the production incident at 2am? Teams need explicit answers before scaling adoption.

  3. Context becomes the bottleneck. An agent is only as good as what it knows about your system. Scaling from one developer’s pet project to a team wide workflow means solving context distribution systematically.

Governance Patterns That Actually Work

The teams doing this well share a common trait: they treat agent generated code with more scrutiny than human generated code, not less. Here are the patterns emerging:

Spec driven development. Rather than giving agents open ended instructions, leading teams write structured specifications before any code generation begins. Kiro’s approach of generating design documents and task breakdowns before implementation is instructive here. The spec becomes both the instruction set for the agent and the acceptance criteria for reviewers. This creates a natural human in the loop checkpoint at the design phase — where human judgment adds the most value.

Tiered review workflows. Not all agent generated code carries equal risk. A utility function with full test coverage is different from a change to your authentication middleware. Teams are implementing tiered review policies: auto merge for low risk changes with passing tests, single reviewer for medium risk, and mandatory senior engineer review for anything touching security boundaries, data models, or public APIs.

Guardrails as code. Static analysis, architectural fitness functions, and custom linting rules become force multipliers when agents are generating code. If your CODEOWNERS file, your ADRs, and your security policies are machine readable, agents can respect them proactively and CI can catch violations deterministically. Invest in codifying your conventions — the ROI compounds when machines are your primary code producers.

Audit trails. Every agent invocation should be logged with its full context: the prompt, the files read, the plan generated, and the diff produced. When something goes wrong in production three weeks later, you need forensics that go beyond git blame.

Context Management at Scale

Here’s the uncomfortable truth: most codebases exceed any agent’s context window by orders of magnitude. A senior engineer navigates a 2 million line monorepo using years of accumulated mental models. An agent gets 128K to 200K tokens and whatever files you explicitly feed it.

Teams scaling agentic workflows are converging on a few patterns:

Architectural decision records (ADRs) as agent context. Your ADRs aren’t just documentation for humans anymore — they’re the institutional memory that agents need to make coherent decisions. Teams maintaining well structured ADRs report significantly better agent output because the agent understands not just what the code does, but why it’s structured that way.

Repository maps and module summaries. Automatically generated structural overviews — dependency graphs, module responsibility summaries, API boundary documentation — give agents navigational context without consuming the entire token budget on source code. Think of it as giving the agent the same “lay of the land” briefing you’d give a new hire on day one.

Scoped context windows. Rather than letting agents see everything, explicitly scope their context to the relevant module, its interfaces, and its tests. This is analogous to the principle of least privilege — agents perform better with focused, relevant context than with a firehose of tangentially related code.

Shared memory across sessions. For complex multi day tasks, teams are experimenting with persistent context stores — structured summaries of previous agent sessions, decisions made, and approaches attempted. This prevents the “amnesia problem” where each new agent session rediscovers constraints that were already resolved.

Measuring Impact Without Gaming Metrics

Lines of code generated per hour is a vanity metric that will actively harm your engineering culture. When AI agents can produce unlimited volume, volume becomes meaningless.

The metrics that matter for agentic development:

  • Cycle time from spec to production. How quickly does a well defined feature move from approved specification to deployed code? This captures the full value chain including review, testing, and deployment — not just generation speed.
  • Defect escape rate. Are agent generated changes introducing more bugs that reach production? Track this separately from human authored code to calibrate your review processes.
  • Review turnaround time. If your bottleneck shifts from writing code to reviewing it, you need to know. A 10x increase in PR volume with the same review capacity just creates a different kind of backlog.
  • Developer satisfaction and cognitive load. Survey your team regularly. Are agents reducing toil and freeing engineers for higher judgment work? Or are they creating a new kind of burden — endless review of mediocre generated code?

Where Human Judgment Remains Non Negotiable

Scaling agent adoption is not about removing humans from the loop. It’s about repositioning humans at the points where their judgment is irreplaceable:

  1. Architecture decisions. Agents can implement patterns, but choosing which patterns to apply — and when to deviate from convention — requires understanding business context, team capabilities, and technical debt trajectories that no context window can fully capture.

  2. Security review. Agents are improving at avoiding common vulnerabilities, but adversarial thinking — “how could this be exploited?” — remains a deeply human skill. Security sensitive code paths need human eyes, period.

  3. Customer facing UX. Agents can generate UI components, but understanding whether the interaction feels right to a user requires empathy and product intuition that remains beyond current model capabilities.

  4. Trade off decisions under uncertainty. When requirements are ambiguous, when you’re choosing between two valid approaches with different long term implications, when you’re deciding what not to build — these are the moments that justify senior engineering salaries.

Practical Takeaways

  1. Codify your conventions now. ADRs, architectural fitness functions, linting rules, and security policies — if they aren’t machine readable, your agents can’t respect them and your CI can’t enforce them.
  2. Implement tiered review before scaling volume. Decide which categories of change need what level of human oversight, and encode that in your workflow tooling.
  3. Invest in context infrastructure. Repository maps, module summaries, and structured specifications pay dividends every time an agent touches your codebase.
  4. Measure outcomes, not output. Track cycle time, defect rates, and developer experience — not lines generated.
  5. Reposition your senior engineers as reviewers and architects. Their highest value work shifts from writing code to ensuring the right code gets written.

Looking Forward

The engineering organizations that will thrive in the agentic era aren’t the ones that adopt agents fastest — they’re the ones that build the governance, context management, and measurement infrastructure to adopt agents sustainably. The tooling is maturing rapidly. The organizational patterns are still being invented. Start building yours now, because the teams that figure out scaled agentic workflows first will have a compounding advantage that’s difficult to replicate.

Your AI coding agent just shipped a 400 line pull request across three microservices, updated the integration tests, and opened a draft PR — all while you were in a planning meeting. Now what? Who reviews it? How do you know it didn’t introduce a subtle security flaw or violate...

Cloud,Architecture,DevOps,AI,FinOps

When Your AI Coding Tools Become a Variable Cloud Bill

Leave a comment

Your engineering team didn’t provision any new infrastructure last quarter. Nobody requested bigger instances or spun up a new microservice. But your cloud bill climbed 40%. The culprit isn’t a rogue developer or a forgotten resource — it’s the AI pair programmer sitting in every engineer’s IDE.

The Amplification Loop Nobody Budgeted For

AI coding assistants — GitHub Copilot, Amazon CodeWhisperer, Cursor, Cody, and the growing roster of alternatives — have fundamentally changed the throughput of individual developers. A senior engineer who previously opened three PRs a day now opens seven. A junior developer who used to spend two hours writing boilerplate produces the same output in twenty minutes, then moves on to the next task.

This is the productivity gain everyone celebrated. What nobody modeled was the downstream infrastructure cost of that productivity.

Here’s the feedback loop:

  1. AI generates more code, faster — developers accept suggestions, scaffold entire modules, write more tests
  2. More code means more pull requests — smaller, more frequent PRs become the norm
  3. More PRs trigger more CI/CD runs — every push kicks off builds, linting, unit tests, integration tests
  4. More CI runs spawn more ephemeral environments — preview deployments, staging replicas, feature branch clusters
  5. More environments consume more compute, networking, and storage — and they stick around longer than anyone realizes

Each step individually looks benign. Together, they create a compounding cost amplification that doesn’t show up as a single line item on your bill. It’s spread across CodeBuild minutes, ECS task hours, EBS snapshots, NAT gateway data transfer, and dozens of other services that each grew “just a little.”

The Numbers Teams Are Seeing

The signal is consistent across organizations adopting AI coding tools at scale. Teams are reporting 30–50% increases in CI/CD compute spend within three to six months of broad AI assistant adoption. One platform engineering team I spoke with saw their CodeBuild costs triple — not because builds got slower, but because build volume exploded.

Consider the math. If your team of 20 engineers averaged 60 PRs per week pre-AI adoption and now averages 120, you’ve doubled your:

  • Build minutes (CodeBuild, GitHub Actions runners, whatever your CI platform)
  • Preview environment hours (ECS tasks, Lambda invocations, RDS snapshots for feature branches)
  • Artifact storage (ECR images, S3 build caches, test result archives)
  • Data transfer (pulling dependencies, pushing containers, syncing across AZs)

None of these individually trigger a cost anomaly alert. A 15% increase in CodeBuild? Normal growth. A 20% bump in ECR storage? Probably just new services. But stack them together and your monthly bill tells a different story.

Why Traditional Cost Controls Miss This

Most FinOps practices are designed to catch two patterns: sudden spikes (anomaly detection) and large single resources (rightsizing recommendations). AI driven cost amplification fits neither pattern.

It’s not a spike — it’s a gradual, distributed increase across many services simultaneously. It’s not one oversized resource — it’s thousands of small, short lived resources that individually cost pennies. Your Cost Explorer dashboard shows everything growing at roughly the same rate, which looks like organic scaling. Except nobody deployed a new product or onboarded new customers.

The traditional question “which service is costing us more?” becomes the wrong question. The right question is “which activity is driving more resource creation?” And most cloud billing tools aren’t designed to answer that.

Practical Mitigations

You don’t need to slow down AI adoption. You need infrastructure guardrails that account for increased developer throughput.

1. Per Developer Environment Budgets

Set monthly compute budgets per developer or per team for ephemeral resources. AWS Budgets supports tag based filtering — tag every CI spawned resource with the developer alias or PR number, then set alerts at 80% of a per person threshold.

# Example AWS Budget with developer-scoped tags
Resources:
  DevBudget:
    Type: AWS::Budgets::Budget
    Properties:
      Budget:
        BudgetName: dev-ephemeral-compute
        BudgetLimit:
          Amount: 500
          Unit: USD
        TimeUnit: MONTHLY
        CostFilters:
          TagKeyValue:
            - "user:developer-alias$dev-team"
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 80
          Subscribers:
            - SubscriptionType: SNS
              Address: !Ref AlertTopic

2. Aggressive TTL Policies on Ephemeral Infrastructure

Every preview environment, feature branch database, and temporary cluster should have a hard TTL. Default to 4 hours, extend on explicit request. Use AWS Lambda with EventBridge Scheduler to sweep and terminate expired resources.

# Tag resources at creation with expiry
aws ec2 create-tags --resources $INSTANCE_ID \
  --tags Key=ttl-expires,Value=$(date -d '+4 hours' -u +%Y-%m-%dT%H:%M:%SZ)

3. Smarter CI Triggers

Not every push needs a full pipeline run. Implement path based triggers that only execute relevant stages. If the AI generated a documentation change, skip the integration test suite. If only tests changed, skip the deployment preview.

# CodePipeline / GitHub Actions path filtering
on:
  pull_request:
    paths:
      - 'src/**'
      - '!src/**/*.md'
      - '!docs/**'

4. Cost Per PR Dashboards

Build visibility into the cost of each pull request. Tag CI resources with the PR number, then query Cost Explorer or use the AWS Cost and Usage Report (CUR) to calculate per PR spend. Surface this in your PR workflow — engineers modify behavior when they see the number.

5. CodeBuild Concurrency Limits

Set explicit concurrency limits on your CodeBuild projects. Without limits, 50 simultaneous AI generated PRs means 50 parallel builds. A concurrency cap of 10 serializes excess builds, smoothing your spend curve without blocking developers indefinitely.

aws codebuild update-project \
  --name my-project \
  --concurrent-build-limit 10

The FinOps Conversation Shift

The meta point here is that AI coding tools transform cloud cost from a provisioning problem into a velocity problem. Traditional capacity planning asked “how much infrastructure do we need for our workload?” Now the question becomes “how much infrastructure does our development activity generate?”

This requires FinOps teams to track a new metric: infrastructure cost per unit of developer output. Not cost per customer request or cost per transaction — cost per PR, cost per deployment, cost per developer hour. These are the leading indicators that predict where your bill is headed before it arrives.

Five Takeaways

  1. Measure CI/CD cost per PR — establish a baseline before AI adoption scales further, then track the trend
  2. Tag everything with developer and PR context — you cannot control what you cannot attribute
  3. Default ephemeral resources to short TTLs — make long lived the exception, not the default
  4. Set concurrency guardrails on build systems — cap parallel builds to prevent bill spikes during high throughput periods
  5. Treat developer throughput as a cost input — model it in your FinOps forecasts the same way you model customer growth

Looking Forward

AI coding tools will only get faster and more capable. The next generation won’t just suggest code — they’ll autonomously create PRs, trigger deployments, and provision infrastructure without a human in the loop. The organizations that survive this shift with predictable cloud bills are the ones building cost guardrails now, while a human still approves each PR.

Your developers aren’t spending more. Their AI pair programmer is. Budget accordingly.

Your engineering team didn’t provision any new infrastructure last quarter. Nobody requested bigger instances or spun up a new microservice. But your cloud bill climbed 40%. The culprit isn’t a rogue developer or a forgotten resource — it’s the AI pair programmer sitting in every engineer’s IDE.

The Amplification...

Technology,DevOps,Software

Goodnotes 5

Leave a comment

Goodnotes 5 was released last week. Goodnotes is my favorite stylus note taking app on the IPad. I’ve tried most of the competitors at least once and revisit them when they release new features. I’ve been on Goodnotes for years and have been using it daily.

Let’s move to the topic of this blog, Goodnotes 5 is a bit buggy. There were a ton of negative comments on Twitter towards the release. The development team has released 7 updates as of the writing of this blog. Goodnotes 5 is not a forced upgrade from version 4. While I’ve not seen all the problems described on Twitter, I’ve seen a few of the issues. I knew installing the initial release, and there were going to be some bugs.

However if you think about the DevOps model release, fix, release, fix, release, fix. The model is built for this type of release and user feedback.

However, many of the twitter complaints, where why was buggy software released. So it made me think about when is software ready for release in the DevOps model? Typically there is a release once code passes, unit tests, integration tests, load tests, functional tests, and GUI Tests. However bugs do reach production and the users, there is no fool-proof plan.

App store doesn’t allow releasing of beta software. However does offer TestFlight, so maybe GoodNotes could have leverage 10,000 of its customers to beta test the software and avoided the negative backlash on Twitter.

Goodnotes 5 was released last week. Goodnotes is my favorite stylus note taking app on the IPad. I’ve tried most of the competitors at least once and revisit them when they release new features. I’ve been on Goodnotes for years and have been using it daily.

Let’s move...

AWS,Certification,DevOps

AWS Certified DevOps Engineer - Professional

Leave a comment

Sat the AWS Certified DevOps Engineer - Professional Exam last this afternoon.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best solution for deployments which comprised CloudFormation, Elastic Beanstalk and OpsWorks.   Every one of those questions had 2 good answers, it came down to which was more correct based on the keywords cost, speed, redundancy, roll back capabilities.  

I did the course on acloud.guru and a lot of AWS pages. At some point I will make a page of all the links I collected when studying for this exam.

The exam took me about two-thirds of the allowed time, I read fast and have a tendency to flag questions I don’t know the answer to and come back later and work thru them. This exam, I flagged 20 questions. Most of them I could figure out, once I thought about them for a while. But flagging questions and going back helps manage the time.

Upon submission, I got the “Congratulations! You have successfully completed the AWS Certified DevOps Engineer - Professional…”

I got my score email very quickly:

Overall Score: 82%

Topic Level Scoring:

1.0 Continuous Delivery and Process Automation: 79%
2.0 Monitoring, Metrics, and Logging:  87%
3.0 Security, Governance, and Validation:  75%
4.0 High Availability and Elasticity:  91%

That now makes my 7th AWS Certification.

Sat the AWS Certified DevOps Engineer - Professional Exam last this afternoon.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best solution for deployments which comprised CloudFormation, Elastic Beanstalk and OpsWorks.   Every one of those questions had 2...

DevOps,Security

Devops needs SecDevOps

Leave a comment

DevOps is well defined and has a great definition Wikipedia. We could argue all day about who is really doing DevOps(see this post for context). Let’s assume that there is efficient and effective DevOps organization, If this is the case, DevOps requires a partner in security. Security needs to manage compliance, security governance, requirements and risks. This requires functions in development, operations and analysis. How can security keep up with effective DevOps? Building DevOps organization for security which we call SecDevOps. SecDevOps is about automating and monitoring compliance, security governance, requirements and risks, and especially updating analysis as DevOps changes the environment.

As organization support DevOps but don’t seem as ready to support SecDevOps, how can security organization evolve to support DevOps?

DevOps is well defined and has a great definition Wikipedia. We could argue all day about who is really doing DevOps(see this post for context). Let’s assume that there is efficient and effective DevOps organization, If this is the case, DevOps requires a partner in security. Security needs...

DevOps,Security,Technology

Security as Code

Leave a comment

One of the things I’ve been fascinated of late is the concept of Security as Code.   I’ve just started to read the book DevOpSec by Jim Bird.   One of the things the book talks about is injecting security into the CI/CD pipeline for applications.  Basically merging developers and security, as DevOps merged developers and operations.   I’ve argued for years DevOps is a lot of things, but fundamentally it was a way for operations to become part of the development process which led to the automation of routine operational tasks and recovery.  So now if we look at DevOpsSec, this would assume security is part of the development process. I mean more than just the standard code analysis using Veracode.  What would it mean if security processes and recovery could be automated?  

Security Operations Centers (SOCs) where people are interpreting security events and reacting.  Over the last few years, much of the improvements in SOCs has been made via AI and machine learning reducing the head count required to operate a SOC.   What if security operations were automated?   Could some code be generated based on the security triggers and provided to the developer for review and incorporation into the next release?

We talk about infrastructure as code, where some data can be generated to create rules and infrastructure using automation.   Obviously on AWS you can install security tool based AMIs, Security Groups and NACLs with Cloudformation.  My thoughts go to firewall based AMIs, appliances  for external access.   The appliance access-lists required are complex, require enormous review and processing within an organization.  Could access lists be constructed based on a mapping of the code and automatically generated for review?  Could the generated access list be compared against existing access-list for deduplication detection.

It’s definitely an interesting topic and hopefully evolves over the next few years. 

One of the things I’ve been fascinated of late is the concept of Security as Code.   I’ve just started to read the book DevOpSec by Jim Bird.   One of the things the book talks about is injecting security into the CI/CD pipeline for applications.  Basically merging developers and security,...

Cloud,DevOps,SaaS

SaaS based CI/CD

Leave a comment

Let’s start with some basics of software development.    It still seems no matter what methodology of software development lifecycle that is followed it includes some level of definition, development, QA, UAT, and Production Release.   Somewhere in the process, there is a merge of multiple items into a release.   This still means your release to production could be monolithic.

The mighty big players like  Google,  Facebook, and Netflix (click any of them to see their development process) have revolutionized the concept of Continous Integration (CI) and Continous Deployment (CD).

I want to question the future of CI/CD,  instead of consolidating a release, why not release a single item into production, validate over a defined period of time and push the next release.   This entire process would happen automatically based on a queue (FIFO) system.

Taking it to the world of corporate IT and SaaS Platforms.   I’m really thinking about software like Salesforce Commerce Cloud,  or Oracle’s NetSuite.      I would want the SaaS platform to provide me this FIFO system to load my user code updates.  The system would push and update the existing code, while it continues to handle the requests and the users wouldn’t see discrepancies.    Some validation would happen, the code would activate and a timer would start on the next release.  If validation failed the code could be rolled back automatically or manually.

Could this be a reality?

Let’s start with some basics of software development.    It still seems no matter what methodology of software development lifecycle that is followed it includes some level of definition, development, QA, UAT, and Production Release.   Somewhere in the process, there is a merge of multiple items into a release....

AWS,Containers,DevOps,Technology

Serverless 2018

Leave a comment

Serverless is becoming the 2018 technology hype.   I remember when containers were gaining traction in 2012, and Docker in 2013.  At technology conventions, all the cool developers were using containers.   It solved a lot of challenges, but it was not a silver bullet. (But that’s a blog article for another day.)

Today after an interview I was asking myself,  have Containers lived up to the hype?   They are great for CI/CD, getting rid of system administrator bottlenecks, helping with rapid deployment, and some would argue fundamental to DevOps.  So I started researching the hype.   People over at  Cloud Foundry published a container report in  2017 and 2016.

Per the 2016 report, “our survey, a majority of companies (53%) had either deployed (22%) or were in the process of evaluating (31%) containers.”

Per the 2017 report, “increase of 14 points among users and evaluators for a total of 67 percent using  (25%) or evaluating (42%).”

As a former technology VP/director/manager, I was always evaluating technology which had some potential to save costs, improve processes, speed development and improve production deployments.   But a 25% adaption rate and a 3% uptick over last year, is not moving the technology needle.

However, I am starting to see the same trend, Serverless is the new exciting technology which is going to solve the development challenges, save costs, improve the development process and you are cool if you’re using it.       But is it really Serverless or just a simpler way to use a container?

AWS Lambda is basically a container.  (Another blog article will dig into the underpinnings of Lambda.)   Where does the container run? ** A Server. **

Just means I don’t have to understand the underlying container, server etc.etc.etc.     So is it truly serverless?   Or is it just the 2018 technology hype to get all us development geeks excited, we don’t need to learn Docker or Kubernetes, or ask our Sysadmin friends provision us another server.

Let me know your thoughts.

Serverless is becoming the 2018 technology hype.   I remember when containers were gaining traction in 2012, and Docker in 2013.  At technology conventions, all the cool developers were using containers.   It solved a lot of challenges, but it was not a silver bullet. (But that’s a blog article for another...

DevOps,Microservices,Technology

DevOps

Leave a comment

Is DevOps the most overused word in technology right now?

The full definition from Wikipedia.  Here what DevOps really is about.   It about taking monolithic code with complex infrastructure supported by developers, operational personnel, testers, system administrators and simplifying it, monitoring it and taking automated corrective actions or notification.

It’s really about reducing resources who aren’t helping the business grow and using that headcount toward a position which can help revenue growth.

It’s done in 3 pieces.

Piece 1. The Infrastructure

It starts by simplifying the infrastructure build-out, whether it in the cloud where environments can be spun up and down instantly based on some known configuration like AWS CloudFormation,  using Docker or Kubernettes.   Recently, Function as a Service (FaaS), AWS Lambda,  Google Cloud Functions or Azure Functions. This reduces reliance on a DBA, Unix or Windows System Administrator and Network Engineers.   Now the developer has the space they need instantly.   The developer can deploy their code quicker, which speeds time to market.

Piece 2.  Re-use and Buy vs. Build

Piece 2 of this is the Re-use and Buy vs. Build.   Meaning if someone has a service re-use it, don’t go building your own.    An example is Auth0 for authentication and Google Maps for mapping locations or directions.

Piece 3.  When building or creating software do it as Microservices.

To simplify it you are going to implement microservices.   Basically, you create code that does one thing well.  It’s small, efficient and manageable.    It outputs JSON which can be parsed by upstream Services.   The JSON can extend without causing issues to upstream Services.   This now reduces the size of the code base a developer is touching, as it one service.   It reduces regression testing footprint.      So now the number of testers, unit tests, regression tests and integration tests have been shrunk.   This means faster releases to production, and also means a reduction in resources.

You’re not doing DevOps if any of these conditions apply?

  1. You have monolithic software you’ve put some web services in front of.

  2. Developers are still asking to provision environments to work.

  3. People are still doing capacity planning and analysis.

  4. NewRelic (or any other system)  is monitoring the environment, but no one is aware of what is happening.

  5. Production pushes happen at most once a month because of the effort and amount of things which break.

Doing DevOps

  1. Take the monolithic software and break it into web services.

  2. Developers can provision environments per a Service Catalog as required.

  3. Automate capacity analysis.

  4. Automatic SLAs which trigger notifications and tickets.

  5. NewRelic is monitoring the environment, and it providing data to systems which are self-correcting issues, and there are feedback loops on releases.

  6. Consistently (multiple times a week)  pushing to production to enhance the customer experience.

Is DevOps the most overused word in technology right now?

The full definition from Wikipedia.  Here what DevOps really is about.   It about taking monolithic code with complex infrastructure supported by developers, operational personnel, testers, system administrators and simplifying it, monitoring it and taking automated corrective actions or notification.

It’s really...