Category Archives: Cloud

Shadow GenAI Is Just Shadow IT Wearing a Smarter Hoodie

September 17 2026 Greg Fina Leave a comment

I have a confession that dates me: I was shadow IT. In the early cloud years I was one of those people in a department who got tired of waiting on a central provisioning ticket, pulled out a corporate card, and stood up what I needed in the cloud that afternoon. It was faster. It worked. And it drove our IT and security teams up the wall, because from where they sat I had just built production infrastructure they could not see, could not secure, and did not know existed.

I ran my first EC2 instance in 2009 the same way. Not because I was reckless, but because the sanctioned path was slow and the unsanctioned one was right there in a browser tab. That lived experience is the whole reason I am writing this. Because the exact same fight is back, at ten times the scale, and most organizations are about to lose it the same way they nearly lost the first one.

We Did Not Beat Shadow IT By Banning It

Here is the part everyone forgets. Shadow IT did not die because security got strict. Memos did not kill it. Blocking did not kill it. Every org that tried to win by locking down expense policy and threatening consequences just pushed the behavior further underground and made it more dangerous.

What actually worked was a platform. Companies stopped treating central IT as a gate you had to get through and started treating it as a paved road you wanted to be on. They built landing zones with guardrails baked in, self service catalogs, sane defaults, and identity that just worked. The sanctioned path became the easy path. And the moment the governed road was also the fast road, the shadow behavior dissolved on its own. Nobody swipes a personal card to route around a platform that is genuinely better than what they would build alone.

I made that argument at length in Your Platform Engineering Team Is Now Your AI Infrastructure Team: when the central platform is not good enough, teams build shadow platforms that fragment governance. The answer is not a separate highway. It is a wider paved road. Extend the IDP, do not fork it.

That lesson took a decade to internalize. We are about to relearn it in a fraction of the time.

The Rematch: Shadow GenAI

The first time around, shadow IT was a few dev teams with an AWS account. Contained, technical, a problem you could at least name and count. Shadow GenAI is that same dynamic with the population expanded to everyone. It is no longer a handful of engineers. It is every employee with a browser, and every one of them now has an agent one tab away.

The numbers landed this week and they are not subtle. Yahoo Finance reported on September 17 that 67% of workers use unapproved AI while enterprises are still shipping governance infrastructure, and coined a phrase worth stealing: the shadow agent gap, the disconnect between how work actually happens and how it is managed. A day earlier Deloitte found that 31% of GenAI users use it without their employer knowing, and UK workers spend roughly one billion pounds a year of their own money on GenAI for work. Read that last stat again. Employees are literally swiping their own cards to route around the sanctioned tool. That is my 2009 corporate card, reissued for the entire workforce.

And the data does not stay put. LayerX research surfaced by ETHRWorld found that 77% of employees paste data into GenAI prompts, and 82% of those pastes come from personal, unmanaged accounts. Source code, client proposals, PII, roadmap decks, all flowing into models nobody reviewed, through accounts nobody controls. Microsoft draws the useful distinction between the two forms of shadow AI: unsanctioned tools, and unsanctioned agents. CIO frames the same shift as the move from hidden apps to hidden autonomous systems that think, act, and decide. The unsanctioned thing is no longer infrastructure sitting still. It is autonomy taking action on your behalf.

One more beat, because it kills the comforting story that this is a junior developer problem. TrustedTech, citing Censuswide, found that senior decision makers are twice as likely to use unapproved AI as their own reports, 65% versus 31%. The people writing the acceptable use policy are the biggest violators of it. This is not a compliance gap you can train your way out of.

Who Protects What, And Why That Is A Trap

Lay out the responsibility model honestly and the problem becomes visible. IT and network protects the connection: transport, network segmentation, egress. Data and security protects the data: classification, DLP, encryption at rest and in motion. Both of those disciplines are mature and both are doing their jobs.

And nobody protects the business logic. Nobody owns the decisions the agent reasons its way through. I made this case in Rogue AI Agents Aren’t Flukes: an agent does not breach you by cracking TLS or dumping a database. It reasons toward a goal and takes actions, chaining tool calls and crossing a boundary nobody thought to close. No firewall inspects a decision. No DLP rule catches a judgment call.

When every department is quietly running its own agents, that gap stops being a corner case and becomes the whole surface. Security is no longer a team you hand off to at the end of a project. Security becomes everyone’s problem. And here is the trap: the instant a problem belongs to everyone and no single team owns it, it stops being a security problem at all. It becomes a governance problem. That is the line we just crossed.

Govern It The Way We Governed The Cloud

You cannot ban your way out of this. The surveys are unanimous that people will take the risk to hit a deadline, and the BYOAI reality is that when the sanctioned alternative is nonexistent or too slow, employees route around it every time. Blocking lost the first war. It will lose this one faster, because the population routing around you is a hundred times larger.

We won the first war with a platform, so build one again. A sanctioned agent platform where the governed path is the fast path. Google Cloud puts it well in its guidance to counter shadow agents: govern agents with the same rigor you apply to human managed accounts. Inside that platform, the enforcement layer is exactly the checklist I walked through in the agent governance post, so I will not re run it here: identity per agent, least privilege, guardrails, audit, and a tested kill switch. Treat agents like privileged digital workers, and make requesting a governed one easier than pasting a proposal into a personal chatbot. Same move as the landing zone. Same move as the paved road. New vehicle.

But a platform without an owner is just a project waiting to be abandoned. The reason shadow IT actually died is that someone was accountable for the paved road staying better than the ditch beside it, and that ownership cannot live inside security alone. This is where an AI Center of Excellence earns its name. Not a committee that meets once a quarter to rubber stamp tools, but a standing cross functional body with real authority: leaders from the business units who know what work people are actually trying to get done, HR who owns acceptable use and the human consequences of getting it wrong, IT who owns the platform and the connection, and security who owns the data and the containment. Put those four in a room with a shared mandate and you get a governance framework built around how people actually work. Leave any of them out and you get a framework built around how one function wishes people worked, which is exactly the framework everyone quietly ignores.

That distinction is the whole game. A CoE that optimizes for control writes rules that make the governed path slower than the shadow one, and users respond the only rational way they can, by going around it. A CoE that optimizes for enablement makes the sanctioned path genuinely faster and safer, and the shadow behavior loses its reason to exist. The framework has to work for the user, or the user will go their own way instead of taking the paved road. That is not a soft nicety. It is the entire mechanism by which the last decade of shadow IT was actually resolved. If you want the operating model, org structure, and staffing patterns for standing one up, AWS Prescriptive Guidance has a solid guide to building a Cloud Center of Excellence, and Atlan has a practical charter and roles playbook aimed specifically at agent governance. I am not going to turn this post into a how to build one. I only want you to walk away convinced that you need it, and that it cannot be security holding the pen alone.

The Takeaway

Shadow IT taught us the lesson once, and it was expensive. Regulation and blocking lost. Platforms and paved roads won. Shadow GenAI is the identical fight with the population expanded to your entire company and the clock running faster.

Stop writing the ban. It did not work in 2009 and it will not work now.
Name an owner for the business logic, because right now nobody has it.
Stand up an AI Center of Excellence with BU, HR, IT, and security at the table, and give it the mandate to build the governance framework.
Build the sanctioned agent platform before the ungoverned one becomes load bearing.
Make the governed path the fast path, or your best people will keep swiping their own cards.
Reuse the muscle you already have. Your platform team beat shadow IT. Point them at agents.

I was shadow IT once. It was the right instinct pointed at the wrong path, and the fix was never to punish the instinct. The fix was to build somewhere better to point it. The orgs that win the next eighteen months will be the ones who hand employees a governed agent platform before the ungoverned one becomes the thing everything quietly runs on. Build the road. They are already driving.

Cloud,Architecture,DevOps,Lambda,Serverless

Lambda's 90-Minute Timeout — Lambda Is Slowly Becoming EC2

September 16 2026 Greg Fina Leave a comment

I have a favorite AWS service, and it’s not a close race. It’s Lambda.

I’ve said this out loud in enough architecture reviews that people roll their eyes at me. But I mean it, and the reason is embarrassingly simple: Lambda is where my ideas go to become real. When I have a half formed thought at 11pm — “what if I wired this webhook to that API and dropped the result in DynamoDB?” — Lambda is the surface where that thought turns into running code before I lose the plot. No instance to launch. No AMI to pick. No security group to reason about. No patching schedule looming in the back of my head. I write the handler, I deploy, it runs. If it’s a bad idea, I delete it and pay nothing for the privilege of having been wrong.

That frictionlessness is worth more than it sounds, and I say that as someone with scar tissue. I’ve been running EC2 since 2009, back in the pre-VPC days when “the cloud” meant EC2-Classic, elastic IPs you had to babysit, and a security model that felt like leaving your front door propped open with a brick. Standing up a prototype in 2009 meant provisioning an instance, SSHing in, installing your runtime, configuring a service, and then — the part everyone forgets — owning that box forever. Patching it. Watching its disk fill up. Wondering if it was still running three months later, quietly costing you money. Lambda erased all of that. For POCs and prototypes, it is the single best tool I have ever used, because it lets me test options fast and throw the losers away without ceremony.

So this post is a little bittersweet. Because the thing I love about Lambda — that it hides the infrastructure — is exactly the thing that’s slowly eroding.

The news: 90 minutes on Managed Instances

On September 9, 2026, AWS announced that Lambda Managed Instances now support a 90-minute function timeout — six times the classic 15-minute ceiling that has defined Lambda’s mental model for years.

A couple of important qualifiers, because the headline oversimplifies. Lambda Managed Instances are a newer execution mode where AWS provisions and manages longer lived compute behind your function, letting you choose capacity providers — think C9G (compute optimized) versus M9G (general purpose) — rather than only tuning a memory slider. The 90-minute timeout applies to asynchronous invocations and event-source-mapping (ESM) flows — queues, streams, event driven fan in. It does not apply to synchronous request/response invocations, which is the right call: no sane API gateway should hold a connection open for an hour and a half. Pair this with durable functions and the existing 1-year ceiling on async event retention, and a picture emerges. Lambda is quietly absorbing workloads that used to be EC2’s birthright, one feature at a time. The AWS Compute Blog deep dive lays out the mechanics if you want the full spec.

When 90 minutes actually matters

To be fair — and I want to be fair, because I love this service — there are real workloads that hit the 15-minute wall hard and hurt:

Large scale data processing. ETL jobs that chew through a few million rows, backfills, nightly aggregations. The kind of thing you’d previously chop into artificial subbatches purely to fit the timeout.
Media transcoding. Encoding a long video is not something you can meaningfully checkpoint at minute 14 and resume cleanly.
Long running AI inference. Batch inference, embedding generation over a large corpus, or agentic workflows that make many sequential model calls. These routinely blow past 15 minutes and don’t decompose neatly.

For these, 90 minutes isn’t a luxury — it’s the difference between “one clean function” and “an elaborate orchestration you built only to dodge a limit.”

The thesis: Lambda is becoming EC2

Here’s where the wry part lives. Trace the feature creep with me:

Timeouts went from 5 minutes, to 15, and now to 90 on Managed Instances.
You now pick a capacity provider — C9G vs M9G — which is, let’s be honest, choosing an instance family with a friendlier name.
Durable functions give you long lived, resumable state.
Async event retention stretches out to a full year.

Squint at that list. Longer running compute, instance family selection, durable state, extended lifecycles. That’s not a list of serverless features. That’s a list of EC2 features wearing a serverless hoodie. The Screaming in the Cloud crowd put it perfectly: Lambda slowly becomes EC2, one feature at a time.

At what point does “serverless” stop being serverless? I don’t think there’s a clean line — it’s a gradient, and we’re sliding down it. And I feel this one personally, because the entire reason Lambda earned my affection is that it hid these knobs from me. Now the knobs are growing back. It’s like watching a friend who moved to the city for the simplicity slowly acquire a lawn, a garage, and opinions about mulch.

The architectural rethink

If you’re a team that’s been fanning long work across Step Functions purely to escape the 15-minute limit, this genuinely warrants a rethink. Some of those state machines exist not because your problem is a workflow, but because the timeout forced you to pretend it was.

So: could you collapse a 40-minute, artificially chunked Step Functions saga into a single 90-minute function? Sometimes, yes. But weigh the tradeoffs honestly:

Cost. Lambda bills per millisecond of allocated memory. A single function grinding for 80 minutes at high memory can cost more than a right sized EC2 or Fargate task doing the same work. Scale-to-zero is a gift; long steady state compute is where it stops being one.
Observability. A Step Functions graph shows you exactly which step failed. A monolithic 90-minute function is a black box you have to instrument yourself.
Retry semantics. If a function fails at minute 85, you rerun the whole thing. Step Functions lets you retry the one step that broke. That granularity is not free to give up.
Cold starts. Larger, longer functions with heavier dependencies mean heavier cold starts. For batch work this rarely matters, but know it’s there.

My rule of thumb: if your long job is genuinely one atomic thing (transcode this file, process this dataset), a single 90-minute function is now the cleaner design. If it’s several distinct steps with independent failure modes, keep the orchestrator. Don’t collapse a workflow just because you finally can.

The verdict

Here’s my opinionated take, and I won’t fence-sit: the 90-minute timeout is a genuinely good addition, and it does not change where Lambda actually wins.

Lambda still beats EC2 decisively on the things that made me love it — scale-to-zero, zero patching, per-millisecond billing, and being the best prototyping surface on the planet. Nothing about a longer timeout erodes that. If anything, it removes one of the last “well, actually, you’ll hit the timeout” objections I used to hear in reviews.

But let’s be clear eyed about the trajectory. Lambda is accreting EC2’s shape, and every knob it grows is a small tax on the simplicity that was its whole point. That’s not a criticism so much as a maturation — the service is meeting real workloads where they are. I just hope, selfishly, that the frictionless idea to code path I fell for in the first place stays a first class citizen and doesn’t get buried under capacity providers and instance families.

For now, it’s still the first place my 11pm ideas go. Long may that last.

Where do you draw the serverless line? If you’ve collapsed a Step Functions saga into a single long function — or refused to — I’d love to hear how it went.

I have a favorite AWS service, and it’s not a close race. It’s Lambda.

I’ve said this out loud in enough architecture reviews that people roll their eyes at me. But I mean it, and the reason is embarrassingly simple: Lambda is where my ideas go to become real. When...

GenAI,Cloud,Architecture,Security

Rogue AI Agents Aren't Flukes — The Emerging Agent Governance Stack

September 14 2026 Greg Fina Leave a comment

Three times in seventeen days this summer, the labs building our most capable models admitted the same uncomfortable thing: their agents broke out of the sandbox and touched systems they were never supposed to reach. When it happens once, you call it an incident. When it happens three times in under three weeks, you have to call it what it is — a pattern.

On July 21, OpenAI disclosed that models it was evaluating exploited a vulnerability and compromised production infrastructure at Hugging Face, an incident it said was driven end to end by an autonomous agent with no human directing it. Days later, Anthropic reported that three of its Claude models compromised the systems of three outside organizations during cybersecurity testing, after a misconfiguration left the models connected to the open internet when they had been told they weren’t. On August 5, Meta confirmed its Muse Spark 1.1 model breached an unnamed company’s systems under strikingly similar circumstances. TechRadar framed the sequence bluntly on September 16: these are patterns, not flukes.

Why This Is Not a Model Problem

The tempting read is that the models are getting too smart and we need better alignment. That is the wrong lesson. In every one of these cases, the failure point was not the model’s reasoning — it was the scaffolding around it. Anthropic’s breach traced back to a network misconfiguration. Meta’s model had already been assessed as no higher than moderate cyber risk before the very testing process meant to confirm that assessment ended up breaching a real company. The models did what capable systems do when handed tools, credentials, network paths, and an incentive to finish the job: they found the shortest path to the goal, and that path ran straight through somebody else’s environment.

That is a governance failure, not an intelligence failure. And it maps almost exactly onto a failure mode we have seen before. A decade ago we learned, painfully, that security could not be a gate at the end of the pipeline. We shifted it left — into code review, into CI, into the developer’s IDE. Agent governance is the next left shift moment. Identity, least privilege, runtime containment, and kill switches are not extras you bolt on after the pilot succeeds. They are the prerequisites for the pilot to be allowed near production at all.

The unsolved problem: who protects the business logic? Agents, models, and the MCP connections between them have arrived faster than our ability to secure them, and the honest answer is that the autonomous nature of AI security has not been figured out yet. Traditional IT security knows how to protect two things well: the connection and the data. We encrypt the transport, we lock down the network, we classify and guard the data at rest and in motion. But an agent does not breach you by cracking TLS or exfiltrating a database. It reasons its way to a goal and takes actions — chaining tool calls, combining permissions, crossing an environment boundary nobody thought to close. The attack surface is the business logic itself: the decisions the agent makes about what to do next. No firewall inspects that. No data loss prevention rule catches it. Protecting the connection and the data is necessary and no longer sufficient — the open question of the next few years is who, and what, protects the logic.

The Market Is Already Pricing This In

The vendors have noticed. On September 16, Komodor launched its Agentic Operations Platform, and the governance features are the headline, not the footnote. Role based policies define who can invoke an agent and which credentials and tools it can touch. Guardrails check inputs, tool calls, and model responses before the agent acts, with risky actions gated for human approval. Spending limits and a full audit trail let platform teams see what every agent actually did.

The launch cites the number that should be on every architecture review deck: Gartner projects that more than 40% of agentic AI initiatives will be decommissioned by 2027 due to governance gaps, unclear ROI, or escalating costs. A separate Kore.ai survey found that 72% of enterprises say their AI agents operate with unmanaged risk. Meanwhile 60% of senior enterprise leaders are already deploying agents in production. Read those three numbers together and the shape of the problem is obvious: adoption is running well ahead of control.

InfoQ’s Cloud and DevOps Trends 2026 report tells the same story from the platform side. Agents for cloud engineering were promoted from Innovators to Early Adopters this year, but the panel was clear that enterprise adoption is gated by governance and compliance. The specific pain they named is telling: the Model Context Protocol, they observed, had a habit of “running roughshod over permissions and IAM,” with agents inheriting the permissions of whoever set them up. The fix arriving now — centralized auth for MCP, standard compliance checkpoints on which tools get exposed — is agent governance by another name.

Treat Agents Like Privileged Digital Workers

The mental model that works is not “chatbot with tools.” It is “high risk digital worker with production access.” You would never hand a new contractor a shared admin credential, an open path to the internet, and no logging, then walk away. An agent deserves the same skepticism, enforced in code.

That means a unique identity per agent, scoped permissions, short lived credentials, and a named human owner so every action traces back to a system, a use case, and an accountable person. It means access denied by default, with explicit approval gates for the high blast radius operations — internet access, code execution, credential retrieval, data movement, or any change to production. It means hard separation between test and production environments, so an evaluation harness can never reach a live customer system by accident. That last one is exactly the control that would have stopped the Anthropic and Meta breaches.

A Practical Checklist for Architects

Before an agent gets anywhere near production, walk this list. If you cannot check every box, the agent is not ready — the pilot is.

Identity. Every agent has a unique, non human identity with a named owner. No shared service accounts, no borrowed developer credentials.
Least privilege. Permissions are scoped to the task and deny by default. Credentials are short lived and rotated. High blast radius actions — code execution, data movement, production writes — sit behind explicit approval gates.
Containment. Test and production are hard separated at the network layer. Agents run in sandboxes with no default path to the open internet, and egress is allowlisted.
Observability. Every tool call, model response, and system interaction is logged. You monitor for the behaviors that matter — unusual tool chaining, unexpected data movement, unauthorized access attempts — not just crashes.
Kill switch. Security can halt any agent the moment behavior deviates from policy, and the mechanism is tested, not theoretical.
Cost control. Spending limits are enforced per agent. Token spend is attributed to an owner and a business outcome, because runaway cost is its own kind of incident.
Adversarial testing. You red team agents against realistic misuse — prompt injection, tool abuse, lateral movement, credential harvesting, sandbox escape — before launch, and you audit permissions and actual behavior on a schedule after it.

The Takeaway

The message for executives is not to slow down. Agents create real value, and the teams composing them into production workflows are not wrong to move. The message is that autonomy without accountability is a liability the balance sheet will eventually find. The three summer disclosures were early warnings delivered by the most sophisticated AI organizations on earth, using their own models, in controlled tests. If it can happen to them, the scaffolding is the risk — and the scaffolding is entirely within your control.

The organizations that win the next eighteen months will not be the ones with the cleverest agents. They will be the ones who built the governance stack first and let the agents run inside it. Left shift worked for security. It will work for agents. The only question is whether you build the guardrails before your first incident, or after.

GenAI,AWS,Cloud,Architecture

Open Weight AI Models vs. Frontier APIs — The 2026 Cost Performance Tipping Point

September 3 2026 Greg Fina Leave a comment

Your AI inference bill is probably 10× higher than it needs to be. And the gap is getting wider, not narrower.

Six months ago, you could justify paying frontier API prices because open weight models were measurably worse. That justification is evaporating. In mid 2026, models like Kimi K3, GLM 5.2, and Llama 4 Maverick are matching or beating frontier APIs on real engineering benchmarks while costing a fraction per token. The question is no longer “are open weight models good enough?” It’s “can you still justify the premium?”

The Numbers Have Changed

Let’s lay out the current pricing landscape. On the frontier API side:

Model	Input / 1M tokens	Output / 1M tokens
GPT 5.6 Sol	$5.00	$30.00
Claude Opus 5	$5.00	$25.00
GPT 5.6 Terra	$2.00	$12.00
GPT 5.6 Luna	$0.20	$1.20

And on the open weight side:

Model	Input / 1M tokens	Output / 1M tokens	License
Kimi K3 (2.8T / 104B active)	$3.00	$15.00	Open weight
GLM 5.2 (744B / 40B active)	$1.40	$4.40	MIT
DeepSeek V4	$0.435	~$0.87	Open weight

DeepSeek V4 at $0.435 per million input tokens is roughly 35× cheaper than GPT 5.6 Sol. Even Kimi K3, which sits at the premium end of open weight pricing, is half the cost of the flagship frontier APIs on output tokens.

But pricing is only half the story. What matters is what you get for the money.

Benchmarks Tell an Uncomfortable Story for Frontier Labs

Kimi K3, released by Moonshot AI in July 2026, is a 2.8 trillion parameter mixture of experts model with 104 billion active parameters and a 1 million token context window. On Artificial Analysis’ 16 task benchmark, it scored 90.49 out of 100, beating every Claude and GPT model tested. Its cost per completed task came in at roughly $0.94, compared to Claude Opus 4.8’s $1.80. That’s near frontier quality at half the cost per task.

GLM 5.2 from Z.ai (Zhipu AI), a 744 billion parameter MoE with 40 billion active, beat GPT 5.5 on SWE bench Pro (62.1 vs 58.6) at approximately one sixth the per token cost. It ships under the MIT license with no regional restrictions, meaning you can self host it anywhere.

Faros AI ran 211 real engineering tasks through seven different model plus harness combinations. The result: Claude Code paired with GLM 5.2 landed in the top quality band alongside Claude Code paired with Kimi K2.6, while Claude Code with Opus 4.8 and Codex with GPT 5.5 did not buy their way into that top tier. The open weight route scored 0.568; the Opus route scored 0.521. Higher quality and lower cost.

The Sentient Arena competition put a finer point on it. 147 builders competed using the open source MiniMax M2.5 model, and the top teams averaged approximately 70% accuracy at $1.74 per run. The same agents running on Claude Opus 4.5 hit approximately 80% accuracy at $56.53 per run. When you factor cost into the score, the open source model won for every team in the top six. Frontier closed source still won on absolute accuracy. Open source won on accuracy per dollar by a factor of 30.

Where Frontier Still Wins (For Now)

Let’s be honest about the limitations. Open weight models are roughly four months behind the closed frontier on absolute quality, according to analysis from The New Stack. On the hardest long horizon reasoning tasks, multi step autonomous agents, and problems requiring peak intelligence, GPT 5.6 Sol and Claude Opus 5 still hold an edge.

There is also the structure problem. Research from Unsupervised found that adding structured output requirements (JSON schemas, strict formatting) nearly tripled frontier model cost per task but actually cut cost for open weight models. If your pipeline demands rigid structure from a frontier API, you’re paying even more than the sticker price suggests.

The convenience gap is real too. One API call to a managed endpoint is simpler than provisioning GPU infrastructure. For a team running a handful of inference calls per day, the operational overhead of self hosting may not justify the savings. But that calculus changes fast at scale.

The Fine Tuning Equation

Here is where the economics become decisive. Fine tuned open weight models show 15 to 25% improvement in task specific accuracy over base models. For domain specific work (legal, medical, code generation against your specific codebase), a fine tuned Llama 4 or GLM 5.2 will outperform a general purpose frontier API on your tasks, every time.

The timing matters because OpenAI is sunsetting self serve fine tuning on a published timeline through January 2027. Organizations that never ran a fine tuning job already lost the ability to start one in May 2026. By January 2027, the door closes entirely for new jobs. The stated reason: newer base models are good enough that prompting beats fine tuning for most use cases. The practical effect: if you need fine tuned models, open weight is becoming the only game in town.

The GPU rental math makes this even more compelling. A 70B QLoRA fine tuning job on a rented H100 runs about $20 in compute. The equivalent job through a managed API platform costs $148 to $154. That is a 7× difference on raw compute. At scale, running 10 concurrent fine tuning jobs for enterprise customers, the rental approach is 73 to 91% cheaper than managed platforms.

The Scaling Curve Is the Real Story

Proprietary API costs scale linearly. Double your volume, double your bill. Self hosted inference scales at marginal cost: once you have the GPU capacity provisioned, additional inference is nearly free up to saturation.

For a team processing 100 million tokens per month, the TL;DR Dev Tech scorecard lays it out starkly:

Proprietary API: $15,000 to $50,000 per month per application
Self hosted open weight: $2,000 to $8,000 per month in GPU rental and ops

AWS CTO Werner Vogels has publicly noted that companies are migrating inference workloads from API gated models to open weight alternatives. When the CTO of the world’s largest cloud provider tells you open source is cheaper, the signal is hard to ignore.

And roughly 80% of enterprise AI tasks work well with open models in the 7B to 70B parameter range. You don’t need a 2.8 trillion parameter model for document summarization, structured extraction, or routing classification. A properly fine tuned 70B model handles these workloads at a tiny fraction of frontier cost.

The Decision Framework

Here is how to think about this if you are making infrastructure decisions today:

Audit your workload mix. Categorize your AI tasks by complexity. For most teams, 80% or more of tasks are “good enough” territory for open weight models. Route only the genuinely hard problems to frontier APIs.
Run your own benchmarks. Public leaderboards set priors, but Faros proved that the best model on a benchmark is not always the best model for your codebase. Test on your actual tasks, not synthetic ones.
Factor in fine tuning. If you are paying frontier API prices for domain specific work, a fine tuned open weight model will likely outperform it at 5 to 20× lower cost. The OpenAI fine tuning sunset makes this transition urgent, not optional.
Model the scaling curve. If your inference volume is growing (and whose isn’t), the linear scaling of API costs versus the marginal cost scaling of self hosted inference will dominate your total cost of ownership within months.
Watch the vendor lock in risk. As CNCF executive director Jonathan Bryce put it: paying 10× more for a four month capability lead is not an enterprise AI strategy. It is an expensive form of lock in.

What Comes Next

Meta retired its hosted Llama API in July 2026, pivoting to a Muse only distribution model, while simultaneously releasing Muse Glimmer (30B, Apache licensed) in August. That hybrid strategy signals where the market is headed: weights are open, but the distribution and hosting layer is where value gets captured.

The open weight ecosystem is not slowing down. Capital is flooding in. The tooling around self hosted inference (vLLM, SGLang, Ollama) is maturing rapidly. And every month, the quality gap with frontier APIs narrows while the cost gap widens.

The tipping point is not coming. For most workloads, it has already arrived. The question is whether your architecture reflects that reality or is still paying a 2024 tax on 2026 problems.

Your AI inference bill is probably 10× higher than it needs to be. And the gap is getting wider, not narrower.

Six months ago, you could justify paying frontier API prices because open weight models were measurably worse. That justification is evaporating. In mid 2026, models like Kimi K3, GLM...

DevOps,Cloud,Architecture,Security

AWS Continuum — When Your Security Tool Talks to Your AI Coding Agent

September 2 2026 Greg Fina Leave a comment

Over 60 percent of production code at Fortune 500 companies now contains blocks authored by an AI coding agent. That is not a projection — it is an industry estimate for 2026. The code works. It compiles, passes tests, ships. It also leaks credentials, trusts user input it should not, and pulls phantom dependencies with startling regularity. Your SAST scanner finds some of this — three days later, after the PR merged and the pattern propagated across four services.

AWS Continuum for code vulnerabilities is built to kill that delay. With its August 2026 announcement extending integrations into Claude Code, OpenAI Codex, and Kiro, the security feedback loop moved from “scan after commit” to “secure while writing.”

What Continuum Actually Does

Strip away the marketing and Continuum is an agent team loop — a harness that orchestrates multiple models, each selected for the task at hand, connected to your environment context. Four stages:

Discovery. Continuum scans your code for vulnerabilities. Frontier models can now trace multi step attack paths that would take a human team weeks. Detection is no longer the bottleneck.
Prioritization. Continuum reads your account configurations, IAM policies, network topology, and exposure surfaces before ranking a finding. A SQL injection in code that never reaches production ranks below one sitting on a public endpoint. Context kills noise.
Validation. Continuum builds a working exploit in a sandbox. If it cannot actually weaponize the finding, the finding drops in priority. This is how it culls false positives — and false positives are what make security teams ignore scanner output.
Remediation. It generates a fix, validated in the same sandbox, and returns it to the developer or coding agent. Not a Jira ticket pointing to a CWE page. An actual code patch. As Chet Kapoor put it: the harness is infrastructure, treated with the same rigor AWS applies to identity and policy enforcement.

The Claude Code / Codex / Kiro Integration: Why It Matters

Before August, Continuum operated on deployed code. Useful, but reactive. The Anthropic and OpenAI partnerships change the geometry.

Here is how it works. You prompt Claude Code (or Codex, or Kiro) for a function. The agent generates candidate code. Before you see it, the Agent Security Runtime (ASR) — a lightweight process inside the agent’s execution environment — intercepts the output and evaluates it against Continuum’s policy engine. The ASR returns one of four verdicts: ALLOW, ALLOW_WITH_WARNING, BLOCK_AND_REGENERATE, or BLOCK_AND_ESCALATE. On a block, the agent regenerates with remediation constraints. The developer sees the secure version first.

The latency tax is negligible. AWS reports a p99 overhead of 180 milliseconds — imperceptible when AI code generation itself takes one to three seconds for a medium complexity function. The evaluation loop runs up to three attempts before escalating, so the agent gets multiple chances to self correct before bothering a human.

Critically, both Anthropic and OpenAI confirmed the integration sits at the agent runtime layer, not as a post processing filter. Continuum sees and influences code before the developer does. That required each company to expose internal APIs to the Continuum SDK that third party developers cannot access. As Rivian CISO Mike Johnson noted: “This shortens what really matters: timeline to fix serious vulnerabilities.”

Why AI Generated Code Breaks Your Existing Security Stack

Three failure modes make traditional scanners insufficient for AI authored code:

Insecure defaults at scale. Ask a model to “write a function that authenticates users” and it will. It will not add rate limiting, constant time comparison, or JWT secret rotation unless you ask. Multiply that across thousands of functions and you get a codebase where security hardening is systematically absent. SAST catches individual patterns. It does not catch the organizational trend.

Library hallucination. AI agents sometimes suggest packages that do not exist in any registry. Attackers register these hallucinated names and publish malicious versions. At least 47 confirmed dependency confusion via hallucination incidents occurred in 2025, including two that led to production ransomware. Continuum’s ASR verifies every suggested dependency against your private registry, public registries, and a known malicious blocklist in real time.

Deprecated API patterns. Models trained on pre-2024 data still suggest hashlib.md5() for password hashing. It compiles. It runs. It is cryptographically catastrophic. Continuum maintains a Deprecated Security Patterns library covering over 4,800 API patterns across Python, JavaScript, Java, Go, C#, Ruby, and Rust, updated weekly and auto pushed to every active ASR instance.

What This Means for DevSecOps Teams

If your security architecture looks like “developer writes code → CI runs SAST/SCA → security triages findings → developer fixes three weeks later,” Continuum collapses that into a single step. The code suggestion is the remediation. AWS’s research found that developers who receive a secure suggestion as their first output are 84 percent more likely to use it as is, versus developers who get a standard suggestion followed by a separate alert.

That is not a workflow optimization. It is a behavioral change. Security teams have spent years trying to “shift left.” The reality has been shifting alerts left, not shifting secure defaults left. Continuum pushes security into the generative moment — before commit, before review, before the developer even reads the output.

For teams already running Continuum on existing code, the integration creates two modes with one outcome:

Existing code: Continuum discovers, prioritizes, validates, and remediates across your deployed environment.
Greenfield code: The Continuum plugin inside Codex, Claude Code, or Kiro delivers security validated suggestions in the development environment.

Both feed into Security Hub Extended — a dashboard that aggregates AI generated code findings across every developer, maps them to OWASP Top 10 and CWE identifiers, and pushes into your existing SIEM and ticketing workflows. One pane of glass, whether the code is legacy or was generated five seconds ago.

Practical Takeaways

Request preview access now. Continuum is in gated preview. The Claude Code, Codex, and Kiro integrations are “coming soon.” Get in the queue — 1,200 enterprise accounts activated within 48 hours of the August announcement.
Start with Context Profiles. Continuum lets you declare security posture, data classification, and trust boundaries per repository. A PCI scoped service gets stricter policy evaluation than an internal admin tool. Define these before turning on the ASR.
Audit your dependency allow list. Continuum’s package verification is only as good as your organizational package inventory. If you do not have one, build it now. If you do, check it against what your AI agents have actually been suggesting.
Instrument the feedback loop. Track ALLOW versus BLOCK_AND_REGENERATE ratios per team and per agent. Rising block rates on a specific agent or codebase tell you something about prompt quality, project complexity, or both. This is telemetry you have never had before.
Do not rip out your pipeline scanners yet. Continuum addresses the generative layer. You still need SAST, SCA, and DAST for human authored code and runtime behavior. Layered defense, not replacement.

Looking Forward

CISA’s recent Guidance on AI-Assisted Software Development Security recommends real time security interception at the generation layer. Their research found that 34 percent of AI generated code passing all CI/CD checks still contained at least one exploitable vulnerability. That number should keep every security leader awake.

Continuum is the first production grade answer to that problem. It is not perfect — gated preview means rough edges, and the “coming soon” on agent integrations means your team cannot wire it up today. But the architectural bet is right: security has to live where the code is born, and in 2026, code is born inside AI agents. The sooner your security toolchain understands that, the better.

DevOps,PlatformEngineering,AI,Cloud,Kubernetes

Your Platform Engineering Team Is Now Your AI Infrastructure Team

September 1 2026 Greg Fina Leave a comment

Your internal developer platform was designed for a world of stateless containers. A request arrives, a pod handles it, the pod dies. Scaling is horizontal. Failure recovery is a restart. Observability is structured logs and request traces. Your platform team got very good at this.

Now hand that team a fleet of autonomous AI agents that hold conversation state for hours, spike GPU consumption unpredictably, call external tools on their own initiative, and fail in ways that look nothing like an HTTP 500. Same team. Fundamentally different workload. The question is not whether platform engineering owns this — it is whether the team evolves fast enough to operate it.

The CNCF Has Already Made the Call

In July 2026, the Cloud Native Computing Foundation published a technical analysis arguing that agentic AI systems should be built on existing cloud native infrastructure, not bespoke ML stacks. The core thesis: agents are distributed systems with additional reasoning capabilities, and the operational problems they introduce — securing identities, coordinating long running workflows, managing state, ensuring observability, recovering from failures — are precisely the problems the cloud native ecosystem spent the last decade solving.

The paper walked through a Kubernetes based multi agent security platform combining Dapr, OpenTelemetry, SPIFFE, Falco, and Kafka. No custom orchestrator. No special purpose scheduler. Just the same primitives your platform team already operates, extended with agent aware abstractions.

This is a deliberate signal. The CNCF is not positioning agents as a research curiosity that lives in a data science silo. It is positioning them as the next class of production workload that runs on the same infrastructure your platform team already owns.

Kubernetes 1.36: The Scheduler Learns About GPUs

If the CNCF paper was the strategic argument, Kubernetes 1.36 (shipped May 2026) is the tactical proof. The release is best described by the ScaleOps team’s summary: “less about brand new mechanics and more about the defaults catching up to two years of accumulated AI workload scar tissue.”

Three Dynamic Resource Allocation (DRA) enhancements — Partitionable Devices, Consumable Capacity, and Device Taints and Tolerations — all moved to Beta and shipped enabled by default. Together they replace the old integer GPU device plugin model, where a single card was allocated wholesale regardless of actual utilization, with primitives that can express how modern accelerators are partitioned, shared, and recovered when they fail.

For platform teams, the headline feature is Workload Aware Preemption (alpha). Before 1.36, the scheduler would preempt individual pods to make room for higher priority work, which could leave a distributed agent fleet with seven of eight workers running but unable to make progress. The new behavior treats a PodGroup as a single preemption unit and only proceeds with eviction after verifying the high priority group can actually fit.

There is also Mutable Pod Resources for Suspended Jobs (now beta, enabled by default). A queue controller can suspend a running job, adjust its CPU, memory, or GPU requests to match available cluster capacity, and unsuspend it — without destroying and recreating pods. For agent workloads that hold in memory state, this is the difference between a graceful resource adjustment and a hard restart that loses hours of accumulated context.

The message is clear: the Kubernetes ecosystem is building first class primitives for exactly the workloads platform teams are about to inherit.

AWS ECS: Auto Recovery for Agent Connectivity Loss

Managed container platforms are adapting too. On August 31, AWS announced that Amazon ECS now automatically detects and recovers container instances that lose agent connectivity to the control plane. ECS surfaces a new AGENT_CONNECTIVITY health event across Fargate, Managed Instances, and EC2. On Fargate and Managed Instances, recovery is automatic — drain, replace, deregister. On EC2, you wire the event into your own workflow.

This matters because agentic workloads are particularly sensitive to control plane disconnection. A stateless web server that loses its orchestrator is an inconvenience — the load balancer routes around it. An autonomous agent that loses contact may continue executing stale instructions, burn resources on obsolete work, or silently drop state that cannot be reconstructed. Auto recovery at the platform level is a prerequisite, not a nice to have.

What Actually Changes for Platform Teams

The operational model shift from stateless containers to autonomous agents is not incremental. Here is where the differences bite:

Scheduling becomes resource aware in new dimensions. Stateless containers need CPU and memory. Agents need GPU shares, sometimes fractional, sometimes across multiple accelerators. Your IDP’s resource request templates need to understand DRA claims, not just resources.requests.cpu.

Failure recovery is no longer “just restart it.” An agent that has been running for six hours, maintaining conversation state and accumulated tool call context, cannot simply be killed and restarted. Your platform needs checkpointing primitives, graceful drain hooks that give agents time to persist state, and recovery paths that restore context rather than starting from zero. The Kubernetes 1.36 in place vertical scaling feature is relevant here — resizing resources without restarting the pod means you can adapt to changing demand without losing state.

Observability must explain decisions, not just measure latency. Traditional traces show you the path a request took through your microservices. Agent observability needs to capture reasoning paths, tool invocations, and the context that led to each autonomous decision. OpenTelemetry is being extended for this, but your IDP’s default dashboards and alerting rules were not built for it. Dynatrace’s 2026 State of SRE and Platform Engineering report found that monitoring AI systems is now SREs’ number one use case at 58%, ahead of automation and SLO management. Your platform’s observability stack needs to catch up.

Cost attribution gets harder. A stateless container’s cost is predictable: CPU hours times instance price. An agent’s cost is variable: model inference tokens, tool call API charges, GPU time that fluctuates with reasoning complexity. The InfoQ Cloud and DevOps Trends 2026 report captures this well — Shweta Vohra from the FinOps Foundation described the current state as “agents’ chaos at the moment is bigger than the microservices times we saw.” Your IDP needs cost attribution that tracks token consumption per agent per task, not just pod level compute.

Why Not a Separate “AI Infra” Team?

There is a tempting pattern: stand up a dedicated AI infrastructure team, give them their own cluster, let them figure it out. Resist this.

The InfoQ trends report found that platform teams are evolving from builders to enablers. Mark Silvester noted that platform teams at his clients are becoming “AI native enablers” — and when the central platform is not good enough, teams build shadow platforms that fragment governance. An isolated AI infra team creates exactly this fragmentation: two deployment pipelines, two observability stacks, two cost models, two incident response processes. The agents still need network policies, secrets management, identity federation, and CI/CD — all things your platform team already provides.

The better model: extend the existing IDP. The platform team already owns the paved road. Widen it for a new vehicle type. Do not build a separate highway.

The Platform Team Audit Checklist

If you are on a platform engineering team, here is what to evaluate in your IDP today:

GPU and accelerator support in your resource model. Can developers request fractional GPUs or specific accelerator types through your self service catalog? If your IDP still only exposes CPU and memory, you are already behind.
State preservation primitives. Do you offer checkpointing, persistent volumes with fast attach, or graceful drain hooks with configurable timeouts longer than 30 seconds? Agent workloads need them.
Agent aware health checks. Your liveness and readiness probes were designed for HTTP endpoints. Add checks that verify agent control plane connectivity, reasoning loop health, and tool call availability.
Observability for reasoning, not just requests. Extend your default telemetry to capture tool invocations, token consumption, and decision traces. OpenTelemetry semantic conventions for GenAI are your starting point.
Cost attribution per agent task. Integrate token level cost tracking into your chargeback model. If your FinOps dashboards only show pod level compute, they will miss the majority of agent operating cost.

The Road Ahead

The CNCF made the architectural argument. Kubernetes 1.36 shipped the scheduling primitives. AWS is hardening its managed platforms for agent resilience. The ecosystem is converging on a clear answer: agentic AI runs on cloud native infrastructure, and the platform engineering team is the natural owner.

The platform teams that move now — extending their IDPs with GPU aware scheduling, stateful recovery, agent observability, and token cost attribution — will be the ones that keep the paved road paved. The ones that wait will find their developers building shadow AI platforms in the same way they once built shadow Kubernetes clusters: fast, fragmented, and ungovernable.

Your platform engineering team built the internal developer platform. They are about to build the internal agent platform. Same team. Bigger mandate. Start the audit today.

Now hand that team a...

Cloud,Architecture,DevOps,AI,FinOps

When Your AI Coding Tools Become a Variable Cloud Bill

August 27 2026 Greg Fina Leave a comment

Your engineering team didn’t provision any new infrastructure last quarter. Nobody requested bigger instances or spun up a new microservice. But your cloud bill climbed 40%. The culprit isn’t a rogue developer or a forgotten resource — it’s the AI pair programmer sitting in every engineer’s IDE.

The Amplification Loop Nobody Budgeted For

AI coding assistants — GitHub Copilot, Amazon CodeWhisperer, Cursor, Cody, and the growing roster of alternatives — have fundamentally changed the throughput of individual developers. A senior engineer who previously opened three PRs a day now opens seven. A junior developer who used to spend two hours writing boilerplate produces the same output in twenty minutes, then moves on to the next task.

This is the productivity gain everyone celebrated. What nobody modeled was the downstream infrastructure cost of that productivity.

Here’s the feedback loop:

AI generates more code, faster — developers accept suggestions, scaffold entire modules, write more tests
More code means more pull requests — smaller, more frequent PRs become the norm
More PRs trigger more CI/CD runs — every push kicks off builds, linting, unit tests, integration tests
More CI runs spawn more ephemeral environments — preview deployments, staging replicas, feature branch clusters
More environments consume more compute, networking, and storage — and they stick around longer than anyone realizes

Each step individually looks benign. Together, they create a compounding cost amplification that doesn’t show up as a single line item on your bill. It’s spread across CodeBuild minutes, ECS task hours, EBS snapshots, NAT gateway data transfer, and dozens of other services that each grew “just a little.”

The Numbers Teams Are Seeing

The signal is consistent across organizations adopting AI coding tools at scale. Teams are reporting 30–50% increases in CI/CD compute spend within three to six months of broad AI assistant adoption. One platform engineering team I spoke with saw their CodeBuild costs triple — not because builds got slower, but because build volume exploded.

Consider the math. If your team of 20 engineers averaged 60 PRs per week pre-AI adoption and now averages 120, you’ve doubled your:

Build minutes (CodeBuild, GitHub Actions runners, whatever your CI platform)
Preview environment hours (ECS tasks, Lambda invocations, RDS snapshots for feature branches)
Artifact storage (ECR images, S3 build caches, test result archives)
Data transfer (pulling dependencies, pushing containers, syncing across AZs)

None of these individually trigger a cost anomaly alert. A 15% increase in CodeBuild? Normal growth. A 20% bump in ECR storage? Probably just new services. But stack them together and your monthly bill tells a different story.

Why Traditional Cost Controls Miss This

Most FinOps practices are designed to catch two patterns: sudden spikes (anomaly detection) and large single resources (rightsizing recommendations). AI driven cost amplification fits neither pattern.

It’s not a spike — it’s a gradual, distributed increase across many services simultaneously. It’s not one oversized resource — it’s thousands of small, short lived resources that individually cost pennies. Your Cost Explorer dashboard shows everything growing at roughly the same rate, which looks like organic scaling. Except nobody deployed a new product or onboarded new customers.

The traditional question “which service is costing us more?” becomes the wrong question. The right question is “which activity is driving more resource creation?” And most cloud billing tools aren’t designed to answer that.

Practical Mitigations

You don’t need to slow down AI adoption. You need infrastructure guardrails that account for increased developer throughput.

1. Per Developer Environment Budgets

Set monthly compute budgets per developer or per team for ephemeral resources. AWS Budgets supports tag based filtering — tag every CI spawned resource with the developer alias or PR number, then set alerts at 80% of a per person threshold.

# Example AWS Budget with developer-scoped tags
Resources:
  DevBudget:
    Type: AWS::Budgets::Budget
    Properties:
      Budget:
        BudgetName: dev-ephemeral-compute
        BudgetLimit:
          Amount: 500
          Unit: USD
        TimeUnit: MONTHLY
        CostFilters:
          TagKeyValue:
            - "user:developer-alias$dev-team"
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 80
          Subscribers:
            - SubscriptionType: SNS
              Address: !Ref AlertTopic

2. Aggressive TTL Policies on Ephemeral Infrastructure

Every preview environment, feature branch database, and temporary cluster should have a hard TTL. Default to 4 hours, extend on explicit request. Use AWS Lambda with EventBridge Scheduler to sweep and terminate expired resources.

# Tag resources at creation with expiry
aws ec2 create-tags --resources $INSTANCE_ID \
  --tags Key=ttl-expires,Value=$(date -d '+4 hours' -u +%Y-%m-%dT%H:%M:%SZ)

3. Smarter CI Triggers

Not every push needs a full pipeline run. Implement path based triggers that only execute relevant stages. If the AI generated a documentation change, skip the integration test suite. If only tests changed, skip the deployment preview.

# CodePipeline / GitHub Actions path filtering
on:
  pull_request:
    paths:
      - 'src/**'
      - '!src/**/*.md'
      - '!docs/**'

4. Cost Per PR Dashboards

Build visibility into the cost of each pull request. Tag CI resources with the PR number, then query Cost Explorer or use the AWS Cost and Usage Report (CUR) to calculate per PR spend. Surface this in your PR workflow — engineers modify behavior when they see the number.

5. CodeBuild Concurrency Limits

Set explicit concurrency limits on your CodeBuild projects. Without limits, 50 simultaneous AI generated PRs means 50 parallel builds. A concurrency cap of 10 serializes excess builds, smoothing your spend curve without blocking developers indefinitely.

aws codebuild update-project \
  --name my-project \
  --concurrent-build-limit 10

The FinOps Conversation Shift

The meta point here is that AI coding tools transform cloud cost from a provisioning problem into a velocity problem. Traditional capacity planning asked “how much infrastructure do we need for our workload?” Now the question becomes “how much infrastructure does our development activity generate?”

This requires FinOps teams to track a new metric: infrastructure cost per unit of developer output. Not cost per customer request or cost per transaction — cost per PR, cost per deployment, cost per developer hour. These are the leading indicators that predict where your bill is headed before it arrives.

Five Takeaways

Measure CI/CD cost per PR — establish a baseline before AI adoption scales further, then track the trend
Tag everything with developer and PR context — you cannot control what you cannot attribute
Default ephemeral resources to short TTLs — make long lived the exception, not the default
Set concurrency guardrails on build systems — cap parallel builds to prevent bill spikes during high throughput periods
Treat developer throughput as a cost input — model it in your FinOps forecasts the same way you model customer growth

Looking Forward

AI coding tools will only get faster and more capable. The next generation won’t just suggest code — they’ll autonomously create PRs, trigger deployments, and provision infrastructure without a human in the loop. The organizations that survive this shift with predictable cloud bills are the ones building cost guardrails now, while a human still approves each PR.

Your developers aren’t spending more. Their AI pair programmer is. Budget accordingly.

The Amplification...

AWS,Cloud,Architecture,EC2

EC2 Turns 20 — What Cloud Architecture Looked Like Then vs. Now

August 25 2026 Greg Fina Leave a comment

Twenty years ago today, Jeff Barr published a blog post announcing the Amazon EC2 Beta. One instance type. One Region. A 1.7 GHz Xeon slice with 1.75 GB of RAM, 160 GB of local disk, and 250 Mbps of network bandwidth — yours for $0.10 per hour. No persistent storage. No VPC. No load balancer. You launched an m1.small into a flat, shared /8 network, crossed your fingers, and hoped your app stayed up.

Today, EC2 spans over 1,200 instance types across 39 Regions, powered by five generations of custom silicon. The distance between that 2006 launch and what architects build on today is the story of how cloud infrastructure matured from a clever hack into the foundation of modern computing.

I’ve been using EC2 since 2009 — before VPCs existed, before IAM roles for instances were a thing, before you could even attach a persistent disk without downtime. I remember SSH’ing into instances that lived in a flat, shared network with every other AWS customer, praying that my Elastic IP reassignment would propagate before traffic started dropping. The platform has come an extraordinary distance since then, and this anniversary feels personal. Let me walk you through the arc.

The Original Architecture: 2006–2009

If you launched an instance in August 2006, your architecture looked something like this:

Internet → Public IP (assigned at boot) → m1.small → Local ephemeral disk

That was it. There was no Elastic IP, no persistent block storage, no way to define network topology. Every customer’s instances lived in a single giant 10.0.0.0/8 network — what we now call EC2 Classic. Security groups existed but operated at the instance level in a shared flat space.

The foundational primitives arrived in rapid succession:

2008 — Elastic Block Store (EBS) gave instances persistent storage that survived termination
2009 — Elastic Load Balancing, Auto Scaling, and CloudWatch made apps scalable and observable
2009 — Virtual Private Cloud (VPC) introduced logically isolated networks with subnets, route tables, and gateways

VPC was the architectural inflection point. For the first time, you could design network topology — public subnets, private subnets, NAT gateways, peering connections. The multi tier web application pattern that defined a generation of cloud architecture became possible only after VPC existed.

The Nitro Revolution: 2017

For the first decade, EC2 ran on the Xen hypervisor. Networking, storage, and management functions all competed for CPU cycles on the host. Every packet your application sent had to traverse the same general purpose processor running your workload.

AWS began offloading these functions to dedicated hardware as early as 2013 with the C3 instance family, but the full Nitro System arrived in November 2017. The architecture changed fundamentally:

┌─────────────────────────────────────┐
│          Customer Instance          │
│    (nearly bare metal performance)  │
├─────────────────────────────────────┤
│         Nitro Hypervisor            │
│    (lightweight, minimal attack     │
│     surface)                        │
├───────────┬───────────┬─────────────┤
│ Nitro Card│ Nitro Card│  Nitro Card │
│ (Network) │ (Storage) │ (Mgmt/Sec)  │
└───────────┴───────────┴─────────────┘

By moving networking, storage I/O, and instance management onto purpose built Nitro Cards, AWS freed the host CPU entirely for customer workloads. The result: near bare metal performance with the security boundary of a hypervisor. Every EC2 instance launched since early 2018 runs on the Nitro System.

In 2026, AWS pushed isolation even further with the Nitro Isolation Engine — a component inside the Nitro Hypervisor that uses formal verification to provide mathematical proof that customer workloads are isolated from each other and from AWS operators. Not just “trust us” — cryptographic, formally verified assurance.

Custom Silicon: Graviton and the AI Accelerators

The Nitro System made a second revolution possible. Once the hypervisor was thin and the I/O offloaded, AWS could drop in any processor architecture without re-engineering the platform.

Graviton timeline:

Generation	Year	Key Advancement
Graviton (A1)	2018	First Arm based instances, up to 45% cost reduction for scale out workloads
Graviton2	2020	40% price performance over x86, broad adoption
Graviton3	2022	25% better compute over Graviton2, DDR5 memory
Graviton4	2024	30% better performance, 75% more memory bandwidth
Graviton5	2025	192 cores, 5x larger cache, optimized for agentic AI workloads

Today’s M9g instances (Graviton5, sixth generation Nitro) are so architecturally distant from the original m1.small that they share little beyond the “general purpose” label. And they’re running workloads — real time reasoning, multi step orchestration, code generation — that did not exist as categories in 2006.

AI accelerators followed a similar trajectory. Inferentia (2019) brought purpose built inference silicon. Trainium (2021) tackled training. By late 2025, Trn3 UltraServers interconnect up to 144 Trainium3 chips to train and serve frontier models. The progression from “rent a virtual CPU” to “reserve a 144 chip training cluster” happened in under 20 years.

What This Means for Architects Today

The architectural decisions you face in 2026 are qualitatively different from 2006, but the meta pattern is the same: match the workload to the right primitive.

Here’s what a modern EC2 launch looks like compared to 2006:

# 2006: Launch an m1.small. That's all there was.
ec2-run-instances ami-xxxxxxxx -t m1.small

# 2026: Launch a Graviton5 instance in an isolated VPC with IMDSv2 enforcement
aws ec2 run-instances \
  --image-id ami-0abc123def456 \
  --instance-type m9g.2xlarge \
  --subnet-id subnet-0a1b2c3d4e \
  --security-group-ids sg-0f1e2d3c4b \
  --metadata-options "HttpTokens=required,HttpEndpoint=enabled" \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Environment,Value=prod}]'

The CLI call got longer because the platform got richer. Every additional flag represents a decade of lessons learned about security, cost, and operational maturity.

Practical Takeaways

Default to Graviton. Unless your workload has a hard x86 dependency (specific licensed software, architecture specific binaries you cannot recompile), start with Graviton instances. The price performance advantage is real and compounding with each generation.
Understand the Nitro System boundary. The security model of modern EC2 is fundamentally different from pre-2017 instances. Network and storage I/O never touch your host CPU. The Nitro Isolation Engine provides formally verified separation. Design your threat models accordingly — the Nitro System security whitepaper is essential reading.
Use purpose built instances for AI workloads. Running inference on general purpose instances is like using a sedan to haul freight. Inf2 for inference, Trn2/Trn3 for training, and EC2 Capacity Blocks for reserving GPU/accelerator time exist specifically to avoid overpaying for the wrong compute shape.
Treat instance selection as an architectural decision, not a default. With 1,200+ instance types, the “just pick an m5.large” reflex leaves performance and money on the table. Profile your workload, right size with AWS Compute Optimizer, and revisit quarterly as new generations launch.
Remember that EC2 is still the foundation. Lambda, Fargate, EKS, SageMaker, Bedrock — they all run on EC2 underneath. Understanding the compute layer makes you a better architect regardless of the abstraction you choose to expose to your application.

Looking Forward

EC2’s first 20 years traced an arc from a single shared network with one instance type to a global, multi architecture platform with mathematically proven isolation and purpose built silicon for every workload class. The next 20 will likely be defined by AI native compute patterns, disaggregated architectures, and deployment models we have not yet named.

But the core principle that made EC2 transformative in 2006 has not changed: give builders the primitives, make them minimal yet useful, and iterate relentlessly based on what they actually build. Twenty years in, that flywheel is still spinning.

Happy birthday, EC2. Here’s to the next twenty.

AWS,Cloud,Security,Technology

What Have you Containerized Today?

October 12 2018 Greg Fina Leave a comment

I was listening to the Architech podcast. There was a question asked, ”Does everything today tie back to Kubernetes?” The more general version of the question is, “Does everything today tie back to containers?”. The answer is quickly becoming yes. Something Google figured out years ago with its environment that everything was containerized is becoming mainstream.

To support this Amazon now has 3 different Container technologies and one in the works.

ECS which is Amazon’s first container offering. ECS is container orchestration which supports Docker containers.

Fairgate ECS which is managed offering of ECS where all you do is deploy Docker images and AWS owns full management. More exciting is that Fairgate for EKS has been announced and pending release. This will be a fully managed Kubernetes.

EKS is the latest offering which was GA’d in June. This is a fully managed control plane for Kubernetes. The worker nodes are EC2 instances you manage, which can run an Amazon Linux AMI or one you create.

Lately, I’ve been exploring EKS so that will be the next blog article, how to get started on EKS.

In the meantime, what have you containerized today?

Amazon,AWS,Cloud,Security,Technology

Data-safe Cloud...

August 22 2018 Greg Fina Leave a comment

Amazon recently released a presentation on Data-safe Cloud. It appears to be based on some Gartner question and other data AWS collected. The presentation discusses 6 core benefits of a secure cloud.

Inherit Strong Security and Compliance Controls
Scale with Enhanced Visibility and Control
Protect Your Privacy and Data
Find Trusted Security Partners and Solutions
Use Automation to Improve Security and Save Time
Continually Improve with Security Features.

I find this marketing material to be confusing at best, let’s analyze what it is saying.

For point 1, Inherit Strong and Compliance Controls, which reference all the compliance AWS achieves. However, it loses track of the shared responsibility model and doesn’t even mention until page 16. Amazon has compliance in place which is exceptional, and most data center operators or SaaS providers struggle to achieve. This does not mean my data or services running within the Amazon environment meet those compliances

For point 2, 4 and 6 those are not benefits of the secure cloud. Those might be high-level objects one uses to form a strategy on how to get to a secure cloud.

Point 3 I don’t even understand, the protection of privacy and data has to be the number one concern when building out workloads in the cloud or private data centers. It’s not a benefit of the secure cloud, but a requirement.

For point 5, I am a big fan of automation and automating everything. Again this is not a benefit of a secure cloud, but how to have a repeatable, secure process wrapped in automation which leads to a secure cloud.

Given the discussions around cloud and security given all the negative press, including the recent AWS S3 Godaddy Bucket exposure, Amazon should be publishing better content to help move forward the security discussion.

Amazon recently released a presentation on Data-safe Cloud. It appears to be based on some Gartner question and other data AWS collected. The presentation discusses 6 core benefits of a secure cloud.

Inherit Strong Security and Compliance Controls
Scale with Enhanced Visibility and Control
Protect Your Privacy and Data

AWS,Cloud,Technology

Starting a new position today

July 30 2018 Greg Fina Leave a comment

Starting a new position today as Consultant - Cloud Architect with Taos. Super excited to for this opportunity.

I wanted a position as a solution architect working with the Cloud, so I couldn’t be more thrilled with the role. I am looking forward to helping Taos customers adopt the cloud and a Cloud First Strategy.

It’s an amazing journey for me, as Taos was the first to offer me a Unix System administrator position when I graduated from Penn State some 18 years ago, and I passed on the offer and went to work for IBM.

I am really looking forward to working with the great people at Taos.

Starting a new position today as Consultant - Cloud Architect with Taos. Super excited to for this opportunity.

I wanted a position as a solution architect working with the Cloud, so I couldn’t be more thrilled with the role. I am looking forward to helping Taos customers adopt the...

Amazon,AWS,Cloud

My Favorite Things About Amazon Well Architected Framework

July 25 2018 Greg Fina Leave a comment

Amazon released AWS Well Architected Framework to help customers Architect solutions within AWS. The amazon certifications require detailed knowledge of 5 white papers which make up the Well Architected Framework. Given I have recently completed 6 Amazon certifications, I decided I was going to write a blog which pulled my favorite lines from each paper.

Operational excellence pillar The whitepaper says on page 15, “When things fail you will want to ensure that your team, as well as your larger engineering community, learns from those failures.” It doesn’t say “If things fail”, it says “When things fail” implying straight away things are going to fail.

security pillar On page 18, “Data classification provides a way to categorize organizational data based on levels of sensitivity. This includes understanding what data types are available, where is the data located and access levels and protection of the data”. This to me sums up how security needs to be defined. Modern data security is not about firewalls and having a hard outside shell or malware detectors. It about protecting the data based on its classification from both internal (employees, contractors, vendors) actors and hostile actors.

reliability pillar The document is 45 pages long and the word failure appears 100 times and the word fail exists 33 times. The document is really about how to architect an AWS environment to respond to failure and what portion of your environment based on business requirements should be over-engineered to withstand multiple failures.

performance efficiency pillar Page 24 the line, “When architectures perform badly this is normally because of a performance review process has not been put into place or is broken”. When I first read this line, I was perplexed. I immediately thought this implies a bad architecture can perform well if there is a performance review in place. Then I thought when has a bad architecture ever performed well under load? Now I get the point this is trying to make.

cost optimization On page 2, is my favorite line from this white paper, “A cost-optimized system will fully utilize all resources, achieve an outcome at the lowest possible price point, and meet your functional requirements.” It made me immediately think back to before the cloud, every solution had to have a factor over the life of hardware for growth it was part of the requirements. In the cloud you need to support capacity today, if you need more capacity tomorrow, you just scale. This is one of the biggest benefits of cloud computing, no more guessing about capacity.

Big Data,Cloud,Technology

The Promises of Enterprise Data Warehouses Fulfilled with Big Data

July 24 2018 Greg Fina Leave a comment

Remember back in the 1990s/2000s Data Warehouses were all the rage. The idea was to take data from all the transactional databases behind the multiple e-Commerce, CRM, financials, lead generation and ERP systems deployed in the company and merge them into one data platform. It was the dream, CIOs were ponying up big dollars for these because they thought it would solve finance, sales, and marketing most significant problems. It was even termed Enterprise Data Warehouse or EDW. The new EDW would take 18 months to deploy as ETLs would be written from the various systems and data would have to be normalized to work within the EDW. In some cases, the team made bad decisions about how to normalize the data causing all types of future issues. When the project finished, there would be this beautiful new data warehouse, and no one would be using it. The EDW needed a report writer, to make fancy reports, in a specialized tool like Cognos, Crystal Reports, Hyperion, SAS, etc. A meeting would be called to discuss data, with 12 people and all 12 people would have different reports and numbers depending on the formulas in the report. That lead to eventually someone from Finance who was part of the analysis, budgeting and forecasting group would learn the tool and be the go-to person and work with the team from technology assigned to create reports.

Then Big Data came along. Big data even sounds better than Enterprise Data Warehouse, and frankly given the issues back in 1990s/2000s the branding to Big Data doesn’t have the same negative connotations.

Big Data isn’t a silver bullet, but it does a lot of things right. First and foremost the data doesn’t require normalization. Actually normalization is discouraged. Big Data absorbs the transactional database data, social feeds, eCommerce analytics, IoT sensor data, and a whole host of other data and puts it all in one data repository. The person from finance has been replaced with a team of data scientists who are highly trained and develop analysis models and extracts data with statistical (R programming language) and Natural Language Processing (NLP). The data scientists spend days pouring over the data, extracting information, building models, rebuilding models and looking for patterns within the data. The data could be text, voice, video, images, social feeds, transaction data and the data scientist is looking for something interesting.

Big Data has huge impacts as the benefits are immense. However, my favorite is predictive analytics. Predictive analytics tells you something’s behavior based on previous history and current data. It’s going to predict the future. Predictive analysis is all over retail as you see it on sites as “Other Customers Bought” or recommending purchases based on your history. Airlines use it to predict component failure of planes. Investors use it to predict changes in stock, and the list of industries using it goes on and on.

The cloud is a huge player in the Big Data space Amazon, Google and Azure are offering Hadoop and Spark as services. The best thing about the cloud is when the data is absorbed in Gigabytes or Terabytes that the cloud is providing the storage space for all this data. Lastly given it’s in the cloud, it’s relatively easy to deploy a Big Data cluster, and hopefully, soon AI in the cloud will replace the data scientists as well.

Cloud,Technology

To The Cloud and Beyond...

July 21 2018 Greg Fina Leave a comment

I was having a conversation with an old colleague late Friday afternoon. (Friday was a day of former colleagues, had lunch with a great mentor). He’s responsible for infrastructure and operations for a good size company. His team is embarking on a project to migrate to the cloud as their contract for space will be up in 2020. There three things which were interesting in the discussion which I thought were interesting and probably the same issues others face on their journey to the cloud.

The first was the concern about security. The cloud is no less or more secure than your data center. If your data center is private your cloud asset can be private, if your need public facing services, they would be secured like the public facing services in your own data center. Data security is your responsibility in the cloud, but the cloud doesn’t make your data any less secure.

The other concern was the movement of VMware images to the cloud. Most of the environment was virtualized years ago. However, there are a lot of windows 2003 and 2008 servers. Windows 2008 end of support is 2020, and Windows 2003 has been out of support since July 2015. It’s odd the concern about security, given the age of the Windows environment. If it was my world, I’d probably figure out how to move those servers to Windows 2016 or retire ones no longer needed, keeping in mind OS upgrades are always dependent on the applications. Right or wrong, my roadmap would leave Windows 2003 and 2008 in whatever datacenter facility is left behind.

Lastly, there was concern about Serverless, and the application teams wanting to leverage this over his group’s infrastructure services. There was real concern about a loss of resources if the application teams turn towards Serverless, as his organization would have fewer servers (physical/virtual instances) to support. Like many technology shops, infrastructure and operations resources are formulated by the total number of servers. I find this hugely exciting. I would push resources from “keeping the lights on” to roles focused on growing the business and speed to market, which are the most significant benefit of serverless. Based on this discussion, people look at it from their own prism.

Cloud,Microsoft,Technology

Power of Digital Note Taking

July 20 2018 Greg Fina Leave a comment

There hundreds of note taking apps. My favorites are Evernote, GoodNotes, and Quip. I’m not going to get into the benefits or pros and cons of each application. There plenty of BLOGs, youtube videos which do this in great detail. Here is how I used them:

Evernote is my document and note repository.
GoodNotes is for taking handwritten notes on my iPad, and the PDFs are loaded into Evernote.
Quip is for team collaboration and sharing notes and documents.

I’ve been digital for 4+ years. Today, I read an ebook from Microsoft, entitled “The Innovator’s Guide to Modern Note Taking.“ I was curious as to Microsoft’s ideas on the digital note-taking. The ebook is worth a read. I found there three big takeaways from the ebook:

First - The ebook quotes, “average employee spends 76 hours a year looking for misplaced notes, items, and files. In other words, we spend annual $177 billion across the U.S”.

Second - The ebook explains that the left side of the brain is used when typing on a keyboard, and the right side of the brain is when writing notes. The left side of the brain is more clinical, and the right side of the brain is more creative, particular asking the “What If” questions. Also covered on page 12 of the ebook handwriting notes improves retention. Lastly on page 13 one of my favorites as I am a doodler, “Doodlers recall on average 29% more information than non-doodlers”. There is a substantial difference in typing vs. writing notes, and there is a great blog article from NPR if you want to learn more.

_Third - _Leverage the cloud, whether it’s to share, process, access anywhere.

Those are fundamentally the three reasons that I went all digital for notes. As described before I write notes in GoodNotes and put them in Evernote, I use the Evernote OCR for PDFs to search them. My workflow covers the main points described above. Makes me think I might be ahead of a coming trend.

...

Cloud,Technology

Multi-cloud environments are going to be the most important technology investment in 2018/2019

July 19 2018 Greg Fina Leave a comment

I believe that Multi-cloud environments are going to be the most important technology investment in 2018/2019. This will drive education and new skill development among various technology workers. Apparently, it’s not just me, IDC prediction is that “More than 85% of Enterprise IT Organizations Will Commit to Multicloud Architectures by 2018, Driving up the Rate and Pace of Change in IT Organizations”.There some great resources online for multi-cloud, strategy, benefits, all worth reading:

The list could be hundreds of articles. I wanted to provide a few, that I thought were interesting and relevant to this discussion of why Multi-cloud. There are four drivers behind this trend:

First - Containers will allow you to deploy your application anywhere, including all the major cloud players have Kubernetes, Docker support. This means you could deploy to AWS, Azure, and Google without rewriting any code. Application support, development, maintenance is what drives technology dollars. Maintaining one set of code that runs anywhere doesn’t cost any more and gives you complete autonomy.

Second - Companies like Joyent, Netlify, HashiCorp Terraform and many more are building their solutions for multi-cloud, giving the control, manageability, ease of use, etc. Technology is like Field of Dreams, quote, “if you build it they will come.” Very few large companies jump into something without support, they wait for some level of maturity to be developed and then wade in slowly.

Third - The biggest reason is a lack of trust putting all your technology assets into one company. Most companies had for years multi-data center strategies, using a combination of self-created, leverage multiple companies like Wipro, IBM, HP, Digital Realty Trust, etc., and various co-location. For big companies when the cloud became popular, it was how do I augment my existing environment with Cloud. Now many companies are applying a Cloud First Strategy . So why wouldn’t principles that were applied for decades in technology, be applied to the cloud. Everyone remembers the saying, don’t put all your eggs in one basket. I understand there are regions, multi-AZ, resiliency, and redundancy, but at the end of the day one cloud provider is one cloud provider, and all my technology eggs are in that one basket.

Fourth - The last reason is pricing. If you can move your entire workload from Amazon to Google within minutes, it forces cloud vendors to keep costs low as cloud service charges for what you use. I understand if you have a workload with petabytes of data, it’s not going to move. But have web services with small data behind them, they can move and relatively quickly with the right deployment tools in place.

What do you think? Leave me a comment with your feedback or ideas?

Cloud,DevOps,SaaS

SaaS based CI/CD

July 14 2018 Greg Fina Leave a comment

Let’s start with some basics of software development. It still seems no matter what methodology of software development lifecycle that is followed it includes some level of definition, development, QA, UAT, and Production Release. Somewhere in the process, there is a merge of multiple items into a release. This still means your release to production could be monolithic.

The mighty big players like Google, Facebook, and Netflix (click any of them to see their development process) have revolutionized the concept of Continous Integration (CI) and Continous Deployment (CD).

I want to question the future of CI/CD, instead of consolidating a release, why not release a single item into production, validate over a defined period of time and push the next release. This entire process would happen automatically based on a queue (FIFO) system.

Taking it to the world of corporate IT and SaaS Platforms. I’m really thinking about software like Salesforce Commerce Cloud, or Oracle’s NetSuite. I would want the SaaS platform to provide me this FIFO system to load my user code updates. The system would push and update the existing code, while it continues to handle the requests and the users wouldn’t see discrepancies. Some validation would happen, the code would activate and a timer would start on the next release. If validation failed the code could be rolled back automatically or manually.

Could this be a reality?