Category Archives: AWS

Open Weight AI Models vs. Frontier APIs — The 2026 Cost Performance Tipping Point

Your AI inference bill is probably 10× higher than it needs to be. And the gap is getting wider, not narrower.

Six months ago, you could justify paying frontier API prices because open weight models were measurably worse. That justification is evaporating. In mid 2026, models like Kimi K3, GLM 5.2, and Llama 4 Maverick are matching or beating frontier APIs on real engineering benchmarks while costing a fraction per token. The question is no longer “are open weight models good enough?” It’s “can you still justify the premium?”

The Numbers Have Changed

Let’s lay out the current pricing landscape. On the frontier API side:

Model Input / 1M tokens Output / 1M tokens
GPT 5.6 Sol $5.00 $30.00
Claude Opus 5 $5.00 $25.00
GPT 5.6 Terra $2.00 $12.00
GPT 5.6 Luna $0.20 $1.20

And on the open weight side:

Model Input / 1M tokens Output / 1M tokens License
Kimi K3 (2.8T / 104B active) $3.00 $15.00 Open weight
GLM 5.2 (744B / 40B active) $1.40 $4.40 MIT
DeepSeek V4 $0.435 ~$0.87 Open weight

DeepSeek V4 at $0.435 per million input tokens is roughly 35× cheaper than GPT 5.6 Sol. Even Kimi K3, which sits at the premium end of open weight pricing, is half the cost of the flagship frontier APIs on output tokens.

But pricing is only half the story. What matters is what you get for the money.

Benchmarks Tell an Uncomfortable Story for Frontier Labs

Kimi K3, released by Moonshot AI in July 2026, is a 2.8 trillion parameter mixture of experts model with 104 billion active parameters and a 1 million token context window. On Artificial Analysis’ 16 task benchmark, it scored 90.49 out of 100, beating every Claude and GPT model tested. Its cost per completed task came in at roughly $0.94, compared to Claude Opus 4.8’s $1.80. That’s near frontier quality at half the cost per task.

GLM 5.2 from Z.ai (Zhipu AI), a 744 billion parameter MoE with 40 billion active, beat GPT 5.5 on SWE bench Pro (62.1 vs 58.6) at approximately one sixth the per token cost. It ships under the MIT license with no regional restrictions, meaning you can self host it anywhere.

Faros AI ran 211 real engineering tasks through seven different model plus harness combinations. The result: Claude Code paired with GLM 5.2 landed in the top quality band alongside Claude Code paired with Kimi K2.6, while Claude Code with Opus 4.8 and Codex with GPT 5.5 did not buy their way into that top tier. The open weight route scored 0.568; the Opus route scored 0.521. Higher quality and lower cost.

The Sentient Arena competition put a finer point on it. 147 builders competed using the open source MiniMax M2.5 model, and the top teams averaged approximately 70% accuracy at $1.74 per run. The same agents running on Claude Opus 4.5 hit approximately 80% accuracy at $56.53 per run. When you factor cost into the score, the open source model won for every team in the top six. Frontier closed source still won on absolute accuracy. Open source won on accuracy per dollar by a factor of 30.

Where Frontier Still Wins (For Now)

Let’s be honest about the limitations. Open weight models are roughly four months behind the closed frontier on absolute quality, according to analysis from The New Stack. On the hardest long horizon reasoning tasks, multi step autonomous agents, and problems requiring peak intelligence, GPT 5.6 Sol and Claude Opus 5 still hold an edge.

There is also the structure problem. Research from Unsupervised found that adding structured output requirements (JSON schemas, strict formatting) nearly tripled frontier model cost per task but actually cut cost for open weight models. If your pipeline demands rigid structure from a frontier API, you’re paying even more than the sticker price suggests.

The convenience gap is real too. One API call to a managed endpoint is simpler than provisioning GPU infrastructure. For a team running a handful of inference calls per day, the operational overhead of self hosting may not justify the savings. But that calculus changes fast at scale.

The Fine Tuning Equation

Here is where the economics become decisive. Fine tuned open weight models show 15 to 25% improvement in task specific accuracy over base models. For domain specific work (legal, medical, code generation against your specific codebase), a fine tuned Llama 4 or GLM 5.2 will outperform a general purpose frontier API on your tasks, every time.

The timing matters because OpenAI is sunsetting self serve fine tuning on a published timeline through January 2027. Organizations that never ran a fine tuning job already lost the ability to start one in May 2026. By January 2027, the door closes entirely for new jobs. The stated reason: newer base models are good enough that prompting beats fine tuning for most use cases. The practical effect: if you need fine tuned models, open weight is becoming the only game in town.

The GPU rental math makes this even more compelling. A 70B QLoRA fine tuning job on a rented H100 runs about $20 in compute. The equivalent job through a managed API platform costs $148 to $154. That is a 7× difference on raw compute. At scale, running 10 concurrent fine tuning jobs for enterprise customers, the rental approach is 73 to 91% cheaper than managed platforms.

The Scaling Curve Is the Real Story

Proprietary API costs scale linearly. Double your volume, double your bill. Self hosted inference scales at marginal cost: once you have the GPU capacity provisioned, additional inference is nearly free up to saturation.

For a team processing 100 million tokens per month, the TL;DR Dev Tech scorecard lays it out starkly:

  • Proprietary API: $15,000 to $50,000 per month per application
  • Self hosted open weight: $2,000 to $8,000 per month in GPU rental and ops

AWS CTO Werner Vogels has publicly noted that companies are migrating inference workloads from API gated models to open weight alternatives. When the CTO of the world’s largest cloud provider tells you open source is cheaper, the signal is hard to ignore.

And roughly 80% of enterprise AI tasks work well with open models in the 7B to 70B parameter range. You don’t need a 2.8 trillion parameter model for document summarization, structured extraction, or routing classification. A properly fine tuned 70B model handles these workloads at a tiny fraction of frontier cost.

The Decision Framework

Here is how to think about this if you are making infrastructure decisions today:

  1. Audit your workload mix. Categorize your AI tasks by complexity. For most teams, 80% or more of tasks are “good enough” territory for open weight models. Route only the genuinely hard problems to frontier APIs.

  2. Run your own benchmarks. Public leaderboards set priors, but Faros proved that the best model on a benchmark is not always the best model for your codebase. Test on your actual tasks, not synthetic ones.

  3. Factor in fine tuning. If you are paying frontier API prices for domain specific work, a fine tuned open weight model will likely outperform it at 5 to 20× lower cost. The OpenAI fine tuning sunset makes this transition urgent, not optional.

  4. Model the scaling curve. If your inference volume is growing (and whose isn’t), the linear scaling of API costs versus the marginal cost scaling of self hosted inference will dominate your total cost of ownership within months.

  5. Watch the vendor lock in risk. As CNCF executive director Jonathan Bryce put it: paying 10× more for a four month capability lead is not an enterprise AI strategy. It is an expensive form of lock in.

What Comes Next

Meta retired its hosted Llama API in July 2026, pivoting to a Muse only distribution model, while simultaneously releasing Muse Glimmer (30B, Apache licensed) in August. That hybrid strategy signals where the market is headed: weights are open, but the distribution and hosting layer is where value gets captured.

The open weight ecosystem is not slowing down. Capital is flooding in. The tooling around self hosted inference (vLLM, SGLang, Ollama) is maturing rapidly. And every month, the quality gap with frontier APIs narrows while the cost gap widens.

The tipping point is not coming. For most workloads, it has already arrived. The question is whether your architecture reflects that reality or is still paying a 2024 tax on 2026 problems.

Your AI inference bill is probably 10× higher than it needs to be. And the gap is getting wider, not narrower.

Six months ago, you could justify paying frontier API prices because open weight models were measurably worse. That justification is evaporating. In mid 2026, models like Kimi K3, GLM...

EC2 Turns 20 — What Cloud Architecture Looked Like Then vs. Now

Twenty years ago today, Jeff Barr published a blog post announcing the Amazon EC2 Beta. One instance type. One Region. A 1.7 GHz Xeon slice with 1.75 GB of RAM, 160 GB of local disk, and 250 Mbps of network bandwidth — yours for $0.10 per hour. No persistent storage. No VPC. No load balancer. You launched an m1.small into a flat, shared /8 network, crossed your fingers, and hoped your app stayed up.

Today, EC2 spans over 1,200 instance types across 39 Regions, powered by five generations of custom silicon. The distance between that 2006 launch and what architects build on today is the story of how cloud infrastructure matured from a clever hack into the foundation of modern computing.

I’ve been using EC2 since 2009 — before VPCs existed, before IAM roles for instances were a thing, before you could even attach a persistent disk without downtime. I remember SSH’ing into instances that lived in a flat, shared network with every other AWS customer, praying that my Elastic IP reassignment would propagate before traffic started dropping. The platform has come an extraordinary distance since then, and this anniversary feels personal. Let me walk you through the arc.

The Original Architecture: 2006–2009

If you launched an instance in August 2006, your architecture looked something like this:

Internet → Public IP (assigned at boot) → m1.small → Local ephemeral disk

That was it. There was no Elastic IP, no persistent block storage, no way to define network topology. Every customer’s instances lived in a single giant 10.0.0.0/8 network — what we now call EC2 Classic. Security groups existed but operated at the instance level in a shared flat space.

The foundational primitives arrived in rapid succession:

  • 2008 — Elastic Block Store (EBS) gave instances persistent storage that survived termination
  • 2009 — Elastic Load Balancing, Auto Scaling, and CloudWatch made apps scalable and observable
  • 2009 — Virtual Private Cloud (VPC) introduced logically isolated networks with subnets, route tables, and gateways

VPC was the architectural inflection point. For the first time, you could design network topology — public subnets, private subnets, NAT gateways, peering connections. The multi tier web application pattern that defined a generation of cloud architecture became possible only after VPC existed.

The Nitro Revolution: 2017

For the first decade, EC2 ran on the Xen hypervisor. Networking, storage, and management functions all competed for CPU cycles on the host. Every packet your application sent had to traverse the same general purpose processor running your workload.

AWS began offloading these functions to dedicated hardware as early as 2013 with the C3 instance family, but the full Nitro System arrived in November 2017. The architecture changed fundamentally:

┌─────────────────────────────────────┐
│          Customer Instance          │
│    (nearly bare metal performance)  │
├─────────────────────────────────────┤
│         Nitro Hypervisor            │
│    (lightweight, minimal attack     │
│     surface)                        │
├───────────┬───────────┬─────────────┤
│ Nitro Card│ Nitro Card│  Nitro Card │
│ (Network) │ (Storage) │ (Mgmt/Sec)  │
└───────────┴───────────┴─────────────┘

By moving networking, storage I/O, and instance management onto purpose built Nitro Cards, AWS freed the host CPU entirely for customer workloads. The result: near bare metal performance with the security boundary of a hypervisor. Every EC2 instance launched since early 2018 runs on the Nitro System.

In 2026, AWS pushed isolation even further with the Nitro Isolation Engine — a component inside the Nitro Hypervisor that uses formal verification to provide mathematical proof that customer workloads are isolated from each other and from AWS operators. Not just “trust us” — cryptographic, formally verified assurance.

Custom Silicon: Graviton and the AI Accelerators

The Nitro System made a second revolution possible. Once the hypervisor was thin and the I/O offloaded, AWS could drop in any processor architecture without re-engineering the platform.

Graviton timeline:

Generation Year Key Advancement
Graviton (A1) 2018 First Arm based instances, up to 45% cost reduction for scale out workloads
Graviton2 2020 40% price performance over x86, broad adoption
Graviton3 2022 25% better compute over Graviton2, DDR5 memory
Graviton4 2024 30% better performance, 75% more memory bandwidth
Graviton5 2025 192 cores, 5x larger cache, optimized for agentic AI workloads

Today’s M9g instances (Graviton5, sixth generation Nitro) are so architecturally distant from the original m1.small that they share little beyond the “general purpose” label. And they’re running workloads — real time reasoning, multi step orchestration, code generation — that did not exist as categories in 2006.

AI accelerators followed a similar trajectory. Inferentia (2019) brought purpose built inference silicon. Trainium (2021) tackled training. By late 2025, Trn3 UltraServers interconnect up to 144 Trainium3 chips to train and serve frontier models. The progression from “rent a virtual CPU” to “reserve a 144 chip training cluster” happened in under 20 years.

What This Means for Architects Today

The architectural decisions you face in 2026 are qualitatively different from 2006, but the meta pattern is the same: match the workload to the right primitive.

Here’s what a modern EC2 launch looks like compared to 2006:

# 2006: Launch an m1.small. That's all there was.
ec2-run-instances ami-xxxxxxxx -t m1.small

# 2026: Launch a Graviton5 instance in an isolated VPC with IMDSv2 enforcement
aws ec2 run-instances \
  --image-id ami-0abc123def456 \
  --instance-type m9g.2xlarge \
  --subnet-id subnet-0a1b2c3d4e \
  --security-group-ids sg-0f1e2d3c4b \
  --metadata-options "HttpTokens=required,HttpEndpoint=enabled" \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Environment,Value=prod}]'

The CLI call got longer because the platform got richer. Every additional flag represents a decade of lessons learned about security, cost, and operational maturity.

Practical Takeaways

  1. Default to Graviton. Unless your workload has a hard x86 dependency (specific licensed software, architecture specific binaries you cannot recompile), start with Graviton instances. The price performance advantage is real and compounding with each generation.

  2. Understand the Nitro System boundary. The security model of modern EC2 is fundamentally different from pre-2017 instances. Network and storage I/O never touch your host CPU. The Nitro Isolation Engine provides formally verified separation. Design your threat models accordingly — the Nitro System security whitepaper is essential reading.

  3. Use purpose built instances for AI workloads. Running inference on general purpose instances is like using a sedan to haul freight. Inf2 for inference, Trn2/Trn3 for training, and EC2 Capacity Blocks for reserving GPU/accelerator time exist specifically to avoid overpaying for the wrong compute shape.

  4. Treat instance selection as an architectural decision, not a default. With 1,200+ instance types, the “just pick an m5.large” reflex leaves performance and money on the table. Profile your workload, right size with AWS Compute Optimizer, and revisit quarterly as new generations launch.

  5. Remember that EC2 is still the foundation. Lambda, Fargate, EKS, SageMaker, Bedrock — they all run on EC2 underneath. Understanding the compute layer makes you a better architect regardless of the abstraction you choose to expose to your application.

Looking Forward

EC2’s first 20 years traced an arc from a single shared network with one instance type to a global, multi architecture platform with mathematically proven isolation and purpose built silicon for every workload class. The next 20 will likely be defined by AI native compute patterns, disaggregated architectures, and deployment models we have not yet named.

But the core principle that made EC2 transformative in 2006 has not changed: give builders the primitives, make them minimal yet useful, and iterate relentlessly based on what they actually build. Twenty years in, that flywheel is still spinning.

Happy birthday, EC2. Here’s to the next twenty.

Twenty years ago today, Jeff Barr published a blog post announcing the Amazon EC2 Beta. One instance type. One Region. A 1.7 GHz Xeon slice with 1.75 GB of RAM, 160 GB of local disk, and 250 Mbps of network bandwidth — yours for $0.10 per hour. No persistent storage....

Passed AWS Certified Security - Specialty

It’s been a heck of three weeks—actually, a month. I started studying on June 15th for the Network and AWS Solution Architect Professional, as the networking was expiring first. I decided I focus on one exam at a time. So I did the Professional Architect June 28th, Networking July 6th, and Security on July 16th. All of this while working full time. It reminded me of the effort required to get my Master’s Degree in Computer Science. I’m relieved, as I have my DevOps in November, but at least now there is a break.

Without violating the NDA, let’s talk about the security exam. I took the exam Friday and passed. I did the exam on Pearson Vue. For the exam, I used about 95 minutes, which is half the allocated time. Some questions were real struggles. Hopefully, I’ll remember some of the contexts and research them later for my knowledge. 

The last time I took the security exam in July 2018, I decided on a Friday to take it the following Wednesday. Last time I wrote, “It’s the hardest exam I’ve taken to date. I think it is harder than the Solution Architect - Professional exam.” In 3 weeks, taking the Solution Architect - Professional, Networking Specialist, and Security Speciality. Oh wait, this is the second time I’ve done this. I guess I haven’t learned. I would confirm it’s hard. Is it harder than the Solution Architect Professional in its current form? I don’t know. It’s a more nuisance exam focused on security. AWS has 100,000s pages of documentation on services, Well-Architected, Mitigation strategies, and this exam pulls from those documents. I’m not going to go into details about the questions. But that’s a ton of information to know and understand to achieve this certification. I guess this is why they’re hard, and few people have 11.  

Now the part I will talk about is my preparation. Security is fundamental to AWS. Every service integrates with IAM, most with KMS, and there are many other services like SCPs, Security Hub, Guard Duty, Shield, etc., designed to help protect workloads in AWS and their integration to other services. Last time I probably put 24 hours into studying for the exam. This time it was maybe 18 hours in total. I don’t think I did the preparation justice either time. I think I fell back on my 12 years of AWS experience and the past three weeks of studying for the other exams. Although I knew going into the exam areas like KMS Key Grants, Private CA on ACM, HSM, Secrets Manager were weaknesses, the more I tried to read up and watch videos, the more learning I felt I needed imposter syndrome at work. 

I watched the 96% of acloud.guru security course did watch it at 1.75x- 2x speed. I didn’t slow down. If I didn’t understand a topic, I read or watched something in the resources section below. Again these are resources collected before the exam that I used. 

Resources

It’s been a heck of three weeks—actually, a month. I started studying on June 15th for the Network and AWS Solution Architect Professional, as the networking was expiring first. I decided I focus on one exam at a time. So I did the Professional Architect June 28th, Networking July 6th,...

Security Reference Architect

AWS has the security shared responsibility model.
Shared Responsibility Model

Anyone on the AWS platform understands where this model. However, security on AWS is not easy as AWS has always been a platform of innovation. AWS has released a ton of services AWS Config, Macie, Shield, Web Application Firewall, SCPs. Over the years, Landing zones and then Control Tower which builds security when starting multi-account on AWS. Lastly, the Well Architected Security Pillar to review and confirm your workload is well architected.

Last month, AWS released a comprehensive guide to a Security Reference Architecture. It was built by Professional Services, which is the customer implementation arm of AWS.

I’m not going to try to summarize a 62-page document in a blog article. Mainly the document is about defense-in-depth, which is security at each layer of the workload. There two key observations from the document. The first observation is it does follow Control Tower guidance. Terms have been changed. It requires workloads to be in separate OU from security and infrastructure(shared services). Again these are general security principles that limit blast radius if an application or account is compromised. Security account and log collection account need to be separate. This Control Tower recommended an OU structure. Also, keeping log data in an immutable state is best for audit analysis.

The second observation is it now talks about leverage the Infrastructure account for Egress and Ingress traffic to the internet. This is only possible with Transit Gateway or a Transit VPC, defined in the document but not mentioned as part of the VPC diagram.

Maybe it’s just because of hyperfocus on renewing certifications. However, I notice the bleed-over between networking and security and how proper networking architecture is to start good security hygiene.

AWS has the security shared responsibility model.
Shared Responsibility Model

Anyone on the AWS platform understands where this model. However, security on AWS is not easy as AWS has always been a platform of innovation. AWS has released a ton of services AWS Config,

Transit Gateway and Direct Connect

After studying for Advanced Networking Exam, I pondered a question about global backbones. There is a need for common understanding. So let’s take a step back. Transit gateway was a service introduced at ReInvent 2018. Transit Gateway(TGW) puts a router between VPCs and other networking services. The transit gateway works by putting attachments in each VPC using ENIs. If you’re lost before proceeding, watch the Re: Invent Video. TGW uses attachments is fundamental to the VPC architecture as the VPC doesn’t process traffic from a source destination outside the VPC. So the attachment ENI becomes part of the VPC. So now I have an attachment in the VPC thru a subnet. So instead of terminating my DirectConnect Gateway(DXGW) on a VGW in a VPC, it’s terminated in a TGW. A quick whiteboard of this architecture.
Transit Gateway

This becomes challenging while building a global network because European network would look like this assuming I had three pops in one Europe region:
Transit Gateway with multiple POPs

Still better than Direct Connect Gateways to the VPCs. But there is a limitation Transit Gateways which are peered, don’t dynamically pass routes. This works great if you summarize routes by region. Like the US was all 10.50.0.0/12, and Europe was all 10.100.0.0/12. What doesn’t work is when I have unsummarized routes. But I digress route summarization doesn’t matter to the question. So here is a quick view of our whiteboard architecture of Europe and US regions: Transit Gateway with multiple POPs

The question is if there was dynamic routing, could I use the AWS backbone to haul traffic around the world without having to build my own global network as the two TGWs would exchange my prefixes from the exchange or pop locations?

After studying for Advanced Networking Exam, I pondered a question about global backbones. There is a need for common understanding. So let’s take a step back. Transit gateway was a service introduced at ReInvent 2018. Transit Gateway(TGW) puts a router between VPCs and other networking services. The transit gateway works...

Passed the AWS Certified Advanced Networking – Specialty Exam

I needed to recertify the Advanced Networking specialty. Technically it expired on 6/20. So I decided to focus on the Professional as it would include Networking and Security topics. I need to recertify Security Speciality later this month.

I took the AWS Advanced Networking Speciality on Tuesday and passed.

I took this exam with Pearson VUE. The exam opens 30 minutes before to get checked out. The process with the same as PSI, only there wasn’t a long wait. Personally, the interface in PSI is a little nicer than Pearson VUE. However, the experience of the otherwise of taking the exam is the same as it’s from the comfort of home.

I’m not going to talk about the exam, as that would violate the NDA. There are three observations. First, the exam requires deep AWS networking knowledge. Make sure you get in the console and get hands-on. The exam, as advertised, requires deep understanding and experience, which can only come thru practical hands-on experience. The other observation I would make is that the exam requires knowledge of services touched by networking, which is why acloud.guru course recommends associate level certification. The last comment on this exam has the most deficient written questions and answers of the certification exams I’ve taken. The questions and answers lack clarity found on the other exams.

I took the exam in about 90 minutes, which is half the allocated time. There were enough questions that I struggled to know the correct answer. I had no sense if I had no sense during the exam of a pass or fail.

Now the parts I can talk about, which was my preparation for the exam. In studying, the number of new networking specific services, including Transit Gateway announced Re:Invent 2018, Firewall Manager introduced April 2018 to name a few. The changes in networking services like AWS Shield, VPC FLow Logs, WAF between studying back in 2018 and studying three years later is incredible. Probably the reason, these certifications have to be re-certified every three years. The first time for the exam, I put about 50 hours of preparation into studying for the exam. This time I put maybe 16 hours into studying.

I watched about 80% of the acloud.guru course. I did watch most of it in 1.75x speed. I would slow down if I didn’t understand a topic or wanted more. I also read many whitepapers and FAQs and watched Youtube videos (2x) and linked below.

Resources:

I needed to recertify the Advanced Networking specialty. Technically it expired on 6/20. So I decided to focus on the Professional as it would include Networking and Security topics. I need to recertify Security Speciality later this month.

I took the AWS Advanced Networking Speciality on Tuesday and passed.

I...

AWS Certified Solutions Architect – Professional

I sat the AWS Certified Solutions Architect - Professional exam last Monday.  This was to recertify my expiring AWS Professional Certification, which recertified my Cloud Practioner and Associate Architect.

The exam is challenging. Probably the most time I spent taking an AWS exam, 2 hours and 21 minutes. My original certification came with what was the famous reading comprehension exam, which retired in 2019. This exam didn’t require that level of reading but was harder. Also, I finished the old exam faster than this one.

Let provide three observations on the exam that won’t violate any NDA. First, it feels more SysOps than Architect because the scenarios aren’t as drawn out in the exam, which retired in 2019. Secondly, it tests both the breadth and depth of AWS services. Lastly, back in 2018, I said, “The entire exam is a challenge to pick the more correct answer based on the scenario and question with a driving factor of one more or more of the following, scalability, cost, recovery time, performance, or security.” That statement holds true on this exam.

For preparation, I watched about 50% of the acloud.guru course. I skipped the sections thru comfortable sections. I also read a bunch of whitepapers, FAQs and watched Youtube videos and linked below.

I took the exam with PSI. First, PSI doesn’t start the check till the exam slot, which takes 20-30 minutes. Secondly, the app will suck battery power. Given the requirements, I couldn’t use my regular desk, so be prepared.

Last comment, not that the team at AWS is trying to, but the exam is about getting you stuck on a question, so you run out of time or rush thru and miss core context. I’ve been on the platform a long time and work for AWS, it took me 78% of the allocated time on a Monday night to do the exam, and I read fast. The main takeaway is to develop an exam strategy that works for you and is practiced on the associate level exams before sitting this exam.

List of resources:

I sat the AWS Certified Solutions Architect - Professional exam last Monday.  This was to recertify my expiring AWS Professional Certification, which recertified my Cloud Practioner and Associate Architect.

The exam is challenging. Probably the most time I spent taking an AWS exam, 2 hours and 21 minutes. My original...

graviton2

awsarch.io was switched over to Graviton2 instance types, as there was significant cost savings, something like 20% if my math was correct. There very little to this blog as it uses some Jekyll and apache. All the posts are maintained in a source code repo as they start life as markdown. Jekyll converts the markdown into HTML.

The os takes care of the differences between graviton2 arm based on the prior intel instances. The performance of t instances is not exceptional, but they scale under load like any other instance and super cost-effective.

Software required which is not available can be built using GCC. I think I had to build on a package, and it worked fine. Tools managed by homebrew had no issues.

awsarch.io was switched over to Graviton2 instance types, as there was significant cost savings, something like 20% if my math was correct. There very little to this blog as it uses some Jekyll and apache. All the posts are maintained in a source code repo as they start life...

AWS Certified Database

I passed the AWS Certified Database Speciality Exam in May. That makes my 11th AWS certification. The database specialty seems to have split out the database content from the Big Data exam, which was retired in April of 2020. I did SME work on this exam and completed several workshops before its release.

Update 4/8/2021:
I reviewed the acloud.guru course and it seems to cover the exam topics covered in the exam blueprint

With the removal of the Alexa Certification, I now have all 11 AWS certifications.

I passed the AWS Certified Database Speciality Exam in May. That makes my 11th AWS certification. The database specialty seems to have split out the database content from the Big Data exam, which was retired in April of 2020. I did SME work on this exam and completed several workshops...

AWS Certified Machine Learning – Specialty Links

Image of AWS Machine Learning Speciality Certification

The Machine Learning exam is rather difficult, as discussed previously. The starting point would be the acloud.guru Machine Learning course or Linux Academy courses. Additionally is the training offered by AWS. A chunk of Machine Learning is data and data preparation, so please see my links from Big Data.

Here is a collection of links I put together which helped me with studying for the exam.

General Topics

Handling imbalances in Data
Learning Rate
Neural Networks
Common Machine Learning Algorithms
Another Resource on Machine Learning Algorithms

Machine Learning Concepts

Formulating the Problem
Regression
Regression Model Insights
The Machine Learning Process
Machine Learning Key Concepts
Cross Validation
Splitting Training Data
Training Parameters
Training Faster with Sagemaker Linear Learner
Multiclass Model Insights
Managing Machine Learning Projects Whitepaper
SageMaker Blog
Underfitting and Overfitting
Machine Learning Models
Binary Model Insights
One Hot Encoding

Data

Glue
Glue Crawler
Athena
SparkML
KPL
Kinesis Data Firehose
Kinesis PutRecord

AWS Machine Learning - SageMaker

Data Formats
SageMaker Batch
SageMaker Docker Registry
Built-in Algorithms
Elastic Inference
Elastic Inference
Inference Pipeline Containers
Validating a Model
Training Metrics
CloudTrail
AutoScaling
SageMaker with Step Functions
Hosting Model
SageMaker and IAM
Polly

SageMaker Machine Learning Implementations

Semantic Segmentation
Seq-to-Seq
K-Means Linear Learner
Linear Learner Tuning
BlazingText BlazingText InputOutput
LDA
Factorization Machines
Random Cut Forest
K Nearest Neighbor
Image Classification
Object2Vec
Object Detection
PCA
DeepAR
XGBoost
XGBoost Tuning
XGBoost Parameters
Neural Topic Model

SageMaker TensorFlow Framework

TensorFlow

SageMaker Hyperparameter Tuning

Creating Hyperparameters Tuning Job
Automated Tuning
Hyperparameter Tuning Job
Hypertunning
Image Classification Hyperparameters

Business Intelligence

QuickSight
Chart types

Image of AWS Machine Learning Speciality Certification

The Machine Learning exam is rather difficult, as discussed previously. The starting point would be the acloud.guru Machine Learning course or Linux Academy courses. Additionally is the training offered by AWS. A chunk of Machine Learning...

AWS Certified Machine Learning – Specialty

Image of AWS Machine Learning Speciality Certification

I passed the AWS Certified Machine Learning Speciality Exam on Monday. That makes my 10th AWS certification in the last 18 months.

The Machine Learning Specialty certification is unlike any of the other exams from AWS. The exam doesn’t just focus on AWS specifics but covers a wide range of Machine Learning topics. The exam blueprint provides a basis of this coverage.

The exam is probably the hardest of the 10 I’ve taken to date. The entire exam, I thought I know the material, but I don’t think I know it well enough to pass the exam. My score was good, and it satisfying to add this certification. For the Machine Learning exam, I put in well over 200 hours over the last six months and over 80 hours the four weeks before sitting the exam. Definitely think the Big Data Certification helped on the data preparation sections.

They’re a bunch of links I will share later this week, which I studied. In addition to all the reading, I did acloud.guru’s AWS Certified Machine Learning - Speciality, which provides 40% of the material required to pass the exam. The rest of the exam requires detailed knowledge of Machine Learning. I followed the learning track recommended by AWS for Data Scientist. I also did several sections from Linux Academy Machine Learning, including the great section explaining PCA. Lastly, I took the AWS practice exam. I did look at Whizlabs but was somewhat disappointed in their practice tests.

In 2020, I hope to get a project which will allow me to leverage Machine Learning in SageMaker to solve a complex customer problem.

Image of AWS Machine Learning Speciality Certification

I passed the AWS Certified Machine Learning Speciality Exam on Monday. That makes my 10th AWS certification in the last 18 months.

The Machine Learning Specialty certification is unlike any of the other exams from AWS. The exam doesn’t...

What I Learned About GCP

I’ve been on AWS since February of 2009, and my first bill was for $1.21 for some S3 Storage. Recently, I wanted to understand the Google Cloud Platform, as people talk about Spanner, BigQuery, BigTable, and App Engine. I figured the best way to learn was to challenge myself with a Google certification exam.

Given all my AWS experience, I initially wanted to write a blog article about what I liked and disliked, but I don’t think it’s that simple. There are exciting things within AWS and Google. Both of the platforms are complex, so this by no means is exhaustive. It’s more of what I noticed in my first couple of logins to Google Cloud.

The first thing I noticed was outside the service names how familiar the services were, and it didn’t take much to understand the VPCs, IAM, Billing, monitoring, Kubernetes (GKE), and Storage. The service names are vastly different, where Google calls everything Cloud blah and AWS calls them AWS or Amazon blah. Most of the fundamental principles were the same, especially in primary services like Compute, Storage, and IAM. This terminology probably speaks more to multi-cloud, than anything else.

The second thing I found that the Google Cloud Shell in the browner was outstanding. Google Cloud Shell is a container running which gives you a fully functioning Linux shell with disk space. Cloud Shell can be used for files, configuration files like Kubernetes manifests, and to check out code repositories. The kicker is that it’s embedded into the service and is free. The closest thing AWS offers is the shell inside Cloud9 service, which comes with an added expense. The Cloud Shell is something I liked on GCP.

The third thing I noticed was this concept of projects, which is a folder construct. I’m not sure if I like it. I saw examples where people used seperate folders for dev, test, and Production in the same account. I would be a little concerned given how easy it would be to be in the wrong project and issue commands. I prefer my dev/test to be separate accounts from Production. So I don’t necessarily know if this is a good or bad thing, but trends toward dislike.

Next fourth thing I noticed was the firewall rules. AWS has both the concept of Security Groups and Firewalls (NACLS). GCP only has firewall rules. The rule structure is impressive, as it allows to target by service account, tags, IP addresses. I would have a concern in a larger environment that the Firewall Rule list would be overly complicated and difficult to read and manage. I much prefer smaller nested security groups on AWS. However, the flexible of the GCP Firewall is impressive. I want the concept of tags inside security groups within AWS. So firewall rules are something I liked.

The fifth thing I want to highlight is the instance configuration. While AWS offers fixed CPU and memory instances, GCP offers custom selections for memory and CPU. This could be very interesting if there are a low CPU and high memory workload. I didn’t see significant cost differences between an overprovisioned AWS resource vs. a custom GCP resource. However, I also didn’t do an in-depth, TCO analysis. Again, I see pros and cons to this and probably I am neutral on this subject.

The last thing is the UI. It is different from AWS, and it took some use getting used too. It’s very similar in my experience to the G-Suite Admin or other Google services. I found the configuration of computing to be more changing given it’s a single page with tabs, vs. the AWS workflow. However, other items like Storage seemed to be more friendly. It doesn’t make a lousy user experience. Again I am neutral on this topic, I learned how to use it.

Probably now you are reading this and looking for that summary or in conclusion section. I’m not going to provide it. I remember two decades ago when we wanted to stand up web servers in a data center for a project, and it was going to cost $5,000 before we wrote the first line of code. As struggling college students, this wasn’t going to happen. What I am going to say is to go build something. Its never been easier for a builder to make an idea come to life on a platform you prefer with minimum investment (free tier). If your game is running Cobol inside a Kubernetes container, go do it. If you hate infrastructure go Serverless. Cobol on serverless would me attractive, eh? The power is in your hands. If you don’t have any ideas, go get a cloud certification. There never been a better time for a technologist with cloud experience.

I’ve been on AWS since February of 2009, and my first bill was for $1.21 for some S3 Storage. Recently, I wanted to understand the Google Cloud Platform, as people talk about Spanner, BigQuery, BigTable, and App Engine. I figured the best way to learn was to challenge myself with...

Big Data Certification

Image of AWS Big Data Speciality Certification

I passed the AWS Certified Big Data Speciality Exam on Saturday. That makes my 9th AWS certification in the last 10 months. For a moment I’ll have 9/9 certifications. Machine Learning opens this month, so come tomorrow I’ll have 9/10 Certifications. Machine learning recommended training is Big Data on AWS and Deep Learning on AWS. Given I just completed Big Data, probably schedule this exam for sometime in May.

Big Data Certification Exam is similar to the other specialty exams. While not necessarily as hard as the Professional level exams it does require a detailed level of knowledge. Also unlike the other specialty exams, Big Data requires a breadth and depth of knowledge consistent with the Professional Level exams. I prepared using acloud.guru’s AWS Certified Big Data - Speciality which provides somewhere between 50% - 60% of the required topics around Kinesis, IoT, S3, DynamoDB, EMR, Redshift, and Quicksight. I did review some topics in Linux Academy to reinforce the concepts. The rest of the experience is hands-on or lab learnings. AWS doesn’t offer a practice exam, so I tried the Whizlab practice exams. Whizlab’s typically have issues and provide a false level of confidence as the practice exams are always easier than the actual certification exam.

Acloud.guru covers much information, and it also provides a set of links to critical whitepapers and blog articles. As always without, violating the NDA, they do an excellent job in pointing you to the topics to study. Aside from that material, I read a whole bunch of AWS links, which will be posted at the end of this blog article. Also, there was a great youtube playlist John Creecy put together at https://www.youtube.com/playlist?list=PLlp-qT09uTBcoMpiQkpO-G8GsHOVWyfV0.

I am relatively little experience with Kinesis, EMR, Redshift, and Quicksight, before studying for the exam. I found Kinesis, Redshift, and Elasticsearch fascinating, and will be looking for projects in this space to continue my learning.

Kinesis
https://docs.aws.amazon.com/streams/latest/dev/key-concepts.html https://docs.aws.amazon.com/streams/latest/dev/introduction-to-enhanced-consumers.html https://docs.aws.amazon.com/streams/latest/dev/kinesis-record-processor-ddb.html https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-split.html https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html https://docs.aws.amazon.com/streams/latest/dev/building-consumers.html https://docs.aws.amazon.com/streams/latest/dev/creating-using-sse-master-keys.html https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-concepts.html https://docs.aws.amazon.com/streams/latest/dev/kinesis-producer-adv-retries-rate-limiting.html https://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html https://docs.aws.amazon.com/streams/latest/dev/monitoring-with-kcl.html https://docs.aws.amazon.com/streams/latest/dev/agent-health.html https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-merge.html

Kinesis Firehose
https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html#data-flow-diagrams https://docs.aws.amazon.com/firehose/latest/dev/data-transformation.html https://docs.aws.amazon.com/firehose/latest/dev/create-configure.html https://docs.aws.amazon.com/firehose/latest/dev/record-format-conversion.html https://docs.aws.amazon.com/firehose/latest/dev/data-transformation.html#lambda-blueprints https://docs.aws.amazon.com/firehose/latest/dev/encryption.html

Kinesis Data Analytics
https://docs.aws.amazon.com/kinesisanalytics/latest/dev/what-is.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/streams-pumps.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/authentication-and-access-control.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/stagger-window-concepts.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/tumbling-window-concepts.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/sliding-window-concepts.html https://docs.aws.amazon.com/kinesisanalytics/latest/dev/continuous-queries-concepts.html

IoT
https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html https://docs.aws.amazon.com/iot/latest/developerguide/policy-actions.html https://docs.aws.amazon.com/iot/latest/developerguide/iam-policies.html https://docs.aws.amazon.com/iot/latest/developerguide/iot-provision.html https://docs.aws.amazon.com/iot/latest/developerguide/iot-device-shadows.html https://docs.aws.amazon.com/iot/latest/developerguide/iot-rule-actions.html

ElasticSearch
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-bp.html https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-aws-integrations.html

CloudSearch
https://docs.aws.amazon.com/cloudsearch/latest/developerguide/what-is-cloudsearch.html

EMR
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-overview.html#emr-overview-clusters https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-file-systems.html https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-consistent-view.html https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-enable.html#emr-awskms-keys https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.html https://docs.aws.amazon.com/emr/latest/ManagementGuide/emrfs-configure-sqs-cw.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-flink.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-tez.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hbase.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hcatalog.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-zookeeper.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-phoenix.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-sqoop.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-jupyter-emr-managed-notebooks.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-jupyterhub.html

QuickSight
https://docs.aws.amazon.com/quicksight/latest/user/welcome.html https://docs.aws.amazon.com/quicksight/latest/user/refreshing-imported-data.html https://docs.aws.amazon.com/quicksight/latest/user/joining-tables.html https://docs.aws.amazon.com/quicksight/latest/user/bar-charts.html https://docs.aws.amazon.com/quicksight/latest/user/combo-charts.html https://docs.aws.amazon.com/quicksight/latest/user/heat-map.html https://docs.aws.amazon.com/quicksight/latest/user/line-charts.html https://docs.aws.amazon.com/quicksight/latest/user/kpi.html https://docs.aws.amazon.com/quicksight/latest/user/restrict-access-to-a-data-set-using-row-level-security.html#create-row-level-security https://docs.aws.amazon.com/quicksight/latest/user/tabular.html https://docs.aws.amazon.com/quicksight/latest/user/supported-data-sources.html https://docs.aws.amazon.com/quicksight/latest/user/scatter-plot.html https://docs.aws.amazon.com/quicksight/latest/user/geospatial-data-prep.html

Redshift
https://docs.aws.amazon.com/redshift/latest/dg/tutorial-tuning-tables-distribution.html https://docs.aws.amazon.com/redshift/latest/dg/c_best-practices-best-dist-key.html https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html#rs-about-clusters-and-nodes https://docs.aws.amazon.com/redshift/latest/mgmt/enhanced-vpc-working-with-endpoints.html https://docs.aws.amazon.com/redshift/latest/dg/c_designing-queries-best-practices.html https://docs.aws.amazon.com/redshift/latest/dg/c_best-practices-use-copy.html https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STL_tables.html https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STV_tables.html https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html https://docs.aws.amazon.com/redshift/latest/dg/wlm-short-query-acceleration.html

DynamoDB
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html#bp-partition-key-partitions-adaptive https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables_monitoring.html https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-data-upload.html https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables_reqs_bestpractices.html https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-aggregation.html https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-gsi-overloading.html https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-indexes-gsi-sharding.html

Machine Learning
https://docs.aws.amazon.com/machine-learning/latest/dg/types-of-ml-models.html https://docs.aws.amazon.com/machine-learning/latest/dg/binary-model-insights.html https://docs.aws.amazon.com/machine-learning/latest/dg/regression-model-insights.html https://docs.aws.amazon.com/machine-learning/latest/dg/multiclass-model-insights.html https://docs.aws.amazon.com/machine-learning/latest/dg/ml-model-insights.html https://docs.aws.amazon.com/machine-learning/latest/dg/cross-validation.html https://docs.aws.amazon.com/machine-learning/latest/dg/creating-and-using-datasources.html https://docs.aws.amazon.com/machine-learning/latest/dg/creating-a-data-schema-for-amazon-ml.html https://docs.aws.amazon.com/machine-learning/latest/dg/amazon-machine-learning-key-concepts.html

Pipeline
https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-how-tasks-scheduled.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-concepts-datanodes.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-concepts-databases.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-importexport-ddb-part1.html https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/datapipeline-related-services.html

Data Movement
https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Welcome.html

Athena
https://docs.aws.amazon.com/athena/latest/ug/access.html https://docs.aws.amazon.com/athena/latest/ug/encryption.html#encryption-options-S3-and-Athena https://docs.aws.amazon.com/athena/latest/ug/athena-aws-service-integrations.html

Glue
https://docs.aws.amazon.com/glue/latest/dg/components-overview.html

Image of AWS Big Data Speciality Certification

I passed the AWS Certified Big Data Speciality Exam on Saturday. That makes my 9th AWS certification in the last 10 months. For a moment I’ll have 9/9 certifications. Machine Learning opens this month, so come tomorrow I’ll have 9/10...

Advanced Architecting on AWS

I took Advanced Architecting on AWS for the last three days. The course is part of the learning process for the AWS Certified Solutions Architect – Professional. I already have the certification based on the older version of the exam. The new version of the certification exam went live on February 4th. The course seems to follow the newer certification guide. Overall the course is good as it covers all the services required, the labs were a little disappointing as they lacked complexity. To become proficient and attempt the certification, one would need to a lot more learning and deep diving on the topics covered in this course. It reviews probably 35% of the material required to sit the exam.

Here is my summary by day of the course.

Day One

The morning was spent covering Account Management and multiple accounts, leading to AWS Organizations with service control policies. It finished on billing. The next two discussions where around Advanced Networking Architectures, then VPN and DirectConnect. The afternoon finished with a discussion on Deployments on AWS which was an abbreviation of material covered in the DevOps Course.

Day Two

The morning started with data specifically discussing S3 and Elasticache. Next, it was all about data import into AWS with Snowball, Snowmobile, S3 Transfer Acceleration, Storage Gateways(Tape Gateway, Volume Gateway, and File Gateway), and fished with Data Sync, and Database Migration,

The afternoon was spent on Big Data Architecture and Designing Large Scale Applications and finished with a lab on Blue-Green Deployments on Elastic BeanStalk.

Day Three

The last day was spent on Building Resilient Architectures, and encryption and Data Security. The day ended early with a Lab on KMS. The lab provided some basic KMS and OpenSSL encryption steps.

I thought the course, missed an opportunity to talk about DR architectures.

It’s an interesting course and worth taking if you’re interested in learning more or planning to take the certifications.

I took Advanced Architecting on AWS for the last three days. The course is part of the learning process for the AWS Certified Solutions Architect – Professional. I already have the certification based on the older version of the exam. The new version of the certification exam went...

Using Athena to Query ALB Logs

One of the more interesting AWS Big Data Services is Amazon Athena. Athena can process S3 data in a few seconds. One of the ways I like using it is to look for patterns in ALB access logs.

AWS provides a detailed instruction on how to setup Athena on how to setup ALB access logs. I’m not going to recap the configuration in this blog article, but share 3 of my favorite queries.

What is the most visited page by the client and total traffic on my website:

SELECT sum(received_bytes) as total_received, sum(sent_bytes) as total_sent, client_ip, 
count(client_ip) as client_requests, request_url  
FROM alb_logs 
GROUP BY client_ip, request_url  
ORDER BY total_sent  desc;

How long does it take to process requests on average?

SELECT sum(request_processing_time) as request_pt, sum(target_processing_time) as target_pt,
sum (response_processing_time) respone_pt, 
sum(request_processing_time + target_processing_time + response_processing_time) as total_pt, 
count(request_processing_time) as total_requests,
sum(request_processing_time + target_processing_time + response_processing_time) / count(request_processing_time) as avg_pt,
request_url, target_ip
FROM alb_logs WHERE target_ip <> ''
GROUP BY request_url, target_ip 
HAVING COUNT (request_processing_time) > 4 
ORDER BY avg_pt desc;

This last one is looking for requests the site doesn’t process. It’s usually some person trying to find some vulnerable PHP code.

SELECT count(client_ip) as client_requests, client_ip, target_ip, request_url, 
target_status_code 
FROM alb_logs 
WHERE target_status_code not in ('200','301','302','304') 
GROUP BY client_ip, target_ip, request_url, target_status_code
ORDER BY client_requests desc; 

Athena is a serverless tool, and it sets up in seconds and the charges based on TB scanned with a 10MB minimum for the query.

One of the more interesting AWS Big Data Services is Amazon Athena. Athena can process S3 data in a few seconds. One of the ways I like using it is to look for patterns in ALB access logs.

AWS provides a detailed instruction on how to setup Athena on...

DevOps Engineering on AWS

I took DevOps Engineering on AWS for the last three days. The course is part of the learning process for the AWS Certified DevOps Engineer – Professional Overall the course is excellent it covers substantial material, and the labs are ok. To become proficient, one should do the labs from scratch and build the CloudFormation templates. It reviews 45-50% of the material for the on the DevOps Exam, so each topic requires a deeper dive before sitting the exam.

Here is my summary by day of the course.

Day One

The class started with an introduction to DevOps and the AWS tools which support Devops:

It’s interesting as CodeBuild, CodeDeploy, and CodePipeline are required to replace Jenkins. Their advantage is that it directly integrate with AWS. One question I have is why isn’t there a service like Jfrog Artifactory

One of my favorite topics was DevSecOps which talks about adding security into the DevOps process. There should be a separate certification and course for DevSecOps or SecDevOps.

There was a minimum discussion on Elastic Beanstalk, which was a big part of the old acloud.guru course and had several questions on the old exam.

Lastly, the day focused on various methods for updating applications. In-place updates Rolling updates Blue/Green Deployments Red/Black Deployments

Day Two

The class started with a lab on CloudFormation. The lab was flawed as it had a code deployment via the cfn-init and cfn-hup. The rest of the morning was a deeper dive on the tools discussed throughout Day 1.

Afternoon lab focused on a pipeline, CodeBuild, and CodeDeploy. After the lab, we spent time discussing various testing, CloudWatch Logs, and Opsworks. Most of the discussion was theoretical.

Day Three

The first part of the morning was a 2-hour lab on AWS Opsworks setting up a Chef recipe and scaling out the environment. The rest of the class was devoted to containers, primary ECS, with a lab that deployed an application on containers.

It’s an interesting course and worth taking if you’re doing AWS DevOps or planning to take the certifications.

I took DevOps Engineering on AWS for the last three days. The course is part of the learning process for the AWS Certified DevOps Engineer – Professional Overall the course is excellent it covers substantial material, and the labs are ok. To become proficient, one should do the...

Cloud Practioner

Passed the AWS Cloud Partitioner Certification Exam. Given I have 7 of the 9 certifications before sitting this exam, I didn’t study. The goal before taking the exam was 100% in 20 minutes. I missed 3 questions and took 16 minutes. I took the exam at some point I am going to complete the Big Data Speciality, which will give me all the AWS certifications for a brief moment. The Machine Learning AI beta completed last month and the Alexa Skill Builder just completed its beta. This means by March there could be 10 or 11 AWS Certifications.

Passed the AWS Cloud Partitioner Certification Exam. Given I have 7 of the 9 certifications before sitting this exam, I didn’t study. The goal before taking the exam was 100% in 20 minutes. I missed 3 questions and took 16 minutes. I took the exam at some point I...

DevOps Pro Links

I posted to Github a list of links I found valuble when studying for the AWS DevOps Pro certification exam.

The original blog article about passing the test can be found here AWS Certified DevOps Engineer - Professional

I posted to Github a list of links I found valuble when studying for the AWS DevOps Pro certification exam.

The original blog article about passing the test can be found here AWS Certified DevOps Engineer - Professional

AWS Certification SME

AWS Certification SME program helps AWS Certification team, develop the certification exams. It’s a complicated process which as many steps, but I won’t get into now. However, I have now done two workshops on two different steps, one an item writing workshop back in November and now a Standard setting workshop.

The most interesting aspect is fellow partitioners create the exams with certifications, there are people to facilitate, validate and review the information.

The questions are designed to have you apply AWS experience and knowledge of situations. Someone asked if labs would be a replacement, maybe running thru a hundred labs would be the equivalent of real-world experience.

Doing the course, reading all the FAQs and whitepapers and watching all the 400 reinvent videos would be the minimum.

AWS Certification SME program helps AWS Certification team, develop the certification exams. It’s a complicated process which as many steps, but I won’t get into now. However, I have now done two workshops on two different steps, one an item writing workshop back in November and now a Standard...

Starting New Position with AWS

Today I officially started with Amazon Web Services as a Senior Cloud Architect. The position is with Professional Services working with Strategic Accounts.

I am looking forward to helping AWS customers continue to build on their cloud journey.

Today I officially started with Amazon Web Services as a Senior Cloud Architect. The position is with Professional Services working with Strategic Accounts.

I am looking forward to helping AWS customers continue to build on their cloud journey.

AWS re:Invent 2018

Every year 10s of thousands of AWS customers and prospect customers desend on Las Vegas. For those of us to don’t make the trek Amazon live streams the the daily Key Notes. Those are where AWS announces it’s newest products and changes. Each year I build a list before November as AWS has a tendency to leak smaller items. This year my wish list for AWS was as follows:

  1. Mixing sizes and types in ASG - Announced
  2. DNS fixed for Collapsed AD - Announced
  3. Cross regional replication for Aurora PostGreSQL - Regions expanded  still waiting on the cross regions to be announced 
  4. Lambda and more Lambda integrations  - Announced 
  5. AWS Config adding machine learning based on account.  
  6. Account level S3 bucket control - Partly Announced 
  7. 40Gbps Direct Connect 

There a lot of announcements, far too many to recap if interested in them all go read the AWS News Blog.   I do like to find two announcements which shock me and two things that seem interesting. 

The two items which shocked me were:

  1. DynamoDB added transactional support (ACID).   This means someone could build an e-commerce or banking application which requires consistent transactions on dynamoDB.  
  2. AWS Outposts and AWS RDS on VMware allows you to deploy AWS on-premise and AWS will manage this for you.   I can only assume this is to help with migrations or workloads so sensitive they can’t move off-premise.     It would be interesting to see how AWS manages storage capacity and compute resources as many companies struggle with these and how the management model will work.   However, given the push to move away from traditional data centers, so reserves that course.   It will be interesting to see how it plays out over the next year and what services this provides a company migrating to the cloud. 

On my passions is security, so the two things which interested me are

  • AWS Security Hub and  AWS Control Tower  - I consider these one thing as they will be used in tandem.   Control Center will provide security launch zone for an organization while AWS Security Hub will provide governance and monitoring of security 
  • The ARM processor in the a1 instances which Amazon developed internally.   Based on pricing these instances seem to offer cost advantages to the existing instance types.   

What did you find interesting, amusing or shocking?   What were you looking for which wasn’t announced? 

Every year 10s of thousands of AWS customers and prospect customers desend on Las Vegas. For those of us to don’t make the trek Amazon live streams the the daily Key Notes. Those are where AWS announces it’s newest products and changes. Each year I build a list before November...

AWS Certified DevOps Engineer - Professional

Sat the AWS Certified DevOps Engineer - Professional Exam last this afternoon.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best solution for deployments which comprised CloudFormation, Elastic Beanstalk and OpsWorks.   Every one of those questions had 2 good answers, it came down to which was more correct based on the keywords cost, speed, redundancy, roll back capabilities.  

I did the course on acloud.guru and a lot of AWS pages. At some point I will make a page of all the links I collected when studying for this exam.

The exam took me about two-thirds of the allowed time, I read fast and have a tendency to flag questions I don’t know the answer to and come back later and work thru them. This exam, I flagged 20 questions. Most of them I could figure out, once I thought about them for a while. But flagging questions and going back helps manage the time.

Upon submission, I got the “Congratulations! You have successfully completed the AWS Certified DevOps Engineer - Professional…”

I got my score email very quickly:

Overall Score: 82%

Topic Level Scoring:

1.0 Continuous Delivery and Process Automation: 79%
2.0 Monitoring, Metrics, and Logging:  87%
3.0 Security, Governance, and Validation:  75%
4.0 High Availability and Elasticity:  91%

That now makes my 7th AWS Certification.

Sat the AWS Certified DevOps Engineer - Professional Exam last this afternoon.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best solution for deployments which comprised CloudFormation, Elastic Beanstalk and OpsWorks.   Every one of those questions had 2...

What Have you Containerized Today?

I was listening to the Architech podcast.  There was a question asked, ”Does everything today tie back to Kubernetes?”   The more general version of the question is, “Does everything today tie back to containers?”.    The answer is quickly becoming yes.    Something Google figured out years ago with its environment that everything was containerized is becoming mainstream.

To support this  Amazon now has 3 different Container technologies and one in the works.

ECS which is Amazon’s first container offering.    ECS is container orchestration which supports Docker containers.    

Fairgate ECS which is managed offering of ECS where all you do is deploy Docker images and AWS owns full management.  More exciting is that  Fairgate for EKS has been announced and pending release.  This will be a fully managed Kubernetes.    

EKS is the latest offering which was GA’d in June.   This is a fully managed control plane for Kubernetes.   The worker nodes are EC2 instances you manage, which can run an Amazon Linux AMI or one you create.

Lately, I’ve been exploring EKS so that will be the next blog article, how to get started on EKS.

In the meantime, what have you containerized today?

I was listening to the Architech podcast.  There was a question asked, ”Does everything today tie back to Kubernetes?”   The more general version of the question is, “Does everything today tie back to containers?”.    The answer is quickly becoming yes.    Something Google figured out years ago with its...

Data-safe Cloud...

Amazon recently released a presentation on Data-safe Cloud.  It appears to be based on some Gartner question and other data AWS collected.  The presentation discusses 6 core benefits of a secure cloud.

  1. Inherit Strong Security and Compliance Controls
  2. Scale with Enhanced Visibility and Control
  3. Protect Your Privacy and Data
  4. Find Trusted Security Partners and Solutions
  5. Use Automation to Improve Security and Save Time
  6. Continually Improve with Security Features.  

I find this marketing material to be confusing at best, let’s analyze what it is saying. 

For point 1, Inherit Strong and Compliance Controls, which reference all the compliance AWS achieves.  However, it loses track of the shared responsibility model and doesn’t even mention until page 16.   Amazon has compliance in place which is exceptional, and most data center operators or SaaS providers struggle to achieve.   This does not mean my data or services running within the Amazon environment meet those compliances

For point 2,  4  and 6 those are not benefits of the secure cloud.  Those might be high-level objects one uses to form a strategy on how to get to a secure cloud.  

Point 3 I don’t even understand, the protection of privacy and data has to be the number one concern when building out workloads in the cloud or private data centers.   It’s not a benefit of the secure cloud, but a requirement.  

For point 5, I am a big fan of automation and automating everything.   Again this is not a benefit of a secure cloud, but how to have a repeatable, secure process wrapped in automation which leads to a secure cloud.

Given the discussions around cloud and security given all the negative press, including the recent AWS S3 Godaddy Bucket exposure, Amazon should be publishing better content to help move forward the security discussion.  

Amazon recently released a presentation on Data-safe Cloud.  It appears to be based on some Gartner question and other data AWS collected.  The presentation discusses 6 core benefits of a secure cloud.

  1. Inherit Strong Security and Compliance Controls
  2. Scale with Enhanced Visibility and Control
  3. Protect Your Privacy and Data
  4. ...

AWS Release T3 Instances

Earlier today AWS released t3 instances.   There are a bunch of press releases about the topic.    The performance is supposed to be 30% better than T2.   Hopefully, in the next few days, independently published benchmarks will be released to confirm if the instances are 30% faster.   In the interim go to the Amazon pages for all the details on T3 instances.   The cost is a few cents less.   For example, a reserved instance from T2.small to  T3.small with no upfront went from .17 cents to .15 cents in the US-WEST-2 region.    

Before today awsarch.io ran off T2 instances, to build this blog article it was updated to T3 instances.    AWS makes it easy to change instance type, just shut down the instance and from the AWS console go to Instance Settings->Change Instance type.  Then select the appropriate t3 instance.   It can be done via the AWS CLI as well.

Change Instance

T3 force you to select EBS optimized volumes.  EBS optimized volumes for T3 provide additional IOPS.  Here is the link for the complete EBS optimized information.

T3 EBS Optimized

The T3 instance uses an ENA adapter so before starting your instance change the ENA adapter thru the AWS command line:  

aws ec2 modify-instance-attribute –instance-id --ena-support

Lastly, I notice mount points changed.   Previously the EBS volumes devices in the Linux /dev directory changes.   Before the change to T3 they were /dev/xvdf1, /dev/xvdf2, etc.  Now the devices are /dev/nvme1n1p1, /dev/nvme1n1p2, etc.   Something to keep in mind if you have additional volumes with mount points on the ec2 instance. 

Earlier today AWS released t3 instances.   There are a bunch of press releases about the topic.    The performance is supposed to be 30% better than T2.   Hopefully, in the next few days, independently published benchmarks will be released to confirm if the instances are 30% faster.   In the interim go...

AWS Logging Solution

Amazon generates a lot of logs via VPC Flow Logs, CloudTrail, S3 access logs, CloudWatch (See the end of the blog article for a full list.)   Additionally, there are OS, Application, web server logs.   That is a lot of data which provides valuable insight into your running AWS environment.   What are you doing to manage this log files?  What are you doing with those log files?  What are you doing to analysis these log files?

There are a lot of logging solutions available that integrate with AWS. Honestly, I’m a big fan of Splunk and have set it up multiple times.  However, I wanted to look at something else for this blog article. Something open source and relatively low cost. This blog is going to explain what I did to setup Graylog. Graylog has no charges for the software, but you’re going to get charged for the instance, Kinesis, SQS, and data storage.  It actually a good exercise if to familiarize yourself with AWS services, especially for the Sysops exams.  

Graylog provides great instructions.   I followed the steps remember to use their image which is already self-built on Ubuntu.   One difference with this setup, I didn’t use a 4GB memory systems.   I picked a t2.small which proves 1vCPU and 2GB of memory.    I didn’t notice performance issues.  Remember to allow ports 443 and 9000 in security groups and the Networking ACLs.   I prefer to run this over HTTPS.  And it bugs me when you see NOT SECURE HTTP:  I installed an SSL certificate, and this is how I did it.

  1. Create a DNS name 
  2. Get a free certificate 
  3. Install the Certificate as such 

Now my instance is up, and I can log into the console.  I want to get my AWS logs into Graylog.   To do this is requires the logs sent to Kinesis or SQS.  I am not going to explain the SQS setup as there plenty of resources for the specific AWS Service.   Also, the Graylog Plugin describes how to do this.  Graylog plugin for CloudTrail, CloudWatch and VPC Flow logs is available on Github at Graylog Plugin for AWS.

What about access_logs?  Graylog has the Graylog Collector Sidecar.      I’m not going to rehash the installation instructions here as there are great installation instructions.     Graylog has a great documentation.   Also if you are looking for something not covered here, it will be in the documentation or in their Github project. 

What are you using as your log collection processing service on Amazon?  

List of AWS Servers generating logs:

Amazon S3 Access logs Amazon CloudFront Access logs Elastic Load Balancer (ELB) logs Amazon Relational Database Service (RDS) logs Amazon Elastic MapReduce (EMR) logs Amazon Redshift logs AWS Elastic Beanstalk logs AWS OpsWorks logs (or this link) AWS Import/Export logs AWS Data Pipeline logs AWS CloudTrail logs

Amazon generates a lot of logs via VPC Flow Logs, CloudTrail, S3 access logs, CloudWatch (See the end of the blog article for a full list.)   Additionally, there are OS, Application, web server logs.   That is a lot of data which provides valuable insight into your running AWS...

My Favorite Cloud Update for July

All three code platforms AWS, Google Cloud, Azure release features all the time.    However, Google Cloud took a major leap by providing great tool developers by integrating with IntelliJ.   Google did a great job covering the how it works in there Platform blog which is worth reading.

I have used Eclipse since it was released, prior to that I would use Emacs.   However, for my master program over the last 3 years, I have been using IntelliJ.   It’s become my go-to platform for coding work because IntelliJ is easy to use,  and my various class groups typically use it.   IntelliJ makes it free for students, which is a great way to develop a  user base given its price tag.

Providing an easy to use a tool, which has an existing user based was smart by Google Cloud especially as it continues its to close the gap with AWS.

Finally, I’m not a big fan of Cloud9 on AWS.   What do you think?   Are you an IntelliJ or Cloud9 user?

All three code platforms AWS, Google Cloud, Azure release features all the time.    However, Google Cloud took a major leap by providing great tool developers by integrating with IntelliJ.   Google did a great job covering the how it works in there Platform blog which is worth reading.

...

HTTP Get or Post DDoS attacks

DDoS attacks are too frequent on the internet. A DDoS attack sends more requests that can be processed. Many times, the requestors machine has been compromised to be part of a more massive DDoS network. This article is not going to explain all the various types as there is a whole list of them here. Let’s discuss a particular type of DDoS attack designed to overwhelm your web server. This traffic will appear as legitimate requests using GET or POST Requests. GET would be for /index.html or any other page at 50 requests per minute. A POST would hit your myApi.php and attempt to post data at 50 plus requests per minute.

This is going to focus on some recommendations using AWS and other technologies to stop a recent HTTP DDoS attacks. The first step is to identify the DDoS attack versus regular traffic. The second question is how does one prevent a DDoS HTTP attack.

Identifying a DDoS attack there various DDoS The first step is to understand your existing traffic, if you have 2,000 requests per day and all of a sudden you have 2,000,000 requests in the morning, its a good indication it’s under a DDoS attack. The easiest way to identify this is to look at the access_log and pull this into a monitoring service like Splunk, AllenVaultGraylog, etc. From there trend analysis in real-time would show the issues. If the Web servers are behind an ALB make sure the ALB is logging requests and that those requests are being analysis instead of the web server access logs. ALB still support the X-Forwarded-For so it can be passed.

Preventing a DDoS attack There is no way to truly prevent an HTTP DDoS attack.  Specifically to deal with this event, the following mitigation techniques the were explored:

  1. AWS Shield - this provides advance WAF functions, and there rules to limit.

  2. The free-ware would be to use Apache and NGINX have rate limited for specific IP addresses.   In Apache, this is implemented by a number of modules.   ModSecurity is usually at the top of the list, a great configuration example is available on Github which includes the X-Forwarded-For.

  3. An EC2 instance in front of the web server can be run as a Proxy. The proxy can be configured to suppress the traffic using ModSecurity or other MarketPlace offerings including other WAF options.

  4. The ALB or CloudFront can deploy an AWS WAF.

  5. Lastly, the most expensive option is to deploy Auto-scaling groups to absorb all traffic.

Please leave a comment if there other options which should have been investigated.

To solve this specific issue, an AWS WAF was deployed on the ALB.   One thing to consider is to make sure to prevent attacks from directly hitting the website.   This is easily accomplished by allowing HTTP/HTTPS from anywhere only to the ALB.    ALB and EC2 instance sharing a security group which allows HTTPS/HTTP to everything in that security group.

DDoS attacks are too frequent on the internet. A DDoS attack sends more requests that can be processed. Many times, the requestors machine has been compromised to be part of a more massive DDoS network. This article is not going to explain all the various types as there is a...

Starting a new position today

Starting a new position today as Consultant - Cloud Architect with Taos.   Super excited to for this opportunity.

I wanted a position as a solution architect working with the Cloud, so I couldn’t be more thrilled with the role.   I am looking forward to helping Taos customers adopt the cloud and a Cloud First Strategy.

It’s an amazing journey for me, as Taos was the first to offer me a Unix System administrator position when I graduated from Penn State some 18 years ago, and I passed on the offer and went to work for IBM.

I am really looking forward to working with the great people at Taos.

Starting a new position today as Consultant - Cloud Architect with Taos.   Super excited to for this opportunity.

I wanted a position as a solution architect working with the Cloud, so I couldn’t be more thrilled with the role.   I am looking forward to helping Taos customers adopt the...

My Favorite Things About Amazon Well Architected Framework

Amazon released AWS Well Architected Framework to help customers Architect solutions within AWS.   The amazon certifications require detailed knowledge of 5 white papers which make up the Well Architected Framework.   Given I have recently completed 6 Amazon certifications, I decided I was going to write a blog which pulled my favorite lines from each paper.

Operational excellence pillar The whitepaper says on page 15, “When things fail you will want to ensure that your team, as well as your larger engineering community, learns from those failures.”   It doesn’t say “If things fail”, it says “When things fail” implying straight away things are going to fail.

security pillar On page 18, “Data classification provides a way to categorize organizational data based on levels of sensitivity. This includes understanding what data types are available, where is the data located and access levels and protection of the data”.  This to me sums up how security needs to be defined. Modern data security is not about firewalls and having a hard outside shell or malware detectors.  It about protecting the data based on its classification from both internal (employees, contractors, vendors) actors and hostile actors.

reliability pillar The document is 45 pages long and the word failure appears 100 times and the word fail exists 33 times. The document is really about how to architect an AWS environment to respond to failure and what portion of your environment based on business requirements should be over-engineered to withstand multiple failures.

performance efficiency pillar Page 24 the line, “When architectures perform badly this is normally because of a performance review process has not been put into place or is broken”.   When I first read this line, I was perplexed.  I immediately thought this implies a bad architecture can perform well if there is a performance review in place.  Then I thought when has a bad architecture ever performed well under load?   Now I get the point this is trying to make.

cost optimization On page 2, is my favorite line from this white paper, “A cost-optimized system will fully utilize all resources, achieve an outcome at the lowest possible price point, and meet your functional requirements.”   It made me immediately think back to before the cloud, every solution had to have a factor over the life of hardware for growth it was part of the requirements.    In the cloud you need to support capacity today, if you need more capacity tomorrow, you just scale. This is one of the biggest benefits of cloud computing, no more guessing about capacity.

Amazon released AWS Well Architected Framework to help customers Architect solutions within AWS.   The amazon certifications require detailed knowledge of 5 white papers which make up the Well Architected Framework.   Given I have recently completed 6 Amazon certifications, I decided I was going to write a blog which pulled my...

AWS Certified Security – Specialty


Sat the AWS Certified Security - Speciality Exam this morning.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best security scenario.   It could be renamed the Certified Architect - Security.    Every one of those questions had 2 good answers, it came down to which was more correct and more secure.       It’s the hardest exam I’ve taken to date.   I think it is harder than the Solution Architect - Professional exam. The majority of the exam questions where on KMS, IAM, securing S3, CloudTrail, CloudWatch, multiple AWS account access, Config, VPC, security groups, NACLs, and WAF.

I did the course on acloud.guru and I think the whitepapers and links really helped me in the studying for this exam:

The exam took me about half the allocated time, I read fast and have a tendency to flag questions I don’t know the answer to and come back later and work thru them.    This exam, I flagged 20 questions, highest of any AWS exam taken to date.     Most of them I could figure out, once I thought about them for a while.      Thru the exam, I was unsure of my success or failure.

Upon submission, I got the “Congratulations! You have successfully completed the AWS Certified Security - Specialty exam…”

Unfortunately, I didn’t get my score, I got the email, which says, “Thank you for taking the  AWS Certified Security - Specialty exam. Within 5 business days of completing your exam,”

That now makes my 6th AWS Certification.


Sat the AWS Certified Security - Speciality Exam this morning.  The exam is hard, as it scenario based.   Most of the exam questions were to pick the best security scenario.   It could be renamed the Certified Architect - Security.    Every one of those questions...

AWS Config, KMS and EBS encryption

If you have an AWS deployment, make sure you turn on AWS Config.       It has a whole bunch of built-in rules, and you can add your own to validate the security of your AWS environment as it relates to AWS services.   Amazon provides good documentation, a GitHub repo,  and SumoLogic does a quick How-to turn it on.      It’s straightforward to turn on and use.   AWS provides some pre-configured rules, and that’s what this AWS environment will validate against.  There is a screenshot below of the results.   Aside from turning it on, you have to decide which rules are valid for you.   For instance, not all S3 buckets have business requirements to replicate, so I’d expect this to always be a noncompliant resource.However, one of my findings yesterday was missing EBS encrypted volumes. In order to make EBS volumes encrypted its 9 easy steps:

  1. Make a snapshot of the EBS Volumes.

  2. Copy the snapshot of the EBS Volume to a Snapshot, but select encryption.   Use the AWS KMS key you prefer or Amazon default aws/ebs.

  3. Create an AMI image from the encrypted Snapshot.

  4. Launch the AMI image from the encrypted Snapshot to create a new instance.

  5. Check the new instance is functioning correctly, and there are no issues.

  6. Update EIPs, load balancers, DNS, etc. to point to the new instance.

  7. Stop the old un-encrypted instances.

  8. Delete the un-encrypted snapshots.

  9. Terminate the old un-encrypted instances.

Remember KMS gives you 20,000 request per month for free, then the service is billable.

If you have an AWS deployment, make sure you turn on AWS Config.       It has a whole bunch of built-in rules, and you can add your own to validate the security of your AWS environment as it relates to AWS services.   Amazon provides good documentation, a

Serverless 2018

Serverless is becoming the 2018 technology hype.   I remember when containers were gaining traction in 2012, and Docker in 2013.  At technology conventions, all the cool developers were using containers.   It solved a lot of challenges, but it was not a silver bullet. (But that’s a blog article for another day.)

Today after an interview I was asking myself,  have Containers lived up to the hype?   They are great for CI/CD, getting rid of system administrator bottlenecks, helping with rapid deployment, and some would argue fundamental to DevOps.  So I started researching the hype.   People over at  Cloud Foundry published a container report in  2017 and 2016.

Per the 2016 report, “our survey, a majority of companies (53%) had either deployed (22%) or were in the process of evaluating (31%) containers.”

Per the 2017 report, “increase of 14 points among users and evaluators for a total of 67 percent using  (25%) or evaluating (42%).”

As a former technology VP/director/manager, I was always evaluating technology which had some potential to save costs, improve processes, speed development and improve production deployments.   But a 25% adaption rate and a 3% uptick over last year, is not moving the technology needle.

However, I am starting to see the same trend, Serverless is the new exciting technology which is going to solve the development challenges, save costs, improve the development process and you are cool if you’re using it.       But is it really Serverless or just a simpler way to use a container?

AWS Lambda is basically a container.  (Another blog article will dig into the underpinnings of Lambda.)   Where does the container run? ** A Server. **

Just means I don’t have to understand the underlying container, server etc.etc.etc.     So is it truly serverless?   Or is it just the 2018 technology hype to get all us development geeks excited, we don’t need to learn Docker or Kubernetes, or ask our Sysadmin friends provision us another server.

Let me know your thoughts.

Serverless is becoming the 2018 technology hype.   I remember when containers were gaining traction in 2012, and Docker in 2013.  At technology conventions, all the cool developers were using containers.   It solved a lot of challenges, but it was not a silver bullet. (But that’s a blog article for another...

AWS Certified Solutions Architect – Professional

I sat the AWS Certified Solutions Architect - Professional exam this morning.   This exam is hard, probably the hardest of the AWS exams I have taken to date.    I did it in about half the allowed time.   Generally, the test is challenging as it covers a lot of topics and each answer always had two correct choices.   The entire exam is a challenge to pick the more correct answer based on the scenario and question with a driving factor of one more or more of the following,  scalability, cost, recovery time, performance or security.

I felt like I passed the exam while doing it, but its always a relief to see:

Congratulations! You have successfully completed the AWS Certified Solutions Architect - Professional exam and you are now AWS Certified.

Here is my score breakdown from the exam.

Topic Level Scoring:
1.0 High Availability and Business Continuity: 81%
2.0 Costing: 75%
3.0 Deployment Management: 85%
4.0 Network Design: 85%
5.0 Data Storage: 81%
6.0 Security: 85%
7.0 Scalability & Elasticity: 63%
8.0 Cloud Migration & Hybrid Architecture: 57%

I sat the AWS Certified Solutions Architect - Professional exam this morning.   This exam is hard, probably the hardest of the AWS exams I have taken to date.    I did it in about half the allowed time.   Generally, the test is challenging as it covers a lot of topics and...

AWS Certified SysOps Administrator - Associate

I sat the AWS Certified SysOps Administrator - Associate this morning.   That makes two exams this week in 3 days.

The exam was a little bit harder than the two other Associate exams as it went a level deeper.   It focused on CloudFormation, CloudWatch, and deployment strategies.      There were nine questions I struggled with the right answer, as all nine had two good answers.     There were about 35 questions I knew cold.    There were three questions duplicated on the other associate exams.   All of the network questions I was over-thinking, probably based on the networking exam this week.    Given this, I wasn’t worried when I ended the test.   However,  it’s always a relief when you get the Congratulations! You have successfully completed the AWS Certified SysOps Administrator - Associate.

Within 10 minutes I got my score email:

Congratulations again on your achievement!

Overall Score: 84%
Topic Level Scoring:
1.0 Monitoring and Metrics: 80%
2.0 High Availability: 83%
3.0 Analysis: 100%
4.0 Deployment and Provisioning: 100%
5.0 Data Management: 83%
6.0 Security: 100%
7.0 Networking: 42%

The score reflected over thinking the networking questions.    I wouldn’t recommend sitting two different exams in the same few days.

That make 4 AWS certifications in 3 weeks:

  • AWS Certified SysOps Administrator - Associate
  • AWS Certified Advanced Networking - Specialty
  • AWS Certified Developer - Associate
  • AWS Certified Solutions Architect - Associate (Released February 2018)

Guess now it’s time to focus on the last of the Amazon Certifications I’ll work on for now which is the  AWS Certified Solutions Architect – Professional.

I sat the AWS Certified SysOps Administrator - Associate this morning.   That makes two exams this week in 3 days.

The exam was a little bit harder than the two other Associate exams as it went a level deeper.   It focused on CloudFormation, CloudWatch, and deployment strategies.      There were...

Studying for the AWS Certified SysOps Administrator – Associate

The material for the AWS Certified SysOps Administrator – Associate seems to be a lot of the material cover under the Associate Architect and Associate Developer.   I would have thought the material more focus on setting up and troubleshooting issues with EC2, RDS, ELB, VPC etc.    It also spends a lot of time looking at CloudWatch, but doesn’t really provide strategies for leveraging the logs.  Studying was a combination of the acloud.guru and the official study guide, and the Amazon Whitepapers.

I took the AWS supplied practice test using a free test voucher and score the following:

Congratulations! You have successfully completed the AWS Certified SysOps Administrator Associate - Practice Exam
Overall Score: 90%
Topic Level Scoring: 1.0 Monitoring and Metrics: 100% 2.0 High Availability: 100% 3.0 Analysis: 66% 4.0 Deployment and Provisioning: 100% 5.0 Data Management: 100% 6.0 Security: 100% 7.0 Networking: 66%

It interesting the networking score was so low as I just passed the Network Speciality.

This is the last Associate exam to pass for me.    If I successfully pass it, I will begin the process of studying for the Certified Solution Architect - Professional.    That will probably be my last AWS certification as I’ll look at either starting on something like  TOGAF certification,  Redhat or Linux Institute, Cisco, GCP  or Azure, depending on where my interest lies in a few weeks.

The material for the AWS Certified SysOps Administrator – Associate seems to be a lot of the material cover under the Associate Architect and Associate Developer.   I would have thought the material more focus on setting up and troubleshooting issues with EC2, RDS, ELB, VPC etc.    It also spends a...

Passed the AWS Certified Advanced Networking – Specialty Exam

I passed the AWS Certified Advanced Networking – Specialty Exam this morning.    The exam is hard.   My career started with a  networking as I had multiple Nortel and Cisco Certifications and was studying to the CCIE Lab back then.  But over the last 12 years,  I got away from networking.    Doing this exam was going back to something I loved for a long time, as  BGP, Networking, Load Balancers, WAF makes me excited.

My exam results

Topic Level Scoring:
1.0  Design and implement hybrid IT network architectures at scale: 75%
2.0  Design and implement AWS networks: 57%
3.0  Automate AWS tasks: 100%
4.0  Configure network integration with application services: 85%
5.0  Design and implement for security and compliance: 83%
6.0  Manage, optimize, and troubleshoot the network: 57%

I have limited experience with AWS networking prior to this exam.   I had the standard things likes load balancers, VPCs, Elastic IPs and Route 53.   This exam tests your knowledge of these areas and more.      To prepare I used the acloud.guru course, also the book  AWS Certified Advanced Networking Official Study Guide: Specialty Exam and the Udemy Practice Tests.    With the course and book, I set up VPC peers, Endpoints, nat instances, gateways, CloudFront distributions.    I put about 50 hours into doing the course, reading the book, doing various exercise, and studying etc.

Based on my experience the acloud.guru course is lacking the details on the ELBs, the WAF, private DNS, and implementation within CloudFormation.     The book comes closer to the exam, but also doesn’t cover CloudFormation, WAF or ELBs as deep as the exam.   The Udemy practice tests were close to the exam, but lack some of the more complex scenario questions.

I plan to sit the AWS Certified SysOps Administrator - Associate exam later this week.

I passed the AWS Certified Advanced Networking – Specialty Exam this morning.    The exam is hard.   My career started with a  networking as I had multiple Nortel and Cisco Certifications and was studying to the CCIE Lab back then.  But over the last 12 years,  I got away from networking.   ...

Finally got my AWS Certified Solution Architect - Associate Results

The pdf provided this:

The AWS Certified Solutions Architect - Associate (Released February 2018) (SAA-C01) has a scaled score between 100 and 1,000. The scaled score needed to pass the exam is 720.

I got a 932….

The pdf provided this:

The AWS Certified Solutions Architect - Associate (Released February 2018) (SAA-C01) has a scaled score between 100 and 1,000. The scaled score needed to pass the exam is 720.

I got a 932….

Exam For AWS Certified Developer – Associate

I sat the exam for the AWS Certified Developer - Associate this morning.     I felt lucky as the system kept asking questions I knew in depth.   There were only 4 questions I didn’t know the answer to and took an educated guess.

I did the exam in 20 minutes for 55 questions.    I only review questions I flag, and I only flagged about 8 questions.    I felt really lucky as the exam was playing to my knowledge of DynamoDB, S3, EC2, and IAM.   There were other questions about Lambda, CloudFormation, CloudFront, and API calls.   But the majority of the questions focused on 4 areas of AWS, I knew really well.

At the end of the exam, I got the Congratulations have successfully completed the AWS Certified Developer  – Associate exam.

Also within 15 minutes, I got the email confirming my score:

Congratulations again on your achievement!

Overall Score: 90%
1.0 AWS Fundamentals: 100%
2.0 Designing and Developing: 85%
3.0 Deployment and Security: 87%
4.0 Debugging: 100%

I’m still waiting on my score from my Solution Architect - Associate Exam.    In the meantime, I’ll get back to studying my AWS Networking Speciality.

I sat the exam for the AWS Certified Developer - Associate this morning.     I felt lucky as the system kept asking questions I knew in depth.   There were only 4 questions I didn’t know the answer to and took an educated guess.

I did the exam in 20 minutes...

AWS Practice Test for Certified Developer – Associate

AWS offers practices emails through PSI exams.   Cost $20 and gives you 20 questions for practice.    I did the exam today.   Here is the results email.

Congratulations! You have successfully completed the AWS Certified Developer Associate - Practice Exam
Overall Score: 95%
Topic Level Scoring: 1.0  AWS Fundamentals: 100% 2.0  Designing and Developing: 87% 3.0  Deployment and Security: 100% 4.0  Debugging: 100%

That’s a confidence builder going into the exam tomorrow morning.

AWS offers practices emails through PSI exams.   Cost $20 and gives you 20 questions for practice.    I did the exam today.   Here is the results email.

...

Back to Studying for my Developer Exam

I had scheduled the test for June 14 for AWS Certified Developer – Associate.   I need to stop studying the Network information and finish studying for the developer exam.    I had completed the   https://acloud.guru/ course on Sunday.      I decided to purchase  AWS Certified Developer - Associate Guide: Your one-stop solution to passing the AWS developer’s certification

The book was good, it covers all the major topics for the associate developer certification, but it lacks hands-on lab and there are several errors in the mock exams.

I had scheduled the test for June 14 for AWS Certified Developer – Associate.   I need to stop studying the Network information and finish studying for the developer exam.    I had completed the   https://acloud.guru/ course on Sunday.      I decided to purchase  AWS Certified Developer - Associate Guide: Your...

Waiting for Score Email

Still waiting on the score from the AWS Certified Solution Architect – Associate exam.

However, I started also studying for the AWS Certified Advanced Network - Speciality.

I love networks and networking, especially VPNs and BGP.     So I felt it was a good challenge as well as something I enjoyed doing.    12 years ago, I had multiple Cisco routers on my desk and would run BGP configurations, OSPF and EIGRP configurations.       Maybe I need an AWS DirectConnect…..

Still waiting on the score from the AWS Certified Solution Architect – Associate exam.

However, I started also studying for the AWS Certified Advanced Network - Speciality.

I love networks and networking, especially VPNs and BGP.     So I felt it was a good challenge as well as something I enjoyed...

AWS Certified Solutions Architect – Associate

I sat the AWS Certified Solution Architect - Associate exam.    It was challenging as it covers a broad set of AWS services.   I sat the February 2018 version which is the new one.

At the end of the exam, I got a Congratulations have successfully completed the AWS Certified Solution Architect - Associate exam.

I decided that I would complete the AWS Certified Developer - Associate next.

I sat the AWS Certified Solution Architect - Associate exam.    It was challenging as it covers a broad set of AWS services.   I sat the February 2018 version which is the new one.

At the end of the exam, I got a Congratulations have successfully completed the AWS Certified Solution...

Congratulations! You have successfully completed the AWS Certified Developer Associate - Practice Exam
Overall Score: 95%
Thank you for taking the AWS Certified Solutions Architect - Associate - Practice (Released February 2018) exam. Please examine the following information to determine which topics may require additional preparation.
Overall Score: 80%
Topic Level Scoring: 1.0 Design Resilient Architectures: 100% 2.0 Define Performant Architectures: 71% 3.0 Specify Secure Applications and Architectures: 66% 4.0 Design Cost-Optimized Architectures: 50% 5.0 Define Operationally-Excellent Architectures: 100%

I was a little concerned after the practice exam.   I spent the rest of the evening studying.    There various blogs which talk about the exam, but it seems depending on the day, exam, the location you could need anywhere from a 65% to a 72% to pass the exam.    Based on the practice I didn’t have a lot of room for error.

AWS offers practices emails through PSI exams.   Cost $20 and gives you 20 questions for practice.    I did the practice exam today.   Here is the results email.

Thank you for taking the AWS Certified Solutions Architect - Associate - Practice (Released February 2018) exam. Please examine the...