Archive of posts from February 2019

Advanced Architecting on AWS

February 23 2019 Greg Fina Leave a comment

I took Advanced Architecting on AWS for the last three days. The course is part of the learning process for the AWS Certified Solutions Architect – Professional. I already have the certification based on the older version of the exam. The new version of the certification exam went live on February 4th. The course seems to follow the newer certification guide. Overall the course is good as it covers all the services required, the labs were a little disappointing as they lacked complexity. To become proficient and attempt the certification, one would need to a lot more learning and deep diving on the topics covered in this course. It reviews probably 35% of the material required to sit the exam.

Here is my summary by day of the course.

Day One

The morning was spent covering Account Management and multiple accounts, leading to AWS Organizations with service control policies. It finished on billing. The next two discussions where around Advanced Networking Architectures, then VPN and DirectConnect. The afternoon finished with a discussion on Deployments on AWS which was an abbreviation of material covered in the DevOps Course.

Day Two

The morning started with data specifically discussing S3 and Elasticache. Next, it was all about data import into AWS with Snowball, Snowmobile, S3 Transfer Acceleration, Storage Gateways(Tape Gateway, Volume Gateway, and File Gateway), and fished with Data Sync, and Database Migration,

The afternoon was spent on Big Data Architecture and Designing Large Scale Applications and finished with a lab on Blue-Green Deployments on Elastic BeanStalk.

Day Three

The last day was spent on Building Resilient Architectures, and encryption and Data Security. The day ended early with a Lab on KMS. The lab provided some basic KMS and OpenSSL encryption steps.

I thought the course, missed an opportunity to talk about DR architectures.

It’s an interesting course and worth taking if you’re interested in learning more or planning to take the certifications.

Security,Technology

Violating Security Policies

February 18 2019 Greg Fina Leave a comment

Dark Reading wrote a Blog Architect entitled 6 Reasons Why Employees Violate Security Policies The 6 reasons according to the article are:

Ignorance
Convenience
Frustration
Ambition
Curiosity
Helpfulness

I think they’re neglecting to get to the root of the issue which is draconian security policies which don’t make things more secure. Over the years, I’ve seen similar policies coming from InfoSec groups. It’s common for developers to want to use the tools they’re comfortable with, in an extreme case I’ve seen developers wanting to use Eclipse to do development and Eclipse is forbidden because the only safe editor according to some InfoSec policy is VI (probably slightly exaggerated). Other extreme cases include banning of Evernote or OneNote because it uses cloud storage. I’m assuming in this that someone is not putting all there confidential customer data in a OneNote book.

Given what I’ve seen, employee violates security policies to get work done, the way they want to do it. Maybe that ignorance, convenience, frustration, ambition, or any other topic, or maybe if you’ve used something for 10 years, you don’t want to have to learn something new for development or keeping notes, given there are many other things to learn and do which add value to their job and employer.

Maybe to keep employees from violating InfoSec policies, InfoSec groups instead of writing draconian security policies could focus on identifying security vulnerabilities which are more likely targets of hackers, putting policies, procedures and operational security around them. Lastly, InfoSec could spend time educating what confidential data is and where it is allowed to stored.

Disclaimer: This blog article is not meant to condone, encourage, or motivate people to violate security policies.

Dark Reading wrote a Blog Architect entitled 6 Reasons Why Employees Violate Security Policies The 6 reasons according to the article are:

Ignorance
Convenience
Frustration
Ambition
Curiosity
Helpfulness

I think they’re neglecting to get to the root of the issue which is draconian security policies which don’t...

Technology

What is the difference between a CDO, CTO and a CIO?

February 16 2019 Greg Fina Leave a comment

I got into an interesting discussion on what is the difference between a CDO, CTO, and CIO. The initial discussion started with are all those positions required in an organization. The group eventually agreed the answer was yes. The logic was given everything we do is digital, digital needs multiple seats at the executive table. The reason for this blog article is where do these roles fit within an organization. Let’s take a step back and explained how we defined the roles.

CDO should own e-commerce, mobile environments, and technology customer outreach. In a digital product company, they own the product roadmap. The CDO is responsible for all digital customer touch points. The technology partner for the CDO is the CMO or SVP of Sales. This role should be driving the business, and be a business enabler.

CIO should own the back office technology like email, ERP, messaging, desktops, laptops, printers, networking, service desks, and traditional data centers. Typically technology organization which is the cost centers.

CTO should own the architecture and technology of the platforms. CTO is the technology partner for both the CDO and CIO. Their job should be to have uniformity, coalesce ideas across technology and work with the various stakeholders to ensure proper architecture governance (think TOGAF architecture review boards).

The group when discussing it was pretty emphatic, the CDO should report to the CEO. Now, this is where the issue with the outstanding roles breaks down. The role defined for the CIO is an operational role, making sure essential infrastructure services and users can function. The group was split 50/50, and half the group thought the CIO should report to the CDO, the other half said some other C-level executive, like the CFO or COO.

The more complicated issue is where does the CTO report. The CTO is responsible for the architecture and technology of the platform which makes them a partner of the CDO, but also owns architecture review which makes them a partner of the CIO. So where does the CTO report?

The CDO has an entirely different objective than the CIO. If the CIO reports to the CDO, it would make sense to have the CTO report there. However, what happens when the CIO doesn’t report to the CDO. What happens if the CIO reports to the COO?

After several rounds of mental gymnastics, the group agreed to coalesce around two outcomes. First, the CIO either reports to the CDO, and the CTO reports to the CDO. Basically, CTO and CIO become peers in the same organization. The other was the CIO reports to the CTO and both the CTO and CDO report to the CEO.

AWS,Certification,Big Data,Technology

Using Athena to Query ALB Logs

February 11 2019 Greg Fina Leave a comment

One of the more interesting AWS Big Data Services is Amazon Athena. Athena can process S3 data in a few seconds. One of the ways I like using it is to look for patterns in ALB access logs.

AWS provides a detailed instruction on how to setup Athena on how to setup ALB access logs. I’m not going to recap the configuration in this blog article, but share 3 of my favorite queries.

What is the most visited page by the client and total traffic on my website:

SELECT sum(received_bytes) as total_received, sum(sent_bytes) as total_sent, client_ip, 
count(client_ip) as client_requests, request_url  
FROM alb_logs 
GROUP BY client_ip, request_url  
ORDER BY total_sent  desc;

How long does it take to process requests on average?

SELECT sum(request_processing_time) as request_pt, sum(target_processing_time) as target_pt,
sum (response_processing_time) respone_pt, 
sum(request_processing_time + target_processing_time + response_processing_time) as total_pt, 
count(request_processing_time) as total_requests,
sum(request_processing_time + target_processing_time + response_processing_time) / count(request_processing_time) as avg_pt,
request_url, target_ip
FROM alb_logs WHERE target_ip <> ''
GROUP BY request_url, target_ip 
HAVING COUNT (request_processing_time) > 4 
ORDER BY avg_pt desc;

This last one is looking for requests the site doesn’t process. It’s usually some person trying to find some vulnerable PHP code.

SELECT count(client_ip) as client_requests, client_ip, target_ip, request_url, 
target_status_code 
FROM alb_logs 
WHERE target_status_code not in ('200','301','302','304') 
GROUP BY client_ip, target_ip, request_url, target_status_code
ORDER BY client_requests desc; 

Athena is a serverless tool, and it sets up in seconds and the charges based on TB scanned with a 10MB minimum for the query.

One of the more interesting AWS Big Data Services is Amazon Athena. Athena can process S3 data in a few seconds. One of the ways I like using it is to look for patterns in ALB access logs.

AWS provides a detailed instruction on how to setup Athena on...

Technology,AWS,Certification

DevOps Engineering on AWS

February 8 2019 Greg Fina Leave a comment

I took DevOps Engineering on AWS for the last three days. The course is part of the learning process for the AWS Certified DevOps Engineer – Professional Overall the course is excellent it covers substantial material, and the labs are ok. To become proficient, one should do the labs from scratch and build the CloudFormation templates. It reviews 45-50% of the material for the on the DevOps Exam, so each topic requires a deeper dive before sitting the exam.

Here is my summary by day of the course.

Day One

The class started with an introduction to DevOps and the AWS tools which support Devops:

It’s interesting as CodeBuild, CodeDeploy, and CodePipeline are required to replace Jenkins. Their advantage is that it directly integrate with AWS. One question I have is why isn’t there a service like Jfrog Artifactory

One of my favorite topics was DevSecOps which talks about adding security into the DevOps process. There should be a separate certification and course for DevSecOps or SecDevOps.

There was a minimum discussion on Elastic Beanstalk, which was a big part of the old acloud.guru course and had several questions on the old exam.

Lastly, the day focused on various methods for updating applications. In-place updates Rolling updates Blue/Green Deployments Red/Black Deployments

Day Two

The class started with a lab on CloudFormation. The lab was flawed as it had a code deployment via the cfn-init and cfn-hup. The rest of the morning was a deeper dive on the tools discussed throughout Day 1.

Afternoon lab focused on a pipeline, CodeBuild, and CodeDeploy. After the lab, we spent time discussing various testing, CloudWatch Logs, and Opsworks. Most of the discussion was theoretical.

Day Three

The first part of the morning was a 2-hour lab on AWS Opsworks setting up a Chef recipe and scaling out the environment. The rest of the class was devoted to containers, primary ECS, with a lab that deployed an application on containers.

It’s an interesting course and worth taking if you’re doing AWS DevOps or planning to take the certifications.

Technology,DevOps,Software

Goodnotes 5

February 2 2019 Greg Fina Leave a comment

Goodnotes 5 was released last week. Goodnotes is my favorite stylus note taking app on the IPad. I’ve tried most of the competitors at least once and revisit them when they release new features. I’ve been on Goodnotes for years and have been using it daily.

Let’s move to the topic of this blog, Goodnotes 5 is a bit buggy. There were a ton of negative comments on Twitter towards the release. The development team has released 7 updates as of the writing of this blog. Goodnotes 5 is not a forced upgrade from version 4. While I’ve not seen all the problems described on Twitter, I’ve seen a few of the issues. I knew installing the initial release, and there were going to be some bugs.

However if you think about the DevOps model release, fix, release, fix, release, fix. The model is built for this type of release and user feedback.

However, many of the twitter complaints, where why was buggy software released. So it made me think about when is software ready for release in the DevOps model? Typically there is a release once code passes, unit tests, integration tests, load tests, functional tests, and GUI Tests. However bugs do reach production and the users, there is no fool-proof plan.

App store doesn’t allow releasing of beta software. However does offer TestFlight, so maybe GoodNotes could have leverage 10,000 of its customers to beta test the software and avoided the negative backlash on Twitter.

Let’s move...