Amazon Web Services Platform & DevOps - Automation and Monitoring

Continuing from my previous blog on Amazon Web Services Platform & DevOps: Automation and Monitoring

Monitoring

• With AWS platform, feedback is provided by two services - Amazon CloudWatch and AWS CloudTrail
• They provide monitoring, alerting, and auditing infrastructure
• Amazon CloudWatch:
• Is a monitoring service for AWS cloud resources and applications that we run on AWS
• CloudWatch can monitor the log files for any production incidents
• CloudWatch allows to set alarm and automatically react to any changes to AWS resources (Amazon DynamoDB tables, Amazon RDS DB, etc)
•  It provides visibility of application performance and operational health ensuring your application is running smoothly
• AWS CloudTrail:
• It’s important to understand who is making modifications to your infrastructure
• All AWS interactions are handled through AWS API calls that are monitored and logged by AWS CloudTrail
• With CloudTrail, you can get history of AWS API calls for your account
• All generated log files are stored in an Amazon S3 bucket
• Log files are encrypted using Amazon S3
• The AWS API call history produced by CloudTrail enables security analysis, tracking of resource change, & compliance auditing

Security

• In DevOps environment the focus on security is vital
• Identity and Access Management (IAM): IAM is part of the AWS security infrastructure
• IAM allows you to centrally manage users and security credentials viz passwords, access keys, & permissions that control which AWS services and resources users can access
• IAM can be used to create roles
• IAM role you can define a set of permissions to access the resources that a user or service needs. But instead of attaching the permissions to a specific user or group, you attach them to a named role
• Resources can be associated with roles and services

Amazon Web Services Platform & DevOps : Get Started

• For meeting the demands of the world of Agile business, IT operations need to deploy applications in a reliable, consistent and repeatable way 

• Automation of processes is one of the best way forward in achieving the above. 

• AWS provides the capability to support various DevOps concepts, principles and practices that IT companies can take advantage of to get better business agility.  

• Contents of this paper focuses on the capability provided by AWS platform to support DevOps.

• DevOps is a combination of development and testing operations which is part of SDLC software development life cycle.

• DevOps focuses and stresses majorly on communication, collaboration, and integration between software developers and IT professionals so as to enable rapid developments of products / services, reduce costs, improve quality and reduce risk.

• It’s a cultural change

• If we look back, historically organisations were structured in a way there was very little or poor integration between the various teams like development team, service support team, infrastructure and information security teams as all the teams had different goals.

…moving to AWS platform

• Application code has a defined format and syntax. If the code is not written according to the rules of the programming language, applications cannot be created. Code is saved in a version management system that logs a history of code development, changes, and defect fixes. When code is compiled into applications, expect a consistent application to be created. That is to ensure that the build is repeatable and reliable.
• All configurations should be defined in a declarative way and stored in a version management system, just like application code
• Infrastructure provisioning and deployment should support the use of the infrastructure code
• AWS platform provides a DevOps focused way of creating and maintaining infrastructure. Similar to the way software developers write application code, AWS provides services that enable the creation, deployment and maintenance of infrastructure in a programmatic, descriptive, and declarative way
• DevOps principles are used in practice in AWS CloudFormation . AWS CloudFormation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.
• AWS templates are written in JavaScript Object Notification (JSON). Using the AWS CloudFormation templates, you can define and model AWS resources that can be created and updated.
• Using templates, you can provision infrastructure in a repeatable and reliable way

AWS CloudFormation Templates

• You can create templates for your service / application architectures
• AWS CloudFormation use your templates for provisioning of stacks (services or applications)
• You can also easily update / replicate the stacks on need basis
• You can manage stacks through the AWS Management Console, AWS Command Line Interface, or AWS CloudFormation APIs. Some of the familiar actions include create-stack, describe-stacks, list-stacks, and update-stack
• AWS provides sample Templates & Snippets
• Sample Templates:
• Auto Scaling: 
• Load-based auto scaling: Creates an Auto Scaling group with scaling policies that are based on CPU usage in order to start and stop Amazon EC2 instances, which span multiple Availability Zones. This template also uses Auto Scaling notifications so that you are emailed when scaling events occur.
• Time-based auto scaling (scheduled actions): Creates an Auto Scaling group with recurring schedules that change the minimum and maximum number of Amazon EC2 instances in the group.
• Auto Scaling group with an update policy: Creates an Auto Scaling group with an update policy that keeps 2 instances running during a rolling update.
• Amazon DynamoDB:
• DynamoDB table: Creates a simple DynamoDB table
• Elastic Load Balancing:
• Load-balanced Auto Scaling group: Creates an Elastic Load Balancing load balancer and an Auto Scaling group that receives traffic only from the load balancer.
• Amazon Simple Storage Service (Amazon S3)
• Amazon Elastic Compute Cloud (EC2)
• When you create or update a stack in the console, events are displayed showing the status of the configuration. If an error occurs, the stack is rolled back to its previous state 
• Amazon Simple Notification Service (Amazon SNS) helps to manage these events. Amazon SNS can be used to track stack creation and deletion progress via email and integrate with other processes programmatically

Continuous Deployment

• Continuous deployment is one of the core concept of DevOps strategy, it usually refers to production deployments
• Goal is to enable the automated deployment of production-ready application code
• With continuous delivery practices and tools, code / software can be deployed quickly, repeatedly, and reliably. In case of any deployment failure, it can be automatically rolled back to previous version
• AWS CodeDeploy: its features provides the ability to deploy applications an Amazon E2C fleet with minimum downtime

How it works:

• Application content is packaged and deployed to Amazon S3 along with an Application Specific (AppSpec) file that defines a series of deployment steps that AWS CodeDeploy needs to execute
• Create an application in AWS CodeDeploy & define the instances to which the application should be deployed
• AWS CodeDeploy agent is deployed on each participating Amazon EC2 which polls the AWS CodeDeploy to determine what and when to pull a revision from the specified Amazon S3 bucket
• AWS CodeDeploy agent pulls the packaged application code and deploys it on the instance

Continuous Deployment….

         AWS CodePipeline

• It builds, tests, and deploys your code every time there is a change in code, based on the release process you define
• For AWS CodePipeline, you only pay for what you use with no upfront cost of commitments

• AWS CodePipeline provides numerous benefits that align with the DevOps principle of continuous deployment (fast delivery, enhanced quality, configurable workflow and easy to integrate)

         AWS CodeCommit:

• Is a fully-managed source control service 
• Allows to host secure and highly scalable private Git repositories 
• With CodeCommit you don’t need your own source control system 
• It works effortlessly with your existing Git tools

        Blue–Green Deployment:

• It is a DevOps deployment practice that uses domain name services (DNS) to make application deployments. It starts with an existing (blue) environment while testing a new (green) one.
• Once the new environment has passed all the necessary tests and is ready to go live, you simply redirect traffic from the old environment to the new one via DNS

n

n

n