Managing Cloud Infrastructure with Ease

February 8, 2024

Are you struggling with managing your cloud infrastructure? Do you find yourself writing adhoc scripts to ensure security and cost optimization? If so, Cloud Custodian, also known as c7n, is here to save the day! Cloud Custodian is a rules engine designed to simplify the management of public cloud accounts and resources. In this article, we’ll explore how Cloud Custodian can help you achieve a well-managed cloud infrastructure that is both secure and cost optimized.

What is Cloud Custodian?

Cloud Custodian is an open-source project that consolidates adhoc scripts into a lightweight and flexible tool. It allows you to define policies to manage your cloud infrastructure across AWS, Azure, and GCP environments. With Cloud Custodian, you can ensure real-time compliance to security policies, tag policies, and cost management. It provides features such as encryption and access requirements, garbage collection of unused resources, and off-hours resource management.

Key Features

Cloud Custodian offers a wide range of features to make cloud management easier:

Comprehensive support for public cloud services and resources: Cloud Custodian supports AWS, Azure, and GCP environments, allowing you to manage resources across multiple providers.
Infrastructure as code support: You can run policies on infrastructure as code assets, such as Terraform, to provide feedback directly on developer workstations or within CI pipelines.
Flexible filtering: Cloud Custodian enables you to apply arbitrary filtering on resources with nested boolean conditions, giving you fine-grained control over policy enforcement.
Dry run mode: You can preview the actions that a policy would perform without actually executing them, ensuring that your policies are well-defined before running them.
Integration with cloud native serverless capabilities: Cloud Custodian leverages the serverless capabilities of each cloud provider to enforce policies in real-time.
Metrics and reporting: Cloud Custodian provides native metrics outputs on resources that match a policy and structured outputs into cloud-native object storage.
Intelligent caching: Cloud Custodian optimizes API calls by intelligently caching data, minimizing the impact on your cloud provider’s resources.
Multi-account/subscription/project usage: Cloud Custodian enables you to manage resources across multiple accounts, subscriptions, or projects.
Battle-tested: Cloud Custodian is in production on various large cloud environments, so you can trust its reliability and scalability.

Getting Started

To get started with Cloud Custodian, you need to write a YAML file containing the policies you want to run. Each policy specifies the resource type, filters to control which resources will be affected, actions to be taken on matched resources, and a mode to control policy execution. Cloud Custodian provides documentation and getting started guides for each cloud provider – AWS, Azure, and GCP. These guides walk you through the process of defining policies specific to each provider.

Here’s a quick example of some sample policies for AWS resources:

#yaml
policies:
 - name: s3-cross-account
   description: |
     Checks S3 for buckets with cross-account access and
     removes the cross-account access.
   resource: aws.s3
   region: us-east-1
   filters:
     - type: cross-account
   actions:
     - type: remove-statements
       statement_ids: matched

 - name: ec2-require-non-public-and-encrypted-volumes
   resource: aws.ec2
   description: |
    Provision a lambda and cloud watch event target
    that looks at all new instances and terminates those with
    unencrypted volumes.
   mode:
    type: cloudtrail
    role: CloudCustodian-QuickStart
    events:
      - RunInstances
   filters:
    - type: ebs
      key: Encrypted
      value: false
   actions:
    - terminate

 - name: tag-compliance
   resource: aws.ec2
   description: |
     Schedule a resource that does not meet tag compliance policies to be stopped in four days. Note a separate policy using the`marked-for-op` filter is required to actually stop the instances after four days.
   filters:
    - State.Name: running
    - "tag:Environment": absent
    - "tag:AppId": absent
    - or:
      - "tag:OwnerContact": absent
      - "tag:DeptID": absent
   actions:
    - type: mark-for-op
      op: stop
      days: 4

Once you have defined your policies, you can validate them, test them in dry run mode, and finally run them using the custodian command-line tool. Cloud Custodian provides detailed documentation on how to install and configure the tool for your environment, including examples of different authentication options.

Additional Tools

In addition to the core Cloud Custodian project, the Custodian community has developed a suite of additional tools to enhance your cloud management experience. These tools include:

Org: Multi-account policy execution.
ShiftLeft: Shift Left integration for running policies against Infrastructure as Code assets like Terraform.
PolicyStream: Git history as a stream of logical policy changes.
Salactus: Scale out S3 scanning.
Mailer: A reference implementation of sending messages to users to notify them.
Trail Creator: Retroactive tagging of resource creators from CloudTrail.
TrailDB: CloudTrail indexing and time series generation for dashboarding.
LogExporter: CloudWatch log exporting to S3.
Cask: Easy Custodian execution via Docker.
Guardian: Automated multi-account GuardDuty setup.
Omni SSM: EC2 Systems Manager Automation.
Mugc: A utility used to clean up Cloud Custodian Lambda policies that are deployed in an AWS environment.

These additional tools provide advanced functionality and integration with other cloud management systems. You can refer to the Cloud Custodian documentation for more information on each tool and how to use them.

Get Involved

Cloud Custodian is an open-source project with a vibrant community of contributors. If you’re interested in getting involved, there are several ways to participate:

GitHub: Visit the Cloud Custodian GitHub repository to explore the source code, file issues, and contribute pull requests.
Slack: Join the Cloud Custodian Slack community to chat with other users and developers for help or collaboration on Custodian-related topics.
Mailing List: Subscribe to the Cloud Custodian mailing list for important project announcements and to ask questions.
Reddit: Join the Cloud Custodian subreddit to engage in discussions and share your experiences with the community.
StackOverflow: Ask questions or find answers related to Cloud Custodian on StackOverflow using the cloudcustodian tag.
YouTube Channel: Subscribe to the Cloud Custodian YouTube channel for tutorials and other useful information about the project.

Additionally, the Custodian community has regular community meetings open to all users and developers. Join the mailing list to receive meeting invites and stay up to date with the latest project developments. You can also find community meeting videos and notes in the community resources section of the Cloud Custodian documentation.

Conclusion

Cloud Custodian (c7n) is an essential tool for managing public cloud accounts and resources. It empowers you to define policies that ensure a secure and cost-optimized cloud infrastructure. With its comprehensive support for multiple cloud providers, flexible filtering options, and serverless capabilities, Cloud Custodian streamlines cloud management tasks and improves operational efficiency.

So why wait? Start using Cloud Custodian today and experience the benefits of a well-managed cloud infrastructure!

Have you tried Cloud Custodian? What are your thoughts? Share your experiences and questions in the comments below.

Group Sum