Skip to content

How to use AWS

Jump to bottom
Vitalii Kanivets edited this page Aug 10, 2023 · 4 revisions

This page describes how to install and use Cloud Custodian for AWS.  

Table of Contents

  1. Pre-requisites
    1.1. Install Python 3
    1.2. Create an AWS account and generate an access key
    1.3. Install AWS CLI
  2. Install Cloud Custodian
  3. Scan a project
    3.1. Explore Cloud Custodian
    3.2. Run a policy
    3.3. Cross-region policies
  4. Useful links

Pre-requisites

Install Python 3 

Cloud Custodian supports Python 3.8 and higher. The recommended version is Python 3.8.

On Ubuntu

Run the following command:

sudo apt update
sudo apt-get install python3.8

Then, install pip and venv:

sudo apt-get install python3-pip
sudo apt-get install python3.8-venv

On MacOS with Homebrew

Run the following command:

/usr/bin/ruby -e "$(curl -fsSL http://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install python3
brew install git

Installing python3 will get you the latest version of Python 3 supported by Homebrew, currently Python 3.9.

Create an AWS account and generate an access key 

Prior to installing Cloud Custodian, you need to perform the following steps: 

Steps Instruction
1. Create an AWS account https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/
2. Generate an access key https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html

Install AWS CLI 

To authenticate to Cloud Custodian, you need to install AWS CLI first. 

On Ubuntu:

  1. To install AWS CLI, run the command:
sudo apt-get install awscli
  1. To configure AWS CLI, run the command:
aws configure

On the console, specify the following parameters

  • your AWS access key ID
  • your AWS secret access key
  • Default region name: us-east-1
  • Default output format: json

On MacOS with Homebrew:

  1. To install AWS CLI, run the command:
brew install awscli
  1. To configure AWS CLI, run the command:
aws configure

On the console, specify the following parameters

  • your AWS access key ID
  • your AWS secret access key
  • Default region name: us-east-1
  • Default output format: json

Install Cloud Custodian

  1. Install the latest version from the repository:
git clone https://github.com/cloud-custodian/cloud-custodian.git
git clone https://github.com/epam/ecc-aws-rulepack.git
  1. Set up your own virtual environment.

To do so, use the venv module and upgrade pip to the latest version:

python3.8 -m venv custodian
source custodian/bin/activate
pip install --upgrade pip
  1. Install Cloud Custodian:
pip3 install -e ./cloud-custodian/

ℹ️ Note: To use  Cloud Custodian once again, run the following command:

source custodian/bin/activate

The alternative way to install Cloud Custodian using make:

cd path/to/cloud-custodian
python3.8 -m venv .venv
source .venv/bin/activate
pip install poetry
make install

Scan a project

Cloud Custodian is configured to manage cloud resources using policies. Custodian policies are expressed in YAML file that follows a predetermined schema to describe what you want the Custodian to do. 

There are three main components to a policy:

  • resource:  the type of resource to run the policy against;
  • filters:  criteria to narrow down the set of resources;
  • actions:  directives to take on the filtered set of resources.

Explore Cloud Custodian

To obtain the information below, run the following commands:

  • To see a list of available commands, run custodian -h.
  • To see the complete list of cloud resources against which you can run policies, run  custodian schema.
  • To invoke command-line help with more information about policy schema details, run  custodian schema -h.
  • To see the available resources for a specific cloud provider, run  custodian schema <cloud-provider>, for example: custodian schema aws.
  • To see the available filters and actions for each resource, run custodian schema <cloud-provider>.<resource>.

To run a policy, go to folder ecc-aws-rulepack/policies : 

cd ecc-aws-rulepack/policies

Run a policy

Run the following command to execute the policy with Custodian:

custodian run --cache-period=0 --output-dir=. ecc-aws-013-ensure_access_keys_are_rotated_every_90_days.yml

If successful, you will see the output similar to the following on the command line:

2021-10-08 13:55:58,214: custodian.policy:INFO policy:ecc-aws-013-ensure_access_keys_are_rotated_every_90_days resource:aws.iam-user region:eu-east-1 count:1 time:0.37

You should also find a new policy directory with log and other files. The name of this directory will match the name of the policy you are using. The following files are stored in the directory:

  • custodian-run.log  – log file;
  • metadata.json  – information in JSON format about the scanning process and description of the rule itself;
  • resources.json  – description of resources in JSON format that does not comply with the rule.

Cross-region policies

Some AWS resources are cross-regional (global) and policies with these resources must be run only once in any region of your choice.

Click here to expand the list of cross-region AWS resources and policies

List of cross-region resources (the list can be updated):

  • aws.distribution
  • aws.iam-user
  • aws.iam-policy
  • aws.iam-policy-all
  • aws.iam-group
  • aws.iam-certificate
  • aws.s3
  • aws.s3-light
  • aws.waf
  • aws.wafv2
  • aws.account*

ℹ️ Note: Not all resources that use 'aws.account' custodian resource are global. The rules that check AWS credentials (passwords and access keys) are global. The rules that check such AWS services as GuardDuty, Access analyzer, X-Ray are not global. Also, custom filters for the 'aws.account' custodian resource often are not global.

List of cross-region policies:

⚠️ ...

Useful links

Check the resources below to find more information about Cloud Custodian, AWS, and the rules:

Custodian info

Documentation

Clone this wiki locally