Skip to content

Commit

Permalink
Adding Cloudlist content
Browse files Browse the repository at this point in the history
  • Loading branch information
@ccosta-pd
ccosta-pd committed Feb 2, 2024
1 parent eabba2d commit 5c9b52b
Show file tree
Hide file tree
Showing 8 changed files with 411 additions and 2 deletions.
6 changes: 6 additions & 0 deletions mint.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,12 @@
"iconType": "duotone",
"pages": ["tools/subfinder/overview", "tools/subfinder/install", "tools/subfinder/usage","tools/subfinder/running"]
},
{
"group": "Cloudlist",
"icon": "cloud",
"iconType": "duotone",
"pages": ["tools/cloudlist/overview", "tools/cloudlist/install", "tools/cloudlist/usage","tools/cloudlist/running"]
},
{
"group": "dnsx",
"icon": "tags",
Expand Down
37 changes: 37 additions & 0 deletions tools/cloudlist/install.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Cloudlist Install
description: "Learn how to install Cloudlist and get started"
sidebarTitle: Install
---

<Tabs>
<Tab title="Go">
<Note> Enter the command below in a terminal to install ProjectDiscovery's Cloudlist using Go. </Note>

```bash
go install -v github.com/projectdiscovery/cloudlist/cmd/cloudlist@latest
```
</Tab>
<Tab title="Binary">
```bash
https://github.com/projectdiscovery/cloudlist/releases/
```

<Tip>
- Download the latest binary for your OS.
- Unzip the ready to run binary.
</Tip>

</Tab>
</Tabs>

## Installation Notes
- Cloudlist requires the latest version of [**Go**](https://go.dev/doc/install)
- Add the Go bin path to the system paths. On OSX or Linux, in your terminal use

```
echo export $PATH=$PATH:$HOME/go/bin >> $home/.bashrc
source $home/.bashrc
```
- To add the Go bin path in Windows, [click this link for instructions.](https://www.architectryan.com/2018/03/17/add-to-the-path-on-windows-10/)
- The binary will be located in `$home/go/bin/cloudlist`
21 changes: 21 additions & 0 deletions tools/cloudlist/overview.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: Cloudlist Overview
description: "A multi-cloud tool to identify assets across cloud service providers"
sidebarTitle: Overview
---

Cloudlist is a multi-cloud tool for getting Assets from Cloud Providers. Cloudlist is designed for the blue team to augment Attack Surface Management efforts by maintaining a centralized list of assets across multiple clouds with very little configuration efforts.

## Features

- List Cloud assets with multiple configurations
- Multiple Cloud providers support
- Multiple output format support
- Multiple filters support
- Highly extensible making adding new providers a breeze
- **stdout** support to work with other tools in pipelines

## Support
Questions about using Cloudlist? Issues working through installation? Cool story or use case you want to share? Get in touch!

Check out the [Help](/help) section of the docs or reach out to us on [Discord](https://discord.com/invite/projectdiscovery).
306 changes: 306 additions & 0 deletions tools/cloudlist/running.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,306 @@
---
title: Running Cloudlist
description: "Learn about running Cloudlist with examples and support details"
sidebarTitle: Running
---

For all of the flags and options available for `cloudlist` be sure to check out the [Usage](/tools/cloudlist/usage) page. On this page you can find some examples with output, details of what 'cloudlist' supports, and details on configuration.

<Note> If you have questions, reach out to us through [Help](/help). </Note>

## Basic Examples

```
cloudlist
```

This will list all the assets from configured providers in the configuration file. Specific providers and asset type can also be specified using `provider` and `id` filter.

```console
cloudlist -provider aws,gcp

________ _____ __
/ ____/ /___ __ ______/ / (_)____/ /_
/ / / / __ \/ / / / __ / / / ___/ __/
/ /___/ / /_/ / /_/ / /_/ / / (__ ) /_
\____/_/\____/\__,_/\__,_/_/_/____/\__/ v0.0.1

projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
[INF] Listing assets from AWS (prod) provider.
example.com
example2.com
example3.com
1.1.1.1
2.2.2.2
3.3.3.3
4.4.4.4
5.5.5.5
6.6.6.6
[INF] Found 2 hosts and 6 IPs from AWS service (prod)
```
## Running cloudlist with Nuclei

Scanning assets from various cloud providers with nuclei for security assessments:-

```bash
cloudlist -silent | httpx -silent | nuclei -t cves/
```

# Supported providers

- AWS (Amazon web services)
- EC2
- Route53
- S3
- GCP (Google Cloud Platform)
- Cloud DNS
- GKE
- DO (DigitalOcean)
- Instances
- SCW (Scaleway)
- Instances
- Fastly
- Services
- Heroku
- Applications
- Linode
- Instances
- Azure
- Virtual Machines
- Namecheap
- Domain List
- Alibaba Cloud
- ECS Instances
- Cloudflare
- DNS
- Hashistack
- Nomad
- Consul
- Terraform
- Hetzner Cloud
- Instances
- Openstack
- Instances
- Kubernetes
- Services
- Ingresses

# Configuration file

The default provider config file should be located at `$HOME/.config/cloudlist/provider-config.yaml` and has the following contents as an example. In order to run this tool, the keys need to updated in the config file for the desired providers.

<details>
<summary>Example Provider Config</summary>

```yaml
- provider: do # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: xxxx
# digitalocean_token is the API key for digitalocean cloud platform
digitalocean_token: $DIGITALOCEAN_TOKEN

- provider: scw # provider is the name of the provider
# scaleway_access_key is the access key for scaleway API
scaleway_access_key: $SCALEWAY_ACCESS_KEY
# scaleway_access_token is the access token for scaleway API
scaleway_access_token: $SCALEWAY_ACCESS_TOKEN

- provider: aws # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# aws_access_key is the access key for AWS account
aws_access_key: $AWS_ACCESS_KEY
# aws_secret_key is the secret key for AWS account
aws_secret_key: $AWS_SECRET_KEY
# aws_session_token session token for temporary security credentials retrieved via STS (optional)
aws_session_token: $AWS_SESSION_TOKEN

- provider: gcp # provider is the name of the provider
# profile is the name of the provider profile
id: logs
# gcp_service_account_key is the minified json of a google cloud service account with list permissions
gcp_service_account_key: '{xxxxxxxxxxxxx}'

- provider: azure # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# client_id is the client ID of registered application of the azure account (not requuired if using cli auth)
client_id: $AZURE_CLIENT_ID
# client_secret is the secret ID of registered application of the zure account (not requuired if using cli uth)
client_secret: $AZURE_CLIENT_SECRET
# tenant_id is the tenant ID of registered application of the azure account (not requuired if using cli auth)
tenant_id: $AZURE_TENANT_ID
#subscription_id is the azure subscription id
subscription_id: $AZURE_SUBSCRIPTION_ID
#use_cli_auth if set to true cloudlist will use azure cli auth
use_cli_auth: true

- provider: cloudflare # provider is the name of the provider
# email is the email for cloudflare
email: $CF_EMAIL
# api_key is the api_key for cloudflare
api_key: $CF_API_KEY
# api_token is the scoped_api_token for cloudflare (optional)
api_token: $CF_API_TOKEN

- provider: heroku # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# heroku_api_token is the api key for Heroku account
heroku_api_token: $HEROKU_API_TOKEN

- provider: linode # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# linode_personal_access_token is the personal access token for linode account
linode_personal_access_token: $LINODE_PERSONAL_ACCESS_TOKEN

- provider: fastly # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# fastly_api_key is the personal API token for fastly account
fastly_api_key: $FASTLY_API_KEY

- provider: alibaba # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# alibaba_region_id is the region id of the resources
alibaba_region_id: $ALIBABA_REGION_ID
# alibaba_access_key is the access key ID for alibaba cloud account
alibaba_access_key: $ALIBABA_ACCESS_KEY
# alibaba_access_key_secret is the secret access key for alibaba cloud account
alibaba_access_key_secret: $ALIBABA_ACCESS_KEY_SECRET

- provider: namecheap # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# namecheap_api_key is the api key for namecheap account
namecheap_api_key: $NAMECHEAP_API_KEY
# namecheap_user_name is the username of the namecheap account
namecheap_user_name: $NAMECHEAP_USER_NAME

- provider: terraform # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
#tf_state_file is the location of terraform state file (terraform.tfsate)
tf_state_file: path/to/terraform.tfstate

- provider: hetzner # provider is the name of the provider
# id is the name defined by user for filtering (optional)
id: staging
# auth_token is the is the hetzner authentication token
auth_token: $HETZNER_AUTH_TOKEN

- provider: nomad # provider is the name of the provider
# nomad_url is the url for nomad server
nomad_url: http:/127.0.0.1:4646/
# nomad_ca_file is the path to nomad CA file
# nomad_ca_file: <path-to-ca-file>.pem
# nomad_cert_file is the path to nomad Certificate file
# nomad_cert_file: <path-to-cert-file>.pem
# nomad_key_file is the path to nomad Certificate Key file
# nomad_key_file: <path-to-key-file>.pem
# nomad_token is the nomad authentication token
# nomad_token: <nomad-token>
# nomad_http_auth is the nomad http auth value
# nomad_http_auth: <nomad-http-auth-value>

- provider: consul # provider is the name of the provider
# consul_url is the url for consul server
consul_url: http://localhost:8500/
# consul_ca_file is the path to consul CA file
# consul_ca_file: <path-to-ca-file>.pem
# consul_cert_file is the path to consul Certificate file
# consul_cert_file: <path-to-cert-file>.pem
# consul_key_file is the path to consul Certificate Key file
# consul_key_file: <path-to-key-file>.pem
# consul_http_token is the consul authentication token
# consul_http_token: <consul-token>
# consul_http_auth is the consul http auth value
# consul_http_auth: <consul-http-auth-value>

- provider: openstack # provider is the name of the provider
# id is the name of the provider id
id: staging
# identity_endpoint is Openstack identity endpoint used to authenticate
identity_endpoint: $OS_IDENTITY_ENDPOINT
# domain_name is Openstack domain name used to authenticate
domain_name: $OS_DOMAIN_NAME
# tenant_name is Openstack project name
tenant_name: $OS_TENANT_NAME
# username is Openstack username used to authenticate
username: $OS_USERNAME
# password is Openstack password used to authenticate
password: $OS_PASSWORD

- provider: kubernetes # provider is the name of the provider
# id is the name of the provider id
id: staging
# kubeconfig_file is the path of kubeconfig file
kubeconfig: path/to/kubeconfig
# context is the context to be used from kubeconfig file
context: <context-name>

```

</details>

# Contribution

Please check [PROVIDERS.md](https://github.com/projectdiscovery/cloudlist/blob/main/PROVIDERS.md) and [DESIGN.md](https://github.com/projectdiscovery/cloudlist/blob/main/DESIGN.md) to include support for new cloud providers in Cloudlist.


- Fork this project
- Create your feature branch (`git checkout -b new-provider`)
- Commit your changes (`git commit -am 'Added new cloud provider'`)
- Push to the branch (`git push origin new-provider`)
- Create new Pull Request

# Cloudlist as a library

It's possible to use the library directly in your go programs. The following code snippets outline how to list assets from all or given cloud provider.

```go
package main

import (
"context"
"log"

"github.com/projectdiscovery/cloudlist/pkg/inventory"
"github.com/projectdiscovery/cloudlist/pkg/schema"
)

func main() {
inventory, err := inventory.New(schema.Options{
schema.OptionBlock{"provider": "digitalocean", "digitalocean_token": "ec405badb974fd3d891c9223245f9ab5871c127fce9e632c8dc421edd46d7242"},
})
if err != nil {
log.Fatalf("%s\n", err)
}

for _, provider := range inventory.Providers {
resources, err := provider.Resources(context.Background())
if err != nil {
log.Fatalf("%s\n", err)
}
for _, resource := range resources.Items {
_ = resource // Do something with the resource
}
}
}
```

## Acknowledgments

Thank you for inspiration

* [Smogcloud](https://github.com/BishopFox/smogcloud)
* [Cloudmapper](https://github.com/duo-labs/cloudmapper)

## License

cloudlist is made with 🖤 by the [projectdiscovery](https://projectdiscovery.io) team and licensed under [MIT](https://github.com/projectdiscovery/cloudlist/blob/main/LICENSE.md)
Loading

0 comments on commit 5c9b52b

Please sign in to comment.