↓Skip to main content
Dev Blog Dev Blog
  1. All Posts/

cli

Tools TypeScript

CloudGraph is the free open-source universal GraphQL API and Cloud Security Posture Management (CSPM) tool for AWS, Azure, GCP, and K8s. With CloudGraph you get:

  • Free and effortless compliance checks (i.e. Azure CIS 1.3.1, GCP CIS 1.2, AWS CIS 1.2, AWS CIS 1.3, AWS CIS 1.4, AWS PCI 3.2.1, AWS NIST 800-53 Rev. 4)
  • Type-Safe asset inventories for all of your resources in all of your cloud environments
  • Automatically generated documentation and query validation – know if your query is valid before you send it!
  • Full resource data including relationships between resources so you can understand context
  • Historical snapshots of your data over time
  • A single endpoint to query all of your cloud data at once (i.e. get AWS + GCP data in the same query, or compare AWS stage with AWS prod)
  • Enhanced billing data (AWS only)
  • Enhanced CloudWatch data (AWS EC2 only)

Cloud Graph lets you Know your cloud in 5 minutes. Built and maintained with love by the team at ❤️ AutoCloud ❤️

🌐 Website
💻 Documentation
💰 Get paid to build CloudGraph providers






Join the conversation




Amazing companies using CloudGraph**

** usage does not imply endorsement

Why CloudGraph

AWS, Azure, and GPC have done a wonderful job of building solutions that let engineers like us create systems to power our increasingly interconnected world. Over the last 15 years, products such as EC2, S3, RDS, and Lambda have fundamentally changed how we think about computing, storage, and databasing.

With the proliferation of Kubernetes and Serverless in the last 5 or so years, cloud services have become increasingly abstract on top of racks of physical servers. To end-users, everything on the cloud is just an API, so we don’t necessarily need to know how Lambda Functions or EKS work under the hood to be able to use them for building applications. With a little documentation, API or console access, and a tutorial anyone can pretty much create anything they need.

These abstractions have led to massive improvements in the overall convenience and breadth of CSP service offerings. What was once a painstaking, time-consuming, and error-prone process of provisioning new servers, databases, or filesystems can now be done in seconds with just the click of a button or deployment of IAC. Since everything is just an API abstraction, when a CAP is ready to introduce a new “product” they simply need to expose a new API – yes, I’m of course simplifying slightly 🙂

Anyone familiar with the CSPs knows that service APIs are almost always split into modular namespaces that contain dozens, if not hundreds, of separate API methods for single resources. For example, the AWS EC2 service contains over 500 different API methods, with new ones added occasionally. Any company building substantial systems on a CSP is likely using many, many different services.

While a masterpiece of datacenter architecture, this choice of hundreds of services and configuration options put the burden of knowledge on how to properly use these services squarely on us engineers. As a result, we find ourselves having to constantly stay up to date and learn about all the service offerings or new changes. This takes a significant amount of time and mental energy. As developers, it can be difficult, time-consuming, and frustrating to use the AWS CLI to make 5 different API calls to describe, as an example, an AWS ECS cluster, its services, task definitions, tasks, container definitions, etc. We often find ourselves lost in documentation and having to use half a dozen of APIs to get answers to questions like “What exactly is running in this VPC?”

This means that AWS, Azure, and GCP can feel overwhelming quickly even to seasoned cloud architects. While the CSPs are fantastic at building the actual services that power our businesses, not a lot of headway has been into simplifying the day-to-day UX of querying these hundreds of services in a sane manner.

New solutions like the Cloud Control API for AWS have attempted to create a standardized interface for querying many different types of AWS resources. Unfortunately, the Cloud Control API’s usage is severely limited, and users still need to know how to correctly query their data. This means more time spent reading documentation and understanding how services work and are related to one another.

While the modularity of the CSP APIs is a great logical organization system and does make sense, it’s a burden on end-users in terms of the cognitive overhead and learning curve. Having to remember how hundreds of constantly changing services work and are connected leads to a caffeine addiction and time wasted playing detective.

Wouldn’t it be great if we as DevOps/Cloud engineers had a simpler way to get our data out of AWS, Azure, GCP and the others? One that reflected our need to easily query any data about any service in any account without having to spend hours on docs or stack overflow?

It is for these reasons that we built CloudGraph, the GraphQL API for everything cloud. CloudGraph extracts, normalizes, processes, and enriches your cloud data allowing you to access deep insights across multiple providers effortlessly. Check out our blog post The GraphQL API for everything to learn more.

How It Works

Note that CloudGraph requires READ ONLY permissions to run and as such can never mutate your actual cloud infrastructure. Additionally, none of your cloud environment information is ever sent to or shared with CloudGraph, AutoCloud, or any other third parties.

Under the hood, CloudGraph reaches out to your cloud provider(s), sucks up all of the configuration data, processes it, and stores a copy of this data for you in Dgraph. It then exposes an endpoint at http://localhost:8997 that allows you to write GraphQL Queries against your stored data. These queries not only allow you do to anything that you would do with say, the AWS SDK/CLI, but they also allow you to run much more powerful queries as well. CloudGraph ships with pre-packaged GraphQL query tools including GraphQL Playground and Altair but you can also feel free to use your own. It also includes a schema visualization tool called Voyager so you can understand relationships between entities.

Authentication and Permissions

CloudGraph currently supports AWS, Azure, GCP, K8s, and Tencent (several others coming soon). CloudGraph needs read permissions in order to ingest your data. To keep things easy you can use the same permissions that we use internally when we run CloudGraph to power AutoCloud. Here are the auth guides and details for how to generate credentials for each provider (feel free to leave out AutoCloud specific configuration):

Install

System Requirements

  • Docker

There are 2 ways to install the CloudGraph CLI

Homebrew (Recommended)

You can install CloudGraph using homebrew with the following command:
brew install cloudgraphdev/tap/cg

NPM

  • Requires Node 16+

Use this command to install and update CloudGraph to the latest version.

npm i -g @cloudgraph/cli

You can then add the providers you want (links to provider repos: AWS, Azure, GCP, K8s, Tencent Docs):

cg init aws
cg init azure
cg init gcp
cg init k8s
cg init tencent

You can also add as many as you want all at once

cg init aws azure gcp k8s tencent

And add in compliance policy packs to supplement your data with instant security insights:

cg policy add gcp-cis-1.2.0
cg policy add azure-cis-1.3.1
cg policy add aws-cis-1.2.0
cg policy add aws-cis-1.3.0
cg policy add aws-cis-1.2.0
cg policy add aws-pci-dss-3.2.1
cg policy add aws-nist-800-53-rev4

You can find a list of currently supported policy packs in the Policy Packs repo

Quick Start

You can get up and running with three simple commands:

cg init
  1. This initializes CloudGraph’s configuration. This command will ask you a series of questions about what providers you are using and how you would like CloudGraph configured. 
cg launch
  1. This command launches an instance of Dgraph, the graphdb CloudGraph uses to store data under the hood. Note that there are 2 ways to launch an instance. BOTH of these require Docker to be installed and running. The preferred solution is to use our cg launch convenience command.

Note that if you do not want to use this command, for example, if you want to launch the Dgraph container in interactive mode, you can use the docker command below.

docker run -it -p 8995:5080 -p 8996:6080 -p 8997:8080 -p 8998:9080 -p 8999:8000
  --label cloudgraph-cli-dgraph-standalone -v ~/dgraph:/dgraph --name dgraph dgraph/standalone:v21.03.1
cg scan
  1. Scan for cloud infrastructure for all configured providers. This command will reach out and read all of the metadata on your cloud infrastructure. Note that it is completely normal to see warnings and errors while the cg scan command runs, these are usually caused by permissions issues. That said, if you encounter any problematic errors running CloudGraph you can prepend CG_DEBUG=5 to the beginning of your command as in, CG_DEBUG=5 cg scan. This will print out the verbose logs with more information and save the output to cg-debug.log. Please share your logs with us either by opening an issue on GitHub or let us know in our Slack Workspace.

That’s it, you are all set to start querying! The query tool you selected during the cg init command will then be opened in your preferred browser to run queries, mutations, and visualizations on all of your cloud infrastructure! Note that if…



↑