This module makes it easy to create a Cloud Composer Environment. As the module develops, this README should be updated.
The resources/services/activations/deletions that this module will create/trigger are:
- Create a GCP Composer Environment
This module is meant for use with Terraform 1.3+ and tested using Terraform 1.3+. If you find incompatibilities using Terraform >=1.3, please open an issue.
Current version is 4.0. Upgrade guides:
Cloud Composer currently has two versions: V2 has greater flexibility in the Airflow core services (scheduler, webserver, worker) and has a more practical and scalable infrastructure. Therefore, we recommend prioritizing the use of V2 for new environments.
You can find an overview of the product here and the list of major differences. Plans for Cloud Composer V1 end of support is documented here.
Simple usage is as follows:
module "composer" {
source = "terraform-google-modules/composer/google"
version = "~> 6.1"
project_id = "<PROJECT ID>"
region = "us-central1"
composer_env_name = "composer-env-test"
network = "test-network"
subnetwork = "composer-subnet"
enable_private_endpoint = false
}Functional examples are included in the examples directory.
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| composer_env_name | Name of Cloud Composer Environment | string |
n/a | yes |
| enable_private_endpoint | Configure public access to the cluster endpoint. | bool |
false |
no |
| network | Network where Cloud Composer is created. | string |
n/a | yes |
| project_id | Project ID where Cloud Composer Environment is created. | string |
n/a | yes |
| region | Region where the Cloud Composer Environment is created. | string |
n/a | yes |
| subnetwork | Subetwork where Cloud Composer is created. | string |
n/a | yes |
| zone | Zone where the Cloud Composer Environment is created. | string |
n/a | yes |
| Name | Description |
|---|---|
| airflow_uri | URI of the Apache Airflow Web UI hosted within the Cloud Composer Environment. |
| composer_env_id | ID of Cloud Composer Environment. |
| composer_env_name | The name of the Cloud Composer Environment. |
| gcs_bucket | Google Cloud Storage bucket which hosts DAGs for the Cloud Composer Environment. |
| gke_cluster | Google Kubernetes Engine cluster used to run the Cloud Composer Environment. |
These sections describe requirements for using this module.
The following dependencies must be available:
- Terraform v1.3+
- Terraform Provider for GCP plugin v5.3+
A service account with the following roles must be used to provision the resources of this module:
- Project Editor:
roles/editor - Network Admin:
roles/compute.networkAdmin - Instance Admin:
roles/compute.instanceAdmin.v1 - Service Account User:
roles/iam.serviceAccountUser - Composer Worker:
roles/composer.worker
The Project Factory module and the IAM module may be used in combination to provision a service account with the necessary roles applied.
A project with the following APIs enabled must be used to host the resources of this module:
- Cloud Composer API:
composer.googleapis.com
The Project Factory module can be used to provision a project with the necessary APIs enabled.
Refer to the contribution guidelines for information on contributing to this module.