cloudera.cloud.env module – Manage CDP Environments

Note

This module is part of the cloudera.cloud collection (version 2.5.1).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install cloudera.cloud. You need further requirements to be able to use this module, see Requirements for details.

To use it in a playbook, specify: cloudera.cloud.env.

Synopsis

  • Create, update, and delete CDP Environments

  • Note that changing states, in particular, creating a new environment, can take several minutes.

Requirements

The below requirements are needed on the host that executes this module.

  • cdpy

Parameters

Parameter

Comments

backup_location

aliases: backup_storage_location_base

string

The base location to store backups. This should be an storage uri - i.e. s3a:// for AWS, abfs:// for Azure, gs:// for GCP.

cascade

aliases: cascading

boolean

Flag to delete all connected resources, e.g. Data Services and Data Hubs.

Choices:

  • false ← (default)

  • true

cdp_region

aliases: cdp_endpoint_region, endpoint_region

string

Specify the Cloudera Data Platform endpoint region.

Default: "default"

cloud

string

The cloud provider or platform for the environment.

Requires region, credential, log_location, and log_identity.

If cloud=aws, one of public_key or public_key_id must be present.

If cloud=aws, one of network_cidr or vpc_id must be present.

If cloud=aws, one of inbound_cidr or default_sg and knox_sg must be present.

Choices:

  • "aws"

  • "azure"

  • "gcp"

credential

string

The CDP credential associated with the environment

datahub_start

boolean

Also starts datahubs within this environment when starting the environment

Choices:

  • false

  • true ← (default)

debug

aliases: debug_endpoints

boolean

Capture the CDP SDK debug log.

Choices:

  • false ← (default)

  • true

default_sg

aliases: default, default_security_group

string

Security group where all other hosts are placed.

Mutually exclusive with inbound_cidr.

delay

aliases: polling_delay

integer

The internal polling interval (in seconds) while the module waits for the environment to achieve the declared state.

Default: 15

description

aliases: desc

string

A description for the environment.

endpoint_access_scheme

string

(AWS)The scheme for the workload endpoint gateway. PUBLIC creates an external endpoint that can be accessed over the Internet. Defaults to PRIVATE which restricts the traffic to be internal to the VPC / Vnet. Relevant in Private Networks.

Choices:

  • "PRIVATE"

  • "PUBLIC"

endpoint_access_subnets

list / elements=string

(AWS) The list of subnet IDs to use for endpoint access gateway.

force

boolean

Flag to remove CDP and cloud provider resources, but ignore cloud provider resources deletion errors.

NOTE, this option might leave cloud provider resources after deletion.

Choices:

  • false ← (default)

  • true

freeipa

dictionary

The FreeIPA service for the environment.

instanceCountByGroup

integer

The number of FreeIPA instances to create per group when creating FreeIPA in the environment.

For high-availability, provide a number greater than 2.

Default: 2

multiAz

boolean

Flag to specify that the FreeIPA instances will be deployed across multi-availability zones.

Only applies to AWS environments.

Choices:

  • false ← (default)

  • true

inbound_cidr

aliases: security_cidr

string

CIDR range which is allowed for inbound traffic. Either IPv4 or IPv6 is allowed.

Mutually exclusive with default_sg and knox_sg.

knox_sg

aliases: knox, knox_security_group

string

Security group where Knox-enabled hosts are placed.

Mutually exclusive with inbound_cidr.

log_identity

aliases: instance_profile

string

(AWS) The instance profile ARN assigned the necessary permissions to access the S3 storage location, i.e. log_location.

log_location

aliases: storage_location_base

string

(AWS) The base location to store logs in S3. This should be an s3a:// url.

name

aliases: environment

string / required

The name of the target environment.

Names must begin with a lowercase alphanumeric, contain only lowercase alphanumerics and hyphens, and be between 5 to 28 characters in length.

network_cidr

string

(AWS) The network CIDR. This will create a VPC along with subnets in multiple Availability Zones.

Mutually exclusive with vpc_id and subnet_ids.

profile

string

If provided, the CDP SDK will use this value as its profile.

project

string

Name of Project when deploying environment on GCP

proxy

aliases: proxy_config, proxy_config_name

string

The name of the proxy config to use for the environment.

public_key_id

aliases: public_key, ssh_key, ssh_key_id

string

The public SSH key ID already registered in the cloud provider.

Mutually exclusive with public_key_text.

public_key_text

aliases: ssh_key_text

string

The content of a public SSH key.

Mutually exclusive with public_key_id.

region

string

The cloud platform specified region

state

string

The declarative state of the environment

If state=present, one of cloud or credential must be present.

Choices:

  • "present" ← (default)

  • "started"

  • "stopped"

  • "absent"

subnet_ids

aliases: subnets

list / elements=string

(AWS) One or more subnet identifiers within the VPC.

Mutually exclusive with network_cidr and requires vpc_id.

tags

aliases: environment_tags

dictionary

Tags associated with the environment and its resources.

timeout

aliases: polling_timeout

integer

The internal polling timeout (in seconds) while the module waits for the environment to achieve the declared state.

Default: 3600

tunnel

aliases: enable_tunnel, ssh_tunnel

boolean

Flag to enable SSH tunnelling for the environment.

Choices:

  • false ← (default)

  • true

verify_endpoint_tls

aliases: endpoint_tls

boolean

Verify the TLS certificates for the CDP endpoint.

Choices:

  • false

  • true ← (default)

vpc_id

aliases: vpc

string

(AWS) The VPC ID.

Mutually exclusive with network_cidr and requires subnet_ids.

wait

boolean

Flag to enable internal polling to wait for the environment to achieve the declared state.

If set to FALSE, the module will return immediately.

Choices:

  • false

  • true ← (default)

workload_analytics

boolean

Flag to enable diagnostic information about job and query execution to be sent to Workload Manager for Data Hub clusters created within the environment.

Choices:

  • false

  • true ← (default)

zones

aliases: availability_zones

list / elements=string

(GCP) The list of zones within the region to use for compute instances.

Examples

# Note: These examples do not set authentication details.

# Create an environment
- cloudera.cloud.env:
    name: example-environment
    state: present
    credential: example-credential
    cloud: aws
    region: us-east-1
    log_location: s3a://example-bucket/datalake/logs
    log_identity: arn:aws:iam::123456789123:instance-profile/example-log-role
    backup_location: s3a://example-bucket/datalake/backups
    public_key_id: example-sshkey
    network_cidr: 10.10.0.0/16
    inbound_cidr: 0.0.0.0/0
    tags:
      project: Arbitrary content

# Create an environment with multiAz FreeIPA, but don't wait for completion (see env_info)
- cloudera.cloud.env:
    name: example-environment
    state: present
    wait: no
    credential: example-credential
    cloud: aws
    region: us-east-1
    log_location: s3a://example-bucket/datalake/logs
    log_identity: arn:aws:iam::123456789123:instance-profile/example-log-role
    public_key_id: example-sshkey
    network_cidr: 10.10.0.0/16
    inbound_cidr: 0.0.0.0/0
    freeipa:
      instanceCountByGroup: 3
      multiAz: yes
    tags:
      project: Arbitrary content

# Update the environment's CDP credential
- cloudera.cloud.env:
    name: example-module
    credential: another-credential

# Stop the environment (and wait for status change)
- cloudera.cloud.env:
    name: example-module
    state: stopped

# Start the environment (and wait for status change)
- cloudera.cloud.env:
    name: example-module
    state: started

# Delete the environment (and wait for status change)
  cloudera.cloud.env:
    name: example-module
    state: absent

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key

Description

environment

dictionary

The information about the Environment

Returned: on success

authentication

dictionary

Additional SSH key authentication configuration for accessing cluster node instances of the Environment.

Returned: always

loginUserName

string

SSH user name created on the node instances for SSH access.

Returned: always

Sample: "cloudbreak"

publicKey

string

SSH Public key string

Returned: when supported

Sample: "ssh-rsa AAAAB3NzaC...BH example-public-key"

publicKeyId

string

Public SSH key ID registered in the cloud provider.

Returned: when supported

Sample: "a_labeled_public_key"

cloudPlatform

string

Cloud provider of the Environment.

Returned: always

Sample: "['AWS', 'AZURE', 'GCP']"

credentialName

string

Name of the CDP Credential of the Environment.

Returned: always

Sample: "a-cdp-credential"

crn

string

CDP CRN value for the Environment.

Returned: always

Sample: "crn:cdp:environments:us-west-1:558bc1d2-8867-4357-8524-311d51259233:credential:61eb5b97-226a-4be7-b56d-795d18a043b5"

description

string

Description of the Environment.

Returned: always

Sample: "An example Environment"

environmentName

string

Name of the Environment.

Returned: always

Sample: "a-cdp-environment-name"

freeipa

complex

Details of a FreeIPA instance in the Environment.

Returned: always

crn

string

CRN of the FreeIPA instance.

Returned: always

Sample: "crn:cdp:freeipa:us-west-1:558bc1d2-8867-4357-8524-311d51259233:freeipa:cbab8ee3-00f2-4958-90c1-6f7cc06b4937"

domain

string

Domain name of the FreeIPA instance.

Returned: always

Sample: "example.012345-abcd.cloudera.site"

hostname

string

Hostname of the FreeIPA instance.

Returned: always

Sample: "ipaserver"

serverIP

list / elements=string

IP addresses of the FreeIPA instance.

Returned: always

Sample: ["['10.10.2.40']"]

logStorage

complex

Storage configuration for cluster and audit logs for the Environment.

Returned: always

awsDetails

dictionary

AWS-specific log storage configuration details.

Returned: when supported

instanceProfile

string

AWS instance profile that contains the necessary permissions to access the S3 storage location.

Returned: always

Sample: "arn:aws:iam::381358652250:instance-profile/EXAMPLE-LOG_ROLE"

storageLocationBase

string

Base location to store logs in S3.

Returned: always

Sample: "s3a://example-bucket/datalake/logs"

azureDetails

dictionary

Azure-specific log storage configuration details.

Returned: when supported

managedIdentity

string

Azure managing identity associated with the logger.

This identify should have the Storage Blob Data Contributor role on the given storage account.

Returned: always

Sample: "/subscriptions/01234-56789-abcd/resourceGroups/example-environment-name/providers/ Microsoft.ManagedIdentity/userAssignedIdentities/loggerIdentity"

storageLocationBase

string

Base location to store logs in Azure Blob Storage.

Returned: always

Sample: "abfs://logs@example_location.dfs.core.windows.net"

enabled

boolean

Flag for external log storage.

Returned: always

network

complex

Network details for the Environment

Returned: always

aws

dictionary

AWS networking specifics for the Environment.

Returned: when supported

vpcId

string

VPC identifier.

Returned: always

Sample: "vpc-08785c81e888251df"

azure

dictionary

Azure networking specifics for the Environment.

Returned: when supported

networkId

string

VNet identifier.

Returned: always

Sample: "example-vnet"

resourceGroupName

string

Resource Group name.

Returned: always

Sample: "example-rg"

usePublicIp

boolean

Flag for associating public IP addresses to the resources within the network.

Returned: always

networkCidr

string

Range of private IPv4 addresses that resources will use for the Environment.

Returned: always

Sample: "10.10.0.0/16"

subnetIds

list / elements=string

Subnet identifiers for the Environment.

Returned: always

Sample: ["['subnet-04a332603a269535f', 'subnet-07bbea553ca667b66', 'subnet-0aad7d6d9aa66d1e7']"]

subnetMetadata

complex

Additional subnet metadata for the Environment.

Returned: always

__subnetId__

dictionary

Keyed subnet identifier.

Returned: always

availabilityZone

string

Availability zone (AWS only)

Returned: when supported

Sample: "us-west-2a"

subnetId

string

Identifier for the subnet

Returned: always

Sample: "subnet-04a332603a269535f"

subnetName

string

Name of the subnet

Returned: always

Sample: "subnet-04a332603a269535f"

proxyConfig

dictionary

Proxy configuration of the Environment.

Returned: when supported

crn

string

CDP CRN for the proxy configuration.

Returned: always

Sample: "crn:cdp:environments:us-west-1:558bc1d2-8867-4357-8524-311d51259233:credential:38eeb2b9-6e57-4d10-ad91-f6d9bceecb54"

description

string

Description of the proxy.

Returned: always

Sample: "The proxy configuration description"

host

string

Proxy host.

Returned: always

Sample: "some.host.example.com"

password

string

Proxy user password.

Returned: always

Sample: "secret_password"

port

string

Proxy port.

Returned: always

Sample: "8443"

protocol

string

Proxy protocol.

Returned: always

Sample: "https"

proxyConfigName

string

Name of the proxy configuration.

Returned: always

Sample: "the-proxy-config"

user

string

Proxy user name.

Returned: always

Sample: "the_username"

region

string

Cloud provider region of the Environment.

Returned: always

Sample: "us-east-1"

securityAccess

dictionary

Security control configuration for FreeIPA and Datalake deployment in the Environment.

Returned: always

cidr

string

CIDR range which is allowed for inbound traffic. Either IPv4 or IPv6 is allowed.

Returned: when supported

Sample: "0.0.0.0/0"

defaultSecurityGroupId

string

Security group associated with Knox-enabled hosts.

Returned: when supported

Sample: "/subscriptions/01234-56789-abcd/resourceGroups/example-environment/providers/Microsoft.Network/ networkSecurityGroups/example-default-nsg"

securityGroupIdForKnox

string

Security group associated with all other hosts (non-Knox).

Returned: when supported

Sample: "/subscriptions/01234-56789-abcd/resourceGroups/example-environment/providers/Microsoft.Network/ networkSecurityGroups/example-knox-nsg"

status

string

Status of the Environment.

Returned: always

Sample: "['AVAILABLE', 'CREATE_FAILED', 'CREATION_INITIATED', 'ENV_STOPPED', 'FREEIPA_CREATION_IN_PROGRESS', 'FREEIPA_DELETE_IN_PROGRESS', 'FREEIPA_DELETED_ON_PROVIDER_SIDE', 'START_FREEIPA_FAILED', 'STOP_FREEIPA_STARTED']"

statusReason

string

Description for the status code of the Environment.

Returned: when supported

sdk_out

string

Returns the captured CDP SDK log.

Returned: when supported

sdk_out_lines

list / elements=string

Returns a list of each line of the captured CDP SDK log.

Returned: when supported

Authors

  • Webster Mudge (@wmudge)

  • Dan Chaffelson (@chaffelson)