cloudera.cloud.de_virtual_cluster module – Manage CDP Data Engineering Virtual Clusters

Note

This module is part of the cloudera.cloud collection (version 3.3.0).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install cloudera.cloud.

To use it in a playbook, specify: cloudera.cloud.de_virtual_cluster.

New in cloudera.cloud 1.5.0

Synopsis 

Create, update, suspend, resume, or delete CDP Data Engineering Virtual Clusters.

Parameters 

Parameter	Comments
access_key string	If provided, the Cloudera on cloud API will use this value as its access key. If not provided, the API will attempt to use the value from the environment variable `CDP_ACCESS_KEY_ID`. Required if `private_key` is provided. Mutually exclusive with `credentials_path`.
acl_users string	Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster.
chart_value_overrides list / elements=dictionary	Chart overrides for creating the Virtual Cluster.
chart_name string	Name of the chart to override, for example `dex-app` or `dex-base`.
overrides string	Space-separated key-value pairs for overriding chart values.
cpu_requests string	CPU requests for autoscaling. Required when `state=present` and the Virtual Cluster does not yet exist.
credentials_path string	If provided, the Cloudera on cloud API will use this value as its credentials path. If not provided, the API will attempt to use the value from the environment variable `CDP_CREDENTIALS_PATH`. Default: `"~/.cdp/credentials"`
debug aliases: debug_endpoints boolean	If `true`, the module will capture the Cloudera on cloud HTTP log and return it in the `sdk_out` and `sdk_out_lines` fields. Choices: `false` ← (default) `true`
delay aliases: polling_delay integer	The internal polling interval (in seconds) while the module waits for the Virtual Cluster to achieve the declared state. Default: `30`
discard_spark_configs boolean	Discard the Virtual Cluster’s existing Spark configurations instead of merging them with `spark_configs`. Choices: `false` `true`
enable_compute_override boolean	Enable compute override for the Virtual Cluster. Choices: `false` `true`
endpoint aliases: endpoint_url, url string	The Cloudera on cloud API endpoint to use. If not provided, the API will attempt to use the value from the environment variable `CDP_ENDPOINT_URL`. Mutually exclusive with `endpoint_region`.
endpoint_region aliases: cdp_endpoint_region, cdp_region, region string	Specify the Cloudera on cloud API endpoint region. See Cloudera Control Plane regions for more information. If not provided, the API will attempt to use the value from the environment variable `CDP_REGION`. `default` is an alias for the `us-west-1` region. Mutually exclusive with `endpoint`. Choices: `"default"` `"us-west-1"` ← (default) `"eu-1"` `"ap-1"`
endpoint_tls aliases: verify_endpoint_tls, verify_tls, verify_api_tls boolean	Verify the TLS certificates for the Cloudera on cloud API endpoint. Choices: `false` `true` ← (default)
environment aliases: env string / required	The CDP environment name of the CDE Service in which to create the Virtual Cluster.
full_access_groups list / elements=string	Groups granted full access to the Virtual Cluster.
full_access_users list / elements=string	Users granted full access to the Virtual Cluster.
http_agent aliases: agent_header string	The HTTP user agent to use for Cloudera on cloud API requests. Default: `"cloudera.cloud"`
memory_requests string	Memory requests for autoscaling, for example `30Gi`. Required when `state=present` and the Virtual Cluster does not yet exist.
name string / required	The name of the Virtual Cluster.
private_key string	If provided, the Cloudera on cloud API will use this value as its private key. If not provided, the API will attempt to use the value from the environment variable `CDP_PRIVATE_KEY`. Required if `access_key` is provided.
profile string	If provided, the Cloudera on cloud API will use this value as its profile. If not provided, the API will attempt to use the value from the environment variable `CDP_PROFILE`. Default: `"default"`
runtime_spot_component string	Where the Driver and the Executors run, on-demand or spot instances.
service_name aliases: cluster_name string / required	The name of the CDE Service in which to create the Virtual Cluster.
session_timeout string	Default session timeout, applicable to the All Purpose (ALLP) virtual cluster tier.
spark_configs dictionary	Spark configurations applied to all jobs run in the Virtual Cluster.
spark_os_name string	Spark OS image for the Virtual Cluster. Choices: `"SECURITYHARDENED"` `"REDHAT"`
spark_version string	Spark version for the Virtual Cluster, for example `SPARK3_5_4`.
state string	The declarative state of the Virtual Cluster. `suspended` suspends the Virtual Cluster; setting `state=present` on a suspended Virtual Cluster resumes it. Choices: `"present"` ← (default) `"absent"` `"suspended"`
strict aliases: strict_errors boolean	Legacy CDPy SDK error handling. Choices: `false` ← (default) `true`
tier aliases: vc_tier string	Tier of the Virtual Cluster, for CDE 1.19.0 and beyond. `CORE` enables operational batch jobs. `ALLP` enables both operational batch jobs and interactive sessions. Choices: `"ALLP"` `"CORE"`
timeout aliases: polling_timeout integer	The internal polling timeout (in seconds) while the module waits for the Virtual Cluster to achieve the declared state. Default: `600`
view_only_groups list / elements=string	Groups granted view-only access to the Virtual Cluster.
view_only_users list / elements=string	Users granted view-only access to the Virtual Cluster.
wait boolean	Flag to enable internal polling to wait for the Virtual Cluster to achieve the declared state. If set to `False`, the module will return immediately after initiating the operation. Choices: `false` `true` ← (default)

Examples 

# Note: These examples do not set authentication details.

# Create a Virtual Cluster and wait for it to become active
- cloudera.cloud.de_virtual_cluster:
    name: my-virtual-cluster
    environment: my-cdp-environment
    service_name: my-cde-service
    cpu_requests: "20"
    memory_requests: "80Gi"
    tier: ALLP
    spark_version: SPARK3_5_4
    state: present
    wait: true

# Delete a Virtual Cluster without waiting
- cloudera.cloud.de_virtual_cluster:
    name: my-virtual-cluster
    environment: my-cdp-environment
    service_name: my-cde-service
    state: absent
    wait: false

# Suspend a Virtual Cluster and wait for it to become suspended
- cloudera.cloud.de_virtual_cluster:
    name: my-virtual-cluster
    environment: my-cdp-environment
    service_name: my-cde-service
    state: suspended
    wait: true

# Resume a suspended Virtual Cluster
- cloudera.cloud.de_virtual_cluster:
    name: my-virtual-cluster
    environment: my-cdp-environment
    service_name: my-cde-service
    state: present
    wait: true

# Update the access control and Spark configs of an existing Virtual Cluster
- cloudera.cloud.de_virtual_cluster:
    name: my-virtual-cluster
    environment: my-cdp-environment
    service_name: my-cde-service
    full_access_users:
      - alice
    view_only_groups:
      - viewers
    spark_configs:
      spark.executor.memory: 4g
    state: present

Return Values 

Common return values are documented here, the following are the fields unique to this module:

Key	Description
sdk_out string	Returns the captured CDP SDK log. Returned: when supported
sdk_out_lines list / elements=string	Returns a list of each line of the captured CDP SDK log. Returned: when supported
virtual_cluster dictionary	Description of the Virtual Cluster. Returned: always
accessControl dictionary	Access control details for the Virtual Cluster. Returned: when available
clusterId string	Cluster ID of the CDE Service that contains the Virtual Cluster. Returned: always
creatorEmail string	Email address of the creator of the Virtual Cluster. Returned: when available
resources dictionary	Resource details of the Virtual Cluster. Returned: when available
sparkConfigs dictionary	Spark configurations applied to all jobs run in the Virtual Cluster. Returned: when available
sparkVersion string	Spark version for the Virtual Cluster. Returned: when available
status string	Status of the Virtual Cluster. Returned: always
vcApiUrl string	URL for the Virtual Cluster APIs. Returned: when available
vcId string	Virtual Cluster ID. Returned: always
vcName string	Name of the Virtual Cluster. Returned: always
vcTier string	Tier of the Virtual Cluster (ALLP or CORE). Returned: when available

Authors 

Curtis Howard (@curtishoward)
Ronald Suplina (@rsuplina)

cloudera.cloud.de_virtual_cluster module – Manage CDP Data Engineering Virtual Clusters

Synopsis

Parameters

Examples

Return Values

Authors

Collection Links

Synopsis 

Parameters 

Examples 

Return Values 

Authors 

Collection Links 