cloudera.cloud.de_virtual_cluster module – Create or delete CDP Data Engineering Virtual Clusters

Note

This module is part of the cloudera.cloud collection (version 2.5.1).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install cloudera.cloud. You need further requirements to be able to use this module, see Requirements for details.

To use it in a playbook, specify: cloudera.cloud.de_virtual_cluster.

Synopsis

  • Create or delete CDP Data Engineering Virtual Clusters

Requirements

The below requirements are needed on the host that executes this module.

  • cdpy

Parameters

Parameter

Comments

acl_users

string

Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster.

chart_value_overrides

list / elements=string

Chart overrides for creating a virtual cluster.

chart_name

string

The key/value pair for the chart_name/override

cluster_id

string / required

Cluster id of the CDE service where virtual cluster has to be created.

cpu_requests

string

Cpu requests for autoscaling.

delay

aliases: polling_delay

integer

The internal polling interval (in seconds) while the module waits for the DE virtual cluster to achieve the declared state.

Default: 30

memory_requests

string

Memory requests for autoscaling - eg. 30Gi.

name

string / required

The name of the CDE Virtual Cluster

runtime_spot_component

string

Used to describe where the Driver and the Executors would run, on-demand or spot instances.

spark_version

string

Spark version for the virtual cluster (e.g. SPARK2 or SPARK3).

state

string

The declarative state of the CDE virtual cluster

Choices:

  • "present" ← (default)

  • "absent"

timeout

aliases: polling_timeout

integer

The internal polling timeout (in seconds) while the module waits for the DE virtual cluster to achieve the declared state.

Default: 600

wait

boolean

Flag to enable internal polling to wait for the DE virtual cluster to achieve the declared state.

If set to FALSE, the module will return immediately.

Choices:

  • false

  • true ← (default)

Examples

# Create a CDE virtual cluster
- cloudera.cloud.de_virtual_cluster:
    name: virtual-cluster-name
    cluster_name: cde-service-name
    env: cdp-environment-name
    state: present
    wait: True
    delay: 30
    timeout: 600

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key

Description

virtual_cluster

complex

DE virtual cluster

Returned: always

accessControl

dictionary

Access control object for the Virtual Cluster

Returned: always

users

list / elements=string

Workload usernames of CDP users granted access to the Virtual Cluster.

Returned: always

chartValueOverrides

list / elements=dictionary

Chart overrides for the CDE virtual cluster.

Returned: always

ChartValueOverridesResponse

list / elements=dictionary

Returned: always

chartName

string

Name of the chart that has to be overridden.

Returned: always

overrides

string

Space separated key value-pairs for overriding chart values (colon separated)

Returned: always

clusterId

string

Cluster ID of the CDE service that contains the Virtual Cluster

Returned: always

creationTime

string

Time of creation of the virtual Cluster

Returned: always

creatorEmail

string

Email address of the creator of Virtual Cluster

Returned: always

creatorID

string

ID of the creator of Virtual Cluster

Returned: always

creatorName

string

Name of the creator of the Virtual Cluster

Returned: always

historyServerUrl

string

Spark History Server URL for the Virtual Cluster

Returned: always

livyServerUrl

string

Livy Server URL for the Virtual Cluster

Returned: always

resources

complex

Resources details of CDE virtual cluster.

Returned: always

VcResources

complex

Object to store resources for a CDE service.

Returned: always

actualCpuRequests

string

Actual CPU request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

Returned: always

actualMemoryRequests

string

Actual Memory request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

Returned: always

cpuRequests

string

The CPU requests for VC for running spark jobs.

Returned: always

memRequests

string

The Memory requests for VC for running spark jobs.

Returned: always

safariUrl

string

Safari URL for the Virtual Cluster

Returned: always

sparkVersion

string

Spark version for the virtual cluster

Returned: always

status

string

Status of the Virtual Cluster

Returned: always

vcApiUrl

string

Url for the Virtual Cluster APIs

Returned: always

vcId

string

Virtual Cluster ID

Returned: always

vcName

string

Name of the CDE Virtual Cluster

Returned: always

VcUiUrl

string

URL of the CDE Virtual Cluster UI

Returned: always

Authors

  • Curtis Howard (@curtishoward)