cloudera.cluster.cluster module – Manage the lifecycle and state of a cluster

Note

This module is part of the cloudera.cluster collection (version 5.0.0).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install cloudera.cluster. You need further requirements to be able to use this module, see Requirements for details.

To use it in a playbook, specify: cloudera.cluster.cluster.

New in cloudera.cluster 4.4.0

Synopsis 

Enables cluster management, cluster creation, deletion, and unified control of all services of a cluster.
Create or delete cluster in Cloudera Manager.
Start or stop all services inside the cluster.
If provided the template parameter, the module will populate its parameters using the cluster template contents.
Any other parameters will merge on top of the parameters declared by the cluster template.

Requirements 

The below requirements are needed on the host that executes this module.

cm-client

Parameters 

Parameter	Comments
add_repositories boolean	Flag to add any repositories found in a cluster template should be added to Cloudera Manager. Valid only if template is set. Choices: `false` ← (default) `true`
agent_header aliases: user_agent string	Set the HTTP user agent header when interacting with the CM API endpoint. Default: `"ClouderaFoundry"`
auto_assign aliases: auto_assign_roles boolean	Flag enabling the auto-assignment of role in the cluster. This function honors existing or declared assignments. Choices: `false` ← (default) `true`
auto_tls aliases: tls_enabled, cluster_tls boolean	Flag enabling TLS for the cluster. Choices: `false` `true`
cluster_version string	The runtime version of the cluster.
contexts aliases: data_contexts list / elements=string	List of data contexts for compute-type clusters. Required if type=COMPUTE_CLUSTER.
control_plane dictionary	Private Cloud Control Plane on embedded kubernetes
control_plane_config aliases: values_yaml dictionary / required	A yaml structured dictionary with configuration parameters for the installation.
datalake_cluster_name string / required	The name of the datalake cluster to use for the initial environment in this control plane.
remote_repo_url string / required	The url of the remote repository where the private cloud artifacts to install are hosted.
debug aliases: debug_endpoints boolean	Capture the HTTP interaction logs with the CM API endpoint. Choices: `false` ← (default) `true`
display_name string	The name of the cluster in the Cloudera Manager UI. If not set, the name of the cluster will be used.
force_tls boolean	Flag to force TLS during CM API endpoint discovery. If `False`, discovery will first try HTTP and follow any redirects. Choices: `false` ← (default) `true`
host aliases: hostname string	Hostname of the CM API endpoint. If set, the `host` parameter will trigger CM API endpoint discovery, which will follow redirects. Mutually exclusive with url.
host_templates list / elements=dictionary	List of host template definitions for the cluster for use with hosts.
name string / required	The name of the host template.
role_groups aliases: role_config_groups list / elements=dictionary	List of role groups, base and custom, to associated with the host template.
name aliases: ref, ref_name string	The name of the custom role group. Mutually exclusive with type.
service aliases: service_name, service_ref string / required	The name of the service associated with the role group.
type aliases: role_type string	The role type of the base role group for the service. Mutually exclusive with name.
hosts list / elements=dictionary	List of hosts and their configuration attached to the cluster.
config dictionary	The host configuration values.
host_template string	The host template to apply to the host.
name aliases: host_id, hostname string / required	The hostname or host ID of the host.
role_groups aliases: role_config_groups list / elements=dictionary	List of role groups to associate directly with the host.
name aliases: ref, ref_name string	The name of the custom role group. Mutually exclusive with type.
service aliases: service_name, service_ref string / required	The name of the service associated with the role group.
type aliases: role_type string	The role type of the base role group for the service. Mutually exclusive with name.
roles list / elements=dictionary	List of per-host role instance configuration overrides.
config dictionary / required	The role instance override configuration values.
service aliases: service_name, service_ref string / required	The name of the service of the role instance.
type aliases: role_type string / required	The role type of the role instance.
tags dictionary	The tags labeling the host.
maintenance aliases: maintenance_enabled boolean	Flag enabling the maintenance mode for the cluster. Choices: `false` `true`
message aliases: msg string	Message to log for any changes to the entity. Default: `"Managed by Ansible"`
name aliases: cluster_name string / required	The cluster name.
parcels dictionary	The parcels by version enabled for a cluster. The name of the parcel is the c(key), the version of the parcel is the c(value).
password string / required	Password for access to the CM API endpoint. This parameter is set to `no_log`.
port integer	Port of the CM API endpoint. If set, CM API endpoint discovery will connect to the designated port first and will follow redirects. Default: `7180`
proxy_server aliases: proxy, http_proxy string	Set the HTTP/S proxy server when interacting with the CM API endpoint.
purge boolean	Set all parameters to either their default or declared values. Choices: `false` ← (default) `true`
services list / elements=dictionary	List of services and service configurations to enable for the cluster.
config dictionary	The service configuration values, i.e. the “service-wide” configuration.
display_name string	The display name of the service in the Cloudera Manager UI. If not set, the name will be used.
name aliases: ref, ref_name string	The name or reference of the service. If not set, a name for the service will be auto-generated.
role_groups aliases: role_config_groups list / elements=dictionary	List of role configuration groups for a role type of the service. Both base and custom role configuration groups are supported.
config dictionary	The role group configuration values.
display_name string	The display name of the role group in the Cloudera Manager UI. If not set, the name will be used.
name aliases: ref, ref_name string	The name of the role group. If not set, the base role group, i.e. the default role group, for the role type is used.
type aliases: role_type string / required	The role type for the role group.
tags dictionary	The tags labeling the service.
type string / required	The type of service to enable.
version string	The version of the service to enable.
ssl_ca_cert aliases: tls_cert, ssl_cert path	Path to SSL CA certificate to use for validation.
state string	The state of the cluster. If state=absent, the cluster’s services will be shut down first, then the cluster deleted. Choices: `"present"` ← (default) `"absent"` `"stopped"` `"started"` `"restarted"`
tags dictionary	The tags labeling the cluster.
template aliases: cluster_template path	A cluster template file to populate the module parameters, e.g. as a baseline.
type string	The type of cluster. Choices: `"BASE_CLUSTER"` `"COMPUTE_CLUSTER"` `"EXPERIENCE_CLUSTER"`
url aliases: endpoint, cm_endpoint_url string	The CM API endpoint URL and should include scheme, host, port, and API root path. Mutually exclusive with host.
username aliases: user string / required	Username for access to the CM API endpoint.
verify_tls boolean	Verify the TLS certificates for the CM API endpoint. Choices: `false` `true` ← (default)
version aliases: api_version string	API version of the CM API endpoint. Default: `"True"`

Attributes 

Attribute	Support	Description
check_mode	Support: full	Can run in check_mode and return changed status prediction without modifying target, if not supported the action will be skipped.
diff_mode	Support: full	Will return details on what has changed (or possibly needs changing in check_mode), when in diff mode
platform	Platforms: all	Target OS/families that can be operated against

Examples 

- name: Create a minimal cluster (can be used by other modules to establish services, etc.)
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: 7180
    cluster_name: example-cluster
    cluster_version: "7.1.9"
    cluster_type: BASE_CLUSTER
    state: present

- name: Start all services on a cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-cluster
    state: started

- name: Delete a Cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-cluster
    state: absent

- name: Create a cluster with a direct assignment of a base role group
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-assignment-base
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ROLE_GROUP_ASSIGNMENT
        type: ZOOKEEPER
        role_groups:
          - type: SERVER
            display_name: Server Base Group
            config:
              zookeeper_server_java_heapsize: 134217728 # 128MB
          - name: DIRECT_ROLE_GROUP_ASSIGNMENT
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 33554432  # 32MB
    hosts:
      - name: worker-02.cldr.internal
        role_groups:
          - type: SERVER
            service: ROLE_GROUP_ASSIGNMENT

- name: Create a cluster with a direct assignment of a custom role group
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-assignment-custom
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ROLE_GROUP_ASSIGNMENT
        type: ZOOKEEPER
        role_groups:
          - type: SERVER
            display_name: Server Base Group
            config:
              zookeeper_server_java_heapsize: 134217728 # 128MB
          - name: DIRECT_ROLE_GROUP_ASSIGNMENT
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 33554432  # 32MB
    hosts:
      - name: worker-02.cldr.internal
        role_groups:
          - type: SERVER
            name: DIRECT_ROLE_GROUP_ASSIGNMENT

- name: Create a cluster with a per-host override of a role configuration
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-override
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ZK-EXAMPLE
        type: ZOOKEEPER
        role_groups:
          - name: NON-BASE-SERVER
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 75497472  # 72MB
    hosts:
      - name: worker-01.cldr.internal
        roles:
          - service: ZK-EXAMPLE
            type: SERVER
            config:
              zookeeper_server_java_heapsize: 67108864 # 64MB
        host_template: Example_Template
    host_templates:
      - name: Example_Template
        role_groups:
          - NON-BASE-SERVER
    parcels:
      CDH: "7.1.9"

- name: Create and establish a minimal base cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: Basic_Cluster
    cluster_version: "7.1.9-1.cdh7.1.9.p0.44702451"
    type: BASE_CLUSTER
    state: started
    services:
      - name: core-settings-0
        type: CORE_SETTINGS
        display_name: Core Settings
      - name: zookeeper-0
        type: ZOOKEEPER
        display_name: Zookeeper
        config:
          zookeeper_datadir_autocreate: true
      - name: hdfs-0
        type: HDFS
        config:
          zookeeper_service: zookeeper-0
          core_connector: core-settings-0
        role_groups:
          - type: DATANODE
            config:
              dfs_data_dir_list: /dfs/dn
          - type: NAMENODE
            config:
              dfs_name_dir_list: /dfs/nn
          - type: SECONDARYNAMENODE
            config:
              fs_checkpoint_dir_list: /dfs/snn
      - name: yarn-0
        type: YARN
        config:
          hdfs_service: hdfs-0
          zookeeper_service: zookeeper-0
        role_groups:
          - type: RESOURCEMANAGER
            config:
              yarn_scheduler_maximum_allocation_mb: 4096
              yarn_scheduler_maximum_allocation_vcores: 4
          - type: NODEMANAGER
            config:
              yarn_nodemanager_resource_memory_mb: 4096
              yarn_nodemanager_resource_cpu_vcores: 4
              yarn_nodemanager_local_dirs: /tmp/nm
              yarn_nodemanager_log_dirs: /var/log/nm
          - type: GATEWAY
            config:
              mapred_submit_replication: 3
              mapred_reduce_tasks: 6
    host_templates:
      - name: Master1
        role_groups:
          - service: HDFS
            type: NAMENODE
          - service: HDFS
            type: SECONDARYNAMENODE
          - service: YARN
            type: RESOURCEMANAGER
          - service: YARN
            type: JOBHISTORY
      - name: Worker
        role_groups:
          - service: HDFS
            type: DATANODE
          - service: YARN
            type: NODEMANAGER
          - service: ZOOKEEPER
            type: SERVER
    parcels:
      CDH: "7.1.9-1.cdh7.1.9.p0.44702451"
    hosts:
      - name: master-01.cldr.internal
        host_template: Master1
      - name: worker-01.cldr.internal
        host_template: Worker
      - name: worker-02.cldr.internal
        host_template: Worker
      - name: worker-03.cldr.internal
        host_template: Worker

- name: Create a cluster from a cluster template
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: "7180"
    name: example-cluster
    template: "./files/cluster-template.json"
    add_repositories: true

- name: Create an ECS cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: 7180
    cluster_name: ECS-cluster
    cluster_version: "1.5.1-b626.p0.42068229"
    cluster_type: EXPERIENCE_CLUSTER
    state: present
    parcels:
      ECS: "1.5.1-b626.p0.42068229"
    services:
      - name: docker
        type: DOCKER
        config:
          docker_images_destination_registry_user: registry-user
          defaultDataPath: /mnt/docker
      - name: ecs
        type: ECS
        config:
          app_domain: test.lab.example
          k8s_webui_secret_admin_token: ecs-k8s_webui_secret_admin_token
          cp_prometheus_ingress_user: cloudera-manager
          infra_prometheus_ingress_user: cloudera-manager
          longhorn_replication: 2
          lsoDataPath: /ecs/local
          docker: docker
          cp_prometheus_ingress_password: password1
          infra_prometheus_ingress_password: password1
          defaultDataPath: /ecs/longhorn-storage
          nfs_over_provisioning: 800
    host_templates:
      - name: ecs_master
        role_groups:
          - service_type: DOCKER
            type: DOCKER_SERVER
          - service_type: ECS
            type: ECS_SERVER
      - name: ecs_workers
        role_groups:
          - service_type: DOCKER
            type: DOCKER_SERVER
          - service_type: ECS
            type: ECS_AGENT
    hosts:
      - name: ecs-master-01.test.lab.example
        host_template: ecs_master
      - name: ecs-worker-01.test.lab.example
        host_template: ecs_workers
      - name: ecs-worker-02.test.lab.example
        host_template: ecs_workers
      - name: ecs-worker-03.test.lab.example
        host_template: ecs_workers
    control_plane:
      datalake_cluster_name: PVC-Base
      remote_repo_url: "https://test_website/cdp-pvc-ds/1.5.1"
      control_plane_config:
        ContainerInfo:
          Mode: public
          CopyDocker: false
        Database:
          Mode: embedded
          EmbeddedDbStorage: 50
        Services:
          thunderheadenvironment:
            Config:
              database:
                name: db-env
          mlxcontrolplaneapp:
            Config:
              database:
                name: db-mlx
          dwx:
            Config:
              database:
                name: db-dwx
          cpxliftie:
            Config:
              database:
                name: db-liftie
          dex:
            Config:
              database:
                name: db-dex
          resourcepoolmanager:
            Config:
              database:
                name: db-resourcepoolmanager
          cdpcadence:
            Config:
              database:
                name: db-cadence
          cdpcadencevisibility:
            Config:
              database:
                name: db-cadence-visibility
          clusteraccessmanager:
            Config:
              database:
                name: db-clusteraccessmanager
          monitoringapp:
            Config:
              database:
                name: db-alerts
          thunderheadusermanagementprivate:
            Config:
              database:
                name: db-ums
          classicclusters:
            Config:
              database:
                name: cm-registration
          clusterproxy:
            Config:
              database:
                name: cluster-proxy
          dssapp:
            Config:
              database:
                name: db-dss-app
        Vault:
          Mode: embedded

Return Values 

Common return values are documented here, the following are the fields unique to this module:

Key	Description
cloudera_manager dictionary	Details about Cloudera Manager Cluster Returned: success
cluster_type string	The type of cluster. Returned: always
display_name string	The name of the cluster displayed in the Cloudera Manager UI. Returned: always
entity_status string	Health status of the cluster. Returned: always
maintenance_mode boolean	Maintance mode of cluster. Returned: always
maintenance_owners list / elements=string	List of maintance owners for cluster. Returned: always
name string	The name of the cluster. Returned: always
tags list / elements=string	List of tags for cluster. Returned: always
uuid boolean	The unique ID of the cluster. Returned: always
version string	Version of the cluster installed. Returned: always

Authors 

Ronald Suplina (@rsuplina)
Webster Mudge (@wmudge)