cloudera.cluster.cluster module – Manage the lifecycle and state of a cluster

Note

This module is part of the cloudera.cluster collection (version 4.4.0).

To install it, use: ansible-galaxy collection install cloudera.cluster. You need further requirements to be able to use this module, see Requirements for details.

To use it in a playbook, specify: cloudera.cluster.cluster.

Synopsis

  • Enables cluster management, cluster creation, deletion, and unified control of all services of a cluster.

  • Create or delete cluster in Cloudera Manager.

  • Start or stop all services inside the cluster.

  • If provided the template parameter, the module will populate its parameters using the cluster template contents.

  • Any other parameters will merge on top of the parameters declared by the cluster template.

Requirements

The below requirements are needed on the host that executes this module.

  • cm-client

Parameters

Parameter

Comments

add_repositories

boolean

Flag to add any repositories found in a cluster template should be added to Cloudera Manager.

Valid only if template is set.

Choices:

  • false ← (default)

  • true

agent_header

aliases: user_agent

string

Set the HTTP user agent header when interacting with the CM API endpoint.

Default: "ClouderaFoundry"

auto_assign

aliases: auto_assign_roles

boolean

Flag enabling the auto-assignment of role in the cluster.

This function honors existing or declared assignments.

Choices:

  • false ← (default)

  • true

cluster_version

string

The runtime version of the cluster.

contexts

aliases: data_contexts

list / elements=string

List of data contexts for compute-type clusters.

Required if type=COMPUTE_CLUSTER.

debug

aliases: debug_endpoints

boolean

Capture the HTTP interaction logs with the CM API endpoint.

Choices:

  • false ← (default)

  • true

display_name

string

The name of the cluster in the Cloudera Manager UI.

If not set, the name of the cluster will be used.

force_tls

boolean

Flag to force TLS during CM API endpoint discovery.

If False, discovery will first try HTTP and follow any redirects.

Choices:

  • false ← (default)

  • true

host

aliases: hostname

string

Hostname of the CM API endpoint.

If set, the host parameter will trigger CM API endpoint discovery, which will follow redirects.

Mutually exclusive with url.

host_templates

list / elements=dictionary

List of host template definitions for the cluster for use with hosts.

name

string / required

The name of the host template.

role_groups

aliases: role_config_groups

list / elements=dictionary

List of role groups, base and custom, to associated with the host template.

name

aliases: ref, ref_name

string

The name of the custom role group.

Mutually exclusive with type.

service

aliases: service_name, service_ref

string / required

The name of the service associated with the role group.

type

aliases: role_type

string

The role type of the base role group for the service.

Mutually exclusive with name.

hosts

list / elements=dictionary

List of hosts and their configuration attached to the cluster.

config

dictionary

The host configuration values.

host_template

string

The host template to apply to the host.

name

aliases: host_id, hostname

string / required

The hostname or host ID of the host.

role_groups

aliases: role_config_groups

list / elements=dictionary

List of role groups to associate directly with the host.

name

aliases: ref, ref_name

string

The name of the custom role group.

Mutually exclusive with type.

service

aliases: service_name, service_ref

string / required

The name of the service associated with the role group.

type

aliases: role_type

string

The role type of the base role group for the service.

Mutually exclusive with name.

roles

list / elements=dictionary

List of per-host role instance configuration overrides.

config

dictionary / required

The role instance override configuration values.

service

aliases: service_name, service_ref

string / required

The name of the service of the role instance.

type

aliases: role_type

string / required

The role type of the role instance.

tags

dictionary

The tags labeling the host.

maintenance

aliases: maintenance_enabled

boolean

Flag enabling the maintenance mode for the cluster.

Choices:

  • false

  • true

message

aliases: msg

string

Message to log for any changes to the entity.

Default: "Managed by Ansible"

name

aliases: cluster_name

string / required

The cluster name.

parcels

dictionary

The parcels by version enabled for a cluster.

The name of the parcel is the c(key), the version of the parcel is the c(value).

password

string / required

Password for access to the CM API endpoint.

This parameter is set to no_log.

port

integer

Port of the CM API endpoint.

If set, CM API endpoint discovery will connect to the designated port first and will follow redirects.

Default: 7180

proxy_server

aliases: proxy, http_proxy

string

Set the HTTP/S proxy server when interacting with the CM API endpoint.

purge

boolean

Set all parameters to either their default or declared values.

Choices:

  • false ← (default)

  • true

services

list / elements=dictionary

List of services and service configurations to enable for the cluster.

config

dictionary

The service configuration values, i.e. the “service-wide” configuration.

display_name

string

The display name of the service in the Cloudera Manager UI.

If not set, the name will be used.

name

aliases: ref, ref_name

string

The name or reference of the service.

If not set, a name for the service will be auto-generated.

role_groups

aliases: role_config_groups

list / elements=dictionary

List of role configuration groups for a role type of the service.

Both base and custom role configuration groups are supported.

config

dictionary

The role group configuration values.

display_name

string

The display name of the role group in the Cloudera Manager UI.

If not set, the name will be used.

name

aliases: ref, ref_name

string

The name of the role group.

If not set, the base role group, i.e. the default role group, for the role type is used.

type

aliases: role_type

string / required

The role type for the role group.

tags

dictionary

The tags labeling the service.

type

string / required

The type of service to enable.

version

string

The version of the service to enable.

ssl_ca_cert

aliases: tls_cert, ssl_cert

path

Path to SSL CA certificate to use for validation.

state

string

The state of the cluster.

If state=absent, the cluster’s services will be shut down first, then the cluster deleted.

Choices:

  • "present" ← (default)

  • "absent"

  • "stopped"

  • "started"

  • "restarted"

tags

dictionary

The tags labeling the cluster.

template

aliases: cluster_template

path

A cluster template file to populate the module parameters, e.g. as a baseline.

tls

aliases: tls_enabled, cluster_tls

boolean

Flag enabling TLS for the cluster.

Choices:

  • false

  • true

type

string

The type of cluster.

Choices:

  • "BASE_CLUSTER"

  • "COMPUTE_CLUSTER"

  • "EXPERIENCE_CLUSTER"

url

aliases: endpoint, cm_endpoint_url

string

The CM API endpoint URL and should include scheme, host, port, and API root path.

Mutually exclusive with host.

username

aliases: user

string / required

Username for access to the CM API endpoint.

verify_tls

boolean

Verify the TLS certificates for the CM API endpoint.

Choices:

  • false

  • true ← (default)

version

aliases: api_version

string

API version of the CM API endpoint.

Default: "True"

Attributes

Attribute

Support

Description

check_mode

Support: full

Can run in check_mode and return changed status prediction without modifying target

diff_mode

Support: full

Will return details on what has changed (or possibly needs changing in check_mode), when in diff mode

platform

Platforms: all

Target OS/families that can be operated against

Examples

- name: Create a minimal cluster (can be used by other modules to establish services, etc.)
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: 7180
    cluster_name: example-cluster
    cluster_version: "7.1.9"
    cluster_type: BASE_CLUSTER
    state: present

- name: Start all services on a cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-cluster
    state: started

- name: Delete a Cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-cluster
    state: absent

- name: Create a cluster with a direct assignment of a base role group
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-assignment-base
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ROLE_GROUP_ASSIGNMENT
        type: ZOOKEEPER
        role_groups:
          - type: SERVER
            display_name: Server Base Group
            config:
              zookeeper_server_java_heapsize: 134217728 # 128MB
          - name: DIRECT_ROLE_GROUP_ASSIGNMENT
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 33554432  # 32MB
    hosts:
      - name: worker-02.cldr.internal
        role_groups:
          - type: SERVER
            service: ROLE_GROUP_ASSIGNMENT

- name: Create a cluster with a direct assignment of a custom role group
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-assignment-custom
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ROLE_GROUP_ASSIGNMENT
        type: ZOOKEEPER
        role_groups:
          - type: SERVER
            display_name: Server Base Group
            config:
              zookeeper_server_java_heapsize: 134217728 # 128MB
          - name: DIRECT_ROLE_GROUP_ASSIGNMENT
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 33554432  # 32MB
    hosts:
      - name: worker-02.cldr.internal
        role_groups:
          - type: SERVER
            name: DIRECT_ROLE_GROUP_ASSIGNMENT

- name: Create a cluster with a per-host override of a role configuration
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: example-host-override
    cluster_version: "7.1.9"
    type: BASE_CLUSTER
    state: present
    services:
      - name: ZK-EXAMPLE
        type: ZOOKEEPER
        role_groups:
          - name: NON-BASE-SERVER
            type: SERVER
            display_name: Server Custom Group
            config:
              zookeeper_server_java_heapsize: 75497472  # 72MB
    hosts:
      - name: worker-01.cldr.internal
        roles:
          - service: ZK-EXAMPLE
            type: SERVER
            config:
              zookeeper_server_java_heapsize: 67108864 # 64MB
        host_template: Example_Template
    host_templates:
      - name: Example_Template
        role_groups:
          - NON-BASE-SERVER
    parcels:
      CDH: "7.1.9"

- name: Create and establish a minimal base cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    port: "7180"
    username: "jane_smith"
    password: "S&peR4Ec*re"
    name: Basic_Cluster
    cluster_version: "7.1.9-1.cdh7.1.9.p0.44702451"
    type: BASE_CLUSTER
    state: started
    services:
      - name: core-settings-0
        type: CORE_SETTINGS
        display_name: Core Settings
      - name: zookeeper-0
        type: ZOOKEEPER
        display_name: Zookeeper
        config:
          zookeeper_datadir_autocreate: yes
      - name: hdfs-0
        type: HDFS
        config:
            zookeeper_service: zookeeper-0
            core_connector: core-settings-0
        role_groups:
          - type: DATANODE
            config:
              dfs_data_dir_list: /dfs/dn
          - type: NAMENODE
            config:
              dfs_name_dir_list: /dfs/nn
          - type: SECONDARYNAMENODE
            config:
              fs_checkpoint_dir_list: /dfs/snn
      - name: yarn-0
        type: YARN
        config:
          hdfs_service: hdfs-0
          zookeeper_service: zookeeper-0
        role_groups:
          - type: RESOURCEMANAGER
            config:
              yarn_scheduler_maximum_allocation_mb: 4096
              yarn_scheduler_maximum_allocation_vcores: 4
          - type: NODEMANAGER
            config:
              yarn_nodemanager_resource_memory_mb: 4096
              yarn_nodemanager_resource_cpu_vcores: 4
              yarn_nodemanager_local_dirs: /tmp/nm
              yarn_nodemanager_log_dirs: /var/log/nm
          - type: GATEWAY
            config:
              mapred_submit_replication: 3
              mapred_reduce_tasks: 6
    host_templates:
      - name: Master1
        role_groups:
          - service: HDFS
            type: NAMENODE
          - service: HDFS
            type: SECONDARYNAMENODE
          - service: YARN
            type: RESOURCEMANAGER
          - service: YARN
            type: JOBHISTORY
      - name: Worker
        role_groups:
          - service: HDFS
            type: DATANODE
          - service: YARN
            type: NODEMANAGER
          - service: ZOOKEEPER
            type: SERVER
    parcels:
      CDH: "7.1.9-1.cdh7.1.9.p0.44702451"
    hosts:
      - name: master-01.cldr.internal
        host_template: Master1
      - name: worker-01.cldr.internal
        host_template: Worker
      - name: worker-02.cldr.internal
        host_template: Worker
      - name: worker-03.cldr.internal
        host_template: Worker

- name: Create a cluster from a cluster template
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: "7180"
    name: example-cluster
    template: "./files/cluster-template.json"
    add_repositories: yes

- name: Create an ECS cluster
  cloudera.cluster.cluster:
    host: example.cloudera.com
    username: "jane_smith"
    password: "S&peR4Ec*re"
    port: 7180
    cluster_name: example-cluster
    cluster_version: "1.5.1-b626.p0.42068229"
    cluster_type: EXPERIENCE_CLUSTER
    state: present

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key

Description

cloudera_manager

dictionary

Details about Cloudera Manager Cluster

Returned: success

cluster_type

string

The type of cluster.

Returned: always

display_name

string

The name of the cluster displayed in the Cloudera Manager UI.

Returned: always

entity_status

string

Health status of the cluster.

Returned: always

maintenance_mode

boolean

Maintance mode of cluster.

Returned: always

maintenance_owners

list / elements=string

List of maintance owners for cluster.

Returned: always

name

string

The name of the cluster.

Returned: always

tags

list / elements=string

List of tags for cluster.

Returned: always

uuid

boolean

The unique ID of the cluster.

Returned: always

version

string

Version of the cluster installed.

Returned: always

Authors

  • Ronald Suplina (@rsuplina)

  • Webster Mudge (@wmudge)

Collection links