cloudera.services.smm_kafka_topic module – Manage Kafka topics in Streams Messaging Manager

Note

This module is part of the cloudera.services collection (version 1.0.0).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install git+https://github.com/cloudera-labs/cloudera.services.git.

To use it in a playbook, specify: cloudera.services.smm_kafka_topic.

New in cloudera.services 1.1.0

Synopsis

  • Create, update, or delete Kafka topics through Cloudera Streams Messaging Manager (SMM).

  • An existing topic can have its partitions increased and its topic-level configs changed.

  • Kafka cannot decrease the partitions or change the replication factor of an existing topic, so the module fails if either is requested.

Parameters

Parameter

Comments

client_cert

path

The path to a client certificate for authenticating to the API endpoint.

client_key

path

The path to a client key for authenticating to the API endpoint.

configs

dictionary

A dictionary of topic-level configuration overrides, for example retention.ms or cleanup.policy.

Values are converted to strings before they are sent to Kafka.

Only the listed configs are set, other topic-level overrides already present on the topic are kept.

SMM derives an hours variant for some configs, for example retention.hours for retention.ms. When you change one of them, the old derived value is not sent back, so SMM recalculates it.

Config names are sent to SMM verbatim.

debug

aliases: debug_endpoints

boolean

A flag to enable debug logging of the module’s execution.

Choices:

  • false ← (default)

  • true

force

boolean

A flag to force a refresh of the API request, ignoring any cached results.

Choices:

  • false ← (default)

  • true

force_basic_auth

boolean

A flag to force basic authentication for API requests.

Choices:

  • false ← (default)

  • true

http_agent

aliases: user_agent

string

The User-Agent string to send with API requests.

Default: "cloudera-services-module"

name

string / required

Name of the Kafka topic.

page_size

aliases: default_page_size

integer

The number of items to return per page in a paginated API response.

Default: 100

partitions

aliases: num_partitions

integer

Number of partitions of the topic.

Required when the topic is created.

On an existing topic the number can only be increased.

replication_factor

integer

Number of replicas of each partition.

Required when the topic is created.

It cannot be changed on an existing topic.

state

string

Whether the topic should be present or absent.

Choices:

  • "present" ← (default)

  • "absent"

timeout

aliases: timeout_seconds

integer

The timeout in seconds for any API requests.

Default: 60

url

aliases: endpoint, endpoint_url

string / required

The base URL of the API endpoint, including the port if necessary.

url_password

string

The password for authenticating to the API endpoint.

url_username

string

The username for authenticating to the API endpoint.

use_gssapi

boolean

A flag to enable or disable GSSAPI authentication for API requests.

Choices:

  • false ← (default)

  • true

use_proxy

boolean

A flag to enable or disable the use of a proxy for API requests.

Choices:

  • false

  • true ← (default)

validate_certs

boolean

A flag to enable or disable SSL certificate validation for API requests.

Choices:

  • false

  • true ← (default)

Attributes

Attribute

Support

Description

check_mode

Support: full

This module supports check mode.

diff_mode

Support: full

This module supports diff mode.

Examples

# NOTE: Examples do not include connection and authentication fields

- name: Create a Kafka topic
  cloudera.services.smm_kafka_topic:
    name: "orders"
    partitions: 3
    replication_factor: 2
    configs:
      retention.ms: 604800000
      cleanup.policy: "delete"

- name: Increase the partitions and change a config of an existing topic
  cloudera.services.smm_kafka_topic:
    name: "orders"
    partitions: 6
    configs:
      retention.ms: 86400000

- name: Delete a Kafka topic
  cloudera.services.smm_kafka_topic:
    name: "orders"
    state: "absent"

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key

Description

topic

dictionary

A dictionary containing details about the topic.

Empty when the topic is absent.

Returned: always

configs

dictionary

Topic-level configuration overrides.

Settings inherited from the broker or built-in defaults are not included.

SMM can also report values it derives from a set config, such as retention.hours for retention.ms.

Returned: always

internal

boolean

Whether the topic is a Kafka internal topic.

Returned: when the topic exists

name

string

Name of the topic.

Returned: always

partition_details

list / elements=dictionary

Leader, replicas and in-sync replicas of every partition.

Returned: when the topic exists

isr

list / elements=dictionary

Brokers whose replica is in sync, in the same format as topic.partition_details[].leader.

Returned: always

leader

dictionary

Broker currently leading the partition.

Returned: always

host

string

Broker host name.

Returned: always

id

integer

Broker ID.

Returned: always

is_controller

boolean

Whether the broker is the cluster controller.

Returned: always

port

integer

Broker port.

Returned: always

rack

string

Rack of the broker.

Returned: always

partition

integer

Partition number.

Returned: always

replicas

list / elements=dictionary

Brokers holding a replica of the partition, in the same format as topic.partition_details[].leader.

Returned: always

partitions

integer

Number of partitions of the topic.

Returned: always

replication_factor

integer

Number of replicas of each partition.

Returned: when the topic has at least one partition or the replication factor was requested

Authors

  • Ronald Suplina (@rsuplina)