cloudera.services.atlas_business_metadata_def module – Manage business metadata type definitions in Apache Atlas

Note

This module is part of the cloudera.services collection (version 1.0.0).

It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install git+https://github.com/cloudera-labs/cloudera.services.git.

To use it in a playbook, specify: cloudera.services.atlas_business_metadata_def.

New in cloudera.services 1.1.0

Synopsis

  • Create, update, or delete business metadata type definitions in Apache Atlas.

  • Business metadata is a named group of attributes that can be attached to entities. Each attribute declares the entity types it applies to through its attribute_defs[].options.

Parameters

Parameter

Comments

attribute_defs

list / elements=dictionary

Attributes of the business metadata.

Replaces the existing attributes; Atlas may reject removing attributes that are in use.

Unset suboptions keep the value Atlas already holds for an attribute of the same name.

Each attribute normally sets applicableEntityTypes in its attribute_defs[].options to say which entity types it applies to.

cardinality

string

Cardinality of the attribute.

Choices:

  • "SINGLE"

  • "LIST"

  • "SET"

constraints

list / elements=dictionary

Constraints on the attribute.

params

dictionary

Parameters of the constraint.

Keys are sent to Atlas verbatim.

type

string / required

Type of the constraint, e.g. ownedRef or inverseRef.

default_value

string

Default value of the attribute.

description

string

Description of the attribute.

display_name

string

Display name of the attribute.

include_in_notification

boolean

Whether changes to the attribute are included in notifications.

Choices:

  • false

  • true

index_type

string

Index type of the attribute.

Choices:

  • "DEFAULT"

  • "STRING"

is_indexable

boolean

Whether the attribute is indexed.

Choices:

  • false

  • true

is_optional

boolean

Whether a value is optional.

Choices:

  • false

  • true

is_unique

boolean

Whether values must be unique.

Choices:

  • false

  • true

name

string / required

Name of the attribute.

options

dictionary

Free-form map of string options for the attribute.

Keys and values are sent to Atlas verbatim.

When set, the map replaces all of the attribute’s existing options, so include every option to keep.

All option values are strings, as Atlas stores them. Quote numbers, e.g. "50", and convert variables with the ansible.builtin.string filter.

applicableEntityTypes is the entity types the attribute applies to, as a JSON list in a string, e.g. '["hive_table", "hive_db"]'.

To build applicableEntityTypes from a list, use the ansible.builtin.to_json filter as the last filter in the expression, so the result stays a string.

To read applicableEntityTypes back as a list, use the ansible.builtin.from_json filter.

The value is compared as a string, so keep the same formatting between runs, including spaces after commas, to avoid reporting a change.

maxStrLength is the maximum length of a string attribute, e.g. "50".

search_weight

integer

Search weight of the attribute.

type_name

string / required

Name of the attribute type, e.g. string, int, or the name of an enum.

values_max_count

integer

Maximum number of values.

values_min_count

integer

Minimum number of values.

client_cert

path

The path to a client certificate for authenticating to the API endpoint.

client_key

path

The path to a client key for authenticating to the API endpoint.

debug

aliases: debug_endpoints

boolean

A flag to enable debug logging of the module’s execution.

Choices:

  • false ← (default)

  • true

description

string

Description of the business metadata type definition.

force

boolean

Delete the business metadata without Atlas validating that nothing references it.

Atlas refuses to delete business metadata that has non-indexable attributes unless this is set. The error reads Deletion not allowed for non-indexable Business Metadata.

Orphaned references to the deleted business metadata may remain on entities.

Only used when state=absent.

Choices:

  • false ← (default)

  • true

force_basic_auth

boolean

A flag to force basic authentication for API requests.

Choices:

  • false ← (default)

  • true

http_agent

aliases: user_agent

string

The User-Agent string to send with API requests.

Default: "cloudera-services-module"

name

string

Name of the business metadata type definition.

Required when state=present or state=absent.

options

dictionary

Free-form map of string options for the type definition.

Keys are sent to Atlas verbatim.

page_size

aliases: default_page_size

integer

The number of items to return per page in a paginated API response.

Default: 100

service_type

string

Name of the service that owns the type definition.

state

string

Whether the business metadata type definition should be present or absent.

Choices:

  • "present" ← (default)

  • "absent"

timeout

aliases: timeout_seconds

integer

The timeout in seconds for any API requests.

Default: 60

type_version

string

Version label of the type definition, e.g. 1.0.

url

aliases: endpoint, endpoint_url

string / required

The base URL of the API endpoint, including the port if necessary.

url_password

string

The password for authenticating to the API endpoint.

url_username

string

The username for authenticating to the API endpoint.

use_gssapi

boolean

A flag to enable or disable GSSAPI authentication for API requests.

Choices:

  • false ← (default)

  • true

use_proxy

boolean

A flag to enable or disable the use of a proxy for API requests.

Choices:

  • false

  • true ← (default)

validate_certs

boolean

A flag to enable or disable SSL certificate validation for API requests.

Choices:

  • false

  • true ← (default)

Attributes

Attribute

Support

Description

check_mode

Support: full

This module supports check mode.

diff_mode

Support: full

This module supports diff mode.

Examples

# NOTE: Examples do not include connection and authentication fields
- name: Create business metadata for tables
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    description: Governance details for tables
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table"]'
          maxStrLength: "50"

# The option values are plain strings. applicableEntityTypes holds a JSON list
# encoded as a string, so a list of entity types can be written in three ways.

- name: Apply an attribute to several entity types using a literal JSON string
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table", "hive_db"]'

- name: Use a folded block scalar for a long list of entity types
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options:
          applicableEntityTypes: >-
            ["hive_table", "hive_db", "hive_column", "hive_storagedesc"]

- name: Build the JSON string from a list variable
  vars:
    governed_types:
      - hive_table
      - hive_db
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options:
          # Keep to_json as the last filter so the value stays a string
          # and is not turned back into a list.
          applicableEntityTypes: "{{ governed_types | to_json }}"

- name: Set the maximum string length from a variable
  vars:
    owner_max_length: 50
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table"]'
          # Option values must be strings, so quote literals ("50") and
          # convert variables.
          maxStrLength: "{{ owner_max_length | string }}"

- name: Share one set of options across several attributes
  vars:
    table_scope:
      applicableEntityTypes: '["hive_table"]'
      maxStrLength: "100"
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        is_optional: true
        options: "{{ table_scope }}"
      - name: steward
        type_name: string
        is_optional: true
        options: "{{ table_scope }}"

- name: Widen the entity types an existing attribute applies to
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    attribute_defs:
      - name: owner
        type_name: string
        options:
          # The options map is replaced as a whole, so repeat every option
          # the attribute should keep.
          applicableEntityTypes: '["hive_table", "hive_db", "hive_column"]'
          maxStrLength: "50"

- name: Read the entity types back as a list
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
  register: governance

- name: Show the entity types of each attribute
  ansible.builtin.debug:
    msg: >-
      {{ item.name }} applies to
      {{ (item.options.applicableEntityTypes | from_json) | join(', ') }}
  loop: "{{ governance.business_metadata_def.attribute_defs }}"
  loop_control:
    label: "{{ item.name }}"

- name: Create business metadata with several attribute types
  cloudera.services.atlas_business_metadata_def:
    name: data_quality
    attribute_defs:
      - name: score
        type_name: int
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table"]'
      - name: certified
        type_name: boolean
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table"]'
      - name: reviewers
        type_name: array<string>
        is_optional: true
        options:
          applicableEntityTypes: '["hive_table"]'
          maxStrLength: "100"

- name: Preview a change without applying it
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    description: Governance details for tables and databases
  check_mode: true
  diff: true
  register: preview

- name: Remove business metadata
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    state: absent

- name: Remove business metadata that has non-indexable attributes
  cloudera.services.atlas_business_metadata_def:
    name: data_governance
    state: absent
    # Atlas will not validate references for non-indexable attributes, so
    # orphaned references may remain on entities.
    force: true

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key

Description

business_metadata_def

dictionary

The business metadata type definition.

Returned: when state=present

attribute_defs

list / elements=dictionary

Attributes of the business metadata.

Returned: success

cardinality

string

Cardinality of the attribute.

Returned: success

constraints

list / elements=dictionary

Constraints on the attribute.

Returned: success

params

dictionary

Parameters of the constraint.

Returned: success

type

string

Type of the constraint.

Returned: success

default_value

string

Default value of the attribute.

Returned: success

description

string

Description of the attribute.

Returned: success

display_name

string

Display name of the attribute.

Returned: success

include_in_notification

boolean

Whether changes are included in notifications.

Returned: success

index_type

string

Index type of the attribute.

Returned: success

is_indexable

boolean

Whether the attribute is indexed.

Returned: success

is_optional

boolean

Whether a value is optional.

Returned: success

is_unique

boolean

Whether values must be unique.

Returned: success

name

string

Name of the attribute.

Returned: success

options

dictionary

Free-form options of the attribute.

Returned: success

search_weight

integer

Search weight of the attribute.

Returned: success

type_name

string

Name of the attribute type.

Returned: success

values_max_count

integer

Maximum number of values.

Returned: success

values_min_count

integer

Minimum number of values.

Returned: success

category

string

Type category, always BUSINESS_METADATA.

Returned: success

create_time

integer

Creation time in epoch milliseconds.

Returned: success

created_by

string

User that created the type definition.

Returned: success

description

string

Description of the business metadata type definition.

Returned: success

guid

string

Atlas-assigned identifier.

Returned: success

name

string

Name of the business metadata type definition.

Returned: success

options

dictionary

Free-form options of the type definition.

Returned: success

service_type

string

Name of the service that owns the type definition.

Returned: success

type_version

string

Version label of the type definition.

Returned: success

update_time

integer

Last update time in epoch milliseconds.

Returned: success

updated_by

string

User that last updated the type definition.

Returned: success

version

integer

Atlas version number of the type definition.

Returned: success

Authors

  • Webster Mudge (@wmudge)