Documentation for Cloudera-Labs

These docs are for new users and power users alike to leverage Cloudera’s Ansible implementation for deployment and automation of Cloudera Products within the wider Hybrid Data Cloud ecosystem.

It is generally broken into several sections:

This framework is generally called the Cloudera Ansible Foundry, and it wraps software dependencies, automation code, examples, and documentation into an integrated family.

It further provides a consistent and portable toolkit for working in Python, Bash, Terraform, various CLIs, language clients, ad-hoc commands, and other commonly used tools in the Hybrid Cloud ecosystem.

Cloudera Deploy is the name of the reference implementation in Ansible Playbooks for automating Cloudera Products.

For most users, Cloudera-Deploy is the entrypoint and only component they need interact with, but underneath it there is an extensible framework of components for handling various scenarios for power-users.

To that end, you can use Cloudera-Deploy itself directly, or use it as a starting point for your own implementation, or simply use the components within for your own purposes.

These simple use cases can be run from the Cloudera-Deploy Quickstart without requiring additional skills in advance beyond use of cmdline, docker, and cloud credentials

In these cases, we expect that you are a more experienced DevOps user with particular goals in mind and at a good working knowledge of the technologies in play

This is the standard entry-point for most users, most of the time.

It contains the main Ansible Playbooks, default User Profile, Definition handling, and Run initialization.

It also usually contains other ease-of-use features for new users, like the current Dynamic Inventory implementation, until such time as they are matured into their own separate feature.

A Container image as a common Runner with all dependencies suitable for use locally, with Ansible Tower, against an IDE, or various other circumstances

It is based in a centos8 image produced by the Ansible Community, known as 'Ansible Runner'. We then resolve the difficulty of conflicting dependencies to layer in the various clients, python modules, utilities, and other bits and pieces that make it a useful shell or remote execution environment for working with Hybrid Cloud.

You can also use this as a template for your own specific implementation, but we ask Users to adopt it as their default if they don’t have a reason not too for the simple benefit of not reinventing something that works well and reduces the burden of reproducing errors.

A Python client for the CDP Control Plane, both Public and Private.

This is essentially a convenience wrapper for CDPCLI, which itself is based in a fork of AWSCLI and fully written in Python.

cdpy contains a large number of helper methods which are reused throughout the cloudera.cloud modules, as well as well-structured Ansible-friendly error handling.

This Collection is primarily un-opinionated modules for the CDP Control Plane in Public or Private Cloud.

The point of keeping the modules unopinionated and solely covering the CDP Control Plane interactions is to minimise the dependencies and attack surface for users who aren’t using Public Cloud.

This Collection is highly opinionated Roles for most task sequences around achieving some run-level or dependency satisfaction.

The concept of run-level deployments in cloudera.exe is explained later in the Architecture.

This Collection is focused on Deploying & Configuring Clusters via Cloudera Manager - usually traditional Cloudera clusters.

It is backwards compatible with the Cloudera Playbooks written for Ansible <2.9, whereas this Collection is for >2.10

Example Definitions for use with Cloudera-Deploy, presently bundled here

We plan on publishing more examples soon.