Molecule - Molecule aids in the development and testing of Ansible roles

mail

Molecule glossary

driver
Molecule delegates to the driver the task of creating hosts to operate on (sources : 1, 2).
mail

How to re-use Ansible role variables in Molecule unit tests ?

This is still a work in progress...
roles/logrotate/tasks/main.yml

- name: Install packages
  package:
    name:  "{{ item }}"
    state: present
  with_items: "{{ packagesToInstall }}"
roles/logrotate/defaults/main.yml

packagesToInstall: ['logrotate', 'bzip2']
roles/logrotate/molecule/myScenario/tests/test_myScenario.py
import os
import pytest

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']
    ).get_hosts('all')


@pytest.fixture
def get_vars(host):
    defaults_files = "file=../../defaults/main.yml name=role_defaults"

    ansible_vars = host.ansible(
        "include_vars",
        defaults_files)["ansible_facts"]["role_defaults"]

    return ansible_vars


def test_packagesAreInstalled(get_vars, host):
    for packageName in get_vars['packagesToInstall']:
        package = host.package(packageName)
        assert package.is_installed
mail

molecule/myScenario/molecule.yml advanced configuration

Based on molecule/myScenario/molecule.yml's initial configuration, here are some extra directives :
---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
  options:
    config-file: molecule/default/yamllint.yml
platforms:
  - name: ubuntu1604
    image: ubuntu:16.04
    privileged: True
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8
    options:
      max-line-length: 150
mail

molecule/myScenario/yamllint.yml advanced configuration

Here are some further configuration values for molecule/myScenario/yamllint.yml, based on this basic configuration :
---
extends: myScenario
rules:
  comments: disable
  comments-indentation: disable
  colons:
    max-spaces-before: 0
    max-spaces-after: -1
  line-length:
    max: 150
    level: warning
  truthy: disable
mail

Ansible Lint error : [403] Package installs should not use latest

This is pretty straightforward once you RTFM : use state: present instead of state: latest

mail

Molecule actions

molecule test is pretty verbose. The first lines of output display the "test matrix", listing the actions that will be executed :
--> Validating schema /home/bob/v_molecule/Ansible_Roles/roles/logrotate/molecule/myScenario/molecule.yml.
Validation completed successfully.
--> Test matrix

└── myScenario
    ├── lint
    ├── dependency
    ├── cleanup
    ├── destroy
    ├── syntax
    ├── create
    ├── prepare
    ├── converge
    ├── idempotence
    ├── side_effect
    ├── verify
    ├── cleanup
    └── destroy

--> Scenario: 'myScenario'
--> Action: 'lint'

--> Scenario: 'myScenario'
--> Action: 'dependency'

--> Scenario: 'myScenario'
--> Action: 'cleanup'

--> Scenario: 'myScenario'
--> Action: 'destroy'
Molecule action Description
--> Test matrix
(when running molecule action)
lint syntax checking (yamllint + flake8 + ansible-lint)
└── myScenario
    └── lint
dependency manage the role's dependencies (allows to pull dependencies from Ansible Galaxy if required)
└── myScenario
    └── dependency
cleanup use the provisioner to cleanup any changes made to external systems during the stages of testing (by default : no action configured, nothing to do)
└── myScenario
    └── cleanup
destroy use the provisioner to destroy the instances, based on destroy.yml
  • when called at the beginning : delete any pre-existing container to start with a fresh one
  • when called at the end : destroy the instances completed during the test and delete the network assigned to those instances
└── myScenario
    ├── dependency
    ├── cleanup
    └── destroy
syntax works in a similar way to the --syntax-check flag in the command ansible-playbook --syntax-check playbook.yml
└── myScenario
    └── syntax
create provision the test node(s), based on create.yml
└── myScenario
    ├── dependency
    ├── create
    └── prepare
prepare Use the provisioner to prepare the instances into a particular starting state by running the "prepare" playbook : myRole/molecule/myScenario/prepare.yml
└── myScenario
    └── prepare
converge actually execute the role inside the test node by executing molecule/myScenario/playbook.yml
└── myScenario
    ├── dependency
    ├── create
    ├── prepare
    └── converge
idempotence make sure no unexpected changes are made in multiple runs
└── myScenario
    └── idempotence
side_effect test advanced stuff like HA failover (by default, nothing to do)
this is launched with the command : molecule side-effect
└── myScenario
    └── side_effect
verify executes the tests you wrote earlier in test_myScenario.py
└── myScenario
    └── verify

For details on any of these actions :

molecule action --help

mail

Molecule

Setup :

This procedure and every further documentation will deliberately ignore everything related to Python 2 (methods, commands, tools, ...).

  1. as root, install some packages :
    apt install -y python3-pip
  2. install Docker CE
  3. as a non-root user, setup+activate a Python virtual environment
    Let's say I have : virtualEnvDir="$HOME/v_molecule"
  4. still as a non-root user, and from within the virtual environment :
    pip3 install -U molecule ansible docker-py

Import existing Ansible roles :

  1. Enter your work environment :
  2. import roles / Ansible data :
    git clone remoteRepository
    Cloning into 'Ansible_Roles'...
    done.
  3. So far, my file tree looks like :
    $HOME/
    |---- v_molecule/
    |	|---- Ansible_Roles/
    |	|	|---- roles/
    |	|	|	|---- haproxy
    |	|	|	|---- logrotate
    |	|	|	|---- mysql
    |	|	|	|---- 
    |	|	|	|---- role_n
    Let's pick a role to start working with :
    cd "$HOME/v_molecule/Ansible_Roles/roles/logrotate"
  4. And let's "Molecule-ize" it :
    molecule init scenario --role-name ${PWD##*/} --verifier-name testinfra --driver-name docker --scenario-name myScenario
    --> Initializing new scenario myScenario...
    Initialized scenario in /home/bob/v_molecule/Ansible_Roles/roles/logrotate/molecule/myScenario successfully.
    This creates the roleName/molecule directory with several files with default configurations. Don't forget to add + commit them.
  5. configure Molecule itself (see also molecule.yml advanced configuration)
    commit file's initial version first
    emacs molecule/myScenario/molecule.yml
    ---
    dependency:
      name: galaxy
    driver:			what we'll use to provision the nodes we'll be testing on (details)
      name: docker
    lint:
      name: yamllint
      options:
        config-file: molecule/myScenario/yamllint.yml
    platforms:
      - name: ubuntu1604
        image: ubuntu:16.04
        privileged: True
    provisioner:
      name: ansible
      lint:
        name: ansible-lint
    scenario:
      name: myScenario
    verifier:		the unit test framework we'll use
      name: testinfra
      lint:
        name: flake8
  6. configure Yamllint (see also yamllint.yml advanced configuration) :
    cat << EOF > molecule/myScenario/yamllint.yml
    
    --- extends: myScenario rules: line-length: max: 120 level: warning truthy: disable
    EOF
  7. commit the changes you made to molecule.yml and yamllint.yml
  8. The time has come to check the whole thing works :
    1. cd "$HOME/v_molecule/Ansible_Roles/roles/logrotate"
    2. molecule destroy
      • If it displays :
        cd -- molecule destroy
        -bash: cd: too many arguments
        make sure you have enabled your Python virtual environment
      • This command should display a pretty verbose output "à la Ansible", showing it's doing things. There's actually nothing to destroy (yet!), but as you may try + retry + try again things, it's possible you forget this important step .
    3. This task may take a while the first time since it requires to download the Docker image :
      TASK [Build an Ansible compatible image (new)] *********************************
      changed: [localhost] => (item=molecule_local/ubuntu:16.04)
  9. If you destroy + create again, you'll see :
    TASK [Build an Ansible compatible image (new)] *********************************
    ok: [localhost] => (item=molecule_local/ubuntu:16.04)
That's it : the basic bricks are there and everything works fine (albeit to do nothing so far ). See you in the next chapter.

Clean everything :

Syntax and coding styles :

Before starting the interesting / fun / hard work (unit tests ) you should run : The lint test is pretty likely to end up NOT HAPPY!!! with a lot to complain about :
Better getting rid of this early.

Anything else :

The playbooks themselves —if not designed with Molecule in mind— can misbehave for many reasons. You'll then have to cycle test-check-retry, where molecule test becomes uncomfortable :
  • creates + deletes test nodes every time : increases duration
  • runs unnecessary tests every time : increases duration
  • verbose output : have to scroll back
To quicken things up, the suggested procedure is :
  1. molecule converge (this implies molecule create)
  2. edit the playbooks
  3. repeat the edit + molecule converge as many times as necessary
  4. molecule destroy

Now let's actually test stuff :

The fun begins now with creating some unit tests :
Based on the previous example, we'll write test cases in /home/bob/v_molecule/Ansible_Roles/roles/logrotate/molecule/myScenario/tests/test_myScenario.py, which becomes :
import os
import pytest

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']
).get_hosts('all')


def test_hosts_file(host):
    f = host.file('/etc/hosts')

    assert f.exists
    assert f.user == 'root'
    assert f.group == 'root'


@pytest.mark.parametrize('pkg', [
  'logrotate',
  'bzip2'
])
def test_pkg(host, pkg):
    package = host.package(pkg)

    assert package.is_installed