Ansible modules - Don't reinvent the wheel : use a module!

mail

fail

mail

Be very careful with assert

For those unfamiliar with assert :

Even though we're dealing with assert in the context of Ansible, the overall concept is language-independent and is widely implemented. The bug/limitation/behavior we're discussing here _may be_ specific to Ansible.

How it works :
  1. I make a series of assertions
  2. assert checks these
  3. ...and yells at the wrong ones
Examples :
Technically, assert is close to an if then block, but it has some differences :
  • assert causes an error when an assertion is wrong, whereas if then can do this only if you trigger the error yourself explicitly in the then clause
  • assert generally raises an exception, causing the program to stop or even crash. This way, you can be sure a program continues only if all assertions pass.
  • listing assertions to be evaluated by assert
    • makes the code more readable
    • and documents the expected conditions for the programs to run normally
    more clearly than a series of if then

OK, so what's wrong with assert ?

I just found this at the beginning of a playbook —used in production for months / years : 
  

  - name: Check variable is specified
    assert:
      that:
      - "{{ requiredVariable }} is undefined"
At first sight, the purpose of this snippet looks obvious : stop the playbook if the required variable is missing. But looking more thoroughly, you'll notice : 
      - "{{ requiredVariable }} is undefined"
which is surprising, for 2 reasons :
  1. for this playbook to work right, the requiredVariable variable is mandatory, but this code ensures it's undefined
  2. the assertion requiredVariable is undefined should fail if the playbook is run while requiredVariable is actually defined (which is how it's always been run on our side)
So is going on ?

Explanations :

The following is based on the code of assert.yml, feel free to experiment on your side .
The code says : 
  - set_fact:
      myVariable: 'hello'

  - assert:
      that:
      - myVariable is defined
      - "{{ myVariable }} is undefined"
And both assertions pass with no error.

1st assertion : myVariable is defined

This is just checking a variable named myVariable is defined —which is true— so it passes.

2nd assertion : "{{ myVariable }} is undefined"

There are quotes and curly braces, and it seems it's doing the contrary but... it passes too ! Here is what happens :
  1. because of curly braces, the value of myVariable replaces {{ myVariable }}, which gives : "hello is undefined"
  2. "hello is undefined" is just a string, and is seen by assert as : hello is undefined
  3. hello is undefined is an assertion where we state the variable named hello is undefined. This is true, so it passes.

3rd assertion : undefinedVariable is undefined

Checking the undefinedVariable variable we defined nowhere is undefined : true.

4th assertion : "{{ undefinedVariable }} is undefined"

Quotes and curly braces again : the interpreter tries to replace {{ undefinedVariable }} with the value of undefinedVariable. There is no such value, hence the error :
The task includes an option with an undefined variable. The error was: 'undefinedVariable' is undefined

Conclusion

  • This subtlety can be the cause of hard-to-find bugs.
  • Using assert may lead to a false feeling of safety : the code I found at the beginning of our old playbook :
    • actually checks nothing (sh*t may happen someday...)
    • it doesn't even fail : this code WORKS. It just does not check what it was intended for
  • Failing to fail doesn't mean you succeeded !
mail

Links and resources

mail

shell and command

Their differences :

  • shell is executed by /bin/sh on the remote node
  • command is NOT executed in a shell on the remote node, so shell-specific variables (like $HOME) and commands (>, |, &, ...) are NOT supported.

Subtleties they share :

Since both behave mostly the same way and accept the same hacks, I won't be repeating "shell or command" hereafter .

changed is always True

This is because Ansible has no mechanism for understanding whether the command run by shell actually changed anything (source).
To workaround this, you can instruct Ansible what to consider as a change with changed_when : 
- name: check whether ...
  shell: someCommand
  register: myVariable
  changed_when: false			never report this task as changed
  ignore_errors: true
- shell: someCommand
  register: myVariable
  changed_when: "myVariable.rc != 2"	change based on return code
If you're running a shell command to decide whether or not to run a subsequent command (i.e. to guarantee idempotence) you _may_ have to use ignore_errors: true. This is required in such case because this shell command will return a success / failure return code. However, since hosts with failed tasks are removed from the targets of the playbook, we must instruct Ansible to ignore such errors and continue working on them.

shell warning : [WARNING]: Consider using yum module rather than running yum (source)

Ansible may warn when shell is used to perform an action that ought to be done via one of the numerous Ansible modules. This is right most of the time : using built-in modules is cleaner and is the best solution to achieve idempotence. But the warning is not always appropriate, since the suggested solution doesn't work (or doesn't exist (yet)). Regarding this yum example, the warning :
  • is perfectly legitimate when trying to install / remove / updates packages : use yum instead of shell
  • falls flat if we're trying to run a command that is not (yet) supported by the yum module, such as listing repositories : 
    
shell: yum repolist enabled | grep "{{ redhat_repository_optional }}"
In the latter situation, there is no alternative to using shell, but we'd like to hide this warning anyway. To do so :
  • Ansible actually sends warnings based on a list of keywords following shell:, so let's fool it with which and the $() construct : 
    shell: $(which yum) repolist enabled | grep "{{ redhat_repository_optional }}"
  • other solution : 
    shell: yum repolist enabled | grep "{{ redhat_repository_optional }}" warn=no
    Does the job but poor readability
  • Better : 
    shell: yum repolist enabled | grep "{{ redhat_repository_optional }}"
args:
  warn: no
mail

lineinfile

Regexp-search + replace line only when the regexp matches (source)

  • use backrefs: yes 
    - name: do something
  lineinfile:
    dest: /path/to/file
    regexp: '^what we are looking for$'
    line: 'the new line that will replace the whole line matched by the regexp above'
    backrefs: yes
  • OR update the regexp so that it matches both the original AND changed lines

unsupported parameter for module: path

Before Ansible 2.3, path was dest, which explains those frequent error messages. This is in the manual, actually, but VERY easy to miss if going too fast .