Ansible - Simple IT Automation

ANSIBLE_CALLBACK_WHITELIST

• Used to whitelist callback plugins
• So far, mostly used to display statistics (tasks duration) at the end of a playbook execution :
  ANSIBLE_CALLBACK_WHITELIST='profile_tasks'

ANSIBLE_DEBUG

without surprise, this displays debug information :

• this is extremely verbose
• does not display the very same set of information than -v

ANSIBLE_LOCALHOST_WARNING

Silence warnings that would be output by code like :

---
# ANSIBLE_LOCALHOST_WARNING=false ansible-playbook test.yml
- hosts: 127.0.0.1
  connection: local
  gather_facts: no
  tasks:

  - debug:
      msg: "hello world!"

ANSIBLE_STDOUT_CALLBACK

Specifies the callback used to display the Ansible execution log :

ANSIBLE_STDOUT_CALLBACK=dense

minimal stdout output

ANSIBLE_STDOUT_CALLBACK=yaml

• less "", '', [] and {} in the output
• can also turn \n in the output into actual \n, making error messages way more readable (source, details)

1. define the vault_password_file in ansible.cfg :

   [defaults]
   
   vault_password_file	= path/to/.vault_password

2. set the vault password in path/to/.vault_password :
   pwgen -N 1 -ys 40 > path/to/.vault_password
   chmod 400 path/to/.vault_password
3. create the encrypted file :
   ansible-vault create path/to/encrypted/file
   If you skipped the steps 1 and 2, this will prompt for the vault password.
4. view the encrypted file :
   ansible-vault view path/to/encrypted/file
5. edit the encrypted file :
   ansible-vault edit path/to/encrypted/file
   You can define which editor will be used by setting the EDITOR environment variable.

For your eyes only :

In a play :

1. ANSIBLE_CONFIG : environment variable
2. ansible.cfg : in the current directory (so this may be versioned and shared with the code)
3. ~/.ansible.cfg : personal user settings
4. /etc/ansible/ansible.cfg : system-wide settings

• Ansible uses the first it finds and ignores the others.
• To explicitly specify which configuration file to use :
  ANSIBLE_CONFIG=path/to/ansible.cfg ansible
• To find out which configuration file was actually used by Ansible, just run any basic command in verbose mode :
  ansible localhost -m ping -v

  Using path/to/ansible.cfg as config file
  

[defaults]
# paths
inventory		= path/to/inventoryFile
log_path		= ./ansible.log
lookup_plugins	= path/to/lookup_plugins
#remote_tmp		= /tmp/${USER}/ansible		deprecated ?, details
roles_path		= path/to/roles
vault_password_file	= /var/lib/ansible/.vault_password

# ansible internal stuff
deprecation_warnings		= True
forks				= 20		details
gathering			= smart
host_key_checking		= False
internal_poll_interval	= 0.05
interpreter_python		= auto_silent
retry_files_enabled		= False

[ssh_connection]
pipelining	= True		conflicts with privilege escalation (become + sudo). For details, please 
retries	= 5
ssh_args	= -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=password,publickey

asynchronous

By default, the SSH connection to a slave stays open until the task completes, which is fine for short tasks, but causes problems if task duration > SSH timeout

Use case :

• start a long running task
• check its status later (by polling regularly)

asynchronous tasks are run "detached" from the SSH connection, but still "blocking" the play.

background

Some tasks can be run concurrently and shorten the overall playbook duration. In other words : this allows not blocking on a task to continue on others.

the play continues while a background task is being executed.

async n

set task timeout

• if task is not finished after n seconds, it will be :
  • reported as FAILED
  • terminated
• when a task is killed by timeout, its register directive is not executed, and anything based on the registered value will fail because the expected variable does not exist

poll n

check task status every n seconds

• it's ok if n > actual task duration
• the task behavior changes when n = 0 (see below)

async + poll n (n > 0) = avoid connection timeout (source)

async + poll n (n = 0) = background task (source)

• when the result of a background task is required to continue
• when you want to ensure a background task is over

• since I don't know which of the background or foreground tasks ends first, how to make sure that —at a given point of a play— the background task is finished ?
• how to check the background task exit status ?
• "regular" tasks output is visible in the Ansible execution log, what do I get here ?
• how to retrieve the background task output (stdout, stderr) ?

---
- hosts: 127.0.0.1
  connection: local
  gather_facts: no
  tasks:

  - name: Init. things
    shell: "[ -f myTempFile ] && rm myTempFile || :"

  - name: Job A
    shell: for i in 1 2 3 4 5_end; do sleep 1; echo "job A\t$i\t$(date)" >> myTempFile; done; echo 'job A OK'
    async: 20
    poll: 0
    register: jobA

  - name: Job B
    shell: for i in 1 2 3_end; do sleep 1; echo "job B\t$i\t$(date)" >> myTempFile; done; echo 'job B OK'

  - name: Wait for 'Job A' to end
    async_status:
      jid: '{{ jobA.ansible_job_id }}'
    register: jobAStatus
    until: jobAStatus.finished
    retries: 10		number of attempts
    delay: 1		seconds between attempts

  - debug:		when reaching this point, the task named "Job A" is finished
      var: jobA

  - debug:
      var: jobAStatus

  - debug:
      msg: "{{lookup('file', jobA.results_file) }}"

ANSIBLE_LOCALHOST_WARNING=false ansible-playbook myPlaybook.yml

PLAY [127.0.0.1] ************************************************************************************

TASK [Init. things] *********************************************************************************
changed: [127.0.0.1]

TASK [Job A] ****************************************************************************************
changed: [127.0.0.1]

TASK [Job B] ****************************************************************************************
changed: [127.0.0.1]

TASK [Wait for 'Job A' to end] **********************************************************************
FAILED - RETRYING: Wait for 'Job A' to end (10 retries left).
FAILED - RETRYING: Wait for 'Job A' to end (9 retries left).
changed: [127.0.0.1]

TASK [debug] ****************************************************************************************
ok: [127.0.0.1] => {
    "jobA": {			the variable I registered to use async_status
        "ansible_job_id": "648023500521.29516",
        "changed": true,
        "failed": false,	the background task exit status
        "finished": 0,		I expected this to be 1 ()
        "results_file": "/home/bob/.ansible_async/648023500521.29516",
        "started": 1
    }
}

TASK [debug] ****************************************************************************************
ok: [127.0.0.1] => {
    "jobAStatus": {		the variable I registered to use until with async_status
        "ansible_job_id": "648023500521.29516",
        "attempts": 3,
        "changed": true,
        "cmd": "for i in 1 2 3 4 5_end; do sleep 1; echo \"job A\\t$i\\t$(date)\" >> myTempFile; done; echo 'job A OK'",
        "delta": "0:00:05.016975",
        "end": "2020-05-27 13:41:46.752286",
        "failed": false,
        "finished": 1,
        "rc": 0,
        "start": "2020-05-27 13:41:41.735311",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "job A OK",
        "stdout_lines": [
            "job A OK"
        ]
    }
}

TASK [debug] ****************************************************************************************
ok: [127.0.0.1] => {
    "msg": {			contents of the job's result_file
        "changed": true,
        "cmd": "for i in 1 2 3 4 5_end; do sleep 1; echo \"job A\\t$i\\t$(date)\" >> myTempFile; done; echo 'job A OK'",
        "delta": "0:00:05.016975",
        "end": "2020-05-27 13:41:46.752286",
        "invocation": {
            "module_args": {
                "_raw_params": "for i in 1 2 3 4 5_end; do sleep 1; echo \"job A\\t$i\\t$(date)\" >> myTempFile; done; echo 'job A OK'",
                "_uses_shell": true,
                "argv": null,
                "chdir": null,
                "creates": null,
                "executable": null,
                "removes": null,
                "stdin": null,
                "stdin_add_newline": true,
                "strip_empty_ends": true,
                "warn": true
            }
        },
        "rc": 0,
        "start": "2020-05-27 13:41:41.735311",
        "stderr": "",
        "stdout": "job A OK"
    }
}

PLAY RECAP ******************************************************************************************
127.0.0.1                  : ok=7    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

a regular file

variables for all groups will be defined here.

This is fine unless this file gets really long, complex and barely readable.

a directory (example)

when there are many groups / variables / both, it becomes easier to have a structure like :

[root] playbook root directory

group_vars (directory)

group1.yml variables for the members of the group1 group

group2.yml variables for the members of the group2 group

i.e. : group_vars is a directory with :

• 1 file per hostgroup
• each file has variables for the corresponding group
• each file is named after the group it applies to

- hosts: all
  roles:
    - roleA
    - roleB
    - roleC


- hosts: sql
  roles:
    - roleD
    - roleE
  tags:
    - sqlOnly

It is possible to play roles roleD and roleE on members of the sql host group with :

ansible-playbook -i myInventoryFile --diff --check -t sqlOnly myPlaybook.yml

To specify several tags :

ansible-playbook -t 'tag1,tag2'

always

will always run a task, unless explicitly skipped with --skip-tags always

never

will prevent a task from running unless

• you specifically request to play such tasks with --tags never
• an associated tag is explicitly requested (i.e. never must be associated with another tag)

  - name: Debug
    ansible.builtin.debug:
     msg: '{{ myVariable }}'
    tags: [ never, debug ]

  This task runs when you specifically request the never or debug tag (i.e. --tags never or --tags debug)

tagged

will run tasks that have at least 1 tag

untagged

will run tasks that have no tag

all

will run all tasks

By default, Ansible runs as if --tags all had been specified.

• a play
• or to statically imported tasks and roles (i.e. when using an import_... directive)

--skip-tags

Not only do tags allow to run specific parts of a playbook, but they also allow skipping parts :

• to skip a single tag :
  ansible-playbook myPlaybook.yml [options] --skip-tags tagToSkip
• to skip several tags :
  ansible-playbook myPlaybook.yml [options] --skip-tags 'tag1,tag2'

--start-at-task

execute subparts of a playbook (without relying on tags)

• kevin is sitting at ansibleMaster, where Ansible is installed
• kevin wants to perform some actions, with Ansible, on ansibleSlave

• connects to ansibleSlave as kevin via SSH to do [some work]
• requires kevin to be able to "ssh kevin@ansibleSlave"
• works the same with or without SSH keys

Host ansibleSlave
	User stuart
	IdentityFile ~/.ssh/id_rsa

• Then :
  kevin@ansibleMaster$ ansible-playbook [some work] ansibleSlave
  will do [some work] on ansibleSlave as stuart, still via SSH, and using key authentication (so no password required).
• Otherwise (no key authentication configured), Ansible would need to be instructed to prompt for stuart's password on ansibleSlave with -k.

Commands that require escalated privileges need not specifying sudo : this would be redundant with -b :

ansible all -i ansibleSlave, -bK -m command -a 'whoami'

ansibleSlave | SUCCESS | rc=0 >>
root

Without -b, with sudo :

ansible all -i ansibleSlave, -K -m command -a 'sudo whoami'

ansibleSlave | FAILED | rc=1 >>
sudo: no tty present and no askpass program specified

TASK [setup] *******************************************************************
fatal: [ansibleSlave]: UNREACHABLE! => {"changed": false, "msg": "Authentication failure.", "unreachable": true}

Make sure your local SSH configuration (~/.ssh/config) doesn't interfere

Make sure the remote SSH configuration (/etc/ssh/sshd_config) is still appropriate

Make sure the SSH connection is opened the way you mean it to be

<10.27.25.1> ESTABLISH SSH CONNECTION FOR USER: root

- hosts: all
  remote_user: root		GOTCHA!!!
  roles: