Ansible Training – 10. Ansible-Navigator

Docs: 10.1 Ansible-Builder

Mon, 01 Jan 0001 00:00:00 +0000

In this lab, we will use ansible-builder to build our own execution environment.

Task 1

Ensure to have a new version of python installed. ansible-builder is developing quickly and chances are that you cannot run it with a python version that’s not rather new.
Install all packages needed to use ansible-builder on the controller host.

Solution Task 1

Since we have no Red Hat Subscription available, we install ansible-builder with pip. We install podman as well to be able to use containers.

Update python if needed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


$ sudo dnf module install -y python39
$ sudo dnf module enable python39
$ sudo alternatives --config python3

There are 2 programs which provide 'python3'.

 Selection Command
-----------------------------------------------
*+ 1 /usr/bin/python3.6
 2 /usr/bin/python3.9

Enter to keep the current selection[+], or type selection number: 2
$ python3 --version
Python 3.9.13

Install ansible-builder AFTER you ensured the newer python version to be present.

1
2
3


$ sudo dnf install -y podman python3-pip
$ pip3 install ansible-builder --user
...

Task 2

Create a playbook container.yml that installs podman and pulls the image docker.io/bitnami/mariadb on all db servers.
Run this playbook and observe how it fails because the collection containers.podman is not available in the demo EE ansible-navigator-demo-ee.
For the remainder of this lab, we build our own execution environment containing the collection containers.podman.

Solution Task 2

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


$ cat container.yml
---
- name: test ee with containers.podman collection
 hosts: db
 become: true
 tasks:
 - name: Install podman
 ansible.builtin.dnf:
 name: podman
 state: present
 - name: Pull an image
 containers.podman.podman_image:
 name: docker.io/bitnami/mariadb

$ ansible-navigator run container.yml

 WARNING
 ──────────────────────────────────────────────────────────────────────────────────
 Errors were encountered while running the playbook:
 ERROR! couldn't resolve module/action 'containers.podman.podman_image'. This often
 indicates a misspelling, missing collection, or incorrect module path....
 [HINT] To see the full error message try ':stdout'
 [HINT] After it's fixed, try to ':rerun' the playbook
 ──────────────────────────────────────────────────────────────────────────────────
 Ok

Task 3

Create a new execution environment with the name default-ee. You can find information about the needed configuration files in the official documentation of ansible-builder. If you have a Red Hat account, you have access to their ansible-builder guide.

The new EE should:

be based on the latest stable version of the ansible-runner image from https://quay.io
use the ansible.cfg in the techlab folder
contain the pyfiglet python3 module
contain the collection containers.podman and ansible.posix

Solution Task 3

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


$ cat default-ee.yml
version: 1
build_arg_defaults:
 EE_BASE_IMAGE: "quay.io/ansible/ansible-runner:latest"
 ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: "-c"
ansible_config: 'ansible.cfg'
dependencies:
 python: requirements.txt
 galaxy: requirements.yml

$ cat requirements.txt
pyfiglet

$ cat requirements.yml
collections:
 - containers.podman
 - ansible.posix

Task 4

Build the new execution environment with the files from the last task. The resulting image should have a name of default-ee. This will take a few minutes.
With the option for very verbose (-v3) set, you can observe what ansible-builder does in the background.

If you are interested in the details about how the execution environment is built:

Have a look at the newly created folder context/ and see the files created there.
Have a look at your local podman registry and see, the images downloaded for creating your EE.

Tip

If the creation fails due to “no space left on device”, remove unneeded images. For example, you could remove the demo EE ansible-navigator-demo-ee installed by ansible-navigator (podman rmi quay.io/ansible/.ansible-navigator-demo-ee:0.6.0).

Solution Task 4

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47


$ ansible-builder build -f default-ee.yml -v3 -t default-ee
...
Complete! The build context can be found at: /home/ansible/techlab/context
$ podman image list --filter reference=default-ee
localhost/default-ee latest 04a2ff8e9e37 About an hour ago 830 MB

$ tree context/
context/
├── _build
│ ├── ansible.cfg
│ ├── requirements.txt
│ └── requirements.yml
└── Containerfile

1 directory, 4 files

$ cat context/Containerfile
ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:latest
ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest

FROM $EE_BASE_IMAGE as galaxy
ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS=-c
USER root

ADD _build/ansible.cfg ~/.ansible.cfg

ADD _build /build
WORKDIR /build

RUN ansible-galaxy role install -r requirements.yml --roles-path /usr/share/ansible/roles
RUN ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path /usr/share/ansible/collections

FROM $EE_BUILDER_IMAGE as builder

COPY --from=galaxy /usr/share/ansible /usr/share/ansible

ADD _build/requirements.txt requirements.txt
RUN ansible-builder introspect --sanitize --user-pip=requirements.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt
RUN assemble

FROM $EE_BASE_IMAGE
USER root

COPY --from=galaxy /usr/share/ansible /usr/share/ansible

COPY --from=builder /output/ /output/
RUN /output/install-from-bindep && rm -rf /output/wheels

Task 5

Inspect the image of your new EE with ansible-navigator.
Check the included ansible version and verify that the collections containers.podman and ansible.posix are present.

Solution Task 5

1
2
3
4
5
6


$ ansible-navigator images

 NAME TAG EXECUTION ENVIRONMENT CREATED SIZE
0│ansible-navigator-demo-ee 0.6.0 True 8 months ago 1.35 GB
1│ansible-runner latest True 14 hours ago 807 MB
2│default-ee latest True 2 hours ago 830 MB

Choose 2:

1
2
3
4
5
6
7


 DEFAULT-EE:LATEST DESCRIPTION
0│Image information Information collected from image inspection
1│General information OS and python version information
2│Ansible version and collections Information about ansible and ansible collections
3│Python packages Information about python and python packages
4│Operating system packages Information about operating system packages
5│Everything All image information

Choose 2:

1
2
3
4
5
6
7
8
9


DEFAULT-EE:LATEST (INFORMATION ABOUT ANSIBLE AND ANSIBLE COLLECTIONS)
│---
1│ansible:
2│ collections:
3│ details:
4│ ansible.posix: 1.5.1
5│ containers.podman: 1.10.1
6│ version:
7│ details: core 2.12.5.post0

Task 6

Change your configuration file ansible-navigator.yml to use your newly created EE default-ee.
Podman’s default behaviour is to pull any image tagged with latest when starting a container from it. Since we didn’t set up a proper registry at localhost we want to avoid this behaviour by a setting in the config file ansible-navigator.yml.

Tip

The stable version of ansible-navigator doesn’t support the same options as the latest version. Be sure to look into the documentation of the stable version since that is what we have installed. There’s also a documentation of the latest version where you can have a glimpse at the upcoming features.

Run the playbook container.yml and verify that the image podman pull public.ecr.aws/bitnami/mariadb was pulled on the db servers. Provide a cmdline option to run it in stdout mode.
Can you run your previous playbook site.yml with the new EE?

Solution Task 6

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


$ cat ansible-navigator.yml
---
ansible-navigator:
 ansible:
 config:
 help: False
 path: /home/ansible/techlab/ansible.cfg
 cmdline: "--forks 20"
 color:
 enable: True
 execution-environment:
 container-engine: podman
 enabled: True
 image: default-ee:latest #<---
 pull-policy: never #<---

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


$ ansible-navigator run container.yml -m stdout

PLAY [test ee with containers.podman collection] *******************************

TASK [Gathering Facts] *********************************************************
ok: [node2]

TASK [Install podman] **********************************************************
ok: [node2]

TASK [Pull an image] ***********************************************************
ok: [node2]

PLAY RECAP *********************************************************************
node2 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

$ ansible db -b -a "podman images"
node2 | CHANGED | rc=0 >>
REPOSITORY TAG IMAGE ID CREATED SIZE
public.ecr.aws/bitnami/mariadb latest c6cb896c1070 11 hours ago 93.5 MB

Note that if you pulled the image as user root on the db servers, you will not see it in the output of podman images unless it’s run as user root as well.

Because we included the collection ansible.posix in our EE earlier, we can run tasks with the ansible.posix.firewalld module. Therefore, running the playbook site.yml from the ansible-navigator lab would work as well!

1
2


$ ansible-navigator run site.yml
...

All done?

Think about why the container was able to connect to the servers over SSH without providing a password
Have a look at the ansible-builder GitHub page
Have a look at the docs of ansible-navigator to see how the tool will evolve

Docs: 10.2 Ansible-Runner

Mon, 01 Jan 0001 00:00:00 +0000

In this lab, we will learn about ansible-runner, the component of AAP Controller / AWX that actually runs Ansible playbooks.

Task 1

Install all packages needed to use ansible-runner command line tool on the controller host. (In case of python problems, have a look at the ansible-builder lab).
Show the help context of the ansible-runner command.

Solution Task 1

Since we have no Red Hat Subscription available, we install ansible-runner with pip. We install podman as well to be able to use containers.

1
2
3


$ pip3 install ansible-runner --user
...
$ ansible-runner --help

Note that ansible-runner is already present when you installed ansible-navigator in the labs before.

Task 2

Set up the folder structure needed by ansible-runner to find your inventory and put your playbook in the correct folder as well.
Use ansible-runner to run the play site.yml from Lab 10.0 Task 3.

Solution Task 2

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


$ pwd
/home/ansible/techlab
$ tree
.
├── inventory
│ └── hosts
└── project
 └── site.yml

$ ansible-runner run /home/ansible/techlab/ -p site.yml
PLAY [Run tasks on webservers] *************************************************

TASK [Gathering Facts] *********************************************************
ok: [node1]
...
...
...
node1 : ok=5 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Task 3

Add a task to site.yml playbook that waits for 120 seconds. The tasks should be at the end of the play running on the web group web.
Use ansible-runner cli to run the playbook site.yml.
Have a look at the running process in the artifacts directory. Find information about the facts cached for node1.
Find information about a random job event.
Show the standard output of the ansible run`

Solution Task 3

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


$ cat site.yml
...
- name: Sleep for 120 seconds
 ansible.builtin.pause:
 seconds: 120
...
$ ansible-runner start /home/ansible/techlab/ -p site.yml
$ tree
.
└── 82b7743d-72db-4760-b163-e24257a5ff78
 ├── command
 ├── fact_cache
 │ └── node1
 ├── job_events
 │ ├── 10-8cbb439d-831f-44e7-ad68-d067847efcfd.json
 │ ├── 11-62869478-2c8c-4cac-a566-6e09ca1aa295.json
 │ ├── 12-5a420566-9318-68c2-fc18-00000000000c.json
 │ ├── 13-ba4910c9-1f35-4687-851f-073bd70f96a1.json
 │ ├── 14-4cedb233-fc76-4019-a6ba-e829d3cf25f7.json
 │ ├── 1-54889cf9-5ceb-4541-930f-73fe24d4a08a.json
 │ ├── 15-5a420566-9318-68c2-fc18-00000000000d.json
 │ ├── 16-80f5f3f7-b1b5-4259-aa67-49213495fe5e.json
 │ ├── 17-505f561d-cce5-42b6-a219-eb7e596c8ef9.json
 │ ├── 18-5a420566-9318-68c2-fc18-00000000000e.json
 │ ├── 19-3ab80b71-789b-4e65-9cb4-6b7d76de2729.json
 │ ├── 2-5a420566-9318-68c2-fc18-000000000008.json
 │ ├── 3-5a420566-9318-68c2-fc18-000000000015.json
 │ ├── 4-790fc1a4-c0f8-4b3a-99cc-b4f4239d9a1f.json
 │ ├── 5-49005502-bf17-4a88-ae9e-e25c45588bf6.json
 │ ├── 6-5a420566-9318-68c2-fc18-00000000000a.json
 │ ├── 7-2764164b-0fbd-4c13-b7ce-0671b308640e.json
 │ ├── 8-6f454b56-c99d-4996-9de2-c349953f9970.json
 │ └── 9-5a420566-9318-68c2-fc18-00000000000b.json
 ├── stderr
 └── stdout

3 directories, 23 files
$

NOTE: the output of the tree command varies depending on when it is run.

$ cat 82b7743d-72db-4760-b163-e24257a5ff78/fact_cache/node1
{
"_ansible_facts_gathered": true,
"ansible_all_ipv4_addresses": [
"5.102.147.253"
],
"ansible_all_ipv6_addresses": [
"fe80::5842:5ff:fe66:93fd"
],
"ansible_apparmor": {
"status": "disabled"
...

$ cat artifacts/82b7743d-72db-4760-b163-e24257a5ff78/job_events/1-54889cf9-5ceb-4541-930f-73fe24d4a08a.json
...

$ cat artifacts/82b7743d-72db-4760-b163-e24257a5ff78/stdout
PLAY [Run tasks on webservers] *************************************************
TASK [Gathering Facts] *********************************************************
ok: [node1]
TASK [install httpd] ***********************************************************
ok: [node1]
TASK [start and enable httpd] **************************************************
ok: [node1]
TASK [start and enable firewalld] **********************************************
ok: [node1]
TASK [open firewall for http] **************************************************
ok: [node1]
TASK [Sleep for 120 seconds] ***************************************************
ok: [node1]
PLAY [Run tasks on dbservers] **************************************************
TASK [Gathering Facts] *********************************************************
ok: [node2]
TASK [prepare motd] ************************************************************
ok: [node2]
PLAY RECAP *********************************************************************
node1 : ok=6 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Task 4

Use ansible-runner cli to run the playbook site.yml inside an execution environment (ee). If you did the ansible-builder labs, you can use the ee you created then. Otherwise, use quay.io/ansible/ansible-navigator-demo-ee. We want to use podman to execute the ee and also use process isolation. See ansible-runner run --help for which options you have to use.

Since it is run inside a container, we have to specify with which user we want to run Ansible. Default would be the root user which is not a best practice. Also, we add a working ssh_key to the correct location in our directory. This is needed in order to open an ssh-connection from inside the ee to the managed nodes.

Tip

If the ee you choose is not present yet, podman will first pull it when running ansible-runner. This can take some time.

create the needed folder, files and its content for using the ee
create the needed folder, files and its content for using ansible as remote_user and the corresponding ssh_key
run the playbook site.yml inside the ee with the ansible-runner cli

Solution Task 4

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52


$ tree
.
├── env
│ ├── settings
│ └── ssh_key
├── inventory
│ └── hosts
└── project
 └── site.yml

$ cat env/settings
---
container_image: default-ee
process_isolation_executable: podman
process_isolation: true

$ cat /home/ansible/.ssh/id_ed25519 > env/ssh_key

$ head env/ssh_key
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn
NhAAAAAwEAAQAAAgEAvAZuI1j7kz+J9bb375jjGdQqqsGc2imuoeTnFOqwLdg6+1LPj9RC
NDE7JpDoTmTRTuEqSyD/CmGawv4tLrOig4Q/sFeV1JsEt9V3fF1s9VJ7VcYq2baSLyrHFQ
...

$ cat project/site.yml
---
- name: Run tasks on webservers
 hosts: web
 become: true
 remote_user: ansible #<-- 
...
- name: Run tasks on dbservers
 hosts: db
 become: true
 remote_user: ansible #<--
...

$ ansible-runner run /home/ansible/techlab/ -p site.yml
Identity added: /runner/artifacts/6fdb3c4d-ee40-4a40-8d6b-57b9a3788d7e/ssh_key_data (Created for convenience ahead of techlab)

PLAY [Run tasks on webservers] *************************************************

TASK [Gathering Facts] *********************************************************
ok: [node1]

...

PLAY RECAP *********************************************************************
node1 : ok=5 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$

Warning

Not defining process_isolation_executable: podman in env/settings would lead to ansible-runner running our playbook NOT inside an ee. The output however looks almost the same.

Task 5

Now we want to have a look at what’s happening in the background while running our playbook with ansible-runner inside an ee.

Start your playbook in the background. Use ansible-runner --help to see which option you can use.
While the playbook runs, have a look at the running processes with watch -n1 'ps -ef | grep ansible- | grep -v grep'
Also while the playbook runs, have a look at the running containers with watch -n1 podman ps -a
While the playbook still runs, stop it with corresponding option and see how the processes are terminated and the running container stopped.

Solution Task 5

1
2
3
4


$ ansible-runner start /home/ansible/techlab/ -p site.yml
...
$ ansible-runner stop /home/ansible/techlab/ -p site.yml
...

Task 6

Now we want to run our playbook site.yml by starting an execution environment with podman and mounting our metadata folder into the correct location inside the ee. Have a look at the documentation for help.

Tip

If you have selinux in enforcing mode, remember to relabel the volumes mounted inside the container with :Z.

Warning

Remove your podman-settings in env/settings. Otherwise, ansible-runner would try to run Ansible inside the container with podman again. This would fail since it’s not installed inside the ansible-runner image.

Use the ee default-ee from before or use quay.io/ansible/ansible-navigator-demo-ee. Remember, that an ee is always based on the ansible-runner reference image.
How do you specify which playbook to run?
Into which directory inside the container do you have to mount your metadata directory?
Remove the podman-settings in the file env/settings. These settings would cause ansible-runner inside the container trying to run in podman again. This would fail because podman is not installed inside the ee.
Run your Ansible project with podman using the ee stated above.

Solution Task 6

With the env variable RUNNER_PLAYBOOK=test.yml.
Into the /runner directory.
$ mv env/settings ../
Run it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


$ podman run --rm -e RUNNER_PLAYBOOK=site.yml -v /home/ansible/techlab:/runner:Z default-ee:latest
Identity added: /runner/artifacts/cf33c64a-c5cf-41dd-8479-e9c0057d8e8f/ssh_key_data
(Created for convenience ahead of techlab)

PLAY [Run tasks on webservers] *************************************************

TASK [Gathering Facts] *********************************************************
ok: [node1]

TASK [install httpd] ***********************************************************
ok: [node1]

TASK [start and enable httpd] **************************************************
ok: [node1]

TASK [start and enable firewalld] **********************************************
ok: [node1]

PLAY [Run tasks on dbservers] **************************************************

TASK [Gathering Facts] *********************************************************
ok: [node2]

TASK [prepare motd] ************************************************************
ok: [node2]

PLAY RECAP *********************************************************************
node1 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

$

Task 7 (Advanced)

Why can’t you run your site.yml playbook with the quay.io/ansible/ansible-runner image?

Solution Task 7

Our playbook contains a tasks that uses the ansible.posix.firewalld module. The ansible-runner ee contains nothing but ansible-core. This means it cannot run the ansible.posix.firewalld module because it’s not present in ansible-core.

You can show the content of the ansible-builder ee with ansible-navigator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


ANSIBLE-RUNNER:LATEST (INFORMATION ABOUT ANSIBLE AND ANSIBLE COLLECTIONS)
0│---
1│ansible:
2│ collections:
3│ details: {}
4│ errors:
5│ - |-
6│ ERROR! - None of the provided paths were usable. Please specify a valid
7│ version:
8│ details: core 2.12.5.post0

See the ansible-navigator lab about how to get there!

Task 8

Now we want to run our playbook directly by using the python module ansible-runner.

Install the python module ansible-runner.
Create a python script run_ansible_run.py that runs your playbook site.yml with ansible-runner.
Ansible-runner should not use an ee yet.
The script should use /home/ansible/techlab/ as the ansible metadata directory.
Make the script executable.
Run the script.
After the Ansible run completed successfully, change things to run it inside an ee.

Solution Task 8

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47


$ pip3 install ansible-runner --user

$ cat run_ansible_run.py
#!/usr/bin/python3
import ansible_runner
ansible_runner.run(
 private_data_dir='/home/ansible/techlab/',
 playbook='site.yml')

$ chmod +x run_ansible_run.py

$ ./run_ansible_run.py
Identity added: /home/ansible/techlab/artifacts/5e703775-5234-491c-b958-09bf0bd2e756/ssh_key_data
(Created for convenience ahead of techlab)

PLAY [Run tasks on webservers] *************************************************

TASK [Gathering Facts] *********************************************************
ok: [node1]

TASK [install httpd] ***********************************************************
ok: [node1]

TASK [start and enable httpd] **************************************************
ok: [node1]

TASK [start and enable firewalld] **********************************************
ok: [node1]

TASK [open firewall for http] **************************************************
ok: [node1]

TASK [Sleep for 120 seconds] ***************************************************
ok: [node1]

PLAY [Run tasks on dbservers] **************************************************

TASK [Gathering Facts] *********************************************************
ok: [node2]

TASK [prepare motd] ************************************************************
ok: [node2]

PLAY RECAP *********************************************************************
node1 : ok=6 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
node2 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
$

For running ansible-runner from the python script inside an ee, just ensure that the settings file is present in the env folder.

1
2
3
4
5


$ cat env/settings
---
container_image: default-ee
process_isolation_executable: podman
process_isolation: true

Now, when you start the run_ansible_run.py script again, Ansible is run inside the defined ee. You can check this with $ watch -n1 podman ps -a while running the script.

All done?

Have a look at the ansible-runner GitHub page
Have a look at the docs of ansible-navigator
Have a look at the documentation of the python module and see what the library is capable of.