I use Ansible to automate just about everything I can in my homelab. I have roles ranging from installing a custom root certificate to configuring a complete Bareos backup system, and everything in between, and the rapid expansion of my playbooks has presented some challenges.
One big issue is that I’ve always just developed roles directly against a target system, iterating on the role until the target system is exactly how I want it and then committing what I have. The next time I run the role, half the time it doesn’t work. Ansible’s idempotence means that as I grow a role, what I’ve already done isn’t being tested again in a now more complex environment, so when it’s run against a clean system, it doesn’t behave the same. This leads to small errors and a lack of idempotence but also means I can easily miss really obvious things, like forgetting to install the necessary dependencies on a clean system. Lastly, roles grow stale over time, and eventually, I end up installing outdated versions of software or failing to install dependencies because the old downloads have been removed.
I’m hoping that Molecule can solve some of these problems. Molecule creates customizable “execution environments” to test your roles in, either using a container engine or full-fat VMs on a local hypervisor or in the cloud. It’s pretty easy to get started with, and makes developing roles a lot easier and faster.
Getting Started with Molecule
Molecule can be installed using
pip, by running
pip install molecule molecule-plugins. The
molecule-plugins package isn’t strictly necessary, by default you can use the
delegated driver and write your own playbook to deploy infrastructure, but the default plugins package add support for Docker, Podman, and others.
Once Molecule is installed, it’s easy to create a new Molecule-enabled role or add a scenario to an existing role. I’m mostly adding Molecule to existing roles for now, so I used the command
molecule init scenario -r time -d containers to add a Molecule scenario for my
time role, which handles setting system timezones across multiple platforms. The command created a new
molecule folder within my role, shown below.
│ └── default
│ ├── converge.yml
│ ├── molecule.yml
│ ├── prepare.yml
│ └── verify.yml
Actually, I added the
prepare.yml file later, but the other three files were created by Molecule. The
molecule.yml file configures the Molecule stages, which include the
verify stages, which are all simply Ansible playbooks. The
molecule.yml contains the following sections:
driver, which is
containers. I use Podman to test my roles in a rootless container, but using the
containers driver instead of
podman allows Molecule to use either Docker or Podman, depending on which one is available (I believe it has a preference for Docker if both are installed). I’ve also had to edit the default
molecule.yml slightly to be able to test this specific role.
First, most of my environment uses Ubuntu Server, so roles are primarily tested on a
docker-ubuntu2204-ansible container, which adds Ansible’s dependencies onto a base Ubuntu container image. Second, I added a
group_vars section to set a variable which would usually be set in the file
group_vars/all.yml. Molecule doesn’t inherit host or group variables, since it’s an isolated execution environment which is only aware of itself and variables from imported roles.
Adding Preparation Steps
For most of my roles, the base
docker-ubuntu2204-ansible container just isn’t enough. Although it packs more than a base Ubuntu container, it’s missing most of the packages needed by my roles. Heck, even the very simple
time role doesn’t work because the
tzdata package isn’t installed. To fix this, I created a
prepare.yml Molecule playbook. This playbook is executed immediately after the container is created, when running the
molecule create command. I could also create a custom
Dockerfile, which might be a bit faster since the
prepare.yml playbook has to be run every time a container is spun up, but in my experience Molecule tried to rebuild the container every time anyway so it didn’t save any time. If you wanted to use a custom container image, Molecule supports a
Dockerfile.j2 can be a Jinja2 template, which can read from an
item variable which represents the specific instance from the
However, I preferred the cleanliness of using one container image rather than customizing it for every role. Using a
prepare.yml playbook also lets you use Ansible modules instead of shell commands:
Converging and Verifying Roles
Most of Molecule’s power comes from the
verify.yml files. The
converge.yml file is a normal playbook, which by default imports the role to be tested, although you can add any other Ansible tasks here as well:
converge.yml playbook completes, the role can be tested, by performing assertions on the Molecule instance. Again, the
verify.yml file is a simple Ansible playbook, which uses the
assert module to test the state of the converged system:
And that’s all that’s needed to set up Molecule testing for a role!
With Molecule fully configured, the role can be tested just by running the
molecule test command. The command
prepares the container, then runs the Ansible role against it during the
converge stage, and lastly runs the
verify playbook to see if the system reached the desired state. Lastly, it destroys the container and exits. This is particularly useful in a CI environment, since it performs every step with just one command, but Molecule is equally powerful for developing and iterating on roles.
molecule create you can spin up and prepare the container, and leave it running to continuously test against. You can run
molecule converge and
molecule idempotence (did I mention Molecule can run your playbook multiple times to test for idempotence?) to test the role as many times as needed, and use
molecule login to spawn a shell in the container to troubleshoot issues with. Lastly, you can write assertions in
verify.yml and run these with
molecule verify, and then run
molecule destroy to tear down the infrastructure (or to start fresh with a new container).
Testing Complex Roles
I’ve only just gotten started developing and testing with Molecule, but using containers, while fast, imposes some limitations on what you can test with the barebones configuration. For example, by default containers do not run an init system such as
systemd, so testing roles which install a
systemd unit won’t work, and that tends to be most of them.
Luckily, most of these restrictions can be worked around. For example, to start the container with
systemd, the container needs to be privileged and have
cgroup access, which can be accomplished using a volume mount. Then,
systemd can be started as PID 1 by calling the
I assume this could be done similarly with device nodes like
/dev/net/tun, which is a bridge I’ll have to cross eventually for another role. That being said, I haven’t found myself spending too much time on Molecule instead of writing the roles themselves. Out of the box, it works how you need it to, and it uses common tools like Docker and Podman for which there is extensive documentation when you run into a limitation you didn’t expect.