Tag: Ansible

Categories
Automation Security

Automating SSH keys with Ansible

I was working with a co-worker to get his SSH key distributed to a large number of systems so we could start managing them with Ansible. For security and logging purposes, we login to the systems with our personal “elevated” account (different from our workstation account), and the elevated accounts are permitted to run “sudo” so their Ansible playbooks can make changes to the system.

Today we were fighting with getting the SSH key generated and distributed to all the systems. I’ve been working in Unix and using SSH for over 20 years, but today we kept making simple typos that kept the keys from working for us. Some of the issues were human error and typos on the command line, others were more subtle involving permissions issues on the files. To help remove this human error, I wrote a two-step playbook to generate an ssh keypair (if they don’t exist), then login to a set of systems and setup that key to permit password-less logins.

Here’s the playbook – named “copykeys.yml” – built to automate the distribution of SSH keys across your environment:

 #!/usr/bin/env ansible-playbook
 ---
 - name: "Create and upload SSH keys"
   hosts: all
   gather_facts: false

   tasks:
   - name: "Ensure ssh key exists"
     openssh_keypair:
       path: "{{ lookup('env','HOME') + '/.ssh/id_ed25519' }}"
       type: ed25519
     delegate_to: localhost
     run_once: true

   - name: "Copy user ssh keys"
     authorized_key:
       user: "{{ lookup('env', 'USER') }}"
       state: present
       key: "{{ lookup('file', lookup('env','HOME') + '/.ssh/id_ed25519.pub') }}"

This playbook first creates an “ed25519” SSH key if it does not exist on the machine the script is run on (the “delegate_to” argument), and once the SSH key is created the public key is distributed to each of the systems the playbook runs against.

To use this, simply call the playbook with either an inventory file (‘-i inventory.ini’) or with a command-line inventory list (‘-i server1,server2,server3’). Here is the command to run:

ansible-playbook copykeys.yml --ask-pass -i server1,server2,server3

Assuming your username and password are accepted by all the servers in the inventory, the public portion of your SSH key will be installed on each of the machines.

You can then test this by performing an Ansible “ping” to validate communication works without the “–ask-pass” option:

ansible all -m ping -i server1,server2,server3

The nice thing about Ansible is that it is easy to debug if you break down the communications and realize that all communications between nodes is using basic SSH. If you’re having communication problems, a simple “ssh servername” from the server you’re running Ansible from will often show if the error is communications based.

If this does not work, review the output that Ansible provides (you might have to add “-vvv” to increase the verbosity level for debugging). In nearly all cases the errors are common SSH error messages, not Ansible error messages. Any modern Unix administrator has probably encountered these errors before and is aware of what the underlying issue is that is breaking the SSH connectivity.

Categories
Automation Programming

Ansible and third-party Python modules

A co-worker wanted a third-party Python modules installed onto the Ansible Tower servers that I maintain.  I don’t like installing any and all packages that people ask for since this is a shared system, but I had to bring myself up to speed on how he could get his module installed to use by his playbooks.

  • The machine that the Ansible playbook is executed from only needs to have Ansible and a set of pre-requisite Python modules installed on it.  For this document, we’ll refer to that system as the “Ansible Server” or just “the server”.
  • The machine(s) that the Ansible playbook make work on and perform changes to need ssh and a small sub-set of Python modules (usually the core Python packages).  We’ll refer to those systems as “the clients”.

To make things more confusing there are two types of “modules” that will be referenced:

  • An “Ansible module” is a package that Ansible uses on the server to execute steps on the clients.  These are usually written in Python and the “core” Ansible modules are included and maintained by the Ansible developers.
    • There are some Ansible modules that may be included with Ansible but they are maintained by the community.  These are usually specialized modules specific to a hardware or software vendor and are usually maintained by the vendor or others interested in automating that vendors tools.
  • The other “module” referenced in this document are add-ons to Python and are called “Python modules”.
    • A Python module may perform some low-level task (e.g. network connection, DNS lookup, etc) and are NOT Ansible specific.

Documentation for the Ansible modules are located here: https://docs.ansible.com/ansible/latest/modules/modules_by_category.html

The request mentioned the need for the “GitHub3.py” Python module so the “github_release” Ansible module would work.  The documentation for the “github_release” module has a requirements section and it notes that also.  The documentation page also notes that this is a preview module (“This module is not guaranteed to have a backwards compatible interface.”) and it is also maintained outside of the Ansible core develoers (“This module is maintained by the Ansible Community.”).

So, how do we add this module?  I’m glad you asked!

The first thing to understand is that all the requirements for this module have to be installed on the clients, not on the Ansible servers.  While this sounds like more work, it really isn’t and it keeps the Ansible servers free from conflicts that different users might have requiring different Python module versions.  The key to all this is the use of Python “Virtual Environments” (or “venvs”).  These virtual environments are walled-off areas that have their own Python executable and associated modules; it’s even possible to have different versions Python installed in different venvs for testing.

In the playbook that needs to use an Ansible module that has special Python module dependencies, there are a few steps to take that we’ll go over in detail below:

  1. Ensure pip is installed
  2. Install Python virtual environment packages into the venv
  3. Setup the virtual environment
  4. Install base Tower packages into venv
  5. Install the Python module specifically needed into venv
  6. Use the new venv to execute the Ansible module

Step 1 – Ensure pip is installed

This is a basic step and will vary by OS but the “pip” package is needed for the “pip:” modules later.

  - name: "Ensure pip is installed"
    yum:
      name: python2-pip
      state: installed

Step 2 – Install Python virtual environment packages

This is also OS dependent, but it installs the “python-virtualenv” package so Python can build venvs.


  - name: "Install Python virtual environment packages"
    yum:
      name:
        - python-virtualenv
      state: installed

Step 3 – Setup the virtual environment

This step does the initial work to build the virtual environment.  The venv is just a directory structure (in this case “/tmp/test01.venv”) that contains helper files and some wrapper scripts and configuration defaults.

  - name: "Setup the initial virtual environment"
    pip:
      name:
        - setuptools
      extra_args: --upgrade
      virtualenv: "/tmp/test01.venv"

Step 4 – Install base Tower packages into venv

Strictly speaking, if you’re not using this venv with Ansible Tower it is not necessary, but it will make the playbook usable in more places.


  - name: "Install base Tower packages"
    pip:
      name:
        - python-memcached
        - psutil
      umask: "0022"
      state: present
      virtualenv: "/tmp/test01.venv"
      virtualenv_site_packages: yes
      extra_args: --ignore-installed

Step 5 – Install the Python module specifically needed into venv

Finally we’re at the step where we’re installing the Python module we need.  This Python module (like the others earlier) are only installed into the venv directory structure.

  - name: "Install github3.py module"
    pip:
      name:
        - github3.py
      virtualenv: "/tmp/test01.venv"

Step 6 – Put the new venv with the Python module to work

The key at this step is the “vars:” section that tells the Ansible execution environment to use the “python” binary found in the “venv” on the remote system, “/tmp/test01.venv/bin/python” in this case.

  - name: "Download latest relase of ISOwithKS"
    # https://github.com/dglinder/ISOwithKS.git
    vars:
      ansible_python_interpreter: "/tmp/test01.venv/bin/python"
    github_release:
      user: "dglinder"
      repo: "ISOwithKS"
      action: latest_release

PLEASE NOTE: The “github_release:” example above does NOT work due to something un-related to the venv created.

How does this work?

When the playbook runs it connects to all of the clients and makes sure the “python2-pip” and “python-virtualevnv” packages are installed, it then builds the bare virtual environment into “/tmp/test01.venv/” and populates that venv with additional modules, then installs the Python modules necessary to execute the Ansible module.  The Ansible module is executed using the “python” executable in the newly built venv.

Note that ALL of these steps are preformed on the Ansible clients, no changes are made to the Ansible server.  In testing, the initial execution of these steps took about 40-50 seconds to get to the final step – most of that time was due to downloading packages from the Pip repository (on the Internet).  Subsequent runs that were able to re-use the venv directory took 20-25 seconds to get to the same location.

Caveats

One big shortcoming of this process is the necessity of the Ansible clients to have access to download the packages from an Internet location.  If the clients are shielded from the Internet, it may be necessary to setup a proxy server they can use (if permitted).

It might be necessary to perform the venv build on a single server with Internet access, then replicate that venv directory structure to each of the clients.  (These workarounds have not been validated, so test and report back any success or failures.)