Ansible Setup & Playbooks

Synced from the repo

This page is generated from ansible/README.md. Edit it there — changes here are overwritten on the next build.

This directory contains a simple test playbook to verify your Ansible and Semaphore UI setup.

Directory Structure

ansible/
├── ansible.cfg                    # Ansible configuration
├── collections/
│   └── requirements.yml           # Collections requirements
├── group_vars/                    # Group-specific variables
├── host_vars/                     # Host-specific variables
├── inventory/
│   ├── hosts.ini                  # Static inventory (localhost)
│   ├── proxmox.proxmox.yml        # Proxmox dynamic inventory
│   └── README.md                  # Inventory documentation
├── playbooks/
│   └── test-playbook.yml          # Test playbook
├── roles/
│   └── requirements.yml           # Roles requirements
├── requirements.yml               # Root requirements (roles & collections)
└── README.md                      # This file

Prerequisites

• Ansible installed on the system
• Docker installed (for Docker-related tasks)
• Appropriate permissions for Docker commands

What the Test Playbook Does

The test-playbook.yml performs the following tasks:

1. Gathers and displays system information (hostname, OS, CPU, memory)
2. Gets current date and time
3. Creates a test directory at /tmp/ansible-test
4. Creates timestamped test files
5. Checks Docker installation status
6. Checks Docker service status (requires sudo)
7. Lists running Docker containers
8. Provides a summary of the test run

Running Locally

From Command Line

# Navigate to the ansible directory
cd /home/unknown224/ubuntu-server-config/ansible

# Run the playbook (note the new path)
ansible-playbook playbooks/test-playbook.yml

# Run with verbose output
ansible-playbook playbooks/test-playbook.yml -v

# Run with specific inventory
ansible-playbook playbooks/test-playbook.yml -i inventory/hosts.ini

Check Syntax

# Validate playbook syntax
ansible-playbook playbooks/test-playbook.yml --syntax-check

# Dry run (check mode)
ansible-playbook playbooks/test-playbook.yml --check

List Tasks

# List all tasks in the playbook
ansible-playbook playbooks/test-playbook.yml --list-tasks

# List all hosts
ansible-playbook playbooks/test-playbook.yml --list-hosts

Using with Semaphore UI

1. Create a New Repository

• Name: ubuntu-server-config
• URL: Point to this git repository
• Branch: main (or your current branch)
• SSH Key: Select appropriate key (or “None” for public repos)

2. Create a New Project

• Name: Ansible Test Project
• Repository: Select the repository created above

3. Create an Inventory

• Name: localhost
• Type: File
• Inventory: ansible/inventory/hosts.ini

4. Create a Variable Group (Environment)

Semaphore uses Variable Groups for environment variables and secrets.

Name: Default Environment

Variables (JSON):

{
  "ANSIBLE_CONFIG": "ansible/ansible.cfg"
}

For secrets (like Proxmox credentials):

• Add sensitive values to Variable Groups
• Secrets are automatically masked in logs
• Example:

{
  "PROXMOX_TOKEN_SECRET": "your-secret-here",
  "ANSIBLE_CONFIG": "ansible/ansible.cfg"
}

5. Create a Task Template

• Name: Test Playbook
• Playbook Filename: ansible/playbooks/test-playbook.yml
• Inventory: Select the inventory created above
• Repository: Select your repository
• Environment: Select your environment (if created)
• Vault Password: Leave empty (not needed for this test)

6. Run the Template

Execute the template from Semaphore UI and monitor the output in real-time.

Expected Output

The playbook will:

• Display system information
• Create files in /tmp/ansible-test/
• Show Docker container status
• Complete successfully with a summary message

Troubleshooting

Permission Errors

If you encounter permission errors for Docker commands:

# Add your user to the docker group
sudo usermod -aG docker $USER

# Or run with become/sudo
ansible-playbook playbooks/test-playbook.yml --become --ask-become-pass

Connection Issues

If you see SSH connection errors:

• Verify ansible_connection=local is set in inventory
• Check ansible.cfg has correct settings
• Ensure you’re running from the ansible directory

Python Interpreter Issues

If you see Python interpreter warnings:

# Check Python 3 location
which python3

# Update inventory if needed with correct path

Cleaning Up

# Remove test files
rm -rf /tmp/ansible-test

# Remove log file
rm ansible.log

# Remove fact cache
rm -rf /tmp/ansible_facts

Installing Dependencies

Install Collections and Roles

Semaphore will automatically install dependencies from requirements.yml files. To install them manually:

# Install all requirements (from root requirements.yml)
ansible-galaxy install -r requirements.yml

# Install only roles
ansible-galaxy role install -r roles/requirements.yml

# Install only collections
ansible-galaxy collection install -r collections/requirements.yml

# Verify installed collections
ansible-galaxy collection list

# Verify installed roles
ansible-galaxy role list

Next Steps

After successful testing:

1. Create more playbooks - Add production playbooks in playbooks/
2. Add roles - Create custom roles in roles/ or install from Galaxy
3. Configure variables - Use group_vars/ and host_vars/ for environment-specific configs
4. Add secrets - Set up Ansible Vault for sensitive data
5. Create templates - Add Jinja2 templates for configuration files
6. Add handlers - Implement service restart handlers
7. Schedule tasks - Use Semaphore’s scheduler for automated runs

Step CA Client Setup

This repository includes an Ansible role to set up Step CA clients on Ubuntu/Debian systems. The role is modular, reusable, and follows Ansible best practices.

Role Structure

roles/step_ca_client/               # Ansible role for Step CA client setup
├── tasks/
│   ├── main.yml                    # Main task orchestration
│   ├── install.yml                 # Install step CLI from official repository
│   ├── bootstrap.yml               # Bootstrap CA trust
│   └── system-cert.yml             # Install certificate system-wide
├── defaults/main.yml               # Default variables
├── meta/main.yml                   # Role metadata
└── README.md                       # Role documentation

playbooks/
├── workflows/
│   └── setup_new_machine.yml       # Workflow using the role
├── tasks/                          # Legacy task playbooks (for reference)
└── setup-step-ca-client.yml        # Legacy playbook (for compatibility)

Prerequisites

Before running the playbook, you need:

1. Step CA server running at stepca.toolsera.lan (or your configured URL)

2. CA fingerprint - Get it from your Step CA server:

   step certificate fingerprint $(step path)/certs/root_ca.crt

   Example output: 702a094e239c9eec6f0dcd0a5f65e595bf7ed6614012825c5fe3d1ae1b2fd6ee

Running the Role

Using the Workflow Playbook (Recommended)

Run the complete new machine setup workflow:

# Run on localhost
ansible-playbook setup_new_machine.yml \
  -e step_ca_fingerprint=YOUR_FINGERPRINT_HERE

# Run on specific hosts
ansible-playbook -i inventory/hosts.ini setup_new_machine.yml \
  -e step_ca_fingerprint=YOUR_FINGERPRINT_HERE

# Run on Proxmox-discovered hosts (e.g., all web servers)
ansible-playbook -i inventory/proxmox.proxmox.yml setup_new_machine.yml \
  --limit tag_web \
  -e step_ca_fingerprint=YOUR_FINGERPRINT_HERE

# Run only specific parts using tags
ansible-playbook setup_new_machine.yml \
  -e step_ca_fingerprint=YOUR_FINGERPRINT \
  --tags step-ca

Using the Role Directly in Your Own Playbooks

---
- name: Configure servers with Step CA
  hosts: all
  roles:
    - role: step_ca_client
      step_ca_fingerprint: 'YOUR_FINGERPRINT_HERE'

Selective Task Execution

Control which tasks run using role variables:

# Install CLI only (skip bootstrap and system cert)
ansible-playbook setup_new_machine.yml \
  -e step_ca_install_cli=true \
  -e step_ca_bootstrap=false \
  -e step_ca_install_system_cert=false

# Bootstrap only (CLI already installed)
ansible-playbook setup_new_machine.yml \
  -e step_ca_install_cli=false \
  -e step_ca_bootstrap=true \
  -e step_ca_install_system_cert=true \
  -e step_ca_fingerprint=YOUR_FINGERPRINT

Advanced Options

# Override CA URL (if using non-default)
ansible-playbook playbooks/setup-step-ca-client.yml \
  -e step_ca_fingerprint=YOUR_FINGERPRINT \
  -e step_ca_url=https://stepca.toolsera.lan:8443

# Dry run (check mode)
ansible-playbook playbooks/setup-step-ca-client.yml --check \
  -e step_ca_fingerprint=YOUR_FINGERPRINT

# Verbose output for troubleshooting
ansible-playbook playbooks/setup-step-ca-client.yml -vv \
  -e step_ca_fingerprint=YOUR_FINGERPRINT

What the Role Does

1. Installs step CLI - Adds official Smallstep APT repository and installs step-cli package
2. Bootstraps CA trust - Validates fingerprint, runs step ca bootstrap, downloads root certificate
3. Installs certificate system-wide - Adds root CA to system trust store (requires sudo)
4. Verifies installation - Tests connectivity to CA and displays comprehensive summary

The role is idempotent - safe to run multiple times, skips already-completed steps.

Configuration

Available Variables

See roles/step_ca_client/defaults/main.yml for all variables:

step_ca_url: 'https://stepca.toolsera.lan'     # CA server URL
step_ca_fingerprint: ''                         # Required: CA fingerprint
step_ca_install_cli: true                       # Install step CLI
step_ca_bootstrap: true                         # Bootstrap CA trust
step_ca_install_system_cert: true               # Install cert system-wide

Override Methods

• group_vars/all.yml: Global defaults for all hosts
• group_vars/webservers.yml: Per-group configuration
• host_vars/hostname.yml: Per-host configuration
• Command line: Use -e variable=value
• Playbook vars: Set in your playbook when including the role

roles:
  - role: step_ca_client
    step_ca_url: 'https://ca.example.com:8443'
    step_ca_fingerprint: 'YOUR_FINGERPRINT'

Using with Semaphore UI

1. Create Variable Group (for fingerprint)

• Go to Variable Groups in your project

• Click “New Variable Group”

• Name: Step CA Configuration

• JSON content:

  {
    "step_ca_fingerprint": "your-actual-fingerprint-here",
    "step_ca_url": "https://stepca.toolsera.lan"
  }

2. Create Task Template

• Go to Task Templates
• Click “New Template”
• Configure:
  • Name: Setup Step CA Client
  • Playbook Filename: ansible/playbooks/setup-step-ca-client.yml
  • Inventory: Select your inventory (static or Proxmox dynamic)
  • Environment: Select Step CA Configuration variable group
  • Extra Variables (optional): Add any overrides

3. Run the Task

• The playbook will install step CLI and configure CA trust on target hosts
• System-wide certificate installation requires sudo privileges

Post-Installation

After running the playbook, clients can:

Request certificates:

step ca certificate yourservice.toolsera.lan srv.crt srv.key

View CA provisioners:

step ca provisioner list

Check CA health:

step ca health

Test HTTPS connections:

curl https://yourservice.toolsera.lan

The root certificate is installed in two locations:

• Step CLI config: ~/.step/certs/root_ca.crt
• System trust store: /usr/local/share/ca-certificates/ (Ubuntu/Debian)

Troubleshooting

Playbook fails with “step_ca_fingerprint variable is required”:

• Ensure you pass -e step_ca_fingerprint=YOUR_FINGERPRINT when running the playbook
• Or add it to your Semaphore Variable Group

“unable to connect” errors:

• Verify DNS resolution: ping stepca.toolsera.lan
• Check CA is accessible: curl -k https://stepca.toolsera.lan
• Verify firewall allows HTTPS (port 443)

Certificate already exists:

• Playbook is idempotent and will skip if already installed
• To reinstall: step certificate uninstall ~/.step/certs/root_ca.crt then re-run

Permission denied during system install:

• Playbook uses become: true for system-wide installation
• Ensure target user has sudo privileges
• Add --ask-become-pass if password is required

Custom APT Repository Setup

This repository includes an Ansible role to configure custom APT repositories with GPG key authentication. By default, it configures the Toolsera Nexus repository.

Role: custom_apt_repo

The role installs GPG signing keys and configures custom APT repositories on Ubuntu/Debian systems.

Quick Start

# Run the complete workflow (includes custom apt repo + step ca)
ansible-playbook setup_new_machine.yml \
  -e step_ca_fingerprint=YOUR_FINGERPRINT

# Run only APT repository configuration
ansible-playbook setup_new_machine.yml \
  --tags apt,repository

Default Configuration

By default, the role configures:

• Repository: https://nexus.toolsera.lan/repository/toolsera/
• Distribution: stable
• Components: main
• GPG Key: Included in role (roles/custom_apt_repo/files/public.gpg.key)

Resulting sources.list entry:

deb https://nexus.toolsera.lan/repository/toolsera/ stable main

Customization

Override variables to configure different repositories:

- hosts: servers
  roles:
    - role: custom_apt_repo
      apt_repo_url: 'https://your-repo.example.com/debian/'
      apt_repo_distribution: 'focal'
      apt_repo_components: 'main contrib'

Available Variables

See roles/custom_apt_repo/defaults/main.yml:

apt_repo_url: 'https://nexus.toolsera.lan/repository/toolsera/'
apt_repo_distribution: 'stable'
apt_repo_components: 'main'
apt_repo_key_file: 'public.gpg.key'
apt_repo_key_dest: '/etc/apt/trusted.gpg.d/toolsera-repo.asc'
apt_repo_sources_file: '/etc/apt/sources.list.d/toolsera.list'
apt_repo_options: ''  # e.g., 'arch=amd64,arm64'

Updating the GPG Key

To update the GPG signing key:

1. Replace ansible/roles/custom_apt_repo/files/public.gpg.key
2. Re-run the playbook

Post-Installation

After configuration:

# Verify repository is configured
cat /etc/apt/sources.list.d/toolsera.list

# Check GPG key
ls -la /etc/apt/trusted.gpg.d/toolsera-repo.asc

# Install packages from your repository
apt update
apt install your-package-name

For more details, see roles/custom_apt_repo/README.md.

Proxmox Dynamic Inventory

This repository includes support for Proxmox VE dynamic inventory, which automatically discovers your VMs and containers.

Quick Start

1. Install collections:

ansible-galaxy collection install -r collections/requirements.yml

2. Configure credentials:

   For local testing:

   cp .env.example .env
   # Edit .env and add your PROXMOX_TOKEN_SECRET

   For Semaphore UI (Recommended):

   • Create a Variable Group in Semaphore
   • Add PROXMOX_TOKEN_SECRET and PROXMOX_URL as JSON
   • Secrets are automatically masked in logs

3. Update Proxmox URL in inventory/proxmox.proxmox.yml (or use Variable Group)

4. Test inventory:

ansible-inventory -i inventory/proxmox.proxmox.yml --list

See inventory/README.md for detailed setup instructions including Semaphore integration.

Using Proxmox Inventory with Playbooks

# Run playbook against all Proxmox hosts
ansible-playbook -i inventory/proxmox.proxmox.yml playbooks/your-playbook.yml

# Target specific group (hosts tagged with "web")
ansible-playbook -i inventory/proxmox.proxmox.yml playbooks/deploy.yml --limit tag_web

# Use multiple inventories
ansible-playbook -i inventory/hosts.ini -i inventory/proxmox.proxmox.yml playbooks/site.yml

Useful Ansible Commands

# Static inventory commands
ansible all -i inventory/hosts.ini --list-hosts
ansible all -i inventory/hosts.ini -m ping

# Proxmox dynamic inventory commands
ansible-inventory -i inventory/proxmox.proxmox.yml --graph
ansible all -i inventory/proxmox.proxmox.yml -m ping
ansible tag_web -i inventory/proxmox.proxmox.yml -m ping

# Gather facts from hosts
ansible localhost -m setup

# Run ad-hoc commands
ansible localhost -m command -a "uptime"

Secret Management Best Practices

Using Semaphore Variable Groups

Semaphore UI provides built-in secret management:

• Variable Groups: Store secrets as JSON in project Variable Groups
• Automatic Masking: Sensitive values are masked in logs and UI
• Runtime Injection: Variables passed securely to playbooks at runtime
• No File Storage: Secrets never stored in repository files

Local Development

For local testing outside Semaphore:

• Use .env files (gitignored)
• Never commit .env files to the repository
• Use .env.example as a template

Summary

• Production/Semaphore: Use Variable Groups
• Local Testing: Use .env files
• Documentation: Use .env.example as templates

Resources

• Ansible Documentation
• Semaphore UI Documentation
• Semaphore Variable Groups
• Ansible Best Practices