Skip to main content

Automating with AWS Secrets Manager

Use Case

In this scenario the following is true:

  • You have an Ansible server in AWS
  • You have AWS EC2 instances you'd like to manage as inventory

Define a Secrets Naming Scheme

You'll want your Ansible server to be able to hit the AWS Secrets Manager endpoint and query secrets in a specific region(s). But, you may not want the Ansible server to read all secrets. This is where a naming scheme will be usefule.

You could createa scheme such that the Ansible server will be able to read any secret that starts with the prefix: Ansible/. So in Secrets Manager, you'll want to store your secrets as something like Ansible/windows-admin.

Create an IAM Policy for the Ansible Server Instance

  1. Create a policy
    • Choose a service
      • Secrets Manager
    • Actions
      • Access Level
        • List
        • Read
      • Resources
        • Specific
          • Add ARM
            • Region
            • Account
            • Secret id: Ansible/*
  2. Name the policy
  3. Set a description

Create an IAM Role and Assign the Policy

Using a role, we can assign specific EC2 instances to make calls to AWS services. That way, there are no keys, secrets, or otherwise to bake into the playbook or other variables. Secrets Manager will trust the API call coming from the instance IP address, because it resides under our AWS account in AWS.

  1. Create a role
    • Trusted entity
      • AWS Service > EC2
    • Add permissions
      • Select the policy you created above
    • Give your role a name and save it
  2. Assign the role to your Ansible EC2 server



Using AWS Dynamic Inventory with the AWS Secrets Manager

This will be a separate policy assigned to a role, in order for Ansible server(s) to access the AWS EC2 endpoint. This will allow the Ansible server(s) to query facts about EC2 instances in an AWS account.

Create an IAM Policy for Ansible Server(s)

  1. Go to Services > IAM > Policies
  2. Click Create Policy
    • Service
      • EC2
    • Actions
      • Under List and Read, choose any Describe* action
    • Resources
      • elastic-ip
      • fleet
      • image
      • instance
      • security-group
      • snapshot
      • volume
      • vpc
    • Request Conditions
      • N/A
  3. Give it a name and description and save it
    • Example: Ansible-Dynamic-Inventory-Policy


Assign the Policy to a Role

You can just assign this policy to the role created earlier. Now, the Ansible server has been assigned the new policy and can access the EC2 endpoint without the need for any keys or authentication variables.



Creating an AWS Dynamic Inventory

The minimum requirement for the dynamic inventory to work is that the filename must end with aws_ec2.yml. For example: ~/.ansible/inventory/aws_ec2.yml.


Set AWS Inventory as Your Default

Edit ~/.ansible.cfg and set the default inventory file:

inventory = $HOME/.ansible/inventory/aws_ec2.yml

Edit ~/.ansible/inventory/aws_ec2.yml

plugin: aws_ec2
- us-east-1
windows: platform_details | regex_search('^win', ignorecase=True)
linux: platform_details | regex_search('^lin', ignorecase=True)


Test the Dynamic Inventory

You can run the command ansible-inventory --list and if your Ansible server is able to successfully hit the EC2 endpoint in AWS, it should list hosts and any groups that match your group filters.


Using Group Variables

You can specify your windows and linux group variables the same was as any other inventory file, except in this case, we want to use the AWS Secrets Manager to retrieve credentials for our instances.

You can specify your group variable files in this following manner:

  • ~/.ansible/inventory/group_vars/windows.yml
  • ~/.ansible/inventory/group_vars/linux.yml


Example windows.yml configuration:

credential_query: Ansible/domain_administrator
aws_region: us-east-1
credential: "{{ lookup(lookup_endpoint, credential_query, region=aws_region) | from_json }}"
username: "{{ credential | json_query('Username') }}"

# Dynamic host names using tags from AWS
ansible_host: "{{ tags.Name }}.mydomain.tld"
ansible_connection: winrm
ansilbe_port: 5985
ansible_winrm_scheme: http
ansible_winrm_transport: kerberos

# Credential template
ansible_user: "{{ username }}@MYDOMAIN.TLD"
ansible_password: "{{ credential | json_query('Password') }}"
ansible_become_user: "{{ ansible_user }}"
ansible_become_password: "{{ ansible_password }}"
ansible_winrm_server_cert_validation: ignore

Ansible Kerberos Documentation


Example linux.yml configuration

credential_query: Ansible/linux-admin
aws_region: us-east-1
credential: "{{ lookup(lookup_endpoint, credential_query, region=aws_region) | from_json }}"
ansible_connection: ssh
username: "{{ credential | json_query('Username') }}"
ansible_host: "{{ private_ip_address }}"
ansible_user: "{{ username }}"
ansible_password: "{{ credential | json_query('Password') }}"
ansible_become_user: "{{ ansible_user }}"
ansible_become_password: "{{ ansible_password }}"
ansible_become_method: sudo