Automating with AWS Secrets Manager
Use Case
In this scenario the following is true:
- You have an Ansible server in AWS
- You have AWS EC2 instances you'd like to manage as inventory
Define a Secrets Naming Scheme
You'll want your Ansible server to be able to hit the AWS Secrets Manager endpoint and query secrets in a specific region(s). But, you may not want the Ansible server to read all secrets. This is where a naming scheme will be usefule.
You could createa scheme such that the Ansible server will be able to read any secret that starts with the prefix: Ansible/
. So in Secrets Manager, you'll want to store your secrets as something like Ansible/windows-admin
.
Create an IAM Policy for the Ansible Server Instance
- Create a policy
- Choose a service
- Secrets Manager
- Actions
- Access Level
- List
- Read
- Resources
- Specific
- Add ARM
- Region
- Account
- Secret id:
Ansible/*
- Add ARM
- Specific
- Access Level
- Choose a service
- Name the policy
- Set a description
Create an IAM Role and Assign the Policy
Using a role, we can assign specific EC2 instances to make calls to AWS services. That way, there are no keys, secrets, or otherwise to bake into the playbook or other variables. Secrets Manager will trust the API call coming from the instance IP address, because it resides under our AWS account in AWS.
- Create a role
- Trusted entity
- AWS Service > EC2
- Add permissions
- Select the policy you created above
- Give your role a name and save it
- Trusted entity
- Assign the role to your Ansible EC2 server
Using AWS Dynamic Inventory with the AWS Secrets Manager
This will be a separate policy assigned to a role, in order for Ansible server(s) to access the AWS EC2 endpoint. This will allow the Ansible server(s) to query facts about EC2 instances in an AWS account.
Create an IAM Policy for Ansible Server(s)
- Go to Services > IAM > Policies
- Click Create Policy
- Service
- EC2
- Actions
- Under List and Read, choose any Describe* action
- Resources
- elastic-ip
- fleet
- image
- instance
- security-group
- snapshot
- volume
- vpc
- Request Conditions
- N/A
- Service
- Give it a name and description and save it
- Example: Ansible-Dynamic-Inventory-Policy
Assign the Policy to a Role
You can just assign this policy to the role created earlier. Now, the Ansible server has been assigned the new policy and can access the EC2 endpoint without the need for any keys or authentication variables.
Creating an AWS Dynamic Inventory
The minimum requirement for the dynamic inventory to work is that the filename must end with aws_ec2.yml
. For example: ~/.ansible/inventory/aws_ec2.yml
.
Set AWS Inventory as Your Default
Edit ~/.ansible.cfg
and set the default inventory file:
[defaults]
inventory = $HOME/.ansible/inventory/aws_ec2.yml
Edit ~/.ansible/inventory/aws_ec2.yml
---
plugin: aws_ec2
regions:
- us-east-1
groups:
windows: platform_details | regex_search('^win', ignorecase=True)
linux: platform_details | regex_search('^lin', ignorecase=True)
Test the Dynamic Inventory
You can run the command ansible-inventory --list
and if your Ansible server is able to successfully hit the EC2 endpoint in AWS, it should list hosts and any groups that match your group filters.
Using Group Variables
You can specify your windows and linux group variables the same was as any other inventory file, except in this case, we want to use the AWS Secrets Manager to retrieve credentials for our instances.
You can specify your group variable files in this following manner:
~/.ansible/inventory/group_vars/windows.yml
~/.ansible/inventory/group_vars/linux.yml
Example windows.yml configuration:
---
lookup_endpoint: amazon.aws.aws_secret
credential_query: Ansible/domain_administrator
aws_region: us-east-1
credential: "{{ lookup(lookup_endpoint, credential_query, region=aws_region) | from_json }}"
username: "{{ credential | json_query('Username') }}"
# Dynamic host names using tags from AWS
ansible_host: "{{ tags.Name }}.mydomain.tld"
ansible_connection: winrm
ansilbe_port: 5985
ansible_winrm_scheme: http
ansible_winrm_transport: kerberos
# Credential template
ansible_user: "{{ username }}@MYDOMAIN.TLD"
ansible_password: "{{ credential | json_query('Password') }}"
ansible_become_user: "{{ ansible_user }}"
ansible_become_password: "{{ ansible_password }}"
ansible_winrm_server_cert_validation: ignore
Ansible Kerberos Documentation
Example linux.yml configuration
---
lookup_endpoint: amazon.aws.aws_secret
credential_query: Ansible/linux-admin
aws_region: us-east-1
credential: "{{ lookup(lookup_endpoint, credential_query, region=aws_region) | from_json }}"
ansible_connection: ssh
username: "{{ credential | json_query('Username') }}"
ansible_host: "{{ private_ip_address }}"
ansible_user: "{{ username }}"
ansible_password: "{{ credential | json_query('Password') }}"
ansible_become_user: "{{ ansible_user }}"
ansible_become_password: "{{ ansible_password }}"
ansible_become_method: sudo