Using aws-vault with Linux

There are lots of purported "best practices" out there on the internet regarding AWS API keypairs. In this author's opinion, the term best practices is in many cases either a dubious claim or a concept that isn't actually real, since "best" is often subjective or situation dependent. However, with regard to AWS API keypairs and developer machines, I'll argue that there really is a best way.

I was recently introduced to aws-vault by an acquaintance from the HangOps slack team. Succinctly put, it solves three significant problems:

  • Leaving a keypair lying around on disk, accessible in plain text from your ~/.aws/credentials file, really sucks. It's not necessarily terrible if you're enforcing role assumption and 2fa, and if your disk is encrypted at rest, and if you never walk away from your machine without locking it - but that's several ifs.
  • The 1h timeout on credentials gained from assuming a role sucks.
  • The need for many different AWS profiles to be in use in your browser (demonstrated here with Chrome)

Enter aws-vault - which stores credentials and session data in your keychain and is capable of handling renewal via --server. Usage is well documented for OSX and outside the scope of this article, which tells you how to do it with Gnome (or Cinnamon - which is what I use). I've used the same process on both Ubuntu 16.04/18.04 and Fedora 27/28 with success. While simply using an encrypted file works just fine, I prefer to use the keychain!

Usage

  • Ensure that gnome-keyring is installed.
  • Ensure that that gnome-keyring-daemon is running - I put these lines in my ~/.bash_profile:
if [ -n "$DESKTOP_SESSION" ];then  
    eval $(gnome-keyring-daemon --start)
    export SSH_AUTH_SOCK
fi  

Under Linux, aws-vault defaults to using encrypted files. Therefore, you'll need to specify a backend via env var. In ~/.bash_profile:

export AWS_VAULT_BACKEND=secret-service  

You're set to begin adding credentials:

  • source ~/.bash_profile
  • aws-vault add $someprofilename - This will prompt you for an AWS Access Key ID and AWS Secret Access Key pair

You now have a keypair stored in aws-vault - however, a little more setup is required for assuming roles. This is done a bit like how the ~/.aws/credentials file normally functions, only it's done in ~/.aws/config - aws-vault adds the [profile someprofile], you just need to add profiles which use its keypair for assuming a role:

[profile someprofile]

[profile ops]
role_arn = arn:aws:iam::12345678910:role/some-role  
source_profile = someprofile  
mfa_serial = arn:aws:iam::23456789101:mfa/connor_rielly

[profile k8s]
role_arn = arn:aws:iam::34567891011:role/another-role  
source_profile = someprofile  
mfa_serial = arn:aws:iam::23456789101:mfa/connor_rielly

[profile blog]
role_arn = arn:aws:iam::456789101112:role/third-role  
source_profile = someprofile  
mfa_serial = arn:aws:iam::23456789101:mfa/connor_rielly  

Now you can see that aws-vault retrieves any linked profiles:

$ aws-vault list
Profile                  Credentials              Sessions  
=======                  ===========              ========                 
someprofile              someprofile              -  
ops                      someprofile              -  
k8s                      someprofile              -  
blog                     someprofile              -  

And you're ready grab some credentials with a nice, long TTL:

$ aws-vault exec --server --session-ttl=8h ops
Enter token for arn:aws:iam::23456789101:mfa/connor_rielly: 549236

$ aws-vault list
Profile                  Credentials              Sessions  
=======                  ===========              ========                 
someprofile              someprofile              86743437993111653036  
ops                      someprofile              -  
k8s                      someprofile              -  
blog                     someprofile              - 

$ aws s3 ls
2017-12-15 20:49:38 some-bucket-1  
2017-12-09 23:08:47 some-bucket-2  
2017-07-18 22:59:37 some-bucket-3  

Or to execute subprocesses with AWS credentials injected as Environment Variables:

aws-vault exec k8s -- ansible-playbook standup-asg-cluster.yml -i inventories/dev -e "namespace=tcdev" -e "region=us-west-2"  

Or to login to the console (this opens a browser tab):

$ aws-vault --debug login k8s
2018/03/08 13:01:34 Loading config file /home/crielly/.aws/config  
2018/03/08 13:01:34 Parsing config file /home/crielly/.aws/config  
2018/03/08 13:01:34 Skipping session token and using master credentials directly  
2018/03/08 13:01:34 Looking up keyring for tc  
Enter token for arn:aws:iam::ACCTNUMBER:mfa/connor_rielly: 976384  
2018/03/08 13:01:41 Assuming role arn:aws:iam::ACCTNUMBER:role/org-admin with iam credentials  

You also now have no keypair stored in anything resembling a readable fashion on-disk.

Advanced Usage

I've cribbed and created a couple of bash aliases to make usage more efficient:

awsvl() {  
    aws-vault --debug login $1 --mfa-token=$2 --stdout | xargs -t nohup $(which google-chrome-stable) %U --no-first-run --new-window --disk-cache-dir=$(mktemp -d /tmp/chrome.XXXXXX) --user-data-dir=$(mktemp -d /tmp/chrome.XXXXXX) >/dev/null 2>&1 &
}

awsve() {  
    aws-vault exec --assume-role-ttl=60m --session-ttl=12h $@
}

Usage for awsvl:

$ awsvl k8s 131161
[1] 5836
[crielly@arlakh aws-vault]$ 2018/06/02 11:39:05 [keyring] Considering backends: [secret-service]
2018/06/02 11:39:05 Loading config file /home/crielly/.aws/config  
2018/06/02 11:39:05 Parsing config file /home/crielly/.aws/config  
2018/06/02 11:39:05 Skipping session token and using master credentials directly  
2018/06/02 11:39:05 Looking up keyring for someprofile  
2018/06/02 11:39:05 Assuming role arn:aws:iam::ACCTNUMBER:role/org-admin with iam credentials  
2018/06/02 11:39:05 Using role ****************Z4UA, expires in 14m59.34725927s  
2018/06/02 11:39:05 Creating login token, expires in 12h0m0s  

This will launch a new chrome window using your profile which will not affect any other open chrome windows due to the use of randomized --user-data-dir and --disk-cache-dir. I am not a master of the shell and so this is imperfect - the command will appear to run in the foreground, but you can safely ctrl + c it. However, if you close the terminal in which it was launched, your chrome window closes. My solution is to simply keep an extra terminator tab open for launching aws web panel sessions, so in practice it's a small inconvenience.

Usage for awsve is...it's just a short form for aws-vault exec:

awsve k8s -- ansible-playbook standup-asg-cluster.yml -i inventories/dev -e "namespace=tcdev" -e "region=us-west-2"