Secrets Mgmt: AWS, Azure, and GCP

Storing secrets is one of the major challenges that users face. Most cloud providers have their own set of tools to store secrets. Credentials, API Keys, and Certificates are widely used in applications that we create and use. Projects tend to get bigger and more complex and securing and managing credentials becomes more and more challenging over time: as your project tends to get bigger, so does the number of secrets you’ll need to store. In this article, I am going to show how I am doing it in AWS, Azure, and GCP.

AWS

As AWS is the most popular cloud provider let's start with it. AWS provides two options to store credentials:

• Parameter Store: link
• Secrets Manager: link

Below you can see the comparison of the two:

How to use Parameter Store:

It provides two options: Standard and Advanced(charges apply). For this case, I will be using the Standard option, there are three types of parameters that can be stored:

• String
• StringList
• Secure String

To create a Parameter of type String, use below:

To create a Parameter of type SecureString:

In the case of SecureString, the password is secured using KMS. To use SecureString ensure you have KMS key created or you can use the default KMS key, I use a Customer managed Symmetric key. As can be seen, I created a key ecs/demo

In the Parameter details, I specified the KMS Key ID: ecs/demo

Under the list of Parameters, you would see the parameters created above:

One thing to note here is the name of the Parameter. As parameters can be stored in a hierarchical order, try to put them in a defined structure. There can be multiple ways to define the hierarchy. The way I have defined is: Resource Name, Application Name, and Secret. The benefit of the above approach is that it's easier to find related objects. Let's see an example:

To get a list of all the variables stored under the /ecs hierarchy.

aws ssm get-parameters-by-path --path /ecs --recursive --query 'Parameters[*].[Name]'
[
    [
        "/ecs/demo/secret1"
    ],
    [
        "/ecs/demo/secret2"
    ]
]

To get the Name and Values:

aws ssm get-parameters-by-path --path /ecs --recursive --query 'Parameters[*].[Name,Value]'
[
    [
        "/ecs/demo/secret1",
        "password1"
    ],
    [
        "/ecs/demo/secret2",
      "AQICAHgUAddzcDAtRMJlFEmrGzQjNI1O+RqRVJVKlWROBEA6OgENiQs1SAEOEyLdpsdjywf2AAAATAeBglghkgBZQMEAS4wEQQMk1LGEy3PyVFizlcsAgEQgCSfO+3Ruu9AyCnpW8uRRM+jpX3+iK7eJGVt5Hij5rIgEC9k3ao="
    ]
]

How to use Secrets Manager:

Under Secrets Manager, there are multiple options to store credentials, as can be seen, Secrets Manager is tightly integrated with RDS, DocumentDB, Redshift.

On the next screen, give the name to the secret/ecs/demo/secret1

On the next screen, choose if you need to enable automatic rotation and then save:

List of Secrets:

aws secretsmanager list-secrets --region us-east-2 --query 'SecretList[*].Name'
[
    "/ecs/demo/secret2",
    "/ecs/demo/secret3"
]

Describe a Secret:

aws secretsmanager get-secret-value --secret-id /ecs/demo/secret2 --version-stage AWSCURRENT
{
    "ARN": "arn:aws:secretsmanager:us-east-2:xxxxxxxx:secret:/ecs/demo/secret2-opckgb",
    "Name": "/ecs/demo/secret2",
    "VersionId": "xxxxxxxx-26bd-4091-b96c-xxxxxxx",
    "SecretString": "{\"user1\":\"password1\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": "2020-05-22T11:49:29.790000-07:00"
}

Azure

Azure follows the same process as AWS and once you create a Key Vault, UI is pretty simple to use:

Once you create the key-vault you would see a screen like as below, there are few things to note: Access control(IAM), Keys, Secrets, and Access Policies that I am using, there are other interesting things but I am not covering in this post:

Keys:

You can create a key as below, In my case, I am using a symmetric key (the same key to encrypt and decrypt the data). Microsoft has good documentation around keys: here and how to perform crypto operations: link

Secrets:

You can either store any value and they can be retrieved as key-value pair.

How to use secrets:

The easiest way for an application to access key-vault is by using Managed Identities. The problem is our use case was when resources that cannot use identities. To solve the problem we started using Key to encrypt the value stored in Secrets and then again when needed we decrypted the Secret using the same key.

Azure: Put and Get secrets from Vault

Azure: Encrypt and Decrypt KV secrets

GCP

GCP’s Secret Manager aims to centralize, manage and secure sensitive information in a convenient and secure way. Just as AWS and Azure creating and retrieving secrets is pretty straightforward.

Secret Manager:

GCP: Create Secret

Create a Secret:

GCP: Secret

Retrieve a Secret

Conclusion

Using a Secret Manager prevents access to your secrets. In today’s multi-cloud world one size does not fit all. If you are truly multi-cloud you can also use HashiCorp KeyVault another of my favorite tools. Ensure what you select is cost-efficient and future-proof.