# All right then, keep your secrets in Git with SOPS

Wait, what? Don't panic, SOPS will encrypt them for you.

# Big NO NO

![](https://media.giphy.com/media/8vUEXZA2me7vnuUvrs/giphy.gif align="center")

Secrets (a.k.a. sensitive values), such as passwords or any other potentially harmful information must be protected, we all know that.

That's why you should avoid sharing those or pushing them to a public (or even private) code repository. Right? Well, yes and no.

### But where do you store them?

There are many vault-like services such as [Hashicorp Vault](https://www.vaultproject.io/), [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html), [GCP KMS](https://cloud.google.com/security-key-management), [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/basic-concepts) and the well-known credential storages out there like [1Password](https://1password.com/).

All of the above offer a separate secure location to isolate the secrets from the code referencing them, but also all of them need an extra fetch process; how do you get the actual value? you add a call to the secrets store using the corresponding method and you will have the secret value exported to your app.

Wanna save a step? Enter SOPS.

# Why SOPS?

Because it offers a simplified version of keeping sensitive information. You can directly store the values in your code and use one of AWS KMS, PGP, GCP KMS, Azure Key Vault, Hashicorp Key Vault and [age](https://github.com/FiloSottile/age) (I did not know this one) to encrypt and decrypt. No need to push/pull or store the secrets somewhere else.

# Example

![](https://media.giphy.com/media/BpGWitbFZflfSUYuZ9/giphy.gif align="center")

Let's sample this in a Kubernetes local scenario, using:

* Helm (Helmfile) to deploy a sample chart + Minikube hosting a simple Kubernetes cluster in your local.
    
    * Please refer to [this article about Helmfile + Minikube for a full installation walkthrough](https://blog.mariano.cloud/helm-declaration-environments-helmfile). For this tutorial, I will assume both are installed in your environment.
        
* AWS KMS key.
    
* SOPS.
    

> [Here](https://github.com/marianogg9/playing-with-sops) you can find the reference code and a few Terraform templates to create KMS and IAM resources. I will be using the AWS console for this tutorial, but I added a walkthrough for Terraform [here](https://github.com/marianogg9/playing-with-sops/tree/main/terraform#via-terraform).

## KMS

* Manually in the AWS console:
    
    1. Access AWS KMS service.
        
    2. Go to **Customer-managed** keys and click on **Create key**.
        
    3. Select **Symmetric** + **Encrypt and decrypt** options, then **Next**.
        
    4. Give it an alias and **Next**.
        
    5. Select a **Key administrator** and **Next**.
        
    6. Select a **Key user** (this step can be done later, after creating the IAM user), **Next**.
        
    7. **Finish**.
        

## IAM

* Manually in the AWS console:
    
    1. Access IAM service > **Users**.
        
    2. **Add Users**.
        
    3. Give it a name > **Next**.
        
    4. Select **Attach policies directly** > and then **Create policy** (this will open the new policy wizard).
        
        1. In the new policy wizard, select **JSON** and then paste the following JSON policy definition > **Next**:
            
            ```json
            {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "VisualEditor0",
                        "Effect": "Allow",
                        "Action": [
                            "kms:Decrypt",
                            "kms:Encrypt",
                            "kms:DescribeKey"
                        ],
                        "Resource": "arn:of:your:kms:key"
                    }
                ]
            }
            ```
            
        2. Give the new policy a name and then click on **Create policy**.
            
    5. Now go back to the IAM user creation wizard (previous tab) and click the refresh icon to update the available policies list. Then enter the new policy name in the search field and select it > **Next** > **Create user**.
        
    6. Now go back to the KMS service, select the key you created previously and select **Add** in the **Key Users** section of the key.
        
        1. Select the IAM user created before > **Add**.
            

## Baseline deployment with Helmfile

First, clone the repo:

```bash
git clone https://github.com/marianogg9/playing-with-sops.git
```

[Once you have Minikube running and Helmfile installed](https://blog.mariano.cloud/helm-declaration-environments-helmfile#heading-installation), let's deploy the sample `httpbin` app in the cluster:

```bash
cd playing-with-sops/helmfile/
helmfile -e example apply
```

This will deploy `httpbin` app in your local Kubernetes environment, including a pod, a service, a service account and a secret (we will use it later).

Let's see how environment variables look in the freshly created pod (we will use it to compare with a later step, trust me):

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1683372610085/27a01790-ac37-47c2-957e-f7b38faafe42.png align="center")

## And now: THE SECRETS

![](https://media.giphy.com/media/YQk8nXloVftzW/giphy.gif align="center")

Helmfile supports secrets by using `helm secret` plugin to manage and inject sensitive information into a given release values.

> The caveat here is those secret values are going to be shown in plain text in the `helmfile diff` or `helmfile apply` outputs, so be careful!
> 
> **Keep reading, there is a workaround for that.**

### Helm-secrets plugin

You will need to [install](https://helmfile.readthedocs.io/en/latest/#secrets) `helm-secrets` plugin:

```bash
helm plugin install https://github.com/jkroepke/helm-secrets
```

### SOPS

Installing:

```bash
brew install sops
```

Let's use it now! First, we need to create a `.sops.yaml` with the following content:

```yaml
creation_rules:
  - path_regex: \.yaml$
    kms: 'arn:of:your:kms:key'
```

This is a global configuration file that SOPS will use as a default when encrypting/decrypting files. You can set a regex for filenames and locations and a KMS ARN to use. That KMS is the one we created before.

The above example will accept any names anywhere ending in `.yaml` so it doesn't matter where you create this specific config file.

> Remember always to configure your local AWS CLI to use the KMS Key User we created before. See [this article](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) on how to set up and use it.

Next, create `secrets.yaml` file within `helmfile/` directory, by running the following:

```bash
sops secrets.yaml
```

This will open a text editor with prefilled sample values. Replace the content with:

```yaml
my_var: a top secret value
my_other_var: not that sensitive, but still please don't tell anyone!
```

When you save and close this file, SOPS will encrypt it automatically. You can have a look at its content now, SOPS added metadata referencing the KMS (ARN) used, a timestamp and some more.

### Helmfile takes over

Now for the Helmfile part, let's add `secrets:` section to the release in `helmfile.yaml`, as follows:

%[https://gist.github.com/marianogg9/a372b99b85f76175ab363a44477750b7] 

As a result, Helmfile will merge `values.yaml` and `secrets.yaml` as one and then use it to populate the charts templates to be deployed (in `charts/httpbin/templates/`).

Let's modify the deployment template to make use of the new secrets, by replacing the environment variables values with references to secrets in `secrets.yaml`:

%[https://gist.github.com/marianogg9/60498790876f7835e61bd5993f8c7ec9] 

Now let's see what Helmfile tries to do (`helmfile diff`):

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1683370219796/e65e661b-0fbb-4876-99ef-2959afde5028.png align="center")

As you can see in the output, it is first decrypting the secrets file we specified: `Decrypting secret /your_local_path/playing-with-sops/helmfile/secrets.yaml`.

Ok, let's apply (`helmfile apply`) and then see what changed in the deployment:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1683370354036/2a9a6bfd-387a-4d46-b5ba-18168381bcea.png align="center")

Now both environment vars values were changed to the secrets (encrypted) values.

![](https://media.giphy.com/media/tlGD7PDy1w8fK/giphy.gif align="center")

### A step further for CI/CD

What if you are trying to implement this in a CI/CD pipeline where logs are printed on stdout and so are your secrets?

Well, then you might want to use a Kubernetes secret definition. In this case, `helm secrets` will avoid showing secret values in plain text.

> Use a Kubernetes secret object to manage a sensitive value that you do not want to be printed in plain text in a CI/CD pipeline log.

You can create a new `secret.yaml` template definition in `helmfile/charts/httpbin/templates/` (I included an example in [the repo](https://github.com/marianogg9/playing-with-sops/blob/main/helmfile/secrets.yaml) as well):

%[https://gist.github.com/marianogg9/643d844ed1f557b0e5270667f1f72156] 

The `data` section contains both secrets (from `secrets.yaml`) and encrypts them to [allow Kubernetes API to accept their format](https://kubernetes.io/docs/concepts/configuration/secret/#restriction-names-data).

Then you can reference these secret keys in `deployment.yaml` template:

%[https://gist.github.com/marianogg9/fbbfecf845f0c6a3f1ddbe76d41fc474] 

If we check what Helmfile does:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1683370820865/621c645d-d77b-4c9f-a53a-3073ac1724a9.png align="center")

Now the values are effectively taken from a Kubernetes secret instead of directly from secrets defined in the Helmfile release.

Go ahead and run `helmfile -e example apply` to deploy the changes.

Let's now modify `secrets.yaml` values

```bash
sops secrets.yaml
```

I removed the last character from both `my_var` and `my_other_var` in `secrets.yaml` - it doesn't really matter the change, it is just an example. Save and close the file to have SOPS re-encrypt it.

And now let's see what Helmfile tries to do:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1683371114420/09073ff0-c7e6-4691-92c6-c04c56874f7e.png align="center")

There are no secrets shown in plain text anymore!

![](https://media.giphy.com/media/TNnyxINX87VAKbNYmZ/giphy.gif align="center")

# Conclusion

You can manage secret values directly in Git now, without extra steps to fetch from remote. It is a simple concept and it is practical if you don't have that many sensitive values to justify an external platform setup for them. Of course, it is a possible tool for a given use case, it all depends on the context.

I found out about this way of managing secrets when I started working with Helmfile, it is pretty well integrated and just works. Simplicity, I like it.

[Please have a look at all SOPS features](https://github.com/mozilla/sops#sops-secrets-operations).

> Helmfile also offers secrets [remote fetch](https://github.com/roboll/helmfile/blob/master/docs/remote-secrets.md) natively as well.

# References

* [SOPS](https://github.com/mozilla/sops).
    
* [Helmfile + Minikube in action](https://blog.mariano.cloud/helm-declaration-environments-helmfile).
    
* [Full code example on Github](https://github.com/marianogg9/playing-with-sops).
    
* [Helm plugin](https://helm.sh/docs/topics/plugins/).
    
* [Helmfile](https://helmfile.readthedocs.io/en/latest/).
    

---

Thank you for stopping by! Do you know other ways to do this? Please let me know in the comments, I always like to learn how to do things differently.