Deploy a Production Ready Hashicorp Vault Node

In this scenario, I will walk you step by step to install Hahsicorp Vault and prepare it to be production ready. I will be using Ubuntu 20.04 as my base image to run my vault node.

Step 1 — Install Hashicorp Consul

Based on your environment, you can download and install consul from the official website here. I will use the latest linux version available by the time this article written (Consul 1.9.4):

wget https://releases.hashicorp.com/consul/1.9.4/consul_1.9.4_linux_amd64.zip

unzip the downloaded file (if you downloaded a different version please update the bellow code based on the downloaded file name):

unzip consul_1.9.4_linux_amd64.zip

Now if you run an ls command, you will see a file named consul in the listed results:

ls

Let’s move the file to the bin folder so we can run consul from anywhere. If you are root, you can remove sudo:

sudo mv consul /usr/bin/

To confirm consul works, let’s execute the following command:

consul version

the results should show the current version installed as shown below:

consul version

Step 2 — Configure Consul to run as a systemd service

We’ll configure consul backend to function as a systemd service to prepare it to run as our storage backend for the HashiCorp Vault.

Let’s create a consul configuration file under /etc/systemd/system. You can use your preferred editor to create and edit your files, I will use vim in the following commands:

sudo vim /etc/systemd/system/consul.service

Copy and paste the following snippet:

[Unit]
Description=Consul
Documentation=https://www.consul.io/
[Service]
ExecStart=/usr/bin/consul agent -server -ui -data-dir=/temp/consul -bootstrap-expect=1 -node=vault -bind=SERVER-PRIVATE-IP-ADDRESS -config-dir=/etc/consul.d/
ExecReload=/bin/kill -HUP $MAINPID
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

Make sure to put your server private IP address for the bind value above before saving, in vim case press :wq to save the file.

Note: If you are not familiar with vim, press i to edit the file, insert the server IP address and hit ESC to exit the edit mode. Here is a useful cheat sheet for vim if you need to explore it more.

Little details about the file we just created:

1. Description: just text describing your file, you can write whatever you want as a reference.
2. Documentation: where you can find consul documentation.
3. ExecStart: consul command that will execute at the beginning of the service start , you need to put the private IP address of the server.

Now let’s create a folder for consul:

sudo mkdir /etc/consul.d

We need to create the following json file in it:

sudo vim /etc/consul.d/ui.json

Copy and paste the following snippet:

{
“addresses”: {
“http”: “0.0.0.0”
    }
}

and hit :wq

It will just ensure consul will listen to any available address . Other config files for consul can be put in this directory (consul.d).

Let’s start consul and verify it is working:

sudo systemctl daemon-reload
sudo systemctl start consul
sudo systemctl status consul

Your results should show consul is active(running)

Active (Running) Consul

Let’s ensure consul starts if the server is rebooted:

sudo systemctl enable consul

We can also monitor consul log entries:

sudo journalctl -f -u consul

Now our backend storage is ready and we can move to the next step.

Step 3 — Obtain a free TLS/SSL to encrypt Vault

Although you can use Vault without an encrypted channel in your development setting, you should not do so in your production environment. We are going to obtain a certificate from an open certificate authority (CA) called Let’s Encrypt. You can find more details about them here.

First let’s run an update to make sure our server is up to date with the latest packages:

sudo apt-get update

next we’ll install software-properties-common and add universe repo(This is for ubuntu only, if you are running centos or other OS that is using yum package manager, skip this step):

sudo apt-get install software-properties-common
sudo add-apt-repository universe
sudo apt-get update

Now let’s install certbot:

sudo apt-get install certbot

press Y to continue the installation.

Next step we are going to generate the certificate for our domain. Make sure to create an A record in your registrar with the public IP address as the value for the domain or subdomain you are using. Keep in mind that the below command might fail before the registrar record gets updated, so try after a short while and you will see the command will succeed:

Google Domain Example

AWS R53 Example

sudo certbot certonly — standalone -d <YOUR DOMIN>

Make sure to enter a valid email so you can get your renewal notification and other security notices, then press A if you agree on Let’s Encrypt Terms of service. They will also ask if they can send you news and updates about the organization, for me it’s a No.

Successful registration

The path to your cert will be in /etc/letsencrypt/live/DOMAIN path

sudo ls /etc/letsencrypt/live/DOMAIN

You should see the following files:

DOMAIN directory contents

In the future, if your certificate expired, you can easily renew it as follows:

sudo certbot renew

If you put a valid email above in the creation process, you’ll get an email reminding you to renew your certificate before it expires and prevent service interruption.

Step 4 — Install Vault

Finally, we are ready now to start working on Vault.

Let’s download it from the official website, again choose based on your server environment. I will use the latest linux version available by the time this article written (Vault 1.7.0):

wget https://releases.hashicorp.com/vault/1.7.0-rc1/vault_1.7.0-rc1_linux_amd64.zip

unzip the downloaded file (if you downloaded a different version please update the bellow code based on the downloaded file name)

unzip vault_1.7.0-rc1_linux_amd64.zip

Now if you run an ls command, you will see a file named vault in the listed results:

ls

Let’s move the file to the bin folder so we can run vault from anywhere. If you are root, you can remove sudo:

sudo mv vault /usr/bin/

To confirm consul works, let’s execute the following command

vault version

the results should show the current version installed as shown below

Vault current version

Step 5 — Configure Vault

Create a vault folder under /etc where we gonna create our vault config file:

sudo mkdir /etc/vault
sudo vim /etc/vault/config.hcl

copy and paste the below snippet:

storage “consul” {
address = “127.0.0.1:8500”
path = “vault/”
}
listener “tcp” {
address = “0.0.0.0:443”
tls_disable = 0
tls_cert_file = “/etc/letsencrypt/live/<DOMAIN>/fullchain.pem”
tls_key_file = “/etc/letsencrypt/live/<DOMAIN>/privkey.pem”
}
ui = true

Replace the DOMAIN with your domain name, make sure the file paths are correct. Press :wq to save the file.

The above file is written in hcl, Hashicorp configuration language. If you need more details about it, click here.

Note: You can use any port in the address and not necessarily 443. the address 0.0.0.0 used so it can call any IP.

Now let’s create vault.service file:

sudo vim /etc/systemd/system/vault.service

copy the bellow code in it:

[Unit]
Description=Vault
Documentation=https://www.vault.io/
[Service]
ExecStart=/usr/bin/vault server -config=/etc/vault/config.hcl
ExecReload=/bin/kill -HUP $MAINPID
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

and press :wq to save it. Make sure your path in the ExecStart matches your config file path.

Let’s set VAULT_ADDR value to our domain:

sudo systemctl daemon-reload
export VAULT_ADDR="https://DOMAIN:443"
echo "export VAULT_ADDR="https://DOMAIN:443" >> ~/.bashrc"

Now let’s run vault and check if our configuration is successful:

sudo systemctl start vault
sudo systemctl enable vault
sudo systemctl status vault

sudo systemctl status vault

Now go to your website (https://domain) and check the User Interface of Vault, the one we enabled by setting ui=true in our config.hcl file:

Not initialized Vault

Step 6 — Initialize and unseal Vault

Let’s install vault autocomplete to make our lives easier when running Vault commands:

sudo vault -autocomplete-install
complete -C /usr/bin/vault vault

Next we are going to initialize our Vault server:

vault operator init

You will get 5 unseal keys and an initial root token to unseal your vault:

Your console will show like this:

Initialized, unsealed Vault console

Now use any 3 out of the 5 keys provided above to unseal your Vault:

Unseal Key 1 provided

Unseal Key 2 provided

Unseal Key 3 provided

Now your Vault is unsealed, copy and paste your initial root key and sign in using the Token method

Vault Console

Conclusion

Congratulations! You are now running a secured, production ready Vault node. Though the process was little bit long, but you can now use Vault and start managing your secrets and keys in a secured environment.

If you need the commands used in this article you can get it from this repo.