Oracle dev – microservice secure bootstrapping for shared secrets

When building microservices, to be more precise when building microservices in containers at one point in time you will start hitting the problem of secure bootstrapping and the handling of shared secrets. This blogpost aims to provide some insights into possible design patterns for handling this problem.
Problem outline

When building microservices or functions which will run from a container you want to provide them as little configuration as possible while at the same time you want them to allow to be configured dynamically at the time of startup. To make the problem a bit more interesting, you want to be able to scale up and down containers dynamically. This problem is directly visible when you are building functions in, for example, Oracle project Fn. A function is in effect a docker container which will only live for a very short amount of time.
s
We did mention configuration we should actually split configuration into two parts; one part is actual configuration and one part is secrets. Configuration is stored in a central configuration store (for example consul from Hashicorp) and secrets are stored in a secret management solution (for example vault from Hashicorp). Making a distinct split between secrets and configuration is a best practice.

The simple bootstrap
Let’s say that you need to build a microservice which needs to call another service within your landscape you might potentially need the following two things to allow your microservice to call the other microservice; (1) a URL and (2) a shared secret for authentication.

A simple way to resolve this is to ensure your container or function will have a bootstrap routine which will call the configuration store in a static manner to acquire the needed configuration and the shared secret from a configuration store. Static manner means that your microservice or function has the URL and possibly a key baked into the deployment which allows it to connect to the configuration store, this is shown in the example below.

The above works and already provide a relative better implementation than building in all configuration and keys needed for microservice 1 to communicate with microservice 2.

Bootstrap with service registry 
The below model is a bit more complex, however, it is also providing some more options. In the below diagram all steps are done over HTTPS, you can however implement step 1 also with a socket connect from your docker container or you can select an IP which is only available within the Docker internal network and is not accessible from outside.

 The reason you might want to secure the “service registry” service from the outside world is because it is key to gaining access to everything else.  In the above model the following steps are executed:

1. The bootstrap of microservice-1 registers at the service registry and receives a key and URL for the configuration store
2. The service registry informs the configuration store about the new service that has been booted and which random secret it will use to connect. 
3. Microservice-1 connects to the configuration store while only providing the secret it received from the service registry. Based upon this secret the configuration store knows which configuration and secrets to return to the service. Additionally, it creates a random secret for this specific instance of microservice-1 to be used to communicate with microservice-2
4. Microservice-1 calls microservice-2 while using the configuration and the key it receives from the configuration store.
5. Microservice-2 receives a call from microservice-1 and verifies the secret with the configuration store. 

The benefit of this model is that only a service which is started within Docker can access the service registry and by enforcing this only a service starting within docker can acquire a key to communicate with the dynamic configuration store.