Friday, November 02, 2018

Why innovators need to eat frogs

Innovation is a long process of tackling very complex problems and often problems nobody has been trying to tackle before. Throughout my career I have been working at a number of hard innovation projects, building IOT like solutions way before IOT was a known phrase and developing cryptographic solutions to allow secure communication with chips on government ID cards. In all those cases I followed the mantra of Google X without knowing it, even before Google was a real company and Google X existed.

The mantra at Google X, the moonshot department of Google where people tackle really hard problems is #MonkyFirst. The idea behind this is that if you want a monkey to recite Shakespeare on a pedestal you do not start with building the pedestal. Everyone can build a pedestal; a lot of people have been building a pedestal before Training the monkey is the hard part. If you are unable to train the monkey to recite Shakespeare there is no need to build the pedestal at all.

In short, try to tackle the hard problem first before you spend time and money on tackling the parts of your project that you know will not be that hard.

The mantra #MonkeyFirst is also stated by Mark Twain in a bit of a different form while having the same meaning. Mark Twain wrote; “eat a live frog first thing in the morning and nothing worse will happen to you for the rest of the day”.

Companies start to eat frogs
In companies around the world the common thing to do is to build the pedestal first and after that try to train the monkey. One of the reasons for it is that people tend to desire quick satisfaction and within companies there is commonly a tendency that management want to see tangible results fast. Building the pedestal is something that can quickly be done and it will show results towards management. Showing what you have accomplished is a more convenient message to tell than providing a long list on why it is so hard to train a monkey.

However, if you are unable to train the monkey there is no reason to build a pedestal. Eating the frog or #MonkeyFirst ties a bit into the “fail fast, fail early, fail cheap" concept.


As resources can only be spend once it is in the best interest of a company to make sure that you fail early in a project. If it turns out that you are unable to train the monkey before you spend resources on building a, then useless, pedestal you have saved burning resources without getting a usable output from it.

Say no to your inner self
It is a natural thing to try to get instant satisfaction, it is a natural thing to build the pedestal first and see what you have achieved. However, it is a wiser thing to try and train the monkey first. On a personal level it is difficult to say no to the natural tendency. Within an enterprise it is equally hard to change the mindset to aim for instant satisfaction. Changing the mindset within an enterprise might even be harder than changing your own inner mindset.

However, changing to a culture of eating frogs is very beneficial for enterprises how strive for innovation. Eating frogs will save valuable time and money and ensures that the focus is on projects that have a higher rate of providing a success. 

Oracle dev – microservice secure bootstrapping for shared secrets

When building microservices, to be more precise when building microservices in containers at one point in time you will start hitting the problem of secure bootstrapping and the handling of shared secrets. This blogpost aims to provide some insights into possible design patterns for handling this problem.
Problem outline

When building microservices or functions which will run from a container you want to provide them as little configuration as possible while at the same time you want them to allow to be configured dynamically at the time of startup. To make the problem a bit more interesting, you want to be able to scale up and down containers dynamically. This problem is directly visible when you are building functions in, for example, Oracle project Fn. A function is in effect a docker container which will only live for a very short amount of time.
s
We did mention configuration we should actually split configuration into two parts; one part is actual configuration and one part is secrets. Configuration is stored in a central configuration store (for example consul from Hashicorp) and secrets are stored in a secret management solution (for example vault from Hashicorp). Making a distinct split between secrets and configuration is a best practice.

The simple bootstrap
Let’s say that you need to build a microservice which needs to call another service within your landscape you might potentially need the following two things to allow your microservice to call the other microservice; (1) a URL and (2) a shared secret for authentication.


A simple way to resolve this is to ensure your container or function will have a bootstrap routine which will call the configuration store in a static manner to acquire the needed configuration and the shared secret from a configuration store. Static manner means that your microservice or function has the URL and possibly a key baked into the deployment which allows it to connect to the configuration store, this is shown in the example below.



The above works and already provide a relative better implementation than building in all configuration and keys needed for microservice 1 to communicate with microservice 2.

Bootstrap with service registry 
The below model is a bit more complex, however, it is also providing some more options. In the below diagram all steps are done over HTTPS, you can however implement step 1 also with a socket connect from your docker container or you can select an IP which is only available within the Docker internal network and is not accessible from outside.


 The reason you might want to secure the “service registry” service from the outside world is because it is key to gaining access to everything else.  In the above model the following steps are executed:

  1. The bootstrap of microservice-1 registers at the service registry and receives a key and URL for the configuration store
  2. The service registry informs the configuration store about the new service that has been booted and which random secret it will use to connect. 
  3. Microservice-1 connects to the configuration store while only providing the secret it received from the service registry. Based upon this secret the configuration store knows which configuration and secrets to return to the service. Additionally, it creates a random secret for this specific instance of microservice-1 to be used to communicate with microservice-2
  4. Microservice-1 calls microservice-2 while using the configuration and the key it receives from the configuration store.
  5. Microservice-2 receives a call from microservice-1 and verifies the secret with the configuration store. 

The benefit of this model is that only a service which is started within Docker can access the service registry and by enforcing this only a service starting within docker can acquire a key to communicate with the dynamic configuration store.