Tuesday, January 15, 2019

UX as part of your Enterprise Architecture

Digitalization within enterprises is still growing rapidly, enterprises are more and more adopting digitalization in every aspect of the daily processes and are moving to more intelligent and integrated systems. Even though a lot of work is being done in the backend systems and a lot of systems are developed and modernized to work in the new digital era a large part of the work has to do with UX User experience.

A large number of enterprises are still lacking in building a good and unified user experience for internal users. It has been thought for long that user experience was more applicable for the external systems such as websites, webshops and mobile applications. It is however evenly important to have a good and clear view on the internal user experience.

Internal user experience
Internal users, your employees, will use the systems developed on a daily basis. Ensuring the systems are simple to use, do what they promise and provide an intuitive experience will add to the productivity. Additionally, ensuring that systems are easy to work with and provide a good experience will ensure that your employees are more motivated and adoption of new systems will be higher

UX as an enterprise architecture component
In the past, it was common that every system within an enterprise would have a different experience. Menu structures, screen structures and the way a system behaved was different per application. As an employee normally interacts with multiple systems this can become overwhelming and complex. Additionally, it is relatively common that all internal enterprise user experiences are, to put it mildly, not that good. Most common, every system has a suboptimal interface and an interface design which is different from the rest.

An advised solution is to include standards for UX and interface design into the Enterprise Architecture repository and ensure, depending on your enterprise size, you have dedicated people to support developers and teams to include your enterprise UX blueprints within the internal applications.

When UX and interface design is a part of the enterprise architecture standards and you ensure all applications adhere to the standards the application landscape will start to become uniform. The additional advantage is that you can have a dedicated group of people who build UX components such as stylesheets, icons, fonts, javascripts and other components to be easily adopted and included by application development teams. At the same go, if you have dependency management done correctly, a change to a central UX component will automatically be adopted by all applications.

Having a Unified Enterprise UX is, from a user experience and adoption point of view one of the most important parts to ensure your digital strategy will succeed. 

Add UX consultants to your team
Not every developer is a UX consultant and not every UX consultant is a developer. Ensuring that your enterprise has a good UX team or a least a good UX consultant to support development teams can be of a large advantage. As per Paul Boag the eight biggest advantages of a UX consultant for your company are the following:
  1. UX Consultants Help Better Understand Customers
  2. UX Consultants Audit Websites
  3. UX Consultants Prototype and Test Better Experiences
  4. UX Consultants Will Establish Your Strategy
  5. UX Consultants Help Implement Change
  6. UX Consultants Educate and Inspire Colleagues
  7. UX Consultants Create Design Systems
  8. UX Consultants Will Help Incrementally Improve the Experience

Adopt a UX template
Building a UX strategy from scratch is complex and costly. A common seen approach for enterprises is that they adopt a template and strategy and use this as the foundation for their enterprise specific UX strategy.

As an example of enterprise UI and UX design, Oracle provides Alta UI which is a true enterprise grade user experience which you can adopt as part of your own enterprise UI and UX strategy. An example is shown below:

The benefit of adopting a UX strategy is that, when selected a mature implementation, a lot of the work is already done for you and as an enterprise you can benefit from a well thought through design. Style guides and other components are ready to be adopted and will not require a lot of customizations to be used within your enterprise so you can ensure all your applications have the same design and the same user experience. 

The above shown presentation from Andrejus Baranovskis showcases Oracle Alta UI Patterns for Enterprise Applications and Responsive UI Support

Monday, January 07, 2019

The adoption of chatbots in the enterprise market

We have seen the rise of chatbots in the past couple of years, more and more customer facing websites do implement a chatbot as part of the customer experience. Even though most people have had a negative experiences with chatbots the way they work is improving rapidly. Where chatbots used to be clumsy and not really good this is rapidly changing. The AI models behind chatbots is improving rapidly and they become more and more "human". As the maturity of chatbots is growing we see a growing adoption with chatbots by enterprises for both customer facing as well as internal facing communication.

As part of a Forbes article on the digital transformation trends in 2019 Chatbots have been placed on second place in the list.
  1. Chatbots Good to Great: Hear me out on this one. I know we’ve all had extremely frustrating chatbot experiences as we round out 2018. But the good news is that huge steps continue to be made in the way of natural language processing and sentiment analytics—so many, in fact, that some believe NLP will shake up the entire service industry in ways we’ve never imagined. Think about all the services that could be provided without humans—fast food lines, loan processors, job recruiters! What’s more, NLP allows companies to gather insights and improve their service based on them. Some 40% of large businesses have or will adopt it by the end of 2019—which makes it one of our top 2019 digital transformation trends. Now, I know many are alarmed by where AI and Chatbots may impact the workforce, but I’m also bullish that companies are going to be upskilling their work forces rather than displacing them as machines may be good at delivering on clearcut requests but leave a lot to be desired when it comes to dealing with empathy and human emotion required to deliver great customer experiences."
Introducing a chatbot in the organisation
Enterprises in general are implementing chatbots for two main reasons; improving the efficiency to communicate with customers and improving internal processes. A commonly seen model is that enterprises take a two phase approach to introducing chatbots to the business.

Phase 1 - Internal use
In phase 1 chatbots are implemented and used to optimize internal processes. for example standard internal HR processes, supporting internal requisitions and internal IT support are commonly seen as first adopters of a internal enterprise chatbot.

Phase 2 - External use
In phase 2 chatbots are used externally facing as part of the enterprise website, shopping site or as part of enterprise mobile applications.

In general phase 1 and phase 2 overlap, while the go-live of phase 1 is in effect phase 2 is already being prepared for external use. By creating the correct overlap the momentum of the chatbot team is maintained and the lessons learned from phase 1 are included in phase 2. It is important from both a team velocity as well as an adoption point of view to ensure you keep the momentum and ensure an overlap or a minimal gap between phase 1 and phase 2.

It is not done in a day
Contradicting the popular believe that building and implementing a chatbot is an easy task one will have to prepare for a "real project". Even though the use of a cloud platform and chatbot framework can speed the technical implementation up extremely a healthy part of the work is in ensuring your chatbot has the correct vocabulary and ensuring your conversation design is properly done.

Two aspects are important when developing your chatbot project planning. The first is to ensure you enough space for conversation design and ensuring the right vocabulary. Conversation design will go into design of how a flow of a conversation between your bot and a human will go. Even though this might sound straightforward initially it might be a very good practice to ensure you have an experienced conversation design expert on your team.

The other important part is to include a marity model for your chatbot in your project planning and strategy. The moment you want to launch internal and the moment you want to launch externally might be on a different point in the maturity model. An example of a chatbot maturity model, developed by Leon Smiers at Capgemini, can be seen below.

Use a chatbot framework
Building a chatbot from the ground up, building all the AI and all the other parts needed to make a good chatbot is an amazing project. However, such a project is only interesting from a technical understanding and research point of view and not so much from a business point of view. As a developer who just wants to build and include a chatbot interaction it is a better solution to leverage an existing platform. As an Example, Oracle provides a intelligent chatbot platform.

The below developer conference video showcases how to build a chatbot.

You can find more information and developer code examples via this link to get started quickly with your first intelligent chatbot to include in your enterprise landscape. 

Wednesday, December 12, 2018

Oracle CX - act upon a negative customer experience

Customer experience is becoming more and more important both in B2B as well as in B2C. When a customer is having a bad experience with your brand and/or the service you are providing the changes that this customer will purchase something or will be a returning customer in the future is becoming very unlikely. Additionally, if the customer is having a good customer experience however is lacking the emotional binding with your brand or product the changes that he will become a brand advocate is becoming unlikely.

The challenge companies are facing is that it becomes more and more important to ensure the entire customer experience and the customer journey is perfect and at the same time an emotional binding is created. As a large part of the customer journey is being moved to the digital world a good digital customer journey is becoming vital for ensuring success for your brand.

As more and more companies invest heavily in the digital customer journey it is, by far, not enough to ensure you have a good working and attractive looking online shopping experience.  To succeed one will have to go a step further than the competition and ensure that even a negative experience can be turned into a positive experience.

The negative experience
The below diagram from koobr.com showcases a customer journey which has a negative experience in it. Is shows that the customer wanted to purchase an item and found that the item was not in stock.

This provides two challenges; the first challenge is how to ensure the customer will not purchase the required items somewhere else and the second challenge is how to ensure we turn a negative experience into a positive experience.

Turning things positive
In the example for koobr.com a number of actions are taken on the item not in stock issue.

  • Company sends offer for their website
  • Company emails when item is in stock
  • Company tweets when item is in stock

This all takes the assumption that we know who the customer is or that we can get the user to reveal who he is. In case we do not know the customer, we can display a message stating that the customer can register for an alert when the items is in stock and as soon as the item is in stock a discount will be given. The promise for a discount on the item in the future also helps to make sure the customer will not purchase somewhere else.

Making a connection
The way you can contact the customer when the item is back in stock depends on the fact if we know who the customer is and which contact details we have from this customer. If we assume that we know who this customer is we can provide a discount specific for this customer only or provide another benefit.

The default way of connecting with a customer in a one on one manner is sending out an email to the mail address we have registered for this customer. A lot of other methods are however available and depending on the geographical location and demographic parameters better options can be selected.

As an example;

  • A teenage girl might be more triggered if we send her a private message via Facebook messenger.
  • A young adult male in Europe might be more triggered if we send a private message via WhatsApp.
  • A young adult female in Asia might be more triggered if we use WeChat
  • A Canadian male might want to receive an email as a trigger to be informed about an item that is back in stock
  • A senior citizen might be more attracted if a phone call is made to inform him that the item is back in stock. 

Only depending on email and a generic tweet on twitter will provide some conversion however much less conversion than might be achieved when taking into account more demographic parameters and multiple channels.

Keep learning
One of the most important parts of a strategy as outlined above is that you ensure your company keeps learning and ensures that every action as well as the resulting reaction are captured. In this case, no reaction is also an action. Combining constant monitoring of every action and reaction and a growing profile of your individual customer as well as the entire customer base provides the dataset upon which you can define the best action to counteract a negative experience as well as ensuring a growing emotional bonding between your customer and your brand.

Integrate everything
When building a strategy like this it needs to be supported by a technology stack. The biggest mistake a company can make is building a solution for this strategy in isolation and have a new data silo. Customers are not interested in which department handles which part of the customer journey, the outside view is that of the company as one entity and not as a collection of departments.

Ensuring that your marketing department, sales department, aftercare department, web-care department and even your logistical department and financial department make use of a single set of data and add new information to this dataset is crucial.

To ensure this the strategy needs to make use of an integrated solution, an example of such an integrated solution is the Oracle Cloud stack where for example the Oracle Customer experience social cloud solution is fully integrated with Oracle marketing, services, sales and commerce.

Even though this might be the ideal situation and provides a very good solution for a greenfield implementation a lot of companies will not start in a greenfield, they will adopt a strategy like this in an already existing ecosystem of different applications and data stores.

This means that breaking down data silos within your existing ecosystem and ensuring that they provide a unified view of your customer and all actions directly and indirectly related to the customer experience is vital.

In conclusion
Creating a good customer experience for your customers and building an emotional relationship between customer and brand is vital. Nurturing this is very much depending on the demographical parameters for each individual customer and a good customer experience as well as building a relationship requires having all data available and capturing every event.

Adopting a winning strategy will involve more than selecting a tool, it will require identifying all available data, all data that can potentially be captured and ensuring it is generally available to select the best possible action.

Implementing a full end to end strategy will be a company wide effort and will involve all business departments as well as the IT department. 

Tuesday, December 11, 2018

Oracle DEV – Automated testing with Selenium

When developing a solution, and more specific a web application in this example, part of your CI/CD process should be automated testing. Ensuring automated testing will save a lot of time and money and will support a fail fast principle where developers are made aware of issues in the application in a very early stage of the development process.

Part of a fail fast strategy is developing in a business driven development or test driven development manner and ensure that developers will develop automated tests to validate every part of the application. The growing set of tests can be executed every time the CI/CD automation triggers the automated testing.

Selenium for testing
One of the test automation tools commonly used is Selenium. Selenium is capable of running a web browser and interact as a user would interact with the system while checking every assertion defined in the test code.

Selenium example with XML output
The below example showcases a very small testcase which will execute two tests and which has been tested using a Oracle Linux instance to run the Selenium code. The difference between a standard selenium test and the below example is that it ensures that the test report is generated as XML and is stored in a default location.

Storing the test reports in XML will enable you to combine and report multiple testcases into a single test report while building individual tests instead of one large testcase. Selenium supports multiple development languages to define your testcases, the below example is a Python based testcase.

import unittest
import xmlrunner
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

# set some generic variables used within the wider test scripting
geckodriver = '/usr/local/lib/selenium/drivers/geckodriver'
options = webdriver.FirefoxOptions()

class PythonOrgSearch(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Firefox(executable_path=geckodriver, firefox_options=options)

    def test_search_in_python_org(self):
        driver = self.driver
        self.assertIn("Python", driver.title)
        elem = driver.find_element_by_name("q")
        assert "No results found." in driver.page_source

    def testCaseFindTitle(self):
        driver = self.driver
        self.assertIn("Python", driver.title)
        elem = driver.find_element_by_name("q")
        assert "No results found." not in driver.page_source

    def tearDown(self):

if __name__ == "__main__":
        failfast=False, buffer=False, catchbreak=False)

Executing the test
If you execute the above test you will notice that one case will fail and one will succeed (at this very moment). A standard execution looks like the example below:

[root@testnode12]# python 7seltest.py 

Running tests...
ERROR [15.376s]: test_search_in_python_org (__main__.PythonOrgSearch)
Traceback (most recent call last):
  File "7seltest.py", line 23, in test_search_in_python_org
    assert "No results found." in driver.page_source

Ran 2 tests in 27.462s

FAILED (errors=1)

Generating XML reports...

As an addition you will find the XML report in the ./test-reports directory for future references and to enable you to parse the individual XML reports into a single report. 

Oracle Cloud – add new nodes to your loadbalancer automatically

Services that requiring you to balance requests over multiple backend services are very common. Moving away from monolith based applications and building more smaller components increases the need for good load balancing. Adding to this that within a cloud environment the number of instances can scale up and down whenever required makes it a requirement for a loadbalancer to quickly adopt to the scaling up and scaling down of nodes.

When you design your solution right the configuration of your loadbalancer should automatically adopt changes in your landscape. As an example, if a new instance of a service is created it should automatically result in the fact that this node is added to the loadbalancer configuration. Additionally, when removing an instance this should result in the fact that this instance is removed from the loadbalancer in a graceful manner.

Oracle Cloud Load balancers
As part of the cloud offerings Oracle provides a load balancing service. When deploying your cloud based applications you can leverage the Oracle Cloud load balancing service and ensure that it spans multiple availability domains.

The below diagram showcases a simple deployment with a high available load balancer service available in two availability domains which will balance the load over six machines who are spread over three availability domains

The above diagram showcases a simple implementation of the Oracle load balancer service in the Oracle cloud. You can use this blueprint as a starting point for building your more complex and sophisticated deployments for enterprise deployments in the Oracle cloud. 

Design for automation
The intention of your designs and architecture should be to support full automation of the loadbalancing process. The processes of adding and removing an instance of your backend services should never result in manual actions which need to be performed by a human. 

Taking this into account when you design your solution will give you a good starting point to ensure your solution is elastic and capable of reacting to changes in the landscape without any additional effort. 

When building the logic for your bootstrapping to ensure automated registration and de-registration of your backend nodes you can leverage the APIs from the Oracle Load Balancer. The two main choices you have are using either the REST APIs or the SDK to call out to the loadbalancer from your bootstrap logic. 

The REST APIs provide you an easy way to work with the loadbalancer in a programmatic manner, this will enable you to call the API endpoints with any programming language you like. As an example, you could use bash scripting under Oracle Linux for your bootstrap process and call out to the API endpoints to register and de-register your nodes. More information on the APIs can be found at this page

The SDK(s) for Oracle Cloud infrastructure, including the Oracle Cloud loadbalancer are provided in a number of programming languages such as Java, Go, Python and Ruby. As an example you can read the documentation of the Python SDK used to work with the Oracle Cloud Loadbalancer at this page

Bootstrap your services
A big part of ensuring that your solution is capable of automatically register and de-register instances at the Oracle Cloud Load Balancer is ensuring the bootstrapping of your instance is done right. This is especially of importance when you need to balance load over custom components that run on a virtual environment in the Oracle Compute Cloud. 

The importance in this is to ensure your services, virtual machine images, have a good bootstrapping. The bootstrapping should ensure both the registration and de-registration of an instance with the loadbalancer. 

The importance in registration and de-registration is that you take the service into account and not the running operating system. This means that part of your bootstrap will require checking if the service itself is up and running, only after the service is available you can register the service at the loadbalancer. Having the operating system up and running and not the service will result in requests being routed to the new node while it is not able to serve requests yet.

Design concept positioning
The concept of adding and removing instances automatically to your load balancer can be named as “Automatic load balancing registration” and could form a part of your enterprise architecture blueprint library. Including the concept in your enterprise architecture library and ensuring all your deployments are done in this manner will ensure a mature and unified way of working across all your solutions. 

Tuesday, November 27, 2018

Oracle Development – Proof your bug first

When building applications, or more specific, when resolving bugs in your application the initial reaction of a lot of developer is to resolve the bug first thing. Another approach is to first proof your bug is indeed a tangible bug by writing a test script to proof it. Even though you might be natural inclined to quickly resolve the bug a lot of value can be found in proving it first.

By applying a test-driven development approach the first thing you will do is write a short test script that will inform you that the bug is present. After that you can write the fix for the bug and re-run the test script. Now, with the bug resolved, the test script should inform you that everything is working as expected.

The value in in this approach is that, with every bug you resolve, you will add to the set of tests for your application. When done right you should have an automated test step in your CI/CD pipeline. This means that every time you build, test and deploy a new release of your application the application will be tested for every bug ever found in your application and make sure that it has not occurred again.

Test new functions
Implementing an automated testing strategy in combination with test driven development is not only applicable when resolving bugs, it should also be part of your new functionality development cycle. Every time you develop a new functionality it should be accompanied with a test script.

The test stage in your pipeline
As stated, when you have implemented your CI/CD strategy correctly and made sure that you have a test step in your pipeline every execution of your pipeline should result in all tests being executed automatically.

When the execution of tests is a manual process by developers or even worse a process that needs to be requested with a specific test team the changes are high testing is happening less frequently than it should be.

By ensuring that testing is an automated part and your CI/CD pipeline enforces the execution fo all tests makes sure that you know that your application is tested over and over again to make sure all functionality is available and that bugs that have been resolved in the past do not surface again.

Testing your web application with Oracle Linux
If you develop a web application the use of tools like Selenium can be extremely valuable.  Selenium allows you to run tests fully automatically and has support for multiple browsers.

When writing your selenium test it is good to know that Selenium is fully working on Oracle Linux and that you can use this in a headless mode. The benefit of the headless mode is that you can run virtual Oracle Linux machines and have them execute the Selenium tests without the need to have a graphical user interface running within your Oracle Linux instance.

The below code snippet shows a test case for Selenium written in Python capable of running with FireFox on an Oracle Linux instance without the need to have a graphical user interface.

options = webdriver.FirefoxOptions()

One of the models to run your Selenium test quickly is to use an Oracle Linux based Docker container and have the test running within the context of this container. There will be no need for any graphical user interface and due to the way the Oracle Linux base image for Docker containers is provided the size of your Docker image will be minimal.

Oracle Linux based container testing
As part of your strategy to run your Selenium tests in a headless Oracle Linux Docker container you could consider the following;

Develop a simple Selenium image based upon the Oracle Linux Docker base image. The simple Selenium image should be able to download Selenium tests from a central repository and execute them. This means that you can keep your Selenium image relative simple and you will not have a need to rebuild the image every time you add a test to your test set.

Every time your pipeline will invoke the testing stage you Oracle Linux based container will download the latest version of a test as part of the bootstrap process.

One thing to remember is that you need to ensure that your container will report the test results to a central location to ensure you will have all the test results after the container has stopped.

Friday, November 02, 2018

Why innovators need to eat frogs

Innovation is a long process of tackling very complex problems and often problems nobody has been trying to tackle before. Throughout my career I have been working at a number of hard innovation projects, building IOT like solutions way before IOT was a known phrase and developing cryptographic solutions to allow secure communication with chips on government ID cards. In all those cases I followed the mantra of Google X without knowing it, even before Google was a real company and Google X existed.

The mantra at Google X, the moonshot department of Google where people tackle really hard problems is #MonkyFirst. The idea behind this is that if you want a monkey to recite Shakespeare on a pedestal you do not start with building the pedestal. Everyone can build a pedestal; a lot of people have been building a pedestal before Training the monkey is the hard part. If you are unable to train the monkey to recite Shakespeare there is no need to build the pedestal at all.

In short, try to tackle the hard problem first before you spend time and money on tackling the parts of your project that you know will not be that hard.

The mantra #MonkeyFirst is also stated by Mark Twain in a bit of a different form while having the same meaning. Mark Twain wrote; “eat a live frog first thing in the morning and nothing worse will happen to you for the rest of the day”.

Companies start to eat frogs
In companies around the world the common thing to do is to build the pedestal first and after that try to train the monkey. One of the reasons for it is that people tend to desire quick satisfaction and within companies there is commonly a tendency that management want to see tangible results fast. Building the pedestal is something that can quickly be done and it will show results towards management. Showing what you have accomplished is a more convenient message to tell than providing a long list on why it is so hard to train a monkey.

However, if you are unable to train the monkey there is no reason to build a pedestal. Eating the frog or #MonkeyFirst ties a bit into the “fail fast, fail early, fail cheap" concept.

As resources can only be spend once it is in the best interest of a company to make sure that you fail early in a project. If it turns out that you are unable to train the monkey before you spend resources on building a, then useless, pedestal you have saved burning resources without getting a usable output from it.

Say no to your inner self
It is a natural thing to try to get instant satisfaction, it is a natural thing to build the pedestal first and see what you have achieved. However, it is a wiser thing to try and train the monkey first. On a personal level it is difficult to say no to the natural tendency. Within an enterprise it is equally hard to change the mindset to aim for instant satisfaction. Changing the mindset within an enterprise might even be harder than changing your own inner mindset.

However, changing to a culture of eating frogs is very beneficial for enterprises how strive for innovation. Eating frogs will save valuable time and money and ensures that the focus is on projects that have a higher rate of providing a success. 

Oracle dev – microservice secure bootstrapping for shared secrets

When building microservices, to be more precise when building microservices in containers at one point in time you will start hitting the problem of secure bootstrapping and the handling of shared secrets. This blogpost aims to provide some insights into possible design patterns for handling this problem.
Problem outline

When building microservices or functions which will run from a container you want to provide them as little configuration as possible while at the same time you want them to allow to be configured dynamically at the time of startup. To make the problem a bit more interesting, you want to be able to scale up and down containers dynamically. This problem is directly visible when you are building functions in, for example, Oracle project Fn. A function is in effect a docker container which will only live for a very short amount of time.
We did mention configuration we should actually split configuration into two parts; one part is actual configuration and one part is secrets. Configuration is stored in a central configuration store (for example consul from Hashicorp) and secrets are stored in a secret management solution (for example vault from Hashicorp). Making a distinct split between secrets and configuration is a best practice.

The simple bootstrap
Let’s say that you need to build a microservice which needs to call another service within your landscape you might potentially need the following two things to allow your microservice to call the other microservice; (1) a URL and (2) a shared secret for authentication.

A simple way to resolve this is to ensure your container or function will have a bootstrap routine which will call the configuration store in a static manner to acquire the needed configuration and the shared secret from a configuration store. Static manner means that your microservice or function has the URL and possibly a key baked into the deployment which allows it to connect to the configuration store, this is shown in the example below.

The above works and already provide a relative better implementation than building in all configuration and keys needed for microservice 1 to communicate with microservice 2.

Bootstrap with service registry 
The below model is a bit more complex, however, it is also providing some more options. In the below diagram all steps are done over HTTPS, you can however implement step 1 also with a socket connect from your docker container or you can select an IP which is only available within the Docker internal network and is not accessible from outside.

 The reason you might want to secure the “service registry” service from the outside world is because it is key to gaining access to everything else.  In the above model the following steps are executed:

  1. The bootstrap of microservice-1 registers at the service registry and receives a key and URL for the configuration store
  2. The service registry informs the configuration store about the new service that has been booted and which random secret it will use to connect. 
  3. Microservice-1 connects to the configuration store while only providing the secret it received from the service registry. Based upon this secret the configuration store knows which configuration and secrets to return to the service. Additionally, it creates a random secret for this specific instance of microservice-1 to be used to communicate with microservice-2
  4. Microservice-1 calls microservice-2 while using the configuration and the key it receives from the configuration store.
  5. Microservice-2 receives a call from microservice-1 and verifies the secret with the configuration store. 

The benefit of this model is that only a service which is started within Docker can access the service registry and by enforcing this only a service starting within docker can acquire a key to communicate with the dynamic configuration store. 

Friday, October 05, 2018

Oracle Fn - build your first Go function

In this guide we showcase how easy it is to build a single function using Oracle Fn based upon Go.

The first step in developing a function is to initialize a function with the fn command. Below example shows we init a new function which will be using Go as the programming language and the trigger to call the function will be a http call. We call the function myCoolFunction

[root@projectfn devstuff]# fn init --runtime go --trigger http myCoolFunction
Creating function at: /myCoolFunction
Function boilerplate generated.
func.yaml created.
[root@projectfn devstuff]#

As soon as the initalization is complete we can see that we have a new directory called myCoolFunction which has a number of files inside
[root@projectfn devstuff]# ls -la myCoolFunction/
total 16
drwxr-xr-x. 2 root root  73 Oct  5 18:48 .
drwxr-xr-x. 8 root root  90 Oct  5 18:48 ..
-rw-r--r--. 1 root root 469 Oct  5 18:48 func.go
-rw-r--r--. 1 root root 193 Oct  5 18:48 func.yaml
-rw-r--r--. 1 root root 127 Oct  5 18:48 Gopkg.toml
-rw-r--r--. 1 root root 505 Oct  5 18:48 test.json
[root@projectfn devstuff]# 

If we look at the files the two most important files are the func.go and the func.yaml file. The func.go file will contain the function logic and the func.yaml will contain the configuration of the function.

If we look at the content of func.go we can see that this is a very simple hello world example written in Go which will respond in the form of a JSON response file " Hello World" or in case you provide a JSON payload file with a name it will respond with a "Hello name". In effect the example is very simple and very handy to quickly test if your function is OK and can be used before you start coding your own logic into it.
package main

import (

 fdk "github.com/fnproject/fdk-go"

func main() {

type Person struct {
 Name string `json:"name"`

func myHandler(ctx context.Context, in io.Reader, out io.Writer) {
 p := &Person{Name: "World"}
 msg := struct {
  Msg string `json:"message"`
  Msg: fmt.Sprintf("Hello %s", p.Name),

The content of the func.yaml file will help in the configuration of the function and how it is, for example, accessible externally on what endpoint.

schema_version: 20180708
name: mycoolfunction
version: 0.0.1
runtime: go
entrypoint: ./func
format: json
- name: mycoolfunction-trigger
  type: http

  Now we have to build and deploy the function. What will happen in the background is that a docker container is build and that the application, the function and the trigger is registered within Fn so it can be called. As we have not defined any application name we will call this application mycoolapp. The command required and the result is shown in the example below.

[root@projectfn myCoolFunction]# fn --verbose deploy --app mycoolapp --local
Deploying mycoolfunction to app: mycoolapp
Bumped to version 0.0.4
Building image mycoolfunction:0.0.4 
Current Context:  No context currently in use.
Sending build context to Docker daemon  6.144kB
Step 1/10 : FROM fnproject/go:dev as build-stage
 ---> fac877f7d14d
Step 2/10 : WORKDIR /function
 ---> Using cache
 ---> 910b06b938d1
Step 3/10 : RUN go get -u github.com/golang/dep/cmd/dep
 ---> Using cache
 ---> f6b396d6e1fa
Step 4/10 : ADD . /go/src/func/
 ---> 35a944c2ad0f
Step 5/10 : RUN cd /go/src/func/ && dep ensure
 ---> Running in 8ef4cfb23602
Removing intermediate container 8ef4cfb23602
 ---> 75991cccc0b0
Step 6/10 : RUN cd /go/src/func/ && go build -o func
 ---> Running in 5d38abb76d94
Removing intermediate container 5d38abb76d94
 ---> 87f20cf4d16d
Step 7/10 : FROM fnproject/go
 ---> 76aed4489768
Step 8/10 : WORKDIR /function
 ---> Using cache
 ---> 1629c0d58cc1
Step 9/10 : COPY --from=build-stage /go/src/func/func /function/
 ---> Using cache
 ---> ac97ccf6b37f
Step 10/10 : ENTRYPOINT ["./func"]
 ---> Using cache
 ---> 5c61704790e4
Successfully built 5c61704790e4
Successfully tagged mycoolfunction:0.0.4

Updating function mycoolfunction using image mycoolfunction:0.0.4...
In effect, this is the only thing you need to do to get your first function up and running. To test if it is really working we can call the function as shown below and we will get the result back as expected.
[root@projectfn myCoolFunction]# curl
{"message":"Hello World"}

Wednesday, September 12, 2018

Oracle Cloud - Elastic Search set number of replicas for shards

We see more and more that customers leverage Elastic Search within modern application deployments. Recently we have been experimenting with Elastic Search in the Oracle Cloud. The initial setup was a relative simple cluster setup of a number of nodes. When doing the first resilience test we found out that we where finding that we degraded the cluster state and lost data when turning off a virtual machine in the Oracle Compute Cloud. The reason for this was that we did not set the replication for shards in the correct manner. Ensuring you have set sharding correctly is vital to ensure your Elastic Search Cluster in the Oracle Cloud is resilient against node failure.

Building a cluster in the cloud
When building a Cluster in the Oracle Compute Cloud based upon Oracle Linux this is in effect nothing different from building an Elastic Cluster in any other cloud or in your own datacenter. This holds that the below is applicable to any installation you do.

The general idea when building an Elastic Search Cluster is that you ensure that you have multiple nodes with multiple roles working as one while ensuring that the cluster is capable of loosing a number of its member nodes and will still continue functioning. The main roles within the cluster configuration of Elastic Search are data nodes, master nodes and client nodes.

when configuring your cluster you have to ensure that you are capable of loosing one node and still be able to provide the services to your customers. One of the things we overlooked while building the initial cluster was the level of replica shards.

Use replica shards
When you store an index in Elastic Search this can be broken down into multiple shards. The shards can be distributed over multiple machines (nodes) in the cluster. The idea behind it is two folded. Firstly it helps you to distribute load and secondly it helps you to ensure data is on more than one node at the same time to ensure that the data is available when a node is removed from the cluster.

To ensure the optimal use for both distributing operations as well as ensuring you have a replica of a shard to mitigate against failure you will have to ensure you have a replicated shard. The below image shows this on a high level:

As you can see in the above example we have two shards for a single index, P0 and P1. However, we also have a replica shard for both R0 and R1. In case node 1 is failing for some reason the data is still available for the cluster in the form of shard R0. In this example we have a replication of 1, you can however set the replication much higher. The level of replication will depend on the level of risk you want to take combined with the level of compute distribution you want to achieve against the costs you are willing to have.

As storage is relative cheap in the Oracle Compute Cloud it is advisable to set the replication higher than 1 to ensure you have 2 or more replication shards.

Setting the replication level
You can set the replication level per index in Elastic Search. This will help you to ensure you have more shards for data which is accessed frequently or where you need a higher level of availability. Setting the replication level is done with the parameter number_of_replicas as shown below:

PUT /my_index/_settings
  "number_of_replicas": 1

using shard replication will help you protect against failure of a datanode in your elastic search cluster and it will improve the performance. As storage is relative cheap in the Oracle Cloud it is advisable to ensure you have set number_of_replicas to 2 or higher.

Monday, August 20, 2018

Oracle Jet - security by obfuscation - Do not use it

Obfuscation “the action of making something obscure, unclear, or unintelligible” is often used when trying to secure an application. The idea behind obfuscation is to make the technical working of an application so unclear to attackers that it will become very hard to figure out the actual working.  Even though obfuscation might look like a good measure it provides no real security against people who intend to understand the real working of an application. A good example of security by obfuscation is URL obfuscation.

When applying URL obfuscation as a security measure a common practice is to obfuscate the URL parameters in such a way that they are not easily understandable by someone who has not build the application.

A recent example I found is the below URL which ends with details?id=QEREUS where at first instance it appears that the ID is a randomly generated ID. When having a randomly generated ID this will make it harder to build a script which will access the details site. However, in this specific example the result of the URL is a JSON file, when examining the JSON file another ID was shown. In this case the ID in the JSON file was A01049 while the ID in the URL was QEREUS

By examining a number of different detail pages quickly a pattern emerges, for example, all URL IDs started with a Q and only used a specific subset of the alphabet. In this specific example the actual ID, shown in the JSON file, was an A followed by a sequential number where the developer has chosen to use a substitution method to obfuscate the actual ID. The below list shows the selected substitution method:

  • 0 - E
  • 1 - R
  • 2 - T
  • 3 - Y
  • 4 - U
  • 5 - I
  • 6 - O
  • 7 - P
  • 8 - A
  • 9 – S

When you take a good look at the letters and take a good look at a QWERTY keyboard layout you can figure out why the developer has selected this set of letters to substitute the numbers.

Why people use obfuscation. 
There are very good reasons why one would like to make it hard for externals to guess ID parameters. When you are able to guess the ID parameter it might become very easy to write a script to scrape a website for information. When looking at more modern web based applications, for example Oracle JET based applications, a javascript will call a REST API, this means that the user will be able to gain access to pure JSON (or XML) based information.

In essence there is nothing wrong with external people accessing this information however there might be very good reasons why you do not want to over-promote the use of the pure JSON data outside of the context of your Oracle JET based application.

A better way to do obfuscation
Besides implementing true randomization and more advanced security than obfuscation there are much better ways to do obfuscation in your URLs. An example would be for example simple encryption would work much (much much much) better. The below examples are rijndael-256 encrypted IDs with a base64 encoding

  • A01049 - fiCOcGED9SDiYe9du0XUIu1tsHQNGwWVK9uvI755+fg=
  • A01050 - cQbgwi1mnsYloonV4ZhqPMo3C2ie0+ilmsZFr3mBb3A= 
  • A01050 - U0xs5e4QP6OIclZBCSLuf34WFNRqs3lbtBgkmMiRvkc=

Opposed to the number substitution shown below for the same IDs:

  • A01049 - QEREUS
  • A01050 - QEREIE
  • A01051 - QEREIR 

Should you use obfuscation?
my personal opinion; no. Main reason for saying no is that it provides a false sense of security. It gives you the idea you are safe while someone who is determined will figure it out at one moment in time. There are better and more structural ways of doing things like preventing people from overly active scrape your website. However, if obfuscation is a part of a wider set of security implementations you should think about a very good way of doing obfuscation and not simply rely on a solution like substitution as it will take a very short amount of time for someone to figure out the substitution algorithm