The personal view on the IT world of Johan Louwers, specially focusing on Oracle technology, Linux and UNIX technology, programming languages and all kinds of nice and cool things happening in the IT world.
When we look at the internet and then primarily at the world wide web part of the internet where webpages are offered to users we see there is a good level of interoperability and standards. Webpages are offered primarily on HTML and HTML and HTTP, originally invented by Tim Berners-Lee, are adopted by the majority of services offered online. The protocols lower in the stack like TCP/IP are well defined and clear to everyone and are the foundation that enables communication. when the first steps where taken to create those protocols the number of people involved where not that big and making sure new ideas where adopted where somewhat less troublesome. At this moment more and more people do have access to the internet and do have (some) level of understanding about how to develop new solutions.
When we look at the internet of things we see that a lot of people, organisations and companies are developing solutions that are added to the concept of the internet of things. Most of those solutions do make use of some of the existing standards. for example, most make use of TCP/IP and a large part of them makes use of the HTTP protocol to communicate data. What is missing however is a set of documented standards on what will be communicated and how we can enable interoperability between devices from different vendors.
Qualcomm is trying to make an attempt to start such a standard. Qualcomm developed the alljoyn solution and has now joined the allseen alliance. By doing so they have given over the alljoyn code to the Linux foundation in the hope that this will help fuel the adoption of a unified protocol for the internet of things.
The allseen alliance is currently rapidly growing and is certifying devices that are compliant to the standards of the alliance. members include:LG, Sharp, Haier, Panasonic, Sears Brand Management Corporation, Silicon Image, Cisco, TP-LINK, Canary, doubleTwist, Fon, Harman, HTC, Lifx, Liteon, Sproutling and Wilocit
For years the selection process of ERP systems has primarily been based upon the functionality the ERP system provides. Also the primary party in selecting the ERP system has been the IT department followed by the other departments. However we do see a changing trend. As rightly stated by the Wall Street Journal: "“Functionality is no longer the definition of success. Usability is key". In a recent Forbes article discussing the decision from Avon to stop the implementation of SAP in favor of Oracle JD Edwards it shows that ERP systems selection and implementation is getting more and more influenced by the Apple effect.
The apple effect is the consumerization of IT. Users do expect to have the same experience in the corporate applications as they experience in the consumer applications. This makes that users do expect to have the same kind of ease in using an application and the same user centered approach build into the core of an application and even in the complete portfolio of an IT department. This means that things like user interface design, gamification, agility and self-service solutions become more and more a central part of the corporate solutions and can no longer be seen as an addition or nice-to-have. At the same time users, now digital natives, do expect to be able to work everywhere, always and on every device. Something that is new to corporate IT however something that has to be adopted by IT departments.
Where IT departments where calling the shots previously and where banking on the fact that the user community was not aware about the possibilities this is now changing. With generations coming into the workplace who are used to work with computers and who are digital natives the non-IT departments are getting a stronger voice in the boardroom and are influencing the IT departments. This makes that IT is becoming, as it should, a service organisation to support day to day operations rather then a department that dictates how the business should work.
Combining the consumerization of IT and a business driven IT management strategy is providing IT organisations a stable solution for the future and this should be on the agenda of every CIO. However, as this is a fundamental change in the way IT departments think and operate this is a, clearly necessary however, slow and complex process.
Oracle is already securing its place in the cloud era for some time by providing a number of cloud based solutions as well as software and hardware which enables companies to build private (or hybrid) clouds within their own datacenters. One area however which has not been touched that much by Oracle is the Infrastructure as a Service area.
If you take a close look at the Oracle cloud strategy which has been forming in past couple of years and if you have been looking closely at the products in both hardware and software this should not come as a surprise to many. What you could seen is that Oracle has been working on a lot of solutions that are all part of a cloud / IaaS foundation. In recent years it has become clear that Oracle is using the public market as some sort of testing ground and have the product they put out in the market, end up up in the Oracle cloud solutions. Another giveaway could have been the fact that Oracle has started to embrace OpenStack which was a clear giveaway that IaaS was the way to go in the minds of the Oracle executives.
In the recent second quarter financials call from Oracle given by Larry Ellison the statement made was: "his company intends to be price competitive with established cloud infrastructure-as-a-service players like Amazon, Microsoft and Rackspace". Also stated by Larry was: "We intend to compete aggressively in the commodity infrastructure as a service marketplace". According to Oracle this should start unfolding in the first half year of 2014
This news, even though one could have expected it, will ensure that other IaaS competitors will have to start watching and battling Oracle in this field in which previously Oracle was not a player.
Already for some time virtual assistants are claiming a place in the market. According to Gartner a Virtual assistant is; “a conversational, computer-generated character that simulates a conversation to deliver voice- or text-based information to a user via a Web, kiosk or mobile interface. A VA incorporates natural-language processing, dialogue control, domain knowledge and a visual appearance (such as photos or animation) that changes according to the content and context of the dialogue. The primary interaction methods are text-to-text, text-to-speech, speech-to-text and speech-to-speech”
Digital assistance are playing more and more a role in the way humans interact with systems and the way, for example, customer care processes are shaped. They way people do think about how services should be provide is more and more on an always and everywhere presumption. People do have the need to be able to interact with a company at any moment in time and do expect to be helped directly. Companies who do offer poor quality of service for their customer care and customer support desks will find themselves moving down in the user satisfaction which will result directly to loss in revenue and harm to their brand. A number of options to implement are available; among them is a better way of interacting with the customer via a digital assistant who is present on your website.
Digital assistants have come a long way and due to the maturing of artificial intelligence the usability of digital assistance has grown over the years. By having a digital assistant on your website present you can lower the load on your human Servicedesk and provide customers with a faster way of finding services and answers to their questions online without the need for human interaction. A research paper from Nuance shows that 71% of the consumers prefer a web virtual assistant over a static web experience.
Virtual assistance can play a number of roles on your website all with the goal to enhance the customer experience and at the same time lower the load on your human Servicedesk engineers while at the same time also try to boost business by attracting new customers. This can be done in a number of ways. Traditionally the way of interacting primary with a human representative, more commonly nowadays is that a consumer first is looking at the information presented on the website and if they are unable to locate the correct information or are unable to find answers to their problems they will interact with the Servicedesk, via email or via phone. Depending on the content of your website and depending on the quality of this content and ease of finding you will lower the number of interactions between the Servicedesk and the consumer.
The inability of your customers to locate the information they need is having two negative effects, firs the customer is frustrated because he is not getting the online experience he would like to have, secondly the load to your Servicedesk is increasing due to the fact more customers have a need to interact with you human Servicedesk. Shown in the below diagram you can see the flow (1) customer searching for information on your website, (2) the customer was unable to locate the required information and contacts the Servicedesk and finally (3) the customer is directed by the Servicedesk to the website where the customer can find the correct information.
When you deploy a virtual assistant the changes that there was a need to interact with a human service desk would be much more limited. The digital assistant could take over the role from the human Servicedesk and first try to help the customer by applying artificial intelligence. This would result in the following; the customer visits the website and interacts with the virtual assistant, (2) the virtual assistant directs the customer to the correct location on the website where the information can be found. In case the customer is still not finding the answers he is looking for the digital assistant can establish a connection for the customer with a human Servicedesk representative.
Even though we still keep the option open for customers to interact directly or via the digital assistant with a human Servicedesk the number of requests that need to be handled by the human Servicedesk is, in general, dropping significantly.
Even thought the above method is adding a lot of benefit to the experience of the customer and is limiting the number of calls to the human service center it is considered a passive digital assistant. The reason it is passive is that the customer needs to take the first step to start interacting with the digital assistant. A new and more recent implementation for digital assistants is a more active digital assistant. In this deployment model the digital assistant is trying to start the interaction with the customer even before the customer has opened the website.
A general way of implementing this is related to the analysis of publicly available data which can for example reside on social networks. By analyzing social networks you can identify existing customers who state they have a question, remark, complaint or thought about one of the products or services you provide. Traditionally you would be unaware of this up until the moment the customer initiates the contact with your Servicedesk or with the passive virtual assistant. The same can be applicable for potential new customers, if a person who is not identified as a customer provides on a social network signals that he might be a potential new customer you can identify this and start interacting with this person.
Interaction commonly comes down to replying on the original message from the person on social media and directing this person to the digital assistant. Information that has been collected is already given to the digital assistant to ensure the conversation can be directly on topic. The below video is showing a good example of such an implementation.
When interacting with Oracle support it is quite common that you would like to, or asked to, upload files to the oracle support site. This can be for example be a logfile or the output from a diagnostic script. Generally most people download the files to their workstations and then upload it the the Oracle support site. however, in some cases you would like to be able to directly upload the file from the server to the Oracle support website.
You can now use cURL to upload files to transport.oracle.com. cURL is commonly available on most Linux and UNIX systems. "cURL is a computer software project providing a library and command-line tool for transferring data using various protocols. The cURL project produces two products, libcurl and cURL. It was first released in 1997."
To upload a file to a specific service request you can use the below command to do so:
curl –T –u “” ftps://transport.oracle.com/issue//
It is advised to not include the password in your curl command and you will be promoted for it. For more and detailed information you can refer to MOS document: 1547088.2
When using Oracle Enterprise Manager you have the option to store some information on the target in OEM as target properties. Up to recently I was under the assumption that this was a fixed list of target properties. One of the things I was hoping for was that this list would be expandable so you can add more target properties when needed for a business purpose. For example you might want to have a field to register your business owner, security officer, if a system should be compliant to PCI DSS and other things.
Initially I was unable to find a way and Oracle support reported that this indeed was not an option. However, after Oracle development has reviewed the enhancement request I filed they came up with a solution that is already in place. It turns out you can add additional target property fields however not from the GUI, you can only add them by using emcli.
The command to use to add additional target properties for a target in Oracle Enterprise Manager is to add_target_property. By using this command you can extend the number of fields per target type.
For example, if you wanted to extend all targets of the type "oracle_database" with a new target property to store the name of the system owner you could do so be executing the below command;
IHS Automotive has just released a report stating that the concept of connected cars will represent billions for the automotive industry. A raw estimate is that the automobile data systems will represent approximately 14.5 billion dollar in 2020. As a sidenote IHS researches note that this is only the amount that is directly related to the direct automotive industry however not included in this forecast is the revenue for companies and activities who have a spin off from this data. Inlcuding this it is estimated by IHS that this could run between $16 billion and $80 billion.
IHS expect that the following four categories will be of the most benefit directly for the automotive industry;
Diagnostics
Location
User Experience (UX)/Feature Usage
Advanced Driver Assistance System (ADAS)/Autonomy data
The below graph shows the buildup of data for the automotive industry between the year 2010 and 2020 and it shows a rapid growth that is predicted;
Even though this are impressive numbers it is in no comparison to what the actual revenue could be when we take into account the non-direct automotive revenue. This will not only bring new revenue to already existing companies however it will also fuel new companies to build new solutions and services. Interesting question to ask is however, who is the owner of the data collected. Most likely this will be the car manufacturer as this is somehow to be inserted within a contract with the owner. However, in an ideal world the owner of the car should be the owner of the data and should be able to opt for sharing this data, most likely anonymously, witth the rest of the world.
If the automotive industry could come up with a way that all car data could be shared openly accessible and free this could potentially even more fuel new innovations and could provide a hughe momentum for the adoption of systems like this.
A lot of people working on building private or public clouds will be familiar with OpenStack amd will use it or at least have adopted some of the technological thinking behind OpenStack. Less know might be juju and CEPH. Akash Chandrashekar is working as a solution engineer at Canonical which is the organization behind Ubuntu Linux and he gives a very clear explanation how your can build clouds by using OpenStack, juju and CEPH.
To give some highlevel background before watching the video on the 3 components; OpenStack, juju and CEPH.
OpenStack: OpenStack, a cloud-computing project, aims to provide an infrastructure as a service (IaaS). It is free and open-source software released under the terms of the Apache License. The project is managed by the OpenStack Foundation, a non-profit corporate entity established in September 2012 to promote OpenStack software and its community.
juju: Juju (formerly Ensemble) is a service orchestration management tool developed by Canonical Ltd.. It is an open-source project hosted on Launchpad released under the Affero General Public License (AGPL). Juju concentrates on the notion of service, abstracting the notion of machine or server, and defines relations between those services that are automatically updated when two linked services observe a notable modification. This allows for services to very easily be up and down scaled through the call of a single command. For example, a web service described as a Juju charm that has an established relation with a load balancer can be scaled horizontally with a single juju "add-unit" call, without having to worry about re-configuring the load-balancer to declare the new instances: the charm's event based relations will take care of that.
CEPH: Ceph is a free software storage platform designed to present object, block, and file storage from a single distributed computer cluster. Ceph's main goals are to be completely distributed without a single point of failure, scalable to the exabyte level, and freely-available. The data is replicated, making it fault tolerant.
In case you want to view the slides about the presentation deploying openstack and CEP with Juju at your own speed and comfort please find the slides below as they are shared on slideshare.
Recently someone asked me to help out with some coding parts for a startup they where beginning. Some of the things they needed help with where infrastructure related and some where more in data management and some query modeling and optimization on an already existing datamodel done in a MySQL database.
This gave me some options to start exploring again some things in MySQL which is not my standard database to develop in and use as I am more focusing on Oracle databases. Part of the fun was trying to find out why some queries where not working as expected, main reason for this was that the person who originally designed the datamodel had a love for the VARCHAR datatype.
In the database we do have a table named ovsocial_inst which holds an ID column named inst_m_id. For some reason the original datamodel developer created the column as a VARCHAR even though it is only holding numbers. Now some funny effects do happen when you try to sort.
When you execute the following query:
SELECT
inst_m_id
FROM
ovsocial_inst
ORDER BY
inst_m_id
you will get a result something like:
inst_m_id
1
10
2
3
4
5
6
7
8
9
This is somehow a little strange as long as you do not realize that the values of inst_m_id are treated as text. When you consider it to be text everything makes sense and the query is doing exactly what you ask it to do. However, we do not want it to behave in this manner, we do want it to treat the numbers as numbers even though they are stored in a VARCHAR column. To do so in a sort we can use the following query which converts the VARCHAR into an unsigned.
SELECT
inst_m_id
FROM
ovsocial_inst
ORDER BY
convert(inst_m_id,unsigned) ASC
Now you will get a result as you expect;
inst_m_id
1
2
3
4
5
6
7
8
9
10
Now imagine another scenario. we know that the table ovsocial_inst will be relative small so to assign a new ID to a record we would like to query the table for the max inst_m_id + 1 and we would like to use that value to insert a new record. When you do not consider the fact that the values are written to the table in a VARCHAR manner this process will work until you have 10 records in your table. Reason for this is that if you have 9 records in your table the highest value (the latest in the sort) is 9. This means that the new ID is 10 (9+1). When we hit 10 records something strange will happen. when we hit 10 records the highest value or at least the latest in the sort will be 9. This results in a new ID of 9+1 instead of 10+1.
When your ID field inst_m_id would have been a proper column for holding numbers you would use a query like the one below;
SELECT
(max(inst_m_id)+1) as new_id
FROM
ovsocial_inst
however, the above will give the issue as soon as you hit more then 10 records. To prevent this you will need to construct your query like the one below;
SELECT
max(convert(inst_m_id,unsigned))+1 as new_id
FROM
ovsocial_inst
In this way you will ensure that it keeps working even if you hit the 10 marker.
Big data is playing a major part in the future Oracle strategy for all good reasons and we do see that Oracle is adopting and connecting with big data solutions. Oracle is still one of the major players in the database world and for all good reasons they want to assure a place in the future data market space. To be able to do so they will have to adopt and integrate with big data solutions. Even though some people claim that big data is a hype it is holding some vert valuable points. For example the fact that data is growing in such rapid acceleration that (some) companies have to adopt different and new technologies to handle the amount of data that is available to them.
A major part of the Oracle strategy around big data is focussing on how to funnel big-data into subsets which can be loaded into a data warehouse so it can be analysed. As can be seen from the below example everything is to be funnelled into a data warehouse in this approach and then OBIEE or in-database analytics can be used to analyse this subset of the original acquired data.
Within this strategy a couple of options to implement this are available. For once you can use a full Oracle engineered systems approach as shown in the image below or you could go for a solution in which you design and create parts yourself. In the below image you see how the Oracle Big Data Appliance is used to acquire and organize the data that is collected from a number of sources. From the Oracle Big Data Appliance the pre-processed data is stored within databases inside an Oracle Exadata engineered system. Finally analysts will connect to the Oracle Exalytics machine to make use of their standard Oracle BI tools to query the subsets stored within the Oracle Exadata.
One of the advantages of the above outlined solution is that the systems are designed to work together and the time to create a working solution from a infrastructure perspective is short. Also the systems are designed to provide you with a optimised performance. Also the communication between the components is handled by infiniband, for more information on how to connect exadata and exalytics together with infiniband please do refer to my presentation on this subject on slideshare.
When working with API's and webservices JSON is becoming more and more the standard used for "packaging" data. When you are looking into how API's are sending back data to you it will become evident that JSON is winning over XML.
JavaScript Object Notation, is an open standard format that uses human-readable text to transmit data objects consisting of key:value pairs. It is used primarily to transmit data between a server and web application, as an alternative to XML.
Although originally derived from the JavaScript scripting language, JSON is a language-independent data format, and code for parsing and generating JSON data is readily available in a large variety of programming languages.
The JSON format was originally specified by Douglas Crockford, and is described in RFC 4627 and ECMA-404. The official Internet media type for JSON is application/json. The JSON filename extension is .json.
Issue is that when working with JSON it is very easy to have a small mistake somewhere in your JSON file which will break you entire code or makes your code unresponsive during parsing. If you for example use the json_decode function in PHP and your JSON file is having an issue it will simply return NULL as the value of the variable you used in conjunction with the json_decode function. Some debugging can be done using json_last_error() however a good thing to do first is to check if you JSON file is correct formatted.
Thanks to the guys from arc90 lab you can make use of the JSONLint. JSONLint is an online JSON validator which you can also use under the MIT license and which can be downloaded from github.
When you are operating a business 2 consumer website, or any other form of website for that matter you do want to drive people to your website. When operating a B2C service you do tend to the more consumer based social networks to help you in this approach. In general the believe is that, when you like to spread the word of your service, you need to interact on facebook and twitter. Specially twitter is seen as a driving force and solution to interact with customers and it is generally thought that it would drive visitors to your website.
Below graph is created by Statista and shows what drives traffic to your website.
Not very much surprising is that facebook is a large contributor in driving traffic to your website. What is extreme interesting is that Pinterest is driving a lot of traffic to the publishers websites and that this is a lot more then what is coming from twitter. Issue with the above numbers is that they are not plotted on type of website. Twitter might for example be used much more in B2B and in tech. websites.
Reading the above numbers of how pinterest is driving traffic to websites it is good to understand what the demographic breakdown is of the pinterest user community.
The above demographic data from the pinterest community shows that 80% of the users is female and that the largest age group is 25 - 34. Not surprising is that 50% of the users do have children.
So, what is this meaning. Pinterest is most likely not on the top of your social networks you think you should interact on to drive more customers to your website. However, you should benchmark your target audience to the pinterest community. If you, for example, operate a B2C fashion website which focuses on female clothing you very much should focus on this. If you do want to focus on this there are a number of things you should keep in mind to make this a success.
Have a strategie;
Ensure you have a clear strategy of who you want to engage, how you would like to interact with users on Pinterest. Ensure that you have a clear step by step approach on what you will publish when and how you will ensure that users will start spreading your pictures.
Have a recognizable image style;
When people see an image it would be good if they recognize on the first glimpse that it is coming from your company. It might however not be the best practice to have your logo or you name in full over the image. A identifying style is better and making the images look great / look cool or look funny will make it that people are more willing to re-pin them on their own pinterest boards.
Make sure you can measure;
Having your images on pinterest is nice, however, at some point in time you do want to know if your images and your work is resulting in sales and what the effect is. Ensuring you have the correct techniques in place to capture this and to analyse it will be key of tune your strategy. With good reporting you will be able to find out what is working and what is not and how to change things to get more success.
The overall feeling and initial thoughts when talking about big data is commonly that big data is coming from social media. The most used example of big data is to track what customer think and feel about a company or a product when the write something in social media. This is indeed a very good example and a very usable implementation target for the technical parts that form the basis of handling big data.
However, big data is to be honest much more then social media generated content which can be processed and analyzed. Big data is in general talked about by the four V's. Oracle has identified the 4 V's as Volume, Velocity, Variety and Value while IBM is giving the 4 V's another meaning.
Other sources of big data can be very well the output of the sensors in your factory, sensors in a city, a large number of other sources from within your company or outside of your company. Big data can be the output of devices from the internet of things as described in this blogpost.
Whatever the source of your "big" data the common factors are in most cases that their is a large volume of it, it is coming in a variety of formats and it coming at you fast and continuous. When working with big data the following 3 steps are common;
- Acquire
Their is the need to acquire the data from a number of sources. Within the acquire process their is also the need to store the data. Acquire is not necessarily capturing the data. In the example of sensor data the sensor will capture the data and send it to the acquire process.
- Process
When data is acquired and stored it will need to be processed. In some (most) cases this will be organizing the data so it can be used in analysis however it can also be very well processed for other means then analysis.
- Use
Using the processed data is in many cases analyzing or further analyzing the data that comes out of the process step. However it can also be input for other, non, analytic business processes.
In the below you can see the vision from Oracle on those steps in which they acquire, organize and analyze data.
Even though the above is giving a good example a step is missing from the visual representation. As it is shown now the acquire phase is displayed as stored data in HDFS, NoSQL or an transactional database. What one of the big parts of a good big data strategy should hold is a step before it is stored and that is receiving the data. One of the big parts where a lot of time will have to be dedicated is creating a good technological capture strategy. You will have to have "listeners" to which data producers can talk and who will write the data to the storage.
If you take for example the internet of things you will have a lot of devices that will be sending out data. You will have to have receivers to which those devices talk. As we have stated that that big data is characterized by volume, variety and velocity this means that you will have to build a number of receivers and those receivers should be able to handle a lot of messages and data. This means that your transceivers should be able to, for example, balance the load and work in parallel to be able to cope with the load of all devices that send data to them. Also you will have to ensure that the write speed of the data that the transceivers need to store the data is in line with the supply of data that is send to the transceivers.
As it turns out Vine is currently the fastest growing app in the world. Vine is a app developed by the people behind twitter and it provides you the option to record a video of only a couple of seconds and share this directly on facebook or via twitter. In some sense it can be seen as the Instagram for video content. Previously people had the option to share their daily life via social media primarily via text, added to this are now platforms that enable you to share in the form of pictures you take with your mobile phone. With Vine those options are now expanding to video. Already other social media companies are implementing video capabilities. For example Instagram is adding video capabilities to their app. Instagram, a Facebook company is trying hard to push Vine out of the market as you can also read on this BGR.com arcticle.
A "downside" of this new option however is that analyzing and mining meaning out of a video is much harder compared to doing this from text. Companies are already picking up on implementing systems where they harvest for example twitter messages to find people who talk about their product and brand or who talk about products and services offered also by their company.
One part of the newly upcoming technologies has been looking for a place to fit in. The VCA or Video content Analysis segment of the market has been looking into security and surveillance solutions. Primarily used for analysing surveillance camera feeds. As video content is getting more and more a place in the social media arena the demand for software that can analyse the content of those videos will also grow. This will be a field of expertise where most likely companies who have been working on security VCA solutions can step into the social analytics game.
The major downside of VCA is that it is a resource intensive and currently is still experiencing a high level of faulty outcomes. Far from being mature however this sector of the IT industry could grow to heights as we see the number of video content enablers grow.
When developing an technical deployment architecture for a company it is always good to base your architecture on the best practices. Best practices are offered by a number of sources and when reading the best practices it is good to understand where they are coming from and if there is a "second agenda" in the best practices that is offered.
In general the best best practice is are to be found by independent organizations that provide industry best practices. Next to this a good source of best practices can be the numerous blogs that can be found online. a lot of software vendors do provide best practices and also implementation companies do provide best practices. One thing you have to remember though is that there is always the change that the people from the software vendor and/or an implementation partner do write a best practice without keeping in mind a specific business case.
In real world deployments every additional piece of software and hardware is an additional cost on the budget. This additional cost in the budget should be accompanied with a valid business reason. When someone writes without keeping that in mind, or even with the intention to add additional and costly features to the design, the costs of your deployment architecture might not weigh up to the business benefits and needs.
When reviewing a best practice always try to keep the following things in mind:
- Is the goal used in the best practice in line with my business need
- Is this best practice possibly written with a second agenda
- Is this best practice including higher demands then in my situation
- Is his best practice only focusing on a single vendor solution stack and are the options to include solutions from other vendors.
If you keep the above point in the back of your mind best practices you download from software vendors are very usable.
