Thursday, June 30, 2011

Oracle VM and MySQL templates

Oracle has released a new press release which is stating a new Oracle VM template. They now have created a Oracle VM template which is based on Oracle Linux and will be running a Oracle MySql Enterprise version. This will provide the MySQL community in a even more relaxed position now they can simply provision complete MySQL installations on a Oracle VM platform. In my opinion this is a great move from Oracle to get (A) MySQL more in the picture within the Oracle community itself and (B) provide a even wider adoption of Oracle VM and Oracle Linux now they are expanding it to the MySQL groups within the Oracle community.

The press release can be found at the Oracle site and you can read a part of it below:

News Facts
Enabling more efficient and lower cost deployments of virtualized MySQL environments, Oracle today announced the first Oracle VM Template for MySQL Enterprise Edition.

The new Oracle VM Template for MySQL helps eliminate manual configuration efforts and risks by providing a pre-installed, pre-configured and certified software stack that includes Oracle VM Server for x86, Oracle Linux with the Unbreakable Enterprise Kernel and MySQL Enterprise Edition.

By pre-integrating the world’s most popular open source database with Oracle Linux and Oracle Virtualization technologies, enterprise users and ISVs can quickly and easily deploy and manage a virtualized MySQL database server for Web and cloud-based applications.
Backed by Oracle’s world-class support organization and the result of extensive integration and quality assurance testing, the Oracle VM Template for MySQL Enterprise Edition further demonstrates Oracle’s investment in MySQL and allows users to benefit from a single point of contact for 24/7 technical support for all pre-configured components.

Delivers Pre-Installed, Pre-Configured and Fully Integrated Virtualized Software Stack
Oracle VM Templates greatly simplify installation, configuration and ongoing maintenance operations, allowing organizations to achieve faster time to market and to reduce costs.

Using the Oracle VM Template for MySQL Enterprise Edition to deliver applications allows users to benefit from:
Faster deployments: Users can drastically reduce installation and configuration cycles both for initial deployments and to provision replicated scale-out environments.
Increased reliability: Using a pre-installed and pre-configured software stack, that has already undergone extensive integration and quality testing and is certified for production use, helps ensure a reliable and trusted environment.

Higher uptime: With MySQL’s replication capabilities plus Oracle VM’s built-in high availability features both planned and unplanned downtime can be reduced.
To further support the cost-effective delivery of high-performance web-based applications, the new Oracle VM Template for MySQL Enterprise Edition also provides users with access to the comprehensive suite of MySQL management tools including MySQL Enterprise Monitor, MySQL Query Analyzer, MySQL Workbench, and MySQL Enterprise Backup.

Oracle VM Templates are also available for Oracle Applications, Oracle Fusion Middleware, Oracle Database, Oracle Linux, Oracle Solaris and other Oracle products.

Supporting Quote
"Oracle is committed to helping MySQL users more efficiently deploy and manage high-performance web and cloud-based applications at a lower cost,” said Tomas Ulin, vice president MySQL Engineering, Oracle. “With the introduction of the first Oracle VM Template for MySQL Enterprise Edition, customers can benefit from a fully integrated solution that is backed by Oracle’s world-class support, enabling faster deployments, increased reliability and higher uptime for MySQL applications in virtualized environments.”

Posted via Johan Louwers his Posterous page to his blogger page.

Thursday, June 16, 2011

Generate QR codes with google API


QR codes are used more and more in all kind of applications, where the barcode was used primarily to identify stuff, for example in a logistics system you now see QR codes coming up in all kind of B2C and C2C applications besides only B2B. Reason for this is that most people are now getting the hang of working with smart-phones which are equipped with camera's and free apps can be downloaded to understand the QR code.


You now see Qr codes coming to magazines where you can use your smartphone to go to a specific website for example. Or you can see QR codes on posters int he city, as a extra information link on tickets, in books… the options for which you can use QR codes are almost endless, not only towards customers but also within business applications.


Another reason that QR codes are coming so popular is that you can store a "large" amount of data in a QR code which you could not easily do with a barcode. You can store up to 7089 numeric code characters in a QR code or 4296 alphanumeric characters or 2953 bytes (binary 8 Bit) or 1817 Kanji or Kana characters (Kanji - chinese characters used in Japanese writing / kana - japanese characters) . for a small image that is a large amount of data and with the correct application and the correct mindset you can do a lot with this.


So all find and good,… how do you generate this mess of pixels? You can find a lot of OpenSource applications and libraries you can use to generate a QR code. However if you want a quick solution you can use the Google Chart API to generate it for you. Google is providing a way to request a QR code via a URL pointing towards the chart API.


The base URL for this part of the chart API is https://chart.googleapis.com however this will redirect you to the google search main page if you are not proving some parameters.


First you have to extend it with "/chart?" this will make the URL https://chart.googleapis.com/chart? which is still not very good and will not give you a QR code. secondly you have to play with a couple of parameters.


cht

Must be set to qr, meaning it will look like cht=qr


chs

This states the size of the QR code. x in pixels.


chl

This parameter is holding your data for example if you want to have "abc23" in the QR code your parameter should look like chl=abc123


choe

This indicates the encoding of the data in the QR code. You can pick UTF-8, Shift_JIS or ISO-8859-1. If not provided UTF-8 is used


child

This indicates the level of correction. You can enter the following values for this parameter (if not provided L is used):

L - [Default] Allows recovery of up to 7% data loss

M - Allows recovery of up to 15% data loss

Q - Allows recovery of up to 25% data loss

H - Allows recovery of up to 30% data loss


so, lets say you want to have a QR code with the text "abcdef123456" in a UTF-8 format and a size of 50X50. for this you will have to use the following URL:


https://chart.googleapis.com/chart?cht=qr&chs=50x50&chl=abcdef123456&choe=UTF-8

this will produce the following QR code:


If you for example want to change the size to 100 by 100 you can change the URL to: https://chart.googleapis.com/chart?cht=qr&chs=100x100&chl=abcdef123456&choe=UTF-8 to create this image:

If you play around with the parameters you will see you can have a lot of fun with it and quickly create QR codes without the need to build a custom (or install) application to create QR codes for you. You can even use the URL to automatically download the pictures for with for example a wget in a bash script.

Saturday, June 11, 2011

Solved: Maximum execution time exceeded file.inc on line 934

Some people at Harvard University have greeted a template extension on the basic Drupal installation named OpenScholar. OpenScholar is intended to provide people with a Drupal installation which can be used to host a website for professors and research groups primarily. Looking at the templates they have build I wanted to give it a go. The installation is basically the same as that of standard Drupal. During installation of OpenScholar I encountered a issue which is also in some cases seen when you install Drupal itself. After the first installation page your webserver comes back with a empty page, looking at the HTML source code of the "empty" page it turns out the following error message is thrown:

Fatal error: Maximum execution time of 65 seconds exceeded in /public_html/includes/file.inc on line 934

The maximum exception time is set in your PHP configuration, this indicates the amount of seconds that a script can take to complete its task. You can resolve this by simply changing the value to something higher. Issue with this is that if you are on a hosted environment, like at a hosting provider, you will not have the option to change this value as it is set for all users on that webserver. Some options are available to you however. First is, you can add a setting to your local .htaccess file where you can change it for that specific directory. You have to add a php_value max_execution_time to this file. Secondly you can initiate a override in your PHP code itself. 

However, most hosting providers will have set a setting preventing such overrides in their main PHP configuration. This is a simple and good approach from a hosting provider point of view. This is preventing certain user to consume all CPU power with bad code. For most users those options will not be valid as was in my case as I installed it not on my own server and was trying to install it on a server at a hosting provider.

Meaning we have to look into other options to resolve this issue. Looking at the code we found that the issue is coming from a part of the code that is checking the files on the file system to verify that all files needed for the installation are present. This scanning of the filesystem in some cases take more time than the time set for the max execution time. With some help from Sam Adams and his postings on acquia.com forums I found that you can prevent this scan from happening by hacking the install.inc file. What you have to do is open the file in VI and change the following line:

$installs = drupal_get_install_files($module_list);

into this:

$installs = array();

This will populate the variable $installs with a empty array and it will not call drupal_get_install_files with a list of modules to be checked. After making this change the installation go's without any issue. 

Posted via Johan Louwers his Posterous page to his blogger page.

Thursday, June 09, 2011

Solved: oracle XE web-interface not working

I recently installed Oracle XE on a laptop to play around with. I installed the latest version of Oracle Linux and downloaded the Oracle XE database. After installation all was working fine, including the web-interface you get when you install Oracle XE.

After shutting down the laptop and starting it again the next day for some reason the web interface was not working. Connecting to http://localhost:8080 was resulting in a 404 error page meaning the page could not be found.

After some looking into my listening ports I found that no service was listening on port 8080. Checking the Oracle database listener gave the first clue:
LSNRCTL> status
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 10.2.0.1.0 - Production
Start Date 09-JUN-2011 11:10:25
Uptime 0 days 0 hr. 12 min. 30 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Default Service XE
Listener Parameter File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/admin/listener.ora
Listener Log File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1521)))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

the status command within lsnrctl stated clearly that PLSExtProc was having some trouble. Issue is that if you configure your machine to make use of DHCP and do not state a hostname and you come to a network where the DHCP server will provide you with a hostname the hostname is not matching with the hostname set during the installation of the Oracle XE database.

Solution to this is to state a hostname and do not let your DHCP server push one towards you. After changing this you have to change the content of listener.ora and tnsnames.ora to make sure the hostname matches with the newly entered hostname. When done do a force-reload (or stop and start) of /etc/init.d/oracle-xe.

When you have done those steps you can request the status again via lsnrctl and you will see something like the below outcome. You can also open a browser and connect to localhost:8080 and you should see the web-interface. a.k.a. Problem solved.

LSNRCTL> status
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC_FOR_XE)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 10.2.0.1.0 - Production
Start Date 09-JUN-2011 11:28:44
Uptime 0 days 0 hr. 0 min. 9 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Default Service XE
Listener Parameter File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/admin/listener.ora
Listener Log File /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/network/log/listener.log
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC_FOR_XE)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=johanlinux.capgemini.nl)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=8080))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "XE" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
Service "XEXDB" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
Service "XE_XPT" has 1 instance(s).
Instance "XE", status READY, has 1 handler(s) for this service...
The command completed successfully
LSNRCTL>

Tuesday, June 07, 2011

Create structure with microdata

Already some time ago HTML was created to be able to provide a structured way of creating content for the web. HTML makes sense to your browser and enable you as a user to view the website in a way the developer of the site intended to (or at least that is the intention of the developer. Having a structured way to create documents is great and needed to enable all browsers to show the data in (almost) the same way. On the other hand companies who run search engines, for example Google Search and Microsoft Bing, deploy index robots who crawl all the websites on the web to index what is on it and store this is the datastores which they use to query.

Both, browsers and crawl robots need structured data. In the current version of HTML their is enough structure to display the content in a browser, however for a crawl robot it is in some cases not enough. For example, if you create a image gallery you will most likely state where it is taken, who is on it, and when it is taken. This will be in a simple way that you as a user understand it when it is rendered by your browser from the HTML code. However for a crawl robot and the algorithm used to search the index created by the search robots this can be quite hard to figure this out.

An other example, if you have a store which sells hardware supplies you will most likely a page which states where your shop is and when it is open. It would be ideal for a search engine if you clearly state in a structured way which data on this page is talking about your location, which part about the time you are open and some parts about what kind of a store you are. The good part about modern websites is that most of the websites are rendered based upon data which is in a structured data, for example a database of XML files. However when we display this data to the users on the web via HTML this structure is sadly abandoned.

Putting back this structure enables applications and search engines to benefits greatly. For this reason Microsoft, Google and Yahoo have created schema.org which can be used as a vocabulary for the microdata format. This format enables you to structure the data in such a way that search engines will be able to index your site better and by doing so provide it higher in the results when someone searches for it. Using the microdata format and the vocabulary of schema.org will provide you the best result while keeping the way your data is shown in a browser the same.

Currently the schema.org hierarchy is not completed and will most likely never be completed however it is providing a first draft on how you could structure your data. For example coming back on the picture with the text I stated at the beginning of this page. In html it should most likely look a bit like this:

<html>
<body>
<img src="some_image.png">
<br><b>Description : </b>Johan Louwers inspecting the place before takeoff
<br><b>Location : </b>Hilversum airfield
<br><b>Date : </b>07-MAY-2011
</body>
</html>

This is providing a very simple page showing a picture and telling you who is on this picture, where it is taken and when it is taken. For a human this is no problem to interpret. For a algorithm this can be a little harder. specially for this the microdata can be used with the information from schema.org to make sure you are consistent. To make sure a search index knows what is what and also that it is related to the picture you can also use the code below. The result in your browser will be exactly the same however the indexing and the resulting position within a search engine can be completely different.

<html>
<body>
<div itemscope itemtype="schema.org/ImageObject">
<img src="some_image.png">
<br><b>Description : </b><span itemprop="description"><span itemprop="name">Johan Louwers</span> inspecting the place before takeoff</span>
<br><b>Location : </b><span itemprop="contentLocation">Hilversum airfield</span>
<br><b>Date : </b><time itemprop="publishDate" datetime="2011-05-07">07-MAY-2011</time>
</div>
</body>
</html>

If you look closely at the code you can see that we have placed most of the HTML code within a div;

<div itemscope itemtype="schema.org/ImageObject">

<div>

As you can see we make use of itemscope to tell that this is a div for starting the scope of the item (within the div) and we state that the itemtype we are making use of is a image object as is defined by schema.org . to be precise we are making use of a itemtype defined at schema.org/ImageObject. If you check this web location you will find more detailed information of what the options are you can use to describe an ImageObject.

The description we have change to the following line:

<br><b>Description : </b><span itemprop="description"><span itemprop="name">Johan Louwers</span> inspecting the place before takeoff</span>

In this we nest to things, first we create a itemproperty for the description which is "johan Louwers inspecting the place before takeoff" which is correct, however the "Johan Louwers" part is a person so to inform the interpreter of our microcode that this part of the description is a person we place a itemprop(erty) for person within the itemprop(erty) for description.

The current location is defined as below by using a itemprop(arty) contentLocation which is telling the search engine where the picture is taken. The great part of this is that more and data is getting location aware. By adding a proper location format to your code you will add to this and when someone is searching for for example pictures with the following search string "Hilversum Airfield 2011" your image is more likely to come up with your picture.

<br><b>Location : </b><span itemprop="contentLocation">Hilversum airfield</span>

To make sure that the time when this picture is also indexed correctly we can add the correct time (without having to thing about what date format you want to display and could be used to search on) using the following option:

<br><b>Date : </b><time itemprop="publishDate" datetime="2011-05-07">07-MAY-2011</time>

A lot more options can be displayed or not be displayed and only used to inform the interpreter of your microcode. For example, I think the picture is quite family friendly and nothing inappropriate is shown on it. So I would like to inform the world this is familyfriendly which can be done with the itemprop isFamilyFriendly which is a boolean value, I do however not want to show this explicitly. By using the below code I only make it available in the source of my page however the user will not have anything displayed on his screen

</span isFamilyFriendly="true"/>

Now you can argue that this, and taking into consideration all the other options you can use in your microdata are not really useful for some home build image gallery where you show your holiday pictures. On the other hand, why would you not like to turn up when someone is looking for the pictures of that great vacation you told him about, is that not the soul reason you place them online. If you are not willing to enable all this kind of microdata that is perfectly fine. If you are running a business this is however a whole other point. As a business being found on the internet is currently the difference between survival or not and this will become more and more the case. If someone is looking for something, interested in buying a product, it is most likely this person will go online first and then come to your shop. If he is unable to find your shop online he is very unlikely to find your store and make a purchase. Meaning, if you want to stay ahead of the rest it is very important that you take a good look into microdata or you might find that the competition is winning.

Sunday, June 05, 2011

MongoDB architecture

As already stated in some previous blogposts the MongoDB is somewhat different than the databases you are might used to. The concept a schema, a table and a record is something that is not used within the MongoDB setup. When you are used to work databases like a Oracle database this might come a little strange to you at first however due to the nature of MongoDB you will get the hang of it quite quick.

To visualize the way MongoDB is working you can see the below chart;

Database;
within MongoDB you will be able to create separate databases. A database will use its own users, store it own data and will be its own datafile on your servers filesystem. As a good practice you can keep the rule that evert application should have its own database. a example of how to create a database you can see in this previous blogpost.

Collection;
As stated MongoDB is not using the concept of a table, if you would like to make a reference between MongoDB and a "normal DB" you can state that a collection is a table. A collection holds one or more documents. lets say you want to add the following document to your database {"shipping_number" : "1046", "receivedate" : "12-FEB-2011"} you will have to place this in a collection. You can state that the document in this case is the record (or row). To insert this in the collection shipping you can enter the use the following command;

db.shipping.insert({"shipping_number" : "1046", "receivedate" : "12-FEB-2011"})

One more option is to mention when we are talking about collections, sub-collections. You have the option to logically group collections in a sub-collection. This is not having any technical implication, it is just a handy feature which can make you development and administration a little more easy. For example if you have a lot of collections regarding shipping in your database, for example data on shipments, trucks, locations of warehouses, authorized personal, average loading times, etc etc etc. You might want to group all those collections in a logical way so that people who work on developing code for your shipping module know where to find this this code. If you want to insert the above line however now you would like to make use of sub-collections (for example the sub-collection "ship" you should use the following command:

db.ship.shipping.insert({"shipping_number" : "1046", "receivedate" : "12-FEB-2011"})

Documents;
we already touched the subject of documents, a document can be seen as the record (or row) in your table. In MongoDB this is a document in a collection. meaning that your document which you inserted in the previous example is in essence; {"shipping_number" : "1046", "receivedate" : "12-FEB-2011"}

Key-value pair:
the key value pair can be compared the best with the column and the value of a record. For example {"shipping_number" : "1046", "receivedate" : "12-FEB-2011"} in this document (record) we have 2 key-value pairs (columns), the shipping_number column and the receivedate column. In this example the shipping_number key has the value 1046 and the key receivedate has the value 12-FEB-2011.

If you take this comparison as a guideline this might help you to translate your idea from a standard DB into a MongoDB way of thinking.

Wednesday, June 01, 2011

Creating a new database in MongoDB

When you start with MongoDB it might be beneficial to not have any experience with databases in some cases as MongoDB is just handling things and approaching things in a different way. Within for example Oracle we are used to having schema, a table, records and columns. Within MongoDB you have to forget all of this as they are not working with a schema or a table. The best you can describe a schema within MongoDB is the concept of a "database" in mongo in which you will store things.

So when you are working with MongoDB you have to learn how you can work with databases, create them and switch between the different databases. When you login to a MongoDB you most likely would like to know which databases are currently in MongoDB. To get a list of all databases you can use "show dbs"

> show dbs
admin (empty)
callreg 0.0625GB
local (empty)
test 0.0625GB
>

local, test and admin are installed by default, the callreg database is something I created to do some testing. If you want to switch between one database and another you can use the command "use " for example if we want to switch to the callreg database you can use the command "use callreg".

> use callreg
switched to db callreg
>

Now lets say we want to create a new database specially for handling shipping data, we will name it shipping. Basically you do not create a database within MongoDB as such. you can just switch to the (not yet existing) database by using the command "use shipping". This will bring you to the database shipping. Now if we insert data into this database it will be directly created. In the below example you see that we switch to the shipping database (which is not existing yet) and insert a document.

> use shipping
switched to db shipping
>
>
> db.shipping.insert({"shipping_number" : "1045", "receivedate" : "12-FEB-2011"})
>

if we nog check the list of databases using the "show dbs" command you will see that the database is created.

> show dbs
admin (empty)
callreg 0.0625GB
local (empty)
shipping 0.0625GB
test 0.0625GB
>

This is how you quickly create a new database, switch to the database (even if is not yet existing) and insert something into the database.