The personal view on the IT world of Johan Louwers, specially focusing on Oracle technology, Linux and UNIX technology, programming languages and all kinds of nice and cool things happening in the IT world.
I just received a box from one of our partners containing a Symbol barcode scanner. We have had a meeting with this partner to discuss the usability of handheld scanners in combination with Oracle E-business suite and even do we where convinced that this should work out of the box we still wanted to test this before we should consult our customers to purchase a curtain brand of scanner.
As a Oracle Applications service provider we also do have to consult our customers on what they should have as workstations and other hardware on site and not only focus on the server part in our datacenter.
In short I received a new Symbol scanner and a Zebra label printer to test on Oracle. Today, on day one, I have decided to start testing the barcode scanner. To my surprise this was much easier than I expected. The barcode scanner is, when scanning a barcode, taking the role of your keyboard. Nothing fancy about this, it simply puts the numbers and characters into the location where your cursor is at this moment. This might come not as a surprise to people who have worked and developed with a barcode scanner but if you are new to this it is quite cool to find out.
Also the setup of the entire scanner is quite straight forward, plugin your USB cable into a workstation, synchronize the scanner with the docking station by scanning the barcode on the docking station and you are in business. That is all there is to.
When starting my research I was planning on writing a deep technical blogpost on how you can enable barcode scanning with Oracle, the “problem” is that I can not write such an blogpost as there is nothing deep technical about.
As commodity hardware becomes cheaper and more powerful it becomes tempting to reflexively use relational databases to track and report on large datasets. Rentrak argues the case that a pure-RDBMS solution can be counterproductive, especially for complex analysis of those datasets. I'll describe an alternate approach we currently use to track and report on video-on-demand and linear television viewership.
Over the past two years, Microsoft has been gathering GPS data from volunteer drivers. Each driver has one of our recording GPS receivers on their dashboard for two weeks. So far, we have data from 227 drivers, comprising over 1.7 million time-stamped (latitude, longitude) points. This talk will give a brief overview of several different research projects we have completed based on this data. These projects include:
* Destination Modeling -- We characterize where and when people go and how quickly they fall into a rut of visiting very few new destinations. * Predestination -- We use our destination models to predict where a driver is driving as the trip progresses. * Snap-To-Road -- Inferring which road a driver is on is harder than just snapping to the nearest one. Our snap-to-road algorithm improves by accounting for the time stamps of the GPS data. * Personalized Routes -- We can plan routes that drivers prefer by looking at where they've already driven. * Location Privacy -- Can we protect the privacy of our volunteer subjects by simply anonymizing their data? We show it is possible to infer the identities of some of them from anonymized GPS data, and we demonstrate different strategies for protecting their privacy by corrupting the data.
Google has implemented a street view option in google maps. You are now able to "walk" in the streets of some of the major cities In America. The pictures are most likley taken with a 360 camera mounted on the top of a car while driving in the city. Also they have posted a very nice video on youtube which you can view here.
When running Oracle e-business suite performance can be a problem. There a numerous of things you can check, do and tune to make your applications faster and enhance user satisfaction. Oracle e-business suite can be, depending on the modules you use, a large and resource demanding environment. As it will be in most cases the system your company is depending on to do the day to day business your users will demand a quick working system and they are not willing to spend hours and hours watching screens being buildup on their screens.
One of the things you might want to look at when running a Oracle database and or e-business suite in a UNIX/Linux environment is memory, both physical and virtual memory. Oracle loves memory even more than it loves CPU power, looking at your CPU load and memory use on a server running Oracle will show you that the use of memory can become quite extreme.
You can find an example of “top” on a simple server running a light installation of E-business suite.
The thing you basically want is that your operating system is not using swap space. Swap space in short is virtual memory which is not in one of your memory banks but is allocated on your file system. The problem is that if your physical memory is used completely your operating system will start to move not so intensively used parts “pages” of memory from your physical memory to your virtual memory. Virtual memory is located on disks and physical memory is located on your memory bank chips. This means that if your operating system is starting to use swap space it depends on the speed of your hard drives. A hard drive is still a lot slower than your memory banks so you will have a enormous performance drop.
In theory you could have a very small footprint of physical memory and have a large pool of virtual memory and run an Oracle instance. However this would mean that everything that can not fit into your physical memory space would be transported to your virtual memory space and this would mean that your disk will have to read/write for every request for memory. Every time a memory request is made and this specific page is in your virtual memory a disk request is made, the disk has to find it and have to return it to the requesting process. This method is much more time consuming than reading from the physical memory.
UNIX systems implement a global paging strategy. This means that the operating system will look at all processes on the system when is searching for a page of physical memory on behalf of a process. This strategy has a number of advantages, and one key disadvantage. The advantages of a global paging strategy are: (1) An idle process can be compleatly paged out so it does not hold memory pages that can be better used by another process. (2) A global strategy allows for a better utilization of system memory; each process's page allocations will be closer to their actual working set size. (3) The administrative overhead of managing process or user page quotas is completely absent. (4) The implementation is smaller and faster. The disadvantage of a global strategy is that is is possible for a single ill-behaved process to affect the performance of all processes on the system, simply by allocating and using a large number of pages. A UNIX process can be conceptually divided into two portions; text and data. The text portion contains the machine instructions that the process executes; the data portion contains everything else. These two portions occupy different areas of the process's virtual address space. Both text and data pages are managed by the paging subsystem. This means that at any point in time, only some of the text pages and only some of the data pages of any given process are in RAM. UNIX treats text pages and data pages differently. Since text pages are typically not modified by a process while it executes, text pages are marked read-only. This means that the operating system will generate an error if a process attempts to write to a text page. (Some UNIX systems provide the ability to compile a program which does not have read-only text: consult the man pages on 'ld' and 'a.out' for details.) The fact that text pages are read-only allows the UNIX kernel to perform two important optimizations: text pages are shared between all processes running the same program, and text pages are paged from the filesystem instead of from the paging area. Sharing text pages between processes reduces the amount of RAM required to run multiple instances of the same program. For example, if five processes are running Oracle Forms, only one set of text pages is required for all five processes. The same is true if there are fifty or five hundred processes running Oracle Forms. Paging from the filesystem means that no paging space needs to be allocated for any text pages. When a text page is paged out it is simply over-written in RAM; if it is paged in at a later time the original text page is available in the program image in the file system. On the other hand, data pages must be read/write, and therefore cannot (in general) be shared between processes. This means that each process must have its own copy of every data page. Also, since a process can modify its data pages, when a data page is paged out it must be written to disk before it is over-written in RAM. Data pages are written to specially reserved sections of the disk. For historical reasons, this paging space is called "swap space" on UNIX. Don't let this name confuse you: the swap space is used for paging. The UNIX kernel is in charge of managing which data pages are in RAM and which are in the swap space. The swap space is divided into swap pages, which are the same size as the RAM pages. For example, if a particular system has a page size of 4K, and 40M devoted to swap space, this swap space will be divided up into 10240 swap pages. A page of swap can be in one of three states: it can be free, allocated, or used. A "free" page of swap is available to be allocated as a disk page. An "allocated" page of swap has been allocated to be the disk page for a particular virtual page in a particular process, but no data has been written to the disk page yet -- that is, the corresponding memory page has not yet been paged out. A "used" page of swap is one where the swap page contains the data which has been paged out from RAM. A swap page is not freed until the process which "owns" it frees the corresponding virtual page. On most UNIX systems, swap pages are allocated when virtual memory is allocated. If a process requests an additional 1M of (virtual) memory, the UNIX kernel finds 1M of pages in the swap space, and marks those pages as allocated to a particular process. If at some future time a particular page of RAM must be paged out, swap space is already allocated for it. In other words, every virtual data page is "backed with" a page of swap space. An important consequence of this strategy is if all the swap space is allocated, no more virtual memory can be allocated. In other words, the amount of swap space on a system limits the maximum amount of virtual memory on the system. If there is no swap space available, and a process makes a request for more virtual memory, then the request will fail. The request will also fail if there is some swap space available, but the amount available is less than the amount requested. There are four system calls which allocate virtual memory: these are fork(), exec(), sbrk(), and shmget(). When one of these system calls fails, the system error code is set to EAGAIN. The text message associated with EAGAIN is often "No more processes". (This is because EAGAIN is also used to indicate that the per-user or system-wide process limit has been reached.) If you ever run into a situation where processes are failing because of EAGAIN errors, be sure to check the amount of available swap as well as the number of processes.
If a system has run out of swap space, there are only two ways to fix the problem: you can either terminate some processes (preferably ones that are using a lot of virtual memory) or you can add swap space to your system. The method for adding swap space to a system varies between UNIX variants: consult your operating system documentation or vendor for details. UNIX systems implement, and the Oracle server uses, shared memory. In the UNIX shared memory implementation, processes can create and attach shared memory segments. Shared memory segments are attached to a process at a particular virtual address. Once a shared memory segment is attached to a processes, memory at that address can be read from and written to, just like any other memory in the processes address space. Unlike "normal" virtual memory, changes written to an address in the shared memory segment are visible to every process that has attached to that segment. Shared memory is made up of data pages, just like "conventional" memory. Other that the fact that multiple processes are using the same data pages, the paging subsystem does not treat shared memory pages any differently than conventional memory. Swap space is reserved for a shared memory segment at the time it is allocated, and the pages of memory in RAM are subject to being paged out if they are not in use, just like regular data pages. The only difference between the treatment of regular data pages and shared data pages is that shared pages are allocated only once, no matter how many processes are using the shared memory segment. When discussing the memory usage of a process, there are really two types of memory usage to consider: the virtual memory usage and the physical memory usage. The virtual memory usage of a process is the sum of the virtual text pages allocated to the process, plus the sum of the virtual data pages allocated to the process. Each non-shared virtual data page has a corresponding page allocated for it in the swap space. There is no system-wide limit on the number of virtual text pages, and the number of virtual data pages on the system is limited by the size of the swap space. Shared memory segments are allocated on a system-wide basis rather than on a per-process basis, but are allocated swap pages and are paged from the swap device in exactly the same way as non-shared data. The physical memory usage of a process is the sum of the physical text pages of that process, plus the sum of the physical data pages of that process. Physical text pages are shared among all processes running the same executable image, and physical data pages used for shared memory are shared among among all processes attached to the same shared memory segment. Because UNIX implements virtual memory, the physical memory usage of a process will be lower than the virtual memory usage. The actual amount of physical memory used by a process depends on the behavior of the operating system paging subsystem. Unlike the virtual memory usage of a process, which will be the same every time a particular program runs with a particular input, the physical memory usage of a process depends on a number of other factors. First: since the working set of a process changes over time, the amount of physical memory needed by the process will change over time. Second: if the process is waiting for user input, the amount of physical memory it needs will drop dramatically. (This is a special case of the working set size changing.) Third: the amount of physical memory actually allocated to a process depends on the overall system load. If a process is being run on a heavily loaded system, then the global page allocation policy will tend to make the number of physical memory pages allocated to that process to be very close to the size of the working set. If the same program is run with the same input on a lightly loaded system, the number of physical memory pages allocated to that process will tend to be much larger than the size of the working set: the operating system has no need to reclaim physical pages from that process, and will not do so. The net effect of this is that any measure of physical memory usage will be inaccurate unless you are simulating both the input and the system load of the final system you will be testing. For example, the physical memory usage of a Oracle Forms process will be very different if a user is rapidly moving between 3 large windows, infrequently moving between the same three windows, rapidly typing into a single window, slowly typing into the same window, or if they are reading data off of the screen and the process is sitting idle -- even though the virtual memory usage of the process will remain the same. By the same token, the physical memory usage of an Oracle Forms process will be different if it is the only active process on a system, or if it is one of fifty active Oracle Forms processes on the same system. There are a number of key points to understand about the UNIX virtual memory implementation. (1) Every data page in every process is "backed" by a page in the swap space. This size of the swap space limits the amount of virtual data space on the system; processes are not able to allocate memory if there is not enough swap space available to back it up, regardless of how much physical memory is available on the system. (2) UNIX implements a global paging strategy. This means that the amount of physical memory allocated to a process varies greatly over time, depending on the size of the process's working set and the overall system load. Idle processes may be paged out completely on a busy system. On a lightly loaded system processes may be allocated much more physical memory than they require for their working sets. (3) The amount of virtual memory available on a system is determined by the amount of swap spaces configured for that system. The amount of swap space needed is equal to the sum of the virtual data allocated by all processes on the system at the time of maximum load.
(4) Physical memory is allocated for processes out of the page pool, which is the memory not allocated to the operating system kernel and the buffer cache. The amount of physical memory needed for the page pool is equal to the sum of the physical pages in the working sets of all processes on the system at the time of maximum load.
One other thing you have to remember is that Swap partitions near the front of a hard disk (primary partition) have faster access time. Swap partitions on different harddisk attached to separate controllers allow linux to send the swap data to different destinations. Linux could start a program on one disk and use the swap space on a different disk. An other thing that can become handy when thinking about swap is to have the "rule" that your swap space has to be twice the size of your physical memory. For how much physical memory is required by you Oracle installation you have to check your installation guide on OTN or Metalink.
References: - Oracle : “VIRTUAL MEMORY ON UNIX” Note:2060094.6 - Oracle : "Swap Space on RedHat Advanced Server" Note:225451.1
I just received a box from one of our partners containing a Symbol barcode scanner. We have had a meeting with this partner to discuss the usability of handheld scanners in combination with Oracle E-business suite and even do we where convinced that this should work out of the box we still wanted to test this before we should consult our customers to purchase a curtain brand of scanner. 
