The personal view on the IT world of Johan Louwers, specially focusing on Oracle technology, Linux and UNIX technology, programming languages and all kinds of nice and cool things happening in the IT world.
Showing posts with label parallel computing. Show all posts
Showing posts with label parallel computing. Show all posts
Some time ago I posted a blogpost named "The future of computing is parallelism". Here I was discussing the future of computing and the role of parallelism that will be a large player in future computing and system design. Below you can find 2 video's on Understanding Parallel Computing and the Amdahl’s Law.
Amdahl's law, also known as Amdahl's argument, is named after computer architect Gene Amdahl, and is used to find the maximum expected improvement to an overall system when only part of the system is improved. It is often used in parallel computing to predict the theoretical maximum speedup using multiple processors. It was presented at the AFIPS Spring Joint Computer Conference in 1967.
The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program. For example, if a program needs 20 hours using a single processor core, and a particular portion of 1 hour cannot be parallelized, while the remaining promising portion of 19 hours (95%) can be parallelized, then regardless of how many processors we devote to a parallelized execution of this program, the minimum execution time cannot be less than that critical 1 hour. Hence the speedup is limited up to 20×, as the diagram illustrates.
A very interesting video with James Phillips who is the Chief Strategy Officer and cofounder from NorthScale. Northscale provides elastic data infrastructure software and is closely tied with the guys from couchbase and are the developed on the memcached project.
Membase was developed by several leaders of the memcached project, who had founded a company, NorthScale, expressly to meet the need for an key-value database that enjoyed all the simplicity, speed, and scalability of memcached, but also provided the storage, persistence and querying capabilities of a database. The original membase source code was contributed by NorthScale, and project co-sponsors Zynga and NHN to a new project on membase.org in June 2010.
In computing, memcached is a general-purpose distributed memory caching system that was originally developed by Danga Interactive for LiveJournal, but is now used by many other sites. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read. Memcached runs on Unix, Linux, Windows and MacOSX and is distributed under a permissive free software license.
Memcached's APIs provide a giant hash table distributed across multiple machines. When the table is full, subsequent inserts cause older data to be purged in least recently used (LRU) order. Applications using Memcached typically layer requests and additions into RAM before falling back on a slower backing store, such as a database.
The system is used by sites including YouTube, Reddit, Zynga, Facebook, Orange, and Twitter. Heroku (now part of Salesforce) offers a Couchbase-managed memcached add-on service as part of their platform as a service. Google App Engine, AppScale and Amazon Web Services also offer a memcached service through an API. Memcached is also supported by some popular CMSs such as Drupal, Joomla, and WordPress.
implementing membase and the memcached API's can help you speed up your website enormously. Investigating the options in this field when you are building a high traffic website is very important and can mean the difference between success or failure in my opinion. memcache can be seen as one of the future building blocks and should be taken serious when you develop large scale web applications.
Hadoop is a Apache project aimed to build a framework a framework for running applications on large cluster built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce, where the application is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, it provides a distributed file system (HDFS) that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both MapReduce and the Hadoop Distributed File System are designed so that node failures are automatically handled by the framework.
When we look at where the future of computing and the future of data we can see Hadoop on a very strategic location on the roadmap and within the overall framework. Hadoop is one of the ultimate building blocks in the framework which is responsible for parallelism within this framework and can be seen as one of the main engines for handling big-data.
In the below video Mike Olson is explaining some parts of the future framework of computing and explains Hadoop and some other parts in depth. Mike Olson is a the CEO of Cloudera, Cloudera is one of the leading companies who are investing in the Hadoop community.
predicting the future is always a tricky thing to do and many this is especially the case when you try to predict the future of computing as this is a field in which a lot of people do have an opinion. However, looking at where we currently stand and what the limits of physics are (as currently known) we can do some modest predictions.
As a statement: "the future of computing is parallelism".
If we look at the current speed of processors (per core) we see that the speed (frequency) is leveling out. Reason for this is that if you increase the frequency you will more leakage in your transistors which are on your chip. leakage is recorded to the outside world as heat. So if we where able to run the chips on a higher frequency we will see that the heat will increase up till a level which cannot be cooled in a "normal" way and against normal prices.
What chip manufacturers are doing to cope with this is issue is to build multi-core processors. Having multiple cores running on a acceptable frequency which is providing enough computational power to the system while keeping the heat within an acceptable range is the way forward. We can already see good examples of this in the AMD liano and the nvidia fermi (image below) many-core processors.
The new breed of processors will be many-core processors holding large number of cores. This will require new ways of developing software and programs. With many-core processors you will have the option (and the need) to run your programs over multiple cores to make full use of your hardware. To be able to do so one will have to consider parallelism when developing code. Developing parallel processes is another way of thinking which is currently not adopted by the majority of the developers simply because they can do without. However, as we are getting more and more data (bigdata), processes and computations are getting more complex and users are not willing to wait very long developers will have to think about parallel programming very soon.
You have some languages which are specially designd to cope with many-core processors, one example is CUDA which is developed by nvidia. You do however not need a special language, python is very well able to cope with it and java is also able to just like C for example. Issue is that developers have to start thinking about it and need to get familiar with it (in my opinion).
So what is the future of computing, parallel computing and many-core processes. Thom Dunning is explaining it in more detail in his "future of high performance computing" lecture for the National Center for Supercomputing Applications which you can watch below:
For those who have been watching the lecture by Aaron Kimball from Google "cluster computing and mapreduce lecture 1" might have noticed that a big portion of the first part of the lecture is about networking and why networking is so important to distributed computing.
"Designing real distributed systems requires consideration of networking topology."
Let say you are designing a Hadoop cluster for distributed computing for a company who will be processing lots and lots of information during the night to be able to use this information the next morning for daily business. The last thing you want is that the work is not completed during the night due to a networking problem. A failing switch can, in certain network setups, make that you loose a large portion of your computing power. Thinking about the network setup and making a good network blueprint for your system is a vital part of creating a successful solution.
We take for example the previously mentioned company. This company is working on chip research and during the day people are making new designs and algorithms which needs to be tested in your Hadoop cluster during the night. The next morning when people come in they expect to have the results of the jobs they have placed in the Hadoop queue the previous day. The number of engineers in this company is enormous and the cluster has a use of around 98% during 'non working hours' and 80% during working hours. As you can see a failure of the entire system or a reduction of the computing power will have a enormous impact on the daily work of all the engineers.
A closer look at this theoretical cluster. - The cluster consists out of 960 computing nodes. - A node is a 1u server with 2 quad core processors and 6 gigabyte of ram. - The nodes are placed in 19' racks every rack houses 24 computing nodes. - There are 40 racks with computing nodes.
As you can see, if we would lose a entire rack due to a network failure we would lose 2.5% of the computing power. As the cluster is used 98% during the night we would not have enough computing power to do all the work during the night. Losing a single node will not be a problem, however losing a stack of nodes will result in a major problem during the next day. For this we will have to create a network blueprint in which we can ensure that we will not lose a entire computing stack. If we talk about a computing stack we talk about a rack of 24 servers in this example.
First we will have to take a look at how we will connect the racks, if you look at the Cisco website you will be able to find the Cisco Catalyst 3560 product page. The Cisco Catalyst 3560 has 4 Fiber ports and we will be using those switches to connect the computing stacks. However, to ensure the network redundancy we will use 2 switches for very stack instead of one. As you can see in the diagram below we crosslink the switches. SW-B0 and SW-B1 will both be handling computing stack B, switches SW-C0 and SW-C1 will be handling the network load for computing stack C etc etc. We will be connecting SW-B0 with fiber to SW-C0 and SW-C1. We will also connect SW-B1 with fiber to SW-C0 and SW-C1. In case SW-B0 or SW-B1 will fail the network can still route traffic to the the switches in the B computing stack and also the a computing stack. By creating the network in this way the network will not fail to route traffic to the other stacks, the only thing that will happen is that the surviving switch will have to handle more load.
This setup will however not solve the problem that the nodes that are connected to the failing switch will loose the network connection. To resolve this we will attache every node to two switches. Every computing stack has 24 computing nodes. The switch has 48 ports and we do have 2 switches. To solve this problem we place 2 network interfaces in every node. One will be in standby mode and one will be active. To spread the load at all even numbered nodes the active NIC will be connected to switch 0, at all uneven numbered nodes the active NIC will be connected switch 1. For the inactive (standby) NIC, all the even numbered nodes will be connected to switch 1 and all uneven numbered nodes will be connected to switch 0. In a normal situation the load will be balanced between the two switches, in case one of the two switches fails the standby NIC's will have to become active and all the network traffic to the nodes in the computing stack will be handled by the surviving switch.
To have the NIC's to switch to the surviving network switch and to make sure that operation continue as normal you will have to make sure that the network keeps looking at the servers in the same way as before the moment one of the switches failed. To do this you will have to make sure that the new NIC has the same IP address and MAC address. To do so you can make use of IPAT, IP Address Takeover.
"IP address takeover feature is available on many commercial clusters. This feature protects an installation against failures of the Network Interface Cards (NICs). In order to make this mechanism work, installations must have two NICs for each IP address assigned to a server. Both the NICs must be connected to the same physical network. One NIC is always active while the other is in a standby mode. The moment the system detects a problem with the main adapter, it immediately fails over to the standby NIC. Ongoing TCP/IP connections are not disturbed and as a result clients do not notice any downtime on the server. "
Now we have tackled almost every possible breakdown, however what can happen is that not one switch but both switches in stack break. If we look at the examples above this would mean that the stacks will be separated by the broken stack. To prevent this you will have to make a connection between the first and the last stack as you make between all the stack. By doing so you make a 'ring' of your network. With correct setup of all your switches and making good routing a failover routings your network can also handle the malfunction of a complete stack in combination with both switches in the stack.
Even do this is a theoretical blueprint, developing your network in such a way in combination with writing your own code to controle network flows, scripts to control the IPAT and the switching back of IPAT, thinking about reporting and alerting mechanisms will make a very solid network. If there are any questions about this networking blueprint for cluster computing please do send me a e-mail or post a comment. I will reply with a answer (good or bad) or explain things in more detail in a new post.
The Google Code Channel on Youtube is having some lectures about Cluster Computing and Hadoop. I am currently viewing them all and I can say they are worth watching. The first one in the serie is given by Aaron Kimball, "Problem solving on Large-Scale clusters".
Some quick nice quotes:
- Parallelization is "easy" if processing can be cleanly split into n units. - Processing more data means using more machines at the same time. - Cooperation between processes requires synchronization. - Designing real distributed systems requires consideration of networking topology.
Aaron will go into the fundamentals and in the upcoming lectures we will dive into more details. you can review the presentation here below. You can also find all the shows on the Google code site and some of the questions and answers. I will also post the other shows with some of my comments after I have viewed them.
