Oracle exadata InfiniBand connections

Oracle Exalytics is the Oracle appliance for running business intelligence in memory. Oracle Exalytics can be deployed in existing IT environments by itself or in conjunction with Oracle Exadata and/or Oracle Exalogic to enable extreme performance. A general Exalytics machine equipped with 1 terrabyte of memory which can be used for in memory operations. Even though it is capable of running 1 terrabyte in memory it will still require that data is provided to the system. When you have “stable” data which is not changing frequently this is not a direct problem however if you have a dataset which changes rapidly you can experience performance that is not up to what you expected. In those cases when you have a dataset with a high number of changing data it can be beneficial to connect your Exalytics appliance, or a cluster of Exalytics appliances, to an Oracle ExaData appliance.

One of the benefits of connection you Exalytics appliance to a ExaData appliance is that you can connect them via infiniband. Both appliances are equipped with infiniband connectors. InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices.

By default a Oracle ExaData (full rack) setup is equipped with 3 infiniband switches, one spine switch and 2 leave switches which are connected a shown below which hardens the setup against the failure of one of the switches. This makes the appliance setup more resilient and enables you to update firmware of your switches without the need for downtime.

When connecting a single Exalytics to your exadata appliance it is common practice to connect to both leave switches. You will have 2 infiniband ports on the back of your Exalytics appliance, those will be used to connect to the spine switches. This helps you to make the connection resilient against the failure of one of the switches.

More information can be found on the slidedeck about this subject shown below.

Oracle infiniband cabling for Oracle appliances from Johan Louwers