Oracle Fabric Interconnect - connecting FC storage appliances in HA

The Oracle Fabric Interconnect is a hardware network appliance which enables you to simplify the way servers are connected to the network within your datacenter. The general idea behind the Oracle Fabric Interconnect is that you connect all your servers to one (or two for high availability) Oracle Fabric Interconnect appliances using Infiniband and virtualize all  needed NIC's and HBA's you might require on your server. The effect is that the number of cables you need per server is reduced enormously and that every change to your network infrastructure is suddenly a software change. This enables you to make use of software defined networking instead of pulling physical cables to your servers.

Oracle Fabric Interconnect employs virtualization to enable you to flexibly connect servers to networks and storage. It eliminates the physical storage and networking cards found in every server and replaces them with virtual network interface cards (vNICs) and virtual host bus adapters (vHBAs) that can be deployed on the fly. Applications and operating systems see these virtual resources exactly as they would see their physical counterparts. The result is an architecture that is much easier to manage, far more cost-effective, and fully open.

As stated, Oracle Fabric Interconnect is a solution for networking, however it can also provide solutions to connect to fiber channel storage appliances. If you look at the below image you see how you can expand the number of modules within a Oracle Fabric Interconnect appliance.

This allows to add multiple types of modules. Oracle provide a number of modules which can be used to expand the standard Oracle Fabric Interconnect appliance. 

One of the expansion kits is to enable you to connect your Oracle Fabric Interconnect appliance to a Fibre Channel based storage appliance. For example a EMC storage appliance. A commonly seen implementation scenario for this is the Oracle Private Cloud Appliance where customers like to make use of the Oracle Private Cloud Appliance and will use the "onboard" ZFS storage for storing machine images however want to consume additional storage from existing EMC storage solution they already have deployed within their IT footprint. 

When adding the required I/O modules to the Oracle Fabric Interconnect appliance you can make virtual HBA's available to your servers. The virtual HBA's are actually directly talking via IPoIB to the Oracle Fabric Interconnect appliance I/O modules that connect to the EMC storage appliance. However, you do not have the need to add HBA's to all servers as they are already connected to the Oracle Fabric Interconnect appliance. 

To ensure you make this high available you will have to ensure you do the connections to your EMC storage appliance correct. A number of best practices are available. The best practice for the Oracle Private Cloud Appliance is shown below. 

Even though the above outlines the best practice to deploy a FC connection from a Oracle Private Cloud Appliance, in effect this is also the best practice for connection EMC via FC to your own deployed Oracle Fabric Interconnect appliance solution which you might use to connect all kinds of servers.