Monday, September 19, 2016

Retrieving IaaS Instance Metadata with Oracle Linux

When creating a new IaaS instance in the Oracle Compute cloud Service you will notice that some information is pre-populated for you and some settings are done. For example, the hostname is set, it will have IP information and you will have the keys populated for the OPC account you have defined.

This is done by a first boot script provided by Oracle which will take information from outside the standard operating system and use it to configure the Oracle Linux instance to your liking. For this it will have to have access to a set of information which is not part of the deployment template as this is specific information for your deployment. The way it is doing it is by making use of the instance metadata which is provided by a REST API.

The rest API provides three sets of information;
  • attributes
  • meta-data
  • user-data
The meta-data is used during the first boot of the instance to ensure everything is configured to your needs. Oracle documentation describes this as follows:

"Two types of metadata are stored within your instances: user-defined instance attributes that you can define explicitly while creating instances, and predefined instance metadata fields that are stored by default for all instances. Scripts and applications running on the instances can use the available metadata to perform certain tasks. For example, SSH public keys that are specified while creating an instance are stored as metadata on the instance. A script running on the instance can retrieve these keys and append them to the authorized_keys file of specified users to allow key-based login to the instance using ssh."

When do you need the REST-API

Even though Oracle will take care of all post deployment configuration when you use a standard Oracle provided template the REST API is interesting in a number of cases.

  1. When you create your own custom template and you will have to build (or modify) a custom first boot script. 
  2. When you are building scripting to install certain specific software and are in need of information from the meta-data set. 

In the first instance you will most likely use primarily the meta-data information set, in the second instance you will most likely use a combination of both meta-data and user-data. The user-data set will enable you to provide (via additional JSON) information to the machine via the REST-API. For example, if you want to indicate if a certain application instance should be the master or a slave in a cluster you can use the user-data information set which will state the role of the machine. In this post we will focus on the meta-data information set and not the user-data information set provided via the REST API

In general, if you are building automation in your deployment model, in whatever way or form, you will at one point in time want to use the REST API.

How to access the REST API

The REST API is accessible from within your instance via the address 192.0.0.192 using http. This means that if you try to access http://192.0.0.192 you will be talking to the REST API. As you will not have a graphical user interface you will have to access that by using curl. An example is shown below of how to access the root of the REST API:

[opc@testbox08 ~]$ curl http://192.0.0.192
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2009-04-04
latest
[opc@testbox08 ~]$

The REST API is compatible with the API you will see at for example at Amazon webservice, this makes it very easy for people who already have invested in automation and scripting based upon the standards developed initially by Amazon.

Based upon your preferences it is good practice to use latest or 1.0 as the version. To be fully sure that your automation scripting will always work and not break when a new latest version is launched it is advisable to use a specific version. In our case in the rest of the blogpost we will use the explicit version 1.0

As it is with REST API’s we can visualize the REST API in a tree shape. One thing you have to remember, even though the REST API provides a loosely compatibility with the Amazon REST API it is not the same. Oracle is also warning for this on the documentation itself in the following words:

You may see certain additional metadata fields, such as reservation-id, product-codes, kernel-id, and security-groups, that aren’t documented. Don’t retrieve and use the values in the undocumented fields

Reviewing the meta-data
The below meta-data components are available, all based upon version 1.0. While some are documented some are not and per statement from Oracle you should not rely on them while creating code.

[opc@testbox08 ~]$  curl http://192.0.0.192/1.0/meta-data
ami-id
ami-launch-index
ami-manifest-path
ancestor-ami-ids
block-device-mapping
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
placement
product-codes
public-hostname
public-ipv4
public-keys
ramdisk-id
reservation-id
security-groups
[opc@testbox08 ~]$

Below is an explanation per meta-data component. The majority will work, however some will not work as you might be used to when using Amazon. You will also see that a number of components have the Amazon naming to ensure compatibility with code you might have created for Amazon.

ami-id
The AMI-ID provides you a unique ID for your IaaS instance. The name AMI-ID comes from Amazon Machine Image.

You will be able to access the ami-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ami-idami-launch-index

ami-manifest-path
The ami-manifest-path is something that comes from Amazon and within Amazon it provides a path to the AMI manifest file in Amazon S3. In the Oracle Cloud this is not supported, a AMI manifest is a bit comparable with what oracle understands as a orchestration however not fully the same. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the ami-manifest-path by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ ami-manifest-path

ancestor-ami-ids
This is by default not supported by Oracle Cloud and comes from the Amazon implementation of the API. The AMI IDs of any instances that were rebundled to create this AMI. This value will only exist if the AMI manifest file contained an ancestor-amis key. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the ancestor-ami-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ancestor-ami-id

block-device-mapping
This provides within Amazon the mapping to the block devices used by the instance. However, within Oracle this is currently not support even though it is available as an object in the REST API. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the block-device-mapping by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/block-device-mapping

instance-id
The instance ID is the unique ID for your instance running in the Oracle Cloud. Exaample of an instance-ID is : /Compute-acme/joe.jonathan@example.com/debc974c-852e-4bd2-acd6-45a2de2109fd

You will be able to access the instance-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/instance-id

ami-launch-index
If you started more than one instance at the same time, this value indicates the order in which the instance was launched. The value of the first instance launched is 0. This is a feature from Amazon and is currently not supported within Oracle Cloud officially however; it will provide you a value back which can be used.

You will be able to access the ami-launch-index by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ami-launch-index

instance-type
The instance-type will provide you information about the sizing of the machine. Memory and CPU resources available for the instance.

You will be able to access the instance-type by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/instance-type

kernel-id
Provides the ID of the kernel launched with this instance, if applicable. Officially not supported by Oracle and is a part of the Amazon compatibility. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the kernel-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/kernel-id

local-hostname
The local-hostname will provide the DNS name of the instance as it is used within the Oracle cloud.

You will be able to access the local-hostname by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/local-hostname

local-ipv4
The local-ipv4 will provide you the private IP address of the instance as it is used internally within the Oracle cloud.

You will be able to access the local-ipv4 by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/local-ipv4

placement
Placement will provide you the the Availability Zone in which the instance launched. Officially not supported by Oracle and is a part of the Amazon compatibility. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the placement by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/placement/availability-zone

product-codes
Product-codes provide you with the Product codes associated with the instance, if any. Officially not supported by Oracle and is a part of the Amazon compatibility. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the product-codes by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/product-codes

public-hostname
The public-hostname will provide the DNS name of the instance as it is used to face outwards of the Oracle Cloud

You will be able to access the public-hostname by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/public-hostname

public-ipv4
The public-ipv4 will provide you the public IP address of the instance as it is used to face outwards of the Oracle Cloud

You will be able to access the public-ipv4 by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/public-ipv4

public-keys
SSH public key specified while creating the instance, where{index} is a number starting with 0. public-keys/{index}/openssh-key

You will be able to access the public-keys by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ public-keys/{index}/openssh-key

ramdisk-id
The ID of the RAM disk specified at launch time, if applicable. Officially not supported by Oracle and is a part of the Amazon compatibility. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the ramdisk-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ placement/ ramdisk-id

reservation-id
The ID of the reservation. This is available from the Oracle REST API however Officially not supported by Oracle and is a part of the Amazon compatibility. As it is not supported by Oracle it wise to not base any scripting based upon this.

You will be able to access the ramdisk-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ placement/reservation-id

security-groups
The names of the security groups applied to the instance. This is an Amazon feature and not available at the Oracle Cloud. Amazone uses it to ensure that after launch, you can only change the security groups of instances running in a VPC. Such changes are reflected here and in network/interfaces/macs/mac/security-groups.

You will be able to access the ramdisk-id by executing the following curl command:
curl http://192.0.0.192/1.0/meta-data/ placement/security-groups

No comments: