Monday, January 15, 2018

Oracle Linux - convert XML to JSON

for people who are not that big a fan of XML files and still need to use XML structures every now and than when they get data from another system the xml2json solution might be a good thing to look into. The xml2json util is an opensource utility written in Python which will convert XML data into JSON data, as well as the other way arround. Using this on the Oracle Linux commandline when you code bash scripting can save a lot of time. Especially if you use xml2json in combination with jq you can save a lot of time in most cases.

As an example, we take an example XML file we found on the Microsoft website. And for convenience I create a gist from it so you can more easily grab it from the command line. The XML contains a list of books and the details of the books.

<?xml version="1.0"?>
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<genre>Computer</genre>
<price>44.95</price>
<publish_date>2000-10-01</publish_date>
<description>An in-depth look at creating applications with XML.</description>
</book>
<book id="bk102">
<author>Ralls, Kim</author>
<title>Midnight Rain</title>
<genre>Fantasy</genre>
<price>5.95</price>
<publish_date>2000-12-16</publish_date>
<description>A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world.</description>
</book>
<book id="bk103">
<author>Corets, Eva</author>
<title>Maeve Ascendant</title>
<genre>Fantasy</genre>
<price>5.95</price>
<publish_date>2000-11-17</publish_date>
<description>After the collapse of a nanotechnology society in England, the young survivors lay the foundation for a new society.</description>
</book>
<book id="bk104">
<author>Corets, Eva</author>
<title>Oberon's Legacy</title>
<genre>Fantasy</genre>
<price>5.95</price>
<publish_date>2001-03-10</publish_date>
<description>In post-apocalypse England, the mysterious agent known only as Oberon helps to create a new life for the inhabitants of London. Sequel to Maeve Ascendant.</description>
</book>
<book id="bk105">
<author>Corets, Eva</author>
<title>The Sundered Grail</title>
<genre>Fantasy</genre>
<price>5.95</price>
<publish_date>2001-09-10</publish_date>
<description>The two daughters of Maeve, half-sisters, battle one another for control of England. Sequel to Oberon's Legacy.</description>
</book>
<book id="bk106">
<author>Randall, Cynthia</author>
<title>Lover Birds</title>
<genre>Romance</genre>
<price>4.95</price>
<publish_date>2000-09-02</publish_date>
<description>When Carla meets Paul at an ornithology conference, tempers fly as feathers get ruffled.</description>
</book>
<book id="bk107">
<author>Thurman, Paula</author>
<title>Splish Splash</title>
<genre>Romance</genre>
<price>4.95</price>
<publish_date>2000-11-02</publish_date>
<description>A deep sea diver finds true love twenty thousand leagues beneath the sea.</description>
</book>
<book id="bk108">
<author>Knorr, Stefan</author>
<title>Creepy Crawlies</title>
<genre>Horror</genre>
<price>4.95</price>
<publish_date>2000-12-06</publish_date>
<description>An anthology of horror stories about roaches, centipedes, scorpions and other insects.</description>
</book>
<book id="bk109">
<author>Kress, Peter</author>
<title>Paradox Lost</title>
<genre>Science Fiction</genre>
<price>6.95</price>
<publish_date>2000-11-02</publish_date>
<description>After an inadvertant trip through a Heisenberg Uncertainty Device, James Salway discovers the problems of being quantum.</description>
</book>
<book id="bk110">
<author>O'Brien, Tim</author>
<title>Microsoft .NET: The Programming Bible</title>
<genre>Computer</genre>
<price>36.95</price>
<publish_date>2000-12-09</publish_date>
<description>Microsoft's .NET initiative is explored in detail in this deep programmer's reference.</description>
</book>
<book id="bk111">
<author>O'Brien, Tim</author>
<title>MSXML3: A Comprehensive Guide</title>
<genre>Computer</genre>
<price>36.95</price>
<publish_date>2000-12-01</publish_date>
<description>The Microsoft MSXML3 parser is covered in detail, with attention to XML DOM interfaces, XSLT processing, SAX and more.</description>
</book>
<book id="bk112">
<author>Galos, Mike</author>
<title>Visual Studio 7: A Comprehensive Guide</title>
<genre>Computer</genre>
<price>49.95</price>
<publish_date>2001-04-16</publish_date>
<description>Microsoft Visual Studio 7 is explored in depth,looking at how Visual Basic, Visual C++, C#, and ASP+ are integrated into a comprehensive development environment.</description>
</book>
</catalog>
view raw example_xml hosted with ❤ by GitHub
If we now would like to do something with it in a JSON format we can can use the xml2sjon command as shown below to get a valid JSON structure we can work with.

./xml2json.py -t xml2json -o test1.json test1.xml --strip_text

{
"catalog": {
"book": [{
"@id": "bk101",
"author": "Gambardella, Matthew",
"title": "XML Developer's Guide",
"genre": "Computer",
"price": "44.95",
"publish_date": "2000-10-01",
"description": "An in-depth look at creating applications \n with XML."
}, {
"@id": "bk102",
"author": "Ralls, Kim",
"title": "Midnight Rain",
"genre": "Fantasy",
"price": "5.95",
"publish_date": "2000-12-16",
"description": "A former architect battles corporate zombies, \n an evil sorceress, and her own childhood to become queen \n of the world."
}, {
"@id": "bk103",
"author": "Corets, Eva",
"title": "Maeve Ascendant",
"genre": "Fantasy",
"price": "5.95",
"publish_date": "2000-11-17",
"description": "After the collapse of a nanotechnology \n society in England, the young survivors lay the \n foundation for a new society."
}, {
"@id": "bk104",
"author": "Corets, Eva",
"title": "Oberon's Legacy",
"genre": "Fantasy",
"price": "5.95",
"publish_date": "2001-03-10",
"description": "In post-apocalypse England, the mysterious \n agent known only as Oberon helps to create a new life \n for the inhabitants of London. Sequel to Maeve \n Ascendant."
}, {
"@id": "bk105",
"author": "Corets, Eva",
"title": "The Sundered Grail",
"genre": "Fantasy",
"price": "5.95",
"publish_date": "2001-09-10",
"description": "The two daughters of Maeve, half-sisters, \n battle one another for control of England. Sequel to \n Oberon's Legacy."
}, {
"@id": "bk106",
"author": "Randall, Cynthia",
"title": "Lover Birds",
"genre": "Romance",
"price": "4.95",
"publish_date": "2000-09-02",
"description": "When Carla meets Paul at an ornithology \n conference, tempers fly as feathers get ruffled."
}, {
"@id": "bk107",
"author": "Thurman, Paula",
"title": "Splish Splash",
"genre": "Romance",
"price": "4.95",
"publish_date": "2000-11-02",
"description": "A deep sea diver finds true love twenty \n thousand leagues beneath the sea."
}, {
"@id": "bk108",
"author": "Knorr, Stefan",
"title": "Creepy Crawlies",
"genre": "Horror",
"price": "4.95",
"publish_date": "2000-12-06",
"description": "An anthology of horror stories about roaches,\n centipedes, scorpions and other insects."
}, {
"@id": "bk109",
"author": "Kress, Peter",
"title": "Paradox Lost",
"genre": "Science Fiction",
"price": "6.95",
"publish_date": "2000-11-02",
"description": "After an inadvertant trip through a Heisenberg\n Uncertainty Device, James Salway discovers the problems \n of being quantum."
}, {
"@id": "bk110",
"author": "O'Brien, Tim",
"title": "Microsoft .NET: The Programming Bible",
"genre": "Computer",
"price": "36.95",
"publish_date": "2000-12-09",
"description": "Microsoft's .NET initiative is explored in \n detail in this deep programmer's reference."
}, {
"@id": "bk111",
"author": "O'Brien, Tim",
"title": "MSXML3: A Comprehensive Guide",
"genre": "Computer",
"price": "36.95",
"publish_date": "2000-12-01",
"description": "The Microsoft MSXML3 parser is covered in \n detail, with attention to XML DOM interfaces, XSLT processing, \n SAX and more."
}, {
"@id": "bk112",
"author": "Galos, Mike",
"title": "Visual Studio 7: A Comprehensive Guide",
"genre": "Computer",
"price": "49.95",
"publish_date": "2001-04-16",
"description": "Microsoft Visual Studio 7 is explored in depth,\n looking at how Visual Basic, Visual C++, C#, and ASP+ are \n integrated into a comprehensive development \n environment."
}]
}
}
As we have a JSON strcuture we can also use jq on it. You can find jq also on the Oracle YUM repository for your use. the xml2json utility is however not included in the Oracle YUM repository at this moment so you will have to grab it from github where you will find the xml2json.py file which you need to ensure is on your Oracle Linux system to be able to use it.

No comments: