I just recently came back from a 4 day hackers convention in the Netherlands, har2009, and have been talking to a lot of people. One of the things that came up during some of the discussions was security and the time a vendor needs to patch some things. In most cases a exploit is found and used in the field before the vendor (for example Oracle or a firewall vendor) is aware of it. After it is made aware of the fact that a security issue is available it will take some time to fix it and make sure a patch is available. After the patch is available administrators still need to apply the patch.
So even do vendors are working on making the systems as secure as possible and developers who are developing on those platforms are trying to develop the code as secure as possible you will still see that security breaches will happen. You can simply not find all the errors in the code even if you test, test, retest and retest you will still have bugs in your code and ways the code can be used you never ever have been thinking about.
So simply to say, it is almost impossible to create a system that is a hundred percent secure. So if you as an oracle developer develop a system, think about it as an architect or are responsible as a project manager you will have to know at least some of the basics of security. I will not go into all kinds of coding examples to make your code as secure as possible I will however provide some best practices on mainly network security and architecture.
As an example I will use the below image I found in a blogpost by Steven Chan on “Loopbacks, Virtual IPs and the E-Business Suite”. Now for the this document we are not going into the fact this is a Oracle E-Business Suite setup we just take it for granted that web nodes that are used are just simple web-nodes and we are unsure of what they do, it is not really important for the examples as you should harden en secure every system as much as possible without looking at what the system is doing.
This statement can create already some questions, why should I harden a system which is not holding any mission critical information, is “normal” security not enough? Simply put, no. why not, if you “lower” security on non mission critical systems you have a weak link in your security. Possibly an attacker can compromise this system and use it as a foothold and starting point for future attacks. You should protect every system as was it the most important system in your entire company. Never compromise on security for whatever reason. Budget, time, deadlines, policies not in place,… none of this can ever be a excuse to “lower” security on a system.
In the above picture we see the network as shown in the blogpost of Steven Chan. In basics nothing is wrong with this approach and the network diagram is not intended to display security it is used to explain another issue. We will however use it as the basis of explaining some things. When reading the rest of this article the question might arise that you as a developer will not have the “power” to request all the things I will state. However, in my opinion you should mention them and ask about them when developing a system because you simply would like to have the most secure and stable system possible.
External firewalls:
With external firewalls I mean non local firewalls, so all firewalls which are in place somewhere in the network. I will make a point on internal firewalls later is this article where I will be talking on the subject of firewalls on your Oracle server. As we can see in the network drawing we have 3 firewalls. A firewall on the outside of your network, a firewall between DMZ0 and DMZ1 and one between DMZ1 and DMZ2. DMZ0 holds the reverse proxy, DMZ1 holds a loadbalancer and 2 web nodes. DMZ2 is not really a demilitarized zone because it also holds clients in it as the internal users. Possibly the network drawing is showing another firewall which protects your database server.
However the setup looks to be valid and is protecting you with a layer of firewalls from the external network (the internet in this case) it not complete. You should have another firewall in place to make this more secure. You should place a firewall between the internal users and the rest of DMZ2 which contains web-nodes and a load balancer.
Reasons for this, (A) even do most companies try to trust the users you can never be sure. It is even so that a large portion of the hacking attempts is done from within the company by for example disgruntled employees. (B) The users will most likely have access to internet and so can be compromised by malware and rootkits which could potentially become a gateway into your company. So you cannot trust your users (or the workstations they are using) and you should at least have a firewall between your servers and your clients. It is even advisable to secure your servers from your internal users in the same way as you would do to protect them from the internet.
Internal firewalls:
So as we stated in the firewall section it is great to have firewalls in place to make sure intruders cannot enter your DMZ without any hurdle however when thinking about security and your DMZ you have to keep one thing in mind. What happens if a intruder gets access to your DMZ, what happens when an intruder has compromised a server in your DMZ without you knowing. With the setup as it stands now all other servers are now wide open.
To prevent this it is advisable to have internal firewalls on your servers to harden them for attacks from within the DMZ. So you will have secure islands within the secure DMZ sector. This will make it harder for an attacker to take over multiple servers before being spotted. You can use for example iptables to harden your servers with internal firewalls. The setup of iptables rules can be a little hard if you do this for the first time and even if you have a long history of working with iptables it can be hard to maintain all local iptables settings when you have a large amount of servers. For this it can be handy to use tools like fwbuilder or kmyfirewall.
When setup an internal firewall you can make sure only the ports are open per interface that are needed for a minimum service on this network segment. I will explain more about network segmentation in combination with internal firewalls in the next topic.
Network Segmentation:
As stated in the previous topic of internal firewalls one has to minimize the open ports and services per network segment so only the ports are open that are needed for this segment. With network segmentation you will make use of all network interfaces in your server to create multiple networks. For example you will have a customer network, a data network and a maintenance network.
Users for example will only use the customer network as we know the servers are webnodes so we will only have to use TCP port 443 for HTTPS traffic over TLS/SSL. Users will most likely have no need to access anything else so no need to provide them the option to connect to those ports.
The data network can be a separate network running over a different NIC which can be used to connect to the database from a web-node or connect storage appliances. This will most likely only be used for server to server connections so most likely no real-life users have a need to access. So people who are working on the customer network will not be able to connect to ports on the data network. First because of they are not on the same physical network and even if you have some vlan configuration mistake or something like that you will have your iptables firewall which will restrict them based upon the source IP address. On this network you do NOT have to open for example port 443 or a port for for example SSH or telnet.
The maintenance network will be used by administrators and on this network you can close for example direct access to network storage appliances, web applications and such. You have to open ports like SSH on this network so administrators can access the servers by secure shell and do their work. This network will be the most wanted by attackers because it will have ports open that can be used and exploited to gain access to the console of the severs.
So by physically have some network segmentation you can separate services and groups of users based upon their role. Are they users or are they administrators of the systems, based upon this you can provide them access to a certain network segment with its own routers, switches and entire network topology.
Encryption:
We have talked about firewalls on the outside and in the inside, we have discussed separating networks. However, even if you have done all this you cannot feel secure. When thinking about security you always have to consider the network compromised. So if you consider the network compromised someone can sniff all the network traffic by using a network sniffer. So you should state, very strongly, that you can only use encrypted network connections.
This means, never use telnet and only use SSH, Never use FTP use SFTP or SCP. Never use HTTP use HTTPS instead. Use encrypted SMTP when you are sending out mails. So you can make your network even more secure by making only use of encrypted connections. By using only encrypted connections a possible attacker who is using a package analyzer on your network traffic cannot sniff cleartext passwords.
You should restrict users from using open services like telnet, ftp and such by simply closing the ports with iptables.
So coding a custom application can be a little harder when using a secure and encrypted connection however it will not be so much more work and the benefits are huge. As this post is intended for Oracle developers it might be good to have a look at the Oracle Application Server Administrator’s Guide, Overview of Secure Socket Layer SSL In Oracle Application Server. So huge that if you implement it correctly a possible attacker will only be able to see scrambled data and not a single useful packet of information.
Patching policy:
So we have setup external and internal firewalls, encrypted the network traffic and separated the network into parts, can we feel save? No, not really. Even do we have taken all those steps we have to take into consideration that we still have to open some ports and very open port is a possible point of entry for a attacker. If you are for example running a web node you will have to run a webserver and as this post is intended for Oracle developers you will most likely be running a Oracle Application server which uses Apache. Even do Apache is a really good and secure webserver still it has its security issues. So if the vendor, Oracle, releases a patch you need to give this a very good look and I advise to apply it after testing it and reading the specs a couple of times.
The patch is released for a reason, this reason is solving bugs. Not always security related bugs however they solve some issues. As you will see most exploits are up and running for old versions and systems who are not being patched are getting more and more vulnerable for a attack every time they miss a patch. It can be that your server can be compromised or services are disturbed, in all cases it is not desirable. So again, my advice, whenever a patch is released, give it a good look and if it is not causing a major problem you will have to apply it as soon as possible.
If a patch is giving a real problem with your custom code for example you should not simply decide to not apply the patch. You should solve the problem which is holding you back from applying the patch and after that apply the new code and the patch on the systems. If you decide to not apply a patch you can have the situation that some time later a critical patch is released and you still have to apply the first patch as it is a prerequisite for the second patch. Then you still have to do the code fixing. Or you can decide also not to apply the second patch and…… you are simply making your system more and more insecure by every patch you miss.
Securing code:
This post is about network security and no so much about code security. However, you vendor (oracle) will provide you security patches, on your own code you (and your team) will be the only one who will be able to release security patches so it is important to not only test the standard things in your code. Also have a look at for example SQL injection, buffer overflows, incorrect error handling…. Etc ect.
Password policies:
Not really a topic on network security however something to mention, passwords. It will go without saying that your administrators will have to have some policy in place on passwords, how long it can be, what the strength should be, etc etc. However, a topic what I would like to mention is Public Key Authentication. In this case you “do not need” a password to login to SSH. You will use a public key to authenticate yourself to the server. The good thing about this is for example that you can grant users to login as root without them knowing the password. You add a key to the user root and based upon this key users can login. So if you want to revoke the rights you do not have to change the password and inform all who still have access to this account. You can simply remove the key for this person and for the rest of the users nothing will change.
Public Key Authentication is considered much stronger than password Authentication so it is advisable to use only Key Authentication. This can also be used for for example SCP when you move (automatically) files between servers you can also make use of Key Authentication. So give this a good look.
Monitoring and sniffing:
Now we are getting a save network. However, if we want to be sure that nothing “funny” is happening you want to monitor your network with a intrusion detection system. For example Snort, snort is a opensource IDS which can monitor and detect strange behavior. This way you can monitor anomalies and possible attacks on your network.
Also it can be good to have some log analyzers ready. All your systems will provide log files. Most administrators are only looking into the log files (and the mails to root) when a problem is found. However it can be very beneficial to write your own custom log analyzer and create some portal like environment where you consolidate the log files and analyze them. You can scan for specific error messages and for example failed login attempts. Use your creativity.
So with the combination SNORT and log analyzers you can keep a good look if someone still tries to do some damage in your network.
