Auditing Change Settings Iptables

Question:

There is a problem to create a system for auditing change settings iptables (config file and binary files). After the search for solutions at the moment focused on SELinux. Tripwire and LIDS are not appropriate in my case.

Tripwire - show only the most recent changes, but will not show the user how it was committed, as well as all the previous changes.

LIDS - Requires a patch to the kernel. This can not be performed, as opposed to customer.

I need a realtime monitoring state of iptables as I have a server and there are several more users as root.

Answer:

Iptables

Requiring auditing iptables sounds not all that complicated. However it would be good to realize how iptables works (and who/what works with it). To summarize: Iptables (better: the Netfilter framework) works by having the (/sbin/)iptables binary read the default configuration file (/etc/sysconfig/iptables on RHEL) and supply that data, the ruleset) to the kernel (as the kernel must never be allowed to read userland). The ruleset is loaded at system bootup by the initscript and can be unloaded or reloaded by those that can execute the initscript. A system can also contain one or more applications that provide a user with a remote, commandline, curses or graphical interface to alter the ruleset (the application itself interfacing with the system using the iptables binary), at runtime the ruleset can be altered by users who can execute the iptables iptables binary (or Sudo), and there may be applications that automate ruleset changes (Snort third party applications like Guardian, fail2ban or equivalent) in specific, designated chains for traffic blocking purposes.

Breaking down the requirements

Having realized the auditing subject breaks down in to three distinct areas: binary, on-disk and in-memory configuration, access, and if I get this correctly, your extended auditing requirements are for:
- the iptables binary (DAC, hash),
- the iptables configuration files (DAC, hash, changes),
- present iptables configuration (changes),
- issued iptables commands (user logging),
- log monitoring, (Sudo, any third party applications),
- iptables running state.

Monitoring static files

Any static filesystem item (a regular file residing on a regular fs) has DAC rights (ownership and access rights) and a checksum. In file integrity verification (and auditing) checking these basic attributes help ascertain a file has not been tampered with. (SELinux works on top of DAC rights to make certain only users with the right context can Do Stuff). Well-known OSS file integrity verification applications can be divided in three classes according to response time (looking at internal features): passive (Aide, Integrit, Osiris, tripwire, rpm), active (Samhain, Monit) and instantaneous (Auditd). Passive here means the application has no internal features that would allow it to run as a daemon (like Samhain and Monit can) and has to be run manually or as a cronjob. RPM can be used to verify file integrity but of course in terms of configurability its scope is limited to only those files that are installed by RPM. Like Samhain, monit can check file hashes at an interval. Auditd, the SELinux logging companion, works like iptables in that it uses an in-kernel ruleset to determine what it needs to log. Reading /var/log/audit/audit.log therefore provides alerting on a near-instantaneous level. For example enabling a rule "-w /etc/sysconfig/iptables -p w -k CFG_iptables" in /etc/audit/audit.rules (or use 'auditctl') will result in a logged line with the "CFG_iptables" tag at the moment /etc/sysconfig/iptables is written to. (Logging implies reading logs or parsing them: we'll get to that later on.) 

Hint: 

One remark about risk vs performance: having any file integrity verification application check the system continuously for a large set of files will result in degraded performance due to I/O. It is therefore perfectly possible, in cases maybe even recommended, to determine your strategy. If you use three tiers you could have Auditd concern itself with contents of /etc (and anything mentioned in the CAPP, LSPP, NISPOM or STIG rulesets, see: /usr/share/doc/audit-*/*.rules), run Samhain on a half-hourly schedule targetting crucial binaries and risk areas like HTTP upload or inbound FTP directories and run Aide on a four hour scheme to cover anything else that requires covering.

* Warning: since you indicated having multiple root account users this implies that any root account user can change configuration. It is strongly advisable to enable * independent, remote file integrity verification with Samhain + Beltane or webjob + a file integrity verification application of your choice.

* Caveat: auditing changes in hashes or DAC rights is complementary: it will for instance not show which user changed something or what content changed.

Monitoring dynamic content

Taking iptables as example we know that 'cat /proc/net/ip_tables_names|xargs -iX iptables -n -L -t 'X' ' will output all rules. We know that creating a checksum can be done reading stdin. Therefore running 'cat /proc/net/ip_tables_names|xargs -iX iptables -n -L -t 'X' |sha1sum' will provide the hash for the combined result of outputting all loaded iptables rules. Storing the hash allows for easy comparison later on. Following the same principle you can easily expand checking to a chain or even rule level. Automating this is not hard. In absence of something better I do use a script myself for the purpose of validating dynamic contents of different Netfilter tables. (If you want help with creating such a script feel free to have a go at it, create a separate thread in the LQ /Programming forum and add your code.) And since the hash won't match if the iptables ruleset is not loaded or not loaded properly this also covers the "iptables running state" part).
* Hint: note that if you already deploy remote monitoring a script similar to the 'cat ..|sha1sum' oneliner could be used to drive a /etc/xinetd.d/ script to provide the hash over say HTTP for comparison.

* Warning: for local binaries to produce results or checks to trust, it is required to verify the integrity of the system.

* Caveat: there may be firewall ruleset management applications that allow you to verify ruleset integrity. I just don't know of any. Doing research never hurts. On Sourceforge you will find the ITVal project that promises you help with validating changes to your firewall policy. It just didn't work for me. Feel free to try.

Monitoring content changes

Like GNU/Linux does not come with extended auditing enabled, it has no default means to record, play back or display differences in configuration files. File integrity verification applications can tell you in the most simplest of ways if something changed, not what. (Any tool based on) Rsync or large scale system versioning and configuration management suites (Puppet, Cfengine, et cetera) may help you solve this problem. (All implying trusted remote storage BTW).

* Warning: since you indicated having multiple root account users this implies that any root account user can change configuration. It is strongly advisable to avoid kludges or solutions that solely depend on local means.

Logging, log monitoring, remote logging

Note that when talking about a "root account user" we talk about qualified and authorized systems administrators. This means that root account usage rights given to any user out of laziness (e.g. instead of figuring out more secure means) should be take away immediately and replaced with the appropriate Sudo configuration. Note that this even goes for users who require access to multiple commands that need to be run as root. Sudo can easily be configured to accomodate those types of users. As this system has multiple root account users it is crucial to provide each with a separate unprivileged user account to start out from. No user should be allowed to log in as root on the console (and definately not from remote!). (And also mind your password complexity and aging!)

* Warning: ignoring the above will severely impact security and limit auditing integrity.

While Sudo (when appropriately configured) will log issued commands it will not do so for users that have been allowed root account access. To provide separated per user, independent, timestamped logging of commandline (and output!) you will configure those unprivileged user accounts to use 'Rootsh' to sudo to root by allowing them to run something like 'sudo rootsh -u root -i'. Depending on configuration the logfiles may go to /var/log/rootsh or syslog. In addition to monitoring above this will provide you with enough of an auditing trail to help you determine which user changed what files contents at what time.

* Warning: with multiple root account users you are strongly advised to enable logging to a trusted, remote syslog server for about everything.

Logging means reading logs. Without automating it this soon becomes more than a 24/7 job. For instantaneous response to priority 1 alerts you will need a log watcher like Swatch or SEC (or search Freshmeat, Sourceforge, Nongnu, Berlios) that can alert (and respond) to alerts by filtering for certain regexes. For logging at intervals you could use for instance Logwatch.
* Hint: what you want is close to PCI-DSS type of logging. Be sure to search this forum for threads about it as you will find more information there.

Wrap-up

I hope that from reading the above you understand that by thoroughly assessing the situation you find that "auditing iptables" involves more layers of security than just checking hashes, and that while GNU/Linux does not come with the required silver spooning out of the box it is perfectly possible to convert your requirements into a practically usable audit trail. Your Achilles heel is having multiple root account users which may not be avoidable. Besides I may know some but I don't know everything. So making sure this all works for you will involve more than just blindly implementing what's been offered: thinking about circumvention or evasion, research possible solutions, take the right measures (how would you be alerted to and what would you with a log DoS?) and discussing things.

Have a Linux Problem
Linux Forum - Do you have a Linux Question?

Linux Books
Linux Certification, System Administration, Programming, Networking Books

Linux Home: Linux System Administration Hints and Tips

(c) www.gotothings.com All material on this site is Copyright.
Every effort is made to ensure the content integrity.  Information used on this site is at your own risk.
All product names are trademarks of their respective companies.
The site www.gotothings.com is in no way affiliated with or endorsed by any company listed at this site.
Any unauthorised copying or mirroring is prohibited.