From DevRandom

Auditd

Assumptions: Everything here is tested on Red Hat/Cent OS 5, and 2.6.24.XX kernel

auditd is the userspace side of kernel auditing functions. It can be used to watch file accesses, monitory system calls, log events etc. The closest to a home page it has http://people.redhat.com/sgrubb/audit/

It has some nice presentations done at different Red Hat conferences and some other articles.

Installation

Installing auditd is straightforward but to use it efficiently you need to tune it to your needs. To install use

yum install audit

To start the service do

/etc/init.d/auditd start

Configuration

Installation is a breeze but configuring to your needs requires some tweaking.

auditd.conf can be used to tweak auditd's behaviour. Common settings to review/tune are :-

log_file (default:/var/log/audit/audit.log) & log_format(default:raw)

Set the log file name and the format. The format can be either raw or nolog.

num_logs(default:0)

Auditd does its own rotation and the default is no rotation. This keyword specifies the number of log files to keep if rotate is given as the max_log_file_action. Note, there is a small catch to this setting. Turns out when Auditd is rotating logs it won't process any logs. So on a high volume server you might miss logs during rotation. To avoid this raise the kernel backlog buffers by adding -b <num> to /etc/audit/audit.rules. The default is 64 1K buffers. This cannot be greater than 99.

max_log_file (default:5MB) & max_log_file_action (default:ROTATE)

The first sets the maximum log size in megabytes. Action defines what to do when it reaches that state. Valid values are ignore, syslog, suspend, rotate and keep_logs. If set to ignore, the audit daemon does nothing. syslog means that it will issue a warning to syslog. suspend will cause the audit daemon to stop writing records to the disk. The daemon will still be alive. The rotate option will cause the audit daemon to rotate the logs.

space_left (default:75 MB) & space_left_action (default:SYSLOG)
admin_space_left (default:75 MB) & admin_space_left_action (default:SUSPEND)
disk_full_action (default: SUSPEND) & disk_error_action (default:SUSPEND)

The first 2 can be explained as a warning threshold + action and critical threshold + action. The 3rd one defines what to do if the disk becomes full or disk errors occur when writing The valid actions are ignore - do nothing, syslog - log to syslog, email - send email to action_mail_acct, exec - execute a script, suspend - stop audit logging, single - single user mode, and halt - halt system

action_mail_acct

Valid email address for email actions from above. This require /usr/lib/sendmail

flush & freq

The default is to use incremental and a count of 20 which means it will flush to disk after 20 events. Valid values are none, incremental, data, and sync. This is a trade off between disk I/O and how much log data you can afford to lose if the server loses power or abruptly reboots. The data forces data to be synced at all times & the sync forced both data and meta data to be synced at all times. The sync option will off course cause the most amount of I/O. If you can afford to lose some log data, then leaving it at default would be fine.

name_format & name

This inserts the node name/host name to the log line. Default is to insert nothing. It can be set to hostname, IP address or FQDN. This makes sense if you are passing logs from several machine to a centralized location.

tcp_listen_port

You can tell auditd to listen for events from other machines.

Auditd rules

The main use of auditd is derived from the rules. /etc/audit/audit.rules is used to set configuration like watch rules, buffers etc. auditctl can also be used to change rules on a live system.

General Settings

Backlog (default: 64 1K buffers)

Turns out when Auditd is rotating logs it won't write any logs to file. So on a high volume server you might miss logs during rotation. To avoid this raise the kernel backlog buffers by adding -b <num> to /etc/audit/audit.rules. The default is 64 1K buffers. If the limit is reached the failure flag is consulted

Rate

Messages/sec can be set by -r. if the rate is exceeded the failure flag is consulted.

Failure flag (default: 1=printk)

Determines how the kernel will handle errors. 0 - silent, 1 - printk , 2 - panic Even though panic sounds like a terrible idea, it can be important in high security environments, especially, if someone is trying to circumvent the logging.

Locking rules

By adding -e 2 to end of the rules file, the configuration is locked and cannot be changed by the auditctl command. Attempts to do so will be logged and denied. Note, reverting this will require a reboot. This is another feature that is suited for very high security environments.

Listing current rules

auditctl -l

Watching Files

-w path

Wildcards are not supported and will generate a warning. The way that watches work is by tracking the inode internally. Unlike other syscall rules, watches do not impact performance. For more granular options like audit a specific user accessing a file, use the syscall auditing

-p r|w|x|a

Set permissions filter for a file system watch. r=read, w=write, x=execute, a=attribute change. They are not standard file permissions but the syscall the gets executed for them

-k <key>

The -k adds a key making it easy to search if there are several watches. One use for -k is to define different alert levels; eg. low, medium, high and then tag different rules with those levels.

eg. Watch yum.conf for write and attribute changes Watch /sbin/service for execution Watch /etc/shadow for read, write and attribute changes

Note: The same can be added to /etc/audit/audit.rules without the auditctl command

auditctl -w /etc/yum.conf -p wa  -k yum_watch
auditctl -w /usr/bin/nmap -p x   -k nmap_watch
auditctl -w /etc/shadow   -p rwa -k shadow_watch

To report on watched files. Date format is local to the server's date format.

aureport -f
aureport -f --start 02/18/10 17:42:00
aureport -f --start 02/18/10 17:00:00 --end 02/18/10 17:10:00
aureport -f -ts this-week
aureport -f -ts today

Output will be similar to this

14742. 04/20/10 12:40:01 /etc/shadow 2 yes /usr/sbin/crond -1 977855
14743. 04/20/10 12:40:01 /etc/shadow 2 yes /usr/sbin/crond -1 977851
14744. 04/20/10 12:40:24 /etc/shadow 89 no /opt/splunk/bin/splunkd 500 977863
14745. 04/20/10 12:40:50 /etc/shadow 89 no /opt/splunk/bin/splunkd 500 977864
14746. 04/20/10 12:41:16 /etc/shadow 89 no /opt/splunk/bin/splunkd 500 977865
14747. 04/20/10 12:41:42 /etc/shadow 89 no /opt/splunk/bin/splunkd 500 977866

The 1st column is just an index

The 2nd column is the date

The 3nd column is the time

The 4rd column is the file name

The 5th column is the system call number. To convert this number into name use aureport -f -i

The 6th column is the result of the system call. Success of failure. You can use aureport --failed to list only failed

The 7th column is the process accessing it

The 8th column is the Actual/Audit UID (AUID). If you login as user joe(uid 500) and su to root (uid=0). The AUID is 500.

The 9th column is the event number. You can use ausearch -a <event no> to look up further details

Note : -i is useful as it converts to more human readable value. However the conversion is done at the time of viewing. The log will contain numeric values only. So if uid 500 belongs to john at the time of logging and at a later date, uid 500 is assigned to user joe, there will be a discrepancy.

The above produces more of a summary. You can use 'ausearch' to get more details.

Let's take a closer look.

nmap is set to 0700 permissions.

[root@slice01 ~]# ls -l /usr/bin/nmap
-rwx------ 1 root root 3580248 Jul  6  2009 /usr/bin/nmap

Access by root yields the following log

# ausearch -i -k nmap_watch 

time->Fri Apr 16 16:10:18 2010
node=slice01 type=PATH msg=audit(04/16/10 16:10:18.044:954405) : item=0 name=/usr/bin/nmap inode=721629 dev=08:01 mode=file,700 
ouid=root ogid=root rdev=00:00 
node=slice01 type=CWD msg=audit(04/16/10 16:10:18.044:954405) :  cwd=/root 
node=slice01 type=EXECVE msg=audit(04/16/10 16:10:18.044:954405) : a0=nmap 
node=slice01 type=SYSCALL msg=audit(04/16/10 16:10:18.044:954405) : arch=x86_64 
syscall=execve per=400000 success=yes exit=0 a0=6e2bd0 a1=6e5160 a2=7ae510 a3=0 items=1 
ppid=7318 pid=14772 auid=sri uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts0 
comm=nmap exe=/usr/bin/nmap key=nmap_watch

Access by user with uid 500 who has been granted access results in this

----
time->Fri Apr 16 16:11:44 2010
node=slice01 type=PATH msg=audit(04/16/10 16:11:44.470:954410) : item=0 name=/usr/bin/nmap inode=721629 dev=08:01 mode=file,700 
ouid=root ogid=root rdev=00:00 
node=slice01 type=CWD msg=audit(04/16/10 16:11:44.470:954410) :  cwd=/root 
node=slice01 type=SYSCALL msg=audit(04/16/10 16:11:44.470:954410) : arch=x86_64 
syscall=execve success=no exit=-13(Permission denied) a0=6f26e0 a1=6f2260 a2=6cf8e0 a3=0 items=1 
ppid=13286 pid=15089 auid=sri uid=sri gid=sri euid=sri suid=sri fsuid=sri egid=sri sgid=sri fsgid=sri tty=pts1 
comm=bash exe=/bin/bash key=nmap_watch

success=<yes|no> shows if the system call was successful

Again the -i lists output in more human readable format and converts syscall numbers and user ids into names.

To remove a rule using auditctl you can use the -W switch

auditctl -W /etc/shadow -p rwa -k shadow_watch

Another way to watch files is via system calls. See below.

Watching system calls

You can use this to watch any system call. There are various options available here that can be combined to audit different types of events.

Required switches with syscall monitoring are the -a list,action -A list,action

-a appends to the end of the list

-A appends to the start of the list

The corresponding switch to the -a is the -d switch which can delete rules of a certain type

The list values can be

entry/exit - This determines when to log with respect to the system call invocation

exclude - This can be used to filter events

The action values can be :-

never No audit records will be generated. The order of rules in the audit.rules file matters. Normally you would put suppressions at the top

always Always write out a record

The -S switch defines the system call to watch. It can be name or the system call number

The next switch is the -F which builds a rule field. Several of these can be grouped together to filter various stuff.

Common things to use with this are :-

arch - cpu architecture

auid/uid - The original ID the user logged in with & the user id

euid/egid - Effective user and group ID

path/dir - Full path of the file or directory to watch. Directory watches are recursive

filetype - Type of the file. file, dir, socket, symlink, char, block, or fifo

perm - permission filter for file operation

pid/ppid - process id and parent process id

Watching for ptrace system call. Utilities like strace use it

auditctl -a entry,always -F arch=b64 -S ptrace -k info_scan

Suppressing 32bit clock_gettime & fstat64 system calls

-a entry,never -F arch=b32 -S clock_gettime -k clock_gettime
-a entry,never -F arch=b32 -S fstat64 -k fstat64

Audit files opened by a specific user. The first rule will audit all open files. It is a good idea to watch for both auid and uid. Also the architecture is required with system calls like these

auditctl -a exit,always -S open -F auid=2010
auditctl -a exit,always -F arch=b64 -F auid=2010  -F uid=2010 -F path=/etc/hosts -S open

To search you can use the key or use the -sc switch

ausearch -k info_scan -i
ausearch -sc ptrace -i

To search by user id

ausearch -ua 2010

Authentication Report

Lists all auth attempts and their result. This also includes logins by other means, eg. imap access

aureport -au

To list just logins

aureport -l

To list account modification attempts. This lists only successful attempts. For eg. it won't list an attempt made by an unprivileged user.

aureport -m

Rules for security standards

/usr/share/doc/audit-<version> has pre-defined rules for different standards

Controlled Access Protection Profile (CAPP) - http://www.commoncriteriaportal.org/files/ppfiles/capp.pdf

Labeled Security Protection Profile (LSPP) - http://www.commoncriteriaportal.org/files/ppfiles/lspp.pdf

National Industrial Security Program Operating Manual (NISPOM) - http://www.fas.org/sgp/library/nispom.htm

Security Technical Implementation Guides - http://iase.disa.mil/stigs/stig/index.html

Best Practices

If you want everything audited you should add a boot param of audit=1 to Grub. Otherwise certain processes that start before audit loads will not be monitored.

/var/log/audit should be on its own partition.

Use Syscall rules with care. They will lead to performance degradation if overused

Commands Listing

Adding/Modifying Rules

Watch for files

auditctl -w /etc/yum.conf -p wa  -k yum_watch
auditctl -w /usr/bin/nmap -p x   -k nmap_watch
auditctl -w /etc/shadow   -p rwa -k shadow_watch

Remove a rule using auditctl

auditctl -W /etc/shadow -p rwa -k shadow_watch

Watching for ptrace system call

auditctl -a entry,always -F arch=b64 -S ptrace -k info_scan

Suppress 32bit clock_gettime & fstat64 system calls

-a entry,never -F arch=b32 -S clock_gettime -k clock_gettime
-a entry,never -F arch=b32 -S fstat64 -k fstat64

Audit files opened by a specific user

auditctl -a exit,always -S open -F auid=2010
auditctl -a exit,always -F arch=b64 -F auid=2010  -F uid=2010 -F path=/etc/hosts -S open

Audit unsuccessful attempts for multiple system calls where user id is greater than or equal to 500

auditctl -a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=500 
auditctl -a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=500

Reporting/Searching

List all rules

auditctl -l

List status

auditctl -s

Report on watched files. Date format is local to the server's date format.

aureport -f
aureport -f --start 02/18/10 17:42:00
aureport -f --start 02/18/10 17:00:00 --end 02/18/10 17:10:00
aureport -f -ts this-week
aureport -f -ts today