Wednesday, 6 December 2017

Linux audit files to see who made changes to a file

Configure System Auditing by Auditd.
It's possible to monitor System Calls, Security Events, File Accesses, Commands Executing and so on.


In order to use audit facility you need to use following utilities

=> auditctl – a command to assist controlling the kernel audit system. You can get status, and add or delete rules into kernel audit system. Setting a watch on a file is accomplished using this command:

=> ausearch – a command that can query the audit daemon logs based for events based on different search criteria.

=> aureport – a tool that produces summary reports of the audit system logs.

The audit package contains the user space utilities for storing and searching the audit records generate by the audit subsystem in the Linux Like CentOS/Red Hat and Fedora includes

1 :- Audit package is installe on CentOS-7

[root@lnx_r00t ~]# yum -y install audit  -y
Loaded plugins: fastestmirror
base                                                                                               | 3.6 kB  00:00:00   
extras                                                                                             | 3.4 kB  00:00:00   
updates                                                                                            | 3.4 kB  00:00:00   
Loading mirror speeds from cached hostfile
 * base: centos.mirror.net.in
 * extras: centos.mirror.net.in
 * updates: centos.mirror.net.in
Resolving Dependencies
--> Running transaction check
---> Package audit.x86_64 0:2.4.1-5.el7 will be updated
---> Package audit.x86_64 0:2.7.6-3.el7 will be an update
--> Processing Dependency: audit-libs(x86-64) = 2.7.6-3.el7 for package: audit-2.7.6-3.el7.x86_64
--> Running transaction check
---> Package audit-libs.x86_64 0:2.4.1-5.el7 will be updated
---> Package audit-libs.x86_64 0:2.7.6-3.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================
 Package                        Arch                       Version                         Repository                Size
==========================================================================================================================
Updating:
 audit                          x86_64                     2.7.6-3.el7                     base                     242 k
Updating for dependencies:
 audit-libs                     x86_64                     2.7.6-3.el7                     base                      96 k

Transaction Summary
==========================================================================================================================
Upgrade  1 Package (+1 Dependent package)

Total download size: 338 k
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/2): audit-2.7.6-3.el7.x86_64.rpm                                                                  | 242 kB  00:00:00   
(2/2): audit-libs-2.7.6-3.el7.x86_64.rpm                                                           |  96 kB  00:00:00   
--------------------------------------------------------------------------------------------------------------------------
Total                                                                                     752 kB/s | 338 kB  00:00:00   
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.7.6-3.el7.x86_64                                                                       1/4
  Updating   : audit-2.7.6-3.el7.x86_64                                                                              2/4
  Cleanup    : audit-2.4.1-5.el7.x86_64                                                                              3/4
  Cleanup    : audit-libs-2.4.1-5.el7.x86_64                                                                       4/4
  Verifying  : audit-libs-2.7.6-3.el7.x86_64                                                                        1/4
  Verifying  : audit-2.7.6-3.el7.x86_64                                                                               2/4
  Verifying  : audit-2.4.1-5.el7.x86_64                                                                               3/4
  Verifying  : audit-libs-2.4.1-5.el7.x86_64                                                                        4/4

Updated:
  audit.x86_64 0:2.7.6-3.el7                                                                                            

Dependency Updated:
  audit-libs.x86_64 0:2.7.6-3.el7                                                                                       

Complete!
[root@lnx_r00t ~]#

2:-  Now auditd service start/enable

[root@lnx_r00t ~]# systemctl daemon-reload
[root@lnx_r00t ~]# systemctl start auditd
[root@lnx_r00t ~]# systemctl enable auditd
[root@lnx_r00t ~]# systemctl status auditd
● auditd.service - Security Auditing Service
   Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2017-12-06 11:38:20 IST; 42s ago
     Docs: man:auditd(8)
           https://github.com/linux-audit/audit-documentation
 Main PID: 3121 (auditd)
   CGroup: /system.slice/auditd.service
           ├─3121 /sbin/auditd -n
           ├─3125 /sbin/audispd
           └─3129 /usr/sbin/sedispatch

Dec 06 11:38:20 lnx_r00t augenrules[3122]: enabled 1
Dec 06 11:38:20 lnx_r00t augenrules[3122]: failure 1
Dec 06 11:38:20 lnx_r00t augenrules[3122]: pid 3121
Dec 06 11:38:20 lnx_r00t augenrules[3122]: rate_limit 0
Dec 06 11:38:20 lnx_r00t augenrules[3122]: backlog_limit 320
Dec 06 11:38:20 lnx_r00t augenrules[3122]: lost 0
Dec 06 11:38:20 lnx_r00t augenrules[3122]: backlog 1
Dec 06 11:38:20 lnx_r00t systemd[1]: Started Security Auditing Service.
Dec 06 11:38:29 lnx_r00t systemd[1]: Started Security Auditing Service.
Dec 06 11:38:46 lnx_r00t systemd[1]: Started Security Auditing Service.

3:- Search Logs with ausearch

It's possible to add your own Audit rules, though, but some rules are set by default like System Login, Modification of User Accounts, Sudo Actions and so on, there logs are recorded in /var/log/audit/audit.log.

 [1] The logs are text format, so it's possible to see logs directly.
[root@lnx_r00t ~]# tail -5 /var/log/audit/audit.log
type=SYSCALL msg=audit(1512541275.236:532): arch=c000003e syscall=2 success=yes exit=5 a0=7f3687f54432 a1=80000 a2=1b6 a3=24 items=1 ppid=1 pid=3959 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="systemd-tmpfile" exe="/usr/bin/systemd-tmpfiles" key="password-file"
type=CWD msg=audit(1512541275.236:532):  cwd="/"
type=PATH msg=audit(1512541275.236:532): item=0 name="/etc/passwd" inode=18288920 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00 objtype=NORMAL
type=SERVICE_START msg=audit(1512541275.268:533): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-tmpfiles-clean comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
type=SERVICE_STOP msg=audit(1512541275.268:534): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-tmpfiles-clean comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[root@lnx_r00t ~]#

 [2] Many logs are recorded in audit.log and they are complicated, so ausearch command is provided by Audit package to search specific logs.

###### search USER_LOGIN logs ######
[root@lnx_r00t ~]# ausearch --message USER_LOGIN --interpret
----
type=USER_LOGIN msg=audit(Saturday 02 December 2017 A.293:80) : pid=2615 uid=root auid=admin ses=2 msg='uid=admin exe=/usr/libexec/gdm-session-worker hostname=? addr=? terminal=? res=success'
----
type=USER_LOGIN msg=audit(Saturday 02 December 2017 A.921:102) : pid=3574 uid=root auid=unset ses=unset msg='op=login acct=root exe=/usr/sbin/sshd hostname=? addr=192.168.0.99 terminal=ssh res=failed'
----
type=USER_LOGIN msg=audit(Saturday 02 December 2017 A.559:116) : pid=3617 uid=root auid=root ses=3 msg='op=login id=root exe=/usr/sbin/sshd hostname=192.168.0.99 addr=192.168.0.99 terminal=/dev/pts/1 res=success'
----
type=USER_LOGIN msg=audit(Saturday 02 December 2017 A.238:169) : pid=4387 uid=root auid=root ses=7 msg='op=login id=root exe=/usr/sbin/sshd hostname=192.168.0.99 addr=192.168.0.99 terminal=/dev/pts/1 res=success'
----
type=USER_LOGIN msg=audit(Saturday 02 December 2017 A.594:761) : pid=22744 uid=root auid=unset ses=unset msg='op=login acct=(unknown) exe=/usr/sbin/sshd hostname=? addr=192.168.0.151 terminal=ssh res=failed'
----
type=USER_LOGIN msg=audit(Monday 04 December 2017 6NA.394:345) : pid=4856 uid=root auid=root ses=39 msg='op=login id=root exe=/usr/sbin/sshd hostname=192.168.0.99 addr=192.168.0.99 terminal=/dev/pts/0 res=success'
----
type=USER_LOGIN msg=audit(Wednesday 06 December 2017 .257:84) : pid=2913 uid=root auid=root ses=1 msg='op=login id=root exe=/usr/sbin/sshd hostname=192.168.0.99 addr=192.168.0.99 terminal=/dev/pts/0 res=success'
[root@lnx_r00t ~]#

###### How do I find out who changed or accessed a file /etc/passwd?  ######
   
-f /etc/passwd :- Only search for this file

[root@lnx_r00t ~]# ausearch -f /etc/passwd
----
time->Wed Dec  6 11:44:39 2017
type=PATH msg=audit(1512540879.281:162): item=0 name="/etc/passwd" inode=18288920 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00 objtype=NORMAL
type=CWD msg=audit(1512540879.281:162):  cwd="/root"
type=SYSCALL msg=audit(1512540879.281:162): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=7ffee768376a a2=0 a3=0 items=1 ppid=2913 pid=3564 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="grep" exe="/usr/bin/grep" key="password-file"
----
time->Wed Dec  6 11:44:52 2017
type=PATH msg=audit(1512540892.787:163): item=0 name="/etc/passwd" inode=18288920 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00 objtype=NORMAL
type=CWD msg=audit(1512540892.787:163):  cwd="/root"
type=SYSCALL msg=audit(1512540892.787:163): arch=c000003e syscall=2 success=yes exit=3 a0=7f75f3ca0432 a1=80000 a2=1b6 a3=24 items=1 ppid=2913 pid=3565 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="vim" exe="/usr/bin/vim" key="password-file"
----
time->Wed Dec  6 11:44:52 2017
type=PATH msg=audit(1512540892.787:164): item=0 name="/etc/passwd" inode=18288920 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00 objtype=NORMAL
type=CWD msg=audit(1512540892.787:164):  cwd="/root"
type=SYSCALL msg=audit(1512540892.787:164): arch=c000003e syscall=2 success=yes exit=3 a0=1f96220 a1=0 a2=0 a3=7fffb6965600 items=1 ppid=2913 pid=3565 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="vim" exe="/usr/bin/vim" key="password-file"
----

##### Display Logs with aureport #####

It's possible to display Audit logs summarily with aureport command which is included in Audit package.

 [1] This is how to use aureport command.

[root@lnx_r00t ~]# aureport

Summary Report
======================
Range of time in logs: Friday 08 September 2017 22:42:22.251 - Wednesday 06 December 2017 12:19:23.408
Selected time for report: Friday 08 September 2017 22:42:22 - Wednesday 06 December 2017 12:19:23.408
Number of changes in configuration: 1336
Number of changes to accounts, groups, or roles: 13
Number of logins: 24
Number of failed logins: 5
Number of authentications: 61
Number of failed authentications: 22
Number of users: 3
Number of terminals: 11
Number of host names: 4
Number of executables: 34
Number of commands: 35
Number of files: 4
Number of AVC's: 10
Number of MAC events: 4
Number of failed syscalls: 2
Number of anomaly events: 17
Number of responses to anomaly events: 0
Number of crypto events: 256
Number of integrity events: 0
Number of virt events: 0
Number of keys: 2
Number of process IDs: 1682
Number of events: 24111
[root@lnx_r00t ~]#

 [2]  # display kind of authentication logs

[root@lnx_r00t ~]# aureport -au

Authentication Report
============================================
# date time acct host term exe success event
============================================
1. Friday 08 September 2017 �� �^:|P�a gdm ? ? /usr/libexec/gdm-session-worker yes 360
2. Friday 08 September 2017 �� �^:|P�a root ? ? /usr/libexec/gdm-session-worker yes 380
3. Friday 08 September 2017 �� �^:|P�a root ? ? /usr/libexec/gdm-session-worker yes 408
.........
.........
18. Thursday 28 September 2017 �^:|P�a root 192.168.0.152 ssh /usr/sbin/sshd yes 140
19. Thursday 28 September 2017 �^:|P�a root 192.168.0.152 ssh /usr/sbin/sshd yes 143
20. Thursday 28 September 2017 �^:|P�a root ? ? /usr/libexec/gdm-session-worker yes 176

 [2] # display kind of failure authentication logs

[root@lnx_r00t ~]# aureport -au --failed --summary

Failed Authentication Summary Report
=============================
total  acct
=============================
13  root
9  admin
[root@lnx_r00t ~]#

 [3] # display kind of modification of user accounts logs

[root@lnx_r00t ~]# aureport -m -i

Account Modifications Report
=================================================
# date time auid addr term exe acct success event
=================================================
1. Saturday 02 December 2017 p^%�?�P�a root ? ? /usr/sbin/groupadd ? yes 175
2. Saturday 02 December 2017 p^%�?�P�a root ? ? /usr/sbin/groupadd ? yes 176
3. Saturday 02 December 2017 p^%�?�P�a root ? ? /usr/sbin/useradd ? yes 177
.............
.............
12. Saturday 02 December 2017 p^%�?�P�a root ? pts/1 /usr/sbin/usermod apache yes 204
13. Saturday 02 December 2017 p^%�?�P�a root ? pts/1 /usr/sbin/usermod apache yes 205
[root@lnx_r00t ~]#

 [4]  # display kind of modification of user accounts logs since this month

[root@lnx_r00t ~]# aureport -m -i --start this-month

Account Modifications Report
=================================================
# date time auid addr term exe acct success event
=================================================
1. Saturday 02 December 2017 �f� �P�a root ? ? /usr/sbin/groupadd ? yes 175
2. Saturday 02 December 2017 �f� �P�a root ? ? /usr/sbin/groupadd ? yes 176
3. Saturday 02 December 2017 �f� �P�a root ? ? /usr/sbin/useradd ? yes 177
.............
.............
11. Saturday 02 December 2017 �f� �P�a root ? pts/1 /usr/sbin/useradd ? yes 203
12. Saturday 02 December 2017 �f� �P�a root ? pts/1 /usr/sbin/usermod apache yes 204
13. Saturday 02 December 2017 �f� �P�a root ? pts/1 /usr/sbin/usermod apache yes 205
[root@lnx_r00t ~]#

 [4]  # display kinf of executing logs
[root@lnx_r00t ~]# aureport -x -i

Executable Report
====================================
# date time exe term host auid event
====================================
1. Friday 08 September 2017 R�FiuA�P�a /usr/lib/systemd/systemd ? ? unset 6
2. Friday 08 September 2017 R�FiuA�P�a /usr/lib/systemd/systemd-update-utmp ? ? unset 7
3. Friday 08 September 2017 R�FiuA�P�a /usr/lib/systemd/systemd ? ? unset 8
4. Friday 08 September 2017 R�FiuA�P�a /usr/lib/systemd/systemd ? ? unset 9
5. Friday 08 September 2017 R�FiuA�P�a /usr/lib/systemd/systemd ? ? unset 10
...............
...............
25034. Wednesday 06 December 2017  /usr/sbin/aureport pts0 ? root 20309
25035. Wednesday 06 December 2017  /usr/sbin/aureport pts0 ? root 20310
25036. Wednesday 06 December 2017  /usr/sbin/aureport pts0 ? root 20311
25037. Wednesday 06 December 2017  /usr/sbin/aureport pts0 ? root 20312
[root@lnx_r00t ~]#

 [5]  # display audit report of executing logs from 2017/12/1 to 2017/12/6

[root@lnx_r00t ~]# aureport -x -i --start 12/01/2017 00:00:00 --end 12/06/2017 00:00:00

Executable Report
====================================
# date time exe term host auid event
====================================
1. 12/01/2017 21:47:57 /usr/lib/systemd/systemd-update-utmp ? ? unset 5
2. 12/01/2017 21:47:57 /usr/lib/systemd/systemd ? ? unset 6
3. 12/01/2017 21:47:57 /usr/lib/systemd/systemd ? ? unset 7
4. 12/01/2017 21:47:57 /usr/lib/systemd/systemd ? ? unset 8
5. 12/01/2017 21:47:57 /usr/lib/systemd/systemd ? ? unset 9
.................
.................
1428. 12/04/2017 12:54:06 /usr/lib/systemd/systemd ? ? unset 440
1429. 12/04/2017 12:54:06 /usr/lib/systemd/systemd ? ? unset 441
1430. 12/04/2017 12:54:06 /usr/lib/systemd/systemd ? ? unset 442
1431. 12/04/2017 12:54:06 /usr/lib/systemd/systemd ? ? unset 443
1432. 12/04/2017 12:54:06 /usr/lib/systemd/systemd ? ? unset 444


###### PERFORMANCE TIPS #######


Syscall rules get evaluated for each syscall for each program. If you have 10 syscall rules, every program on your system will delay during a syscall while the audit system evaulates each one. Too many syscall rules will hurt performance. Try to combine as many as you can whenever the filter, action, key, and fields are identical. For example:

auditctl -a exit,always -S open -F success=0

auditctl -a exit,always -S truncate -F success=0

could be re-written as one rule:

auditctl -a exit,always -S open -S truncate -F success=0

Also, try to use file system auditing wherever practical. This improves performance. For example, if you were wanting to capture all failed opens & truncates like above, but were only concerned about files in /etc and didn’t care about /usr or /sbin, its possible to use this rule:

auditctl -a exit,always -S open -S truncate -F dir=/etc -F success=0

This will be higher performance since the kernel will not evaluate it each and every syscall. It will be handled by the filesystem auditing code and only checked on filesystem related syscalls.

######## Write Custom System Audit Rules CentOS 7 ########


To see all syscalls made by a specific program:

auditctl -a entry,always -S all -F pid=1005

To see files opened by a specific user:

auditctl -a exit,always -S open -F auid=510

To see unsuccessful open call’s:

auditctl -a exit,always -S open -F success=0

To watch a file for changes (2 ways to express):

auditctl -w /etc/shadow -p wa
auditctl -a exit,always -F path=/etc/shadow -F perm=wa

To recursively watch a directory for changes (2 ways to express):

auditctl -w /etc/ -p wa
auditctl -a exit,always -F dir=/etc/ -F perm=wa

By:-  L!nx_r00t