| Apache server records all incoming requests
and all requests processed to a log file. The format of the access log
is highly configurable. The location and content of the access log are
controlled by the CustomLog directive.
Default apache access log file location:
RHEL / Red Hat / CentOS / Fedora Linux Apache access file location
- /var/log/httpd/access_log
Debian / Ubuntu Linux Apache access log file location - /var/log/apache2/access.log
FreeBSD Apache access log file location - /var/log/httpd-access.log
To find exact apache log file location, you
can use grep command:
# grep CustomLog /usr/local/etc/apache22/httpd.conf
# grep CustomLog /etc/apache2/apache2.conf
# grep CustomLog /etc/httpd/conf/httpd.conf
Sample output:
# a CustomLog directive (see below).
# CustomLog "/var/log/httpd-access.log" common
CustomLog "/var/log/httpd-access.log" combined
These assume you use a standard log format. If not you may have to
adjust them to suit your format.
Most viewed pages (top ten).
Code:
awk '{print $7}' /path/to/log |sort |uniq -c |sort -rn |head -10
Top ten referrers:
Code:
awk '{print $11}' /path/to/log |sort |uniq -c |sort -rn |head -10
Search logs:
Code:
grep /path/to/log query |awk '{print $8}'|tail -n+5|sort|uniq|tr -d
[1]
And here is a script I wrote a long time ago that incorparates these
and few others.
Code:
#!/bin/bash
# usage
# ./this_script search pattern log_file
# all hosts with 5 or more matches
of the given pattern will be banned
# ./this_script ban_from_log log_file
# all hosts that appear more than
5 times in the given log file will be banned
# ./this_script ban_rfi log_file
# bans all hosts that match the
rfi pattern(rfi's and proxy requests)
# ./this_script most_viewed log_file
# shows the top ten viewed pages
with the number of views
# ./this_script statuses response log_file
# shows the top ten viewed pages
for the given response header e.g. 404, 200 ect
# ./this_script referrers log_file
# shows the top ten referrers and
page views for each
#
# author tomfmason
ban_file=/etc/hosts.deny
function ban_ip() {
exists=`grep ${1} $ban_file`
if [ ! "$exists" ]; then
echo "ALL: ${1}" >> $ban_file
fi
}
function search() {
ret=`grep ${1} ${2} |awk '{print $8}'|tail -n+5|sort|uniq|tr
-d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_from_log() {
ret=`awk '{print $8}' ${1}|tail -n+5|sort|uniq|tr -d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_rfi() {
ret=`awk '{print $1 " " $7}' ${1} |grep -iE '(http|https|ftp)'|awk
'{print $1}'|tail -n+5|sort|uniq`
for r in $ret; do
ban_ip $r
done
}
function most_viewed() {
awk '{print $7}' ${1} |sort |uniq -c |sort -rn |head
-10
}
function statuses() {
awk '{print $7 " " $9}' ${2} |grep -iE '${1}' |sort |
uniq -c | sort -rn |awk '{print $1 " " $2 }' |head -10
}
function referrers() {
awk '{print $11}' ${1} |sort |uniq -c |sort -rn
|head -10
}
if type "$1" | grep -qF "$1 is a function"; then "$@"; fi
Note:
As this was written a long time ago and now I would not suggest using
hosts.deny to ban an ip. Iptables would be a much better choice. Believe
you can fix it yourself ;-) |