IIS Log Files
April 2007
Log files from web servers usually contain a bunch of information, and in fact most statistics packages for log files, expect them to containquite a few more fields that are always collected.
This was the case for a site I recently encountered. For one reason or another, the site is setup to save only the littlest of information about requests: time, client's IP, request method, the URI, and the HTTP status code. Because of this the software that I would normally run to make a report choked, and there happens to be no way to change what fields the software is looking for. Many freeware/shareware tools suffered the same hiccup and there was no time to get the popular free (as in freedom) alternatives, Analog, or AWStats, installed, configured and working correctly.
Luckily, the file format was so simple, it was trivial to create a shell script that counted requests to a given URI and output them sorted. The files were named exYYMMDD.log and contained this:
#Fields: time c-ip cs-method cs-uri-stem sc-status
08:47:08 194.153.113.12 GET /robots.txt 404
Stringing together a combination of a echo, cat, sed, awk, [e]grep, sort, and uniq is all it takes. I've used similar pipelines to list "live referrers," but I never considered it to be a log analyzer. Of course, I then needed to show how big of a dork I am by making it as concise as possible.
for f in *.log;do echo $f|sed -e 's/ex\([0-9][0-9]\)\([0-9][0-9]\)\([0-9][0-9]\).log/\2-\3-20\1/';echo;cat $f|egrep "cfm|/[[:space:]]+"|awk '{print $4;}'|sort|uniq -c|sort;echo;done
178 bytes. The output is decent, and you can filter out files to search for. Here I only looked for cfm (ColdFusion) files and "/". The report looks like this:
04-04-2007
1 /index.cfm/fuseaction/events.viewall
3 /index.cfm
6 /
