How to get status of Nginx HttpProxyModule (proxy_cache) - Just Logging and Collecting Log
Introduction
I run a Unicorn + Nginx server with cache by HttpProxyModule. I'd like to get status of cache, for example, in memcached, we can get status like this.
But, it seems there are no commands for doing that. Finally, my co-worker advised me and the problem was solved.
The method is logging and collecting log. Now, let's do it ;)
1. Logging
Add more log_format and access_log directives to /etc/nginx/nginx.conf.
/etc/nginx/nginx.conf
log_format cache '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" $upstream_cache_status'; access_log /var/log/nginx/cache.log cache;
Reload nginx.
$ sudo service nginx reload
Then log including cache will be outputted like below:
$ cat /var/log/nginx/cache.log 1.2.3.4 ..(snip).. "GET /something HTTP/1.1" 200 ..(snip).. HIT 1.2.3.4 ..(snip).. "GET /another_one HTTP/1.1" 200 ..(snip).. MISS
The last them, HIT and MISS, are status of cache. How great!
An explanation for each status is in here.
2. Collecting log
Write Ruby script as follows:
(It's dirty code, but enough to work)
ache_count = 0 miss_count = 0 expired_count = 0 updating_count = 0 stale_count = 0 hit_count = 0 # Change arbitrarily lines = File::open('/var/log/nginx/cache.log').read.split("\n").last(10000) lines.each do |line| if line.match(/MISS$/) miss_count = miss_count + 1 cache_count = cache_count + 1 end if line.match(/EXPIRED$/) expired_count = expired_count + 1 cache_count = cache_count + 1 end if line.match(/UPDATING$/) updating_count = updating_count + 1 cache_count = cache_count + 1 end if line.match(/STALE$/) stale_count = stale_count + 1 cache_count = cache_count + 1 end if line.match(/HIT$/) hit_count = hit_count + 1 cache_count = cache_count + 1 end end puts "HIT:%12d (%5.2f%%)\nMISS:%11d (%5.2f%%)\nEXPIRED:%8d (%5.2f%%)\nUPDATING:%7d (%5.2f%%)\nSTALE:%10d (%5.2f%%)" % [ hit_count, (hit_count / (cache_count + 0.0)) * 100, miss_count, (miss_count / (cache_count + 0.0)) * 100, expired_count, (expired_count / (cache_count + 0.0)) * 100, updating_count, (updating_count / (cache_count + 0.0)) * 100, stale_count, (stale_count / (cache_count + 0.0)) * 100 ]
Execute script.
$ sudo ruby cache.rb HIT: 3586 (85.63%) MISS: 168 ( 4.01%) EXPIRED: 434 (10.36%) UPDATING: 0 ( 0.00%) STALE: 0 ( 0.00%)
Oh, that's fantastic ;)
Optional
If you'd like to draw a graph of status, fluentd + growthforecast will be nice combination.
At the end
Have a cool Nginx cache life X)