bekkou68 の日記

Gogengo! や IT 技術など。

How to get status of Nginx HttpProxyModule (proxy_cache) - Just Logging and Collecting Log

Introduction

I run a Unicorn + Nginx server with cache by HttpProxyModule. I'd like to get status of cache, for example, in memcached, we can get status like this.
But, it seems there are no commands for doing that. Finally, my co-worker advised me and the problem was solved.

The method is logging and collecting log. Now, let's do it ;)

1. Logging

Add more log_format and access_log directives to /etc/nginx/nginx.conf.

/etc/nginx/nginx.conf

log_format cache '$remote_addr - $remote_user [$time_local] "$request" '
                   '$status $body_bytes_sent "$http_referer" '
                   '"$http_user_agent" "$http_x_forwarded_for" $upstream_cache_status';
access_log  /var/log/nginx/cache.log  cache;

Reload nginx.

$ sudo service nginx reload

Then log including cache will be outputted like below:

$ cat /var/log/nginx/cache.log
1.2.3.4 ..(snip).. "GET /something HTTP/1.1" 200 ..(snip).. HIT
1.2.3.4 ..(snip).. "GET /another_one HTTP/1.1" 200 ..(snip).. MISS

The last them, HIT and MISS, are status of cache. How great!

An explanation for each status is in here.

2. Collecting log

Write Ruby script as follows:
(It's dirty code, but enough to work)

ache_count = 0 
miss_count = 0 
expired_count = 0 
updating_count = 0 
stale_count = 0 
hit_count = 0 

# Change arbitrarily
lines = File::open('/var/log/nginx/cache.log').read.split("\n").last(10000)

lines.each do |line|
  if line.match(/MISS$/)
    miss_count = miss_count + 1 
    cache_count = cache_count + 1 
  end 

  if line.match(/EXPIRED$/)
    expired_count = expired_count + 1 
    cache_count = cache_count + 1 
  end 

  if line.match(/UPDATING$/)
    updating_count = updating_count + 1 
    cache_count = cache_count + 1 
  end 

  if line.match(/STALE$/)
    stale_count = stale_count + 1 
    cache_count = cache_count + 1 
  end 

  if line.match(/HIT$/)
    hit_count = hit_count + 1 
    cache_count = cache_count + 1 
  end 
end

puts "HIT:%12d (%5.2f%%)\nMISS:%11d (%5.2f%%)\nEXPIRED:%8d (%5.2f%%)\nUPDATING:%7d (%5.2f%%)\nSTALE:%10d (%5.2f%%)" % [ 
  hit_count, (hit_count / (cache_count + 0.0)) * 100,
  miss_count, (miss_count / (cache_count + 0.0)) * 100,
  expired_count, (expired_count / (cache_count + 0.0)) * 100,
  updating_count, (updating_count / (cache_count + 0.0)) * 100,
  stale_count, (stale_count / (cache_count + 0.0)) * 100 
]

Execute script.

$ sudo ruby cache.rb 
HIT:        3586 (85.63%)
MISS:        168 ( 4.01%)
EXPIRED:     434 (10.36%)
UPDATING:      0 ( 0.00%)
STALE:         0 ( 0.00%)

Oh, that's fantastic ;)

Optional

If you'd like to draw a graph of status, fluentd + growthforecast will be nice combination.

At the end

Have a cool Nginx cache life X)