www.fabiankeil.de/sourcecode/privoxy-log-parser/

privoxy-log-parser – A parser for Privoxy log messages

privoxy-log-parser reads Privoxy log messages and:

• syntax-highlights recognized log lines,
• reformats some of them for easier comprehension,
• filters out less useful messages, and
• (in some cases) calculates additional information, like the compression ratio or how a filter affected the content size.

With privoxy-log-parser, you should be able to increase Privoxy's log level, without getting confused by the amount of output you get. For example for debug 64, privoxy-log-parser will (by default) only show messages that affect the content. If a filter doesn't cause any hits, privoxy-log-parser will hide the "filter foo caused 0 hits" message.

privoxy-log-parser ships as privoxy-log-parser.pl to make it clear that it's a Perl script, on most operating systems you can strip the extension.

Run perldoc privoxy-log-parser(.pl) to read the documentation.

Example Output

In the screen shot above, privoxy-log-parser makes it easier to see which filters hit the page, and how the content size was affected.

In the screen shot to the left (full version) you see log messages for three different pft requests, the first of them was responsible for the two other ones and is waiting for their results. The colored thread ids make it easy to see which log line belongs to which request.

Statistic mode

Privoxy-Log-Parser can also generate statistics:

fk@t520 ~ $privoxy-log-parser.pl --statistics /usr/jails/privoxy-jail/var/log/privoxy/privoxy.log
Inconsistent request counts: 6004/5977
Client requests total: 5977
Crunches: 673 (11.26%)
Blocks: 536 (8.97%)
Fast redirections: 21 (0.35%)
Connection timeouts: 0 (0.00%)
Connection failures: 0 (0.00%)
Outgoing requests: 5304 (88.74%)
Server keep-alive offers: 4041 (67.61%)
New outgoing connections: 1813 (30.33%)
Reused server connections: 3518 (58.86%; server offers accepted: 87.06%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 1951
Bytes of content transferred to the client: 1339608595
Client requests per connection distribution:
    1207: 1
     271: 2
     137: 3
      69: 4
      65: 5
      54: 6
      38: 7
      23: 8
      16: 9
      13: 10
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~14
Method distribution:
    5804 : GET     
     156 : POST    
      17 : OPTIONS 
Client HTTP versions:
    5977 : HTTP/1.1
HTTP status codes according to 'debug 512' (status codes sent by the server may differ):
    5297 : 200     
     536 : 403     
      59 : 503     
      44 : 404     
      25 : 302     
      16 : 502     
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.