Log3NF – IPv6 support coming real soon

A few years ago, I converted my earlier idea of logging Apache request to 3rd normal form into a fully fledged Mod Perl 2.0 Log Handler – embedding this into Apache. Its essentially a very simple Handler of less than 60 lines, and a stored procedure inside MySQL that normalises the data. Its been running on my personal server since February 2009, and in that time its collected around

  • 1.4 million hits
  • 27,000 unique useragents
  • 57,000 unique paths
  • 9 basic authentiation users
  • 13 HTTP methods
  • 50,000 unique referrer URLs
  • 13 HTTP Status
  • Recorded the transfer of 192,185.9637 Megabytes of (body) data; the average body response is 144 KB.

Wow. The log data on disk is 574 MB, or around 410 bytes per request – including the indexes (this is the size of the MySQL directory containing the data).

All well and good. Now time to get it fit for IPv6, and then improve the reporting. The reporting has two phases:

  • Live reporting from the 3rd normal form for data covering the last few seconds/minutes/hours/days.
  • Summary reporting per day or per month, per statistic, pre-calculated

Anyway, we’re about to pull in IPv6, again storing this as efficiently as possible, and then improve the currently very basi reporting interface…. stay tuned… and see the SVN repository for code…

Now with IPv6 goodness

So, with about 10 mins of reading and 1 min of work, this site is now available with both IPv4 and IPv6. Thanks Bytemark for making it so easy, and of course to all the software stack that just works. If you want to test and ensure you’re using just IPv6, you can browse to ipv6.james.rcpt.to, which I have only published a AAAA address for.

Next up; update my Log3NF Apache module to understand IPv6 addresses as well as the IPv4 if currently does.

Avast! SIP IPTables in mainline kernel

Well, following all the change logs out there for 2.6.18 Linux Kernel release shows that:

Add SIP protocol support to Netfilter (commit), a statistic match which is a combination of the nth and random matches (commit), a quota match (commit), and support for Call Forwarding to the H.323 netfilter module (commit)

Pretty obscure, so let me describe a problem with SIP.

SIP packets, like any other packet, has a header. This header has the standard IPv4 options, of which source address, source port, destination address and destination port are the most blindingly obvious ones. NAT is pretty well performed by most firewall boxes at tweaking these values and replaceing them on traversal of the gateway, and putting things back for responses. No surprises here.

However, the payload of a SIP packet by its design, has the IP address embedded in it. For example, the payload may have:


INVITE sip:3103 at 10.0.0.173 SIP/2.0
Via: SIP/2.0/UDP
10.21.99.221:5060;branch=z9hG4bK6caf7db4
From: "Someone"
<sip:3151 at 10.0.0.221>;tag=as23c4523c
To: <sip:3103 at 10.0.0.173>

The design is so that intermediate gateways can tack on their ‘Via’ headers and the call can be switched virtually between a chain of gateways, each one doing their bit. All well and good until you want to intrduce NAT, as the ‘Via’ then has information for networks the destination cannot necessarily get to.

This patch, I think, does the necessary bits to update the payload to make the payload be corrected as it passes through a NAT.

So, this will probably start to mean that Asterisk people will no longer have to set localnet definitions to be able to correctly operate their internal and external SIP connectivity behind their firewalls.

The implications are even bigger when hard ware vendors of DSL gateways that run Linux kernels also have this enabled – suddenly the Mom and Pop crowd can just plug SIP devices in at home and hey bringo, there goes the Telecomunications industry as we know it (is that good or bad?).

Juniper, hello, Netscreen product line? Your SIP ALG should be doing this?

Viva la revolution!

SNOM and XML

I recently noticed that SNOM have released version 6 of their firmware, which includes their MiniBrowser app in it. This can read XML and follow links, and is perfect for delivering mini apps, such as a phone book, to the display.

So, given an Active directory populated with entries, try this