First off what is n2n ?

n2n is a layer-two peer-to-peer virtual private network (VPN) which allows users to exploit features typical of P2P applications at network instead of application level. This means that users can gain native IP visibility (e.g. two PCs belonging to the same n2n network can ping each other) and be reachable with the same network IP address regardless of the network where they currently belong. In a nutshell, as OpenVPN moved SSL from application (e.g. used to implement the HTTPS protocol) to network protocol, n2n moves P2P from application to network level.

So why do I care ?

Some services you may wish to run on a public cloud such as Gluster do not have (at the time of writing) internal TLS (read: encryption), this n2n allows you to establish peer to peer vpn connections, wihtout the need of a single routing device (with some assume caveats I will cover shortly).

So in short you can have your own private network within the cloud environment without affecting that environment, this allows for:

1. TLS for services otherwise sent “in the clear”
2. Potential for Cluster services and floating IP’s without touching the host network infrastructure.

Installation

We are going to use EPEL, why? because I’m a packager and I will be using redhat for this setup, so admitedly I am a little biased toward RedHat, that said the majority of the following configurations should be portable to other distros, leave a commment if you get stuck I will try to help!

yum -y install n2n

And no I’m not using sudo i.m.o sudo is akin to “training wheels”, and somethign I will only generally use if I have too (such as maintaining an auditable system), you are of course welcome to use sudo yourself, I use “throw away” vm’s for all my experimentation so in these cases the ethos is if it’s broken it gets rebuilt.

SuperNode Setup

First thing’s first we’re going to need at least 1 Supernode, as I uderstand it a Supernode is used to register new peers and to retrieve currently connected peers. Once this list is retrieved the individual nodes will communicate directly (p2p), and not via the supernode.

Caveats to note:

1. If all supernodes are down, only existing peers can communicate, new peers can not.

supernode whilst installed does not at the time of writing provide an init.d/sysvinit script, you may use the following:

place the above in /etc/init.d/supernode and chmod +x i.e.

curl -o /etc/init.d/supernode https://raw.github.com/gist/1986260/b66b38da265ea14aac8d0ef7196a9ba98939716c/supernode.sh && chmod +x /etc/init.d/supernode

(Though I really do recommend you read through this code first before trusting it blindly!)

Note: Annoyingly I had to use the -f (foreground) flag to allow the daemon wrapper to function correctly with this process, there is more than likely a better solution, please feel free to revise the gist it is public.

Now as I have opted to use a non existant n2n account to daemonize the process this will need creating as will the pid directory.

useradd -d /dev/null -s /sbin/nologin n2n
mkdir /var/run/supernode && chown n2n:n2n /var/run/supernode

You will now be able to start your supernode with: /etc/init.d/supernode start.

In my configuration above I have chosen to bind port 1200, you can change this to any port, but remember that your vpn peers will need to be able to access this port. As such you will need the relevant iptables rules

iptables -N N2N
iptables -I INPUT -j N2N
iptables -A N2N -s <vpn peer> -p udp --dport 1200 -j ACCEPT

I highly recomend you limit your firewall to only allow connection from known peers, and that this is done over the internal interface (for which you do not generally pay bandwidth charges).

I also recomend you repeat this process on a 2nd node to provide 2 Supernodes (The maximum allowable) for greater resilliance.

Edge Setup

I have opted for a .conf file approach here, you can of course opt to instead embed everything in the sysvinit script.

DEVICE="n0"
ADDRESS="127.16.0.1"
MAC="00:11:22:33:44:55"
COMMUNITY="N2N"
SHAREDKEY="asdf12345"
SUPER1="1.2.3.4:1200"
SUPER2="1.2.3.5:1200"
PORT="1201"

Place this in /etc/edge.conf, you can negate ADDRESS if you wish to use DHCP, whilst you can also Negate SUPERNODE2 and MAC I do not recomend doing so for the following reasons.

1. Negating Supernode2 means there is only 1 supernode and as such a single point of failiure in the setup
2. Negating MAC is valid, however on loss of connection and restoration a new MAC is generated meaning all existing nodes can not communicate with the restored node untill their local ARP caches are cleared, specifiying a static MAC address ensures immediate restoration of communication.
3. I have made PORT a requirement, it is technically optional but fixing the port makes your iptables / firewall rules far easier.
4. Make sure you actually edit the file and replace the args with VALID ones, especially the SHAREDKEY as the above is in no way secure!
5. Make sure your ip and mac addresses are unique!

We need to prep the pid dir again:

mkdir /var/run/edge && chown n2n:n2n /var/run/edge

place the above in /etc/init.d/edge and chmod +x i.e.

curl -o /etc/init.d/edge https://raw.github.com/gist/1986260/3061d0fb9d6f2ddf1608f01917129d65b8131d33/edge.sh && chmod +x /etc/init.d/edge

(Again I HIGHLY recommend you actually read the code before blindly trusting it!)

Note: the –user option is negated in this init file. This is because we need to actually create a network interface, something that can only be done as root. As such we are reliant on the edge binary to drop privileges itself by providing the -u and -g arguments, these are of course assuming you have allready setup the n2n user, as per above and not just skipped to this section.

Add the Services and set them to run

chkconfig --add supernode
chkconfig --add edge
chkconfig supernode on
chkconfig edge on

Modify other services that are reliant on the VPN

Modify the “Requires” line in the sysvinit script for each service you want to only start once your VPN has been established.

...
# Required-Start: $local_fs $network $supernode $edge
...

Note: Whilst I have opted for requiring supernode here, you do not need this, you can require just your edge service, as the supernode does not have to run on the same device.

You should now be able to reboot and see all required services start up in the correct order.

And done, that’s where I am ending this blog post,

1. we have setup n2n with supernodes and edge
2. generated valid sysvinit scripts

expect future posts to cover more advanced n2n configuration as I discover the options available.