So as many know I am firmly an advocate of git, and a pesemist when it comes to subversion because well it seems to fail for unresolveable reasons either through use, or through fault of it’s own (lock files in meta directories for one …).
Now this is not to say that git is infallable. At any rate here is the best way to use git as an svn client and still maintain access to branches and tags.
git svn init -s <protocol>://<FQDN of server>/<repo path> ./folder_to_checkout_to
This initializes an empty git respository in the folder specified (This can also be handy for migrating from subversion)
The examples above use -s for stdlayout, the 2nd example can be used to specify exact locations of trunk,tags,branches.
Now we need the data:
git svn fetch
One thing to note that whilst tags are present within git, subversion tags do not (at this stage) translate into git tags, as such to checkout svn tags (if you are not yet ready to make the jump to using git and want to maintain a subversion server this will needed).
git checkout -b tags/tag_name tags/tag_name
What does this do? it will checkout the tag from subversion: tag_name and setup a local git branch to track it.
We’re going to build a minimal chroot directory for Fedora 16 using yum and rpm, we are using the ChrootDirectory functionality of Openssh which only came in >= 4.9p1
Credit goes Here for a great article getting me started on this.
Match Group chrooted
ChrootDirectory /chroot/%u
AllowTcpForwarding no
X11Forwarding no
AllowAgentForwarding no
PermitRootLogin no
ForceCommand /bin/bash
And restart the service: systemctl restart sshd.service
In this post I will cover getting openstack nova and glance services installed from EPEL and configured to the point where an image can be started, this assumes
You have a mysql instance installed and running
You have a rabbitmq-server installed and running
You have kvm installed and running (libvirt)
You have selinux set to permissive, as I will not be covering selinux rules here at this time and I do not think disabled is a valid option ;-)
I will also be carrying out mySQL configuration of glance and nova, for 2011.3 (Diablo), though most if not all of this should be portable to the Essex release
yum should take care of all the dependencies here, and install both with a minimal configuration.
Burning and Rebuilding bridges
First thing’s first KVM is going to install with it’s own default bridged networking, this provides NAT.
Which is also noted as being very slow (There is/was an note on the wiki@ linux-kvm.org but I have been unable to locate it at the time of writing)
If you are only setting this up for experimentation you can run with the default networking, simply use vibr0 in your nova.conf instead of br0, and ensure you have ipv4 forwarding enabled.
Burning Bridges
123456789
virsh net-list
Name State Autostart
-----------------------------------------
default active yes
virsh net-destroy default
Network default destroyed
virsh net-undefine default
Network default has been undefined
service libvirtd restart
Building Bridges
The theory here is that this configuration of bridge will give us near native network performance, which if you are setting up for use beyond a throwaway sandbox, you really do not want to start introducing bottlenecks.
Shutdown and disable NetworkManager
123
service NetworkManager stop
chkconfig NetworkManager off
chkconfig network on
If you know of a NetworkManager friendly way of doing the following please let me know!
There is plenty more fun to be had here such as bonded interfaces (I myself have a few systems with bonded interfaces as such becoming br0 -> bond0 -> NIC’s), but that’s for another time.
Note: you may also use brctl for temporary configurations if you are just experimenting.
Caution: my network dropped out immediatly on my testbox, most likely because networkmanager was running, always ensure you can attach to the head of your box when doing network configuration ;-)
Once you have these configurations in place (Ensuring your have replaced the placeholder IP’s and MAC address with valid ones) you can now go for a
1
service network restart
All being well you’ll lose and re-establish connection, of you’ll be attaching a monitor / to kvm over ip.
Configuring Nova
First we’re going to need a blank database, please ensure you change the placeholder password that follows for something more secure, and amend the host if you are using mySQL on the same host as nova.
Setup the database and start the relevant nova services
nova-manage db sync
for i in api network scheduler compute;do service openstack-nova-$i start;donefor i in api network scheduler compute;do chkconfig openstack-nova-$i on;done
Note: you could also use openstack-nova-db-setup instead of “nova-manage db sync”, but it requires mysql-server, which at the time of writing if you have Percona installed will falsely adivse you a need to install mysql-server, Percona need to add: “Provides: mysql-server” to their spec ideally.
Remember this is only a basic setup so a lot of the options are left default such as the network_manager, I will cover their options at a later date.
Onto setting up a basic user (Note: this will be replaced in future posts with keystone)
Once you have made the change, unlike nova all you need do is start glance and it will setup the database.
for i in api registry;do chkconfig openstack-glance-$i on; service openstack-glance-$i start;done
Now were going to need an image, I’m using the BT5-R2 .iso as an example, you could use any of the pre-generated images out there, or even build them using oz
Once the import has completed it should appear in your glance index
1234
glance index
ID Name Disk Format Container Format Size
---------------- ------------------------------ -------------------- -------------------- --------------
1 BT5-R2-Gnome-x64 raw ovf 2762084352
And assuming you setup your nova.conf correctly you should now be able to see this image from nova
123456
nova image-list
+----+------------------+--------+
| ID | Name | Status |
+----+------------------+--------+
| 1 | BT5-R2-Gnome-x64 | ACTIVE |
+----+------------------+--------+
You will also have some default instance sizes aka flavours (commands use american spelling flavor).
And as iso-boot is not currently complete, this example falls down here, as the instance fails to boot from the .iso file, still you now have
Successfully configured nova
Sucessfully configured glance
Have nova using glance
All you need do is load a valid image into glance and boot using nova, so now I will be cheating a little I will create a blank 10GB qcow2 image, import it into glance
boot it and use virt-manager to attach the .iso and reboot.
Now I cheat I used virt-manager to force off the insance, create and attach an IDE cdrom and set it as the primary boot device.
BT5 boots from the ISO and I can even begin to work through the install to hard drive menus, which as irony would have it prompts me that it needs an 11.5GB partition to install upon :D
I will cover producing proper images in my next openstack post, as the size of the storage volume should not be defined by the image in glance, it should be defined by the falvour being started.
n2n is a layer-two peer-to-peer virtual private network (VPN) which allows users to exploit features typical of P2P applications at network instead of application level. This means that users can gain native IP visibility (e.g. two PCs belonging to the same n2n network can ping each other) and be reachable with the same network IP address regardless of the network where they currently belong. In a nutshell, as OpenVPN moved SSL from application (e.g. used to implement the HTTPS protocol) to network protocol, n2n moves P2P from application to network level.
So why do I care ?
Some services you may wish to run on a public cloud such as Gluster do not have (at the time of writing) internal TLS (read: encryption), this n2n allows you to establish peer to peer vpn connections, wihtout the need of a single routing device (with some assume caveats I will cover shortly).
So in short you can have your own private network within the cloud environment without affecting that environment, this allows for:
TLS for services otherwise sent “in the clear”
Potential for Cluster services and floating IP’s without touching the host network infrastructure.
Installation
We are going to use EPEL, why? because I’m a packager and I will be using redhat for this setup, so admitedly I am a little biased toward RedHat, that said the majority of the following configurations should be portable to other distros, leave a commment if you get stuck I will try to help!
yum -y install n2n
And no I’m not using sudo i.m.o sudo is akin to “training wheels”, and somethign I will only generally use if I have too (such as maintaining an auditable system), you are of course welcome to use sudo yourself, I use “throw away” vm’s for all my experimentation so in these cases the ethos is if it’s broken it gets rebuilt.
SuperNode Setup
First thing’s first we’re going to need at least 1 Supernode, as I uderstand it a Supernode is used to register new peers and to retrieve currently connected peers.
Once this list is retrieved the individual nodes will communicate directly (p2p), and not via the supernode.
Caveats to note:
If all supernodes are down, only existing peers can communicate, new peers can not.
supernode whilst installed does not at the time of writing provide an init.d/sysvinit script, you may use the following:
place the above in /etc/init.d/supernode and chmod +x i.e.
(Though I really do recommend you read through this code first before trusting it blindly!)
Note: Annoyingly I had to use the -f (foreground) flag to allow the daemon wrapper to function correctly with this process, there is more than likely a better solution, please
feel free to revise the gist it is public.
Now as I have opted to use a non existant n2n account to daemonize the process this will need creating as will the pid directory.
You will now be able to start your supernode with: /etc/init.d/supernode start.
In my configuration above I have chosen to bind port 1200, you can change this to any port, but remember that your vpn peers will need to be able to access this port.
As such you will need the relevant iptables rules
I highly recomend you limit your firewall to only allow connection from known peers, and that this is done over the internal interface (for which you do not generally pay bandwidth charges).
I also recomend you repeat this process on a 2nd node to provide 2 Supernodes (The maximum allowable) for greater resilliance.
Edge Setup
I have opted for a .conf file approach here, you can of course opt to instead embed everything in the sysvinit script.
Place this in /etc/edge.conf, you can negate ADDRESS if you wish to use DHCP, whilst you can also Negate SUPERNODE2 and MAC I do not recomend doing so for the following reasons.
Negating Supernode2 means there is only 1 supernode and as such a single point of failiure in the setup
Negating MAC is valid, however on loss of connection and restoration a new MAC is generated meaning all existing nodes can not communicate with the restored node untill their local ARP caches are cleared,
specifiying a static MAC address ensures immediate restoration of communication.
I have made PORT a requirement, it is technically optional but fixing the port makes your iptables / firewall rules far easier.
Make sure you actually edit the file and replace the args with VALID ones, especially the SHAREDKEY as the above is in no way secure!
(Again I HIGHLY recommend you actually read the code before blindly trusting it!)
Note: the –user option is negated in this init file. This is because we need to actually create a network interface, something that can only be done as root.
As such we are reliant on the edge binary to drop privileges itself by providing the -u and -g arguments, these are of course assuming you have allready setup the n2n user, as per above and not just skipped to this section.
Add the Services and set them to run
chkconfig --add supernode
chkconfig --add edge
chkconfig supernode on
chkconfig edge on
Modify other services that are reliant on the VPN
Modify the “Requires” line in the sysvinit script for each service you want to only start once your VPN has been established.
Note: Whilst I have opted for requiring supernode here, you do not need this, you can require just your edge service, as the supernode does not have to run on the same device.
You should now be able to reboot and see all required services start up in the correct order.
And done, that’s where I am ending this blog post,
we have setup n2n with supernodes and edge
generated valid sysvinit scripts
expect future posts to cover more advanced n2n configuration as I discover the options available.
To facilitate running wordpress on the smalest possible CloudServer I am using Varnish which using Apache as the backend, now with static files I can get all the blogging functionality without the need for Wordpress nor varnish, yet still uncached content can lead to increased load on the server.
Also wordpress does not lend itself to scalability especially with the at the time of writing schema and sql queries (percona-query-advisor flags up a few wordpress core sql queries as non scalable).
But this does not mean you need remove the option of using Wordpress, for you client for instance keeping wordpress in place can aid in content generation simply through ease of use.
wordpress used as normal
wordpress content is ported to markdown
markdown used to generate html
Now this does have caveats:
you’re going to need to maintain designs in wordpress templates and markdown (_layouts).
you’re going to need to handel any shortcode plugins in your export process.
you’re going to need to handel any other content modifying plugins in your export process.
any UGC / Dynamic requests will still need PHP.
But it essentially replaces the whole caching layer with static content which then can be pushed to CDN.
And with CDN’s now supporting index files (S3, and Coming Soon @ CloudFiles) in essence entire sites can be placed on CDN whilst maintaining ease of content generation.
Now don’t get me wrong, this requires a whole lot of “glue” to get working, but the potential for serving an entire web app from CDN without Origin pull / cache headers etc, saves a lot of systems time in scaling and adressing performance issues, or rather makes them “less critical” as the “business” part of the webapp is all on CDN.
I’m still getting to grips with Jekyll and by extention Octopress to see what it can achieve, so expect more posts.
In this post I will cover getting openstack nova and glance services installed from EPEL and configured to the point where an image can be started, this assumes