This took me awhile to figure out today, so I figure it’s worth a post (and we all know I’m WAAAY behind on those!).
The problem I ran into was with an OpenStack (diablo build) instance that we use internally at my work. To make a long story short, I chose to go around the nova-network method of routing and use my own hardware gateway as opposed to using the software appliance provided by the OpenStack setup.
Now, a brief bit of Background here.
1. Openstack is a “Virtualization Fabric”, i.e., a piece of software that ties together hypervisors with a nice API. It handles the networking, connection to the hypervisors, sercurity and let’s you access everything with an API. This is the same software that the Rackspace Cloud is setup and comparable to what VPS.Net uses (onapp.com).
2. Openstack is comprised of components, nova-network handles the networking, routing and firewalling. Nova-api handles the API calls. Nova-scheduler handles all the scheduling. Nova-compute actually interfaces with the hypervisors and creates/manages the VMs with nova-volume providing an EBS style of volume provisioning.
3. When you provision a VM, it automatically routes through the machine you’ve designated to run nova-network, I didn’t like this.
So, I decided to change how the networking on the VMs routes to the public internet. There’s a service called ‘dnsmasq’ that runs on the nova-network machine which hands out IP and basic configuration information to your VMs when they come up. It gives your VMs their IP, Gateway, subnet mask, resolvers and a few other items as well. So here’s, where things needed to change.
Step 1. Know your real network gateway!
Traditionally in an openstack setup, your VMs will route like this…
VM->Nova-network appliance->network gateway->Internet.
I was modifying it to route as follows…
This is handled by adding one line to your nova.conf file, and creating a super basic configuation file to go along with it. By adding ‘–dnsmasq_config_file=/etc/nova/dnsmasq.conf’ to your nova.conf file, and ensuring that /etc/nova/dnsmasq.conf looks like the following snippet, you’ll 1. route through your own gateway, and 2. use your own resolvers.
dhcp-option 6 should be set to the resolver you want to use, and dhcp-option 3 should be set to the gateway you want to use.
After that, do a ‘killall dnsmasq; /etc/init.d/nova-network restart’, and then restart your instances and you should be good to go!