Fabric and Cuisine help dish up a Vagrant box

The goal of this post will to be create a Vagrant box which will be suitable for deploying a simple Flask app.

Before getting too far it is important to map out how the directory structure is going to look. Our project directory is going to contain a Python virtual environment, a Vagrant virtual machine, and a fabfile.py with our Fabric configuration. So to start off, be prepared for things to look like this:

/parent_directory
    /fabfile.py
    /Vagrantfile

Vagrant

[Vagrant] is an open-source wrapper for VirtualBox that allows the kind of configuration management we are about to embark on. If you are new to Vagrant have a look at the getting started guide for a quick introduction.

Download a vagrant box. I chose a fairly up-to-date 64-bit version of Debian Wheezy from Vagrantbox.es with Virtual Box’s Virtual Tools installed.

NB: This post is only targeted at Debian-based systems. I haven’t verified other Linux distributions or *BSDs.

I suggest using your web browser, curl, or wget to download the image before trying to add the base box. I tried a one-liner vagrant box add with a URL and for some reason it failed. If your Internet connection is as slow as my work connection, it is better to be safe and separately download the Vagrant box and then add it:

$ vagrant box add Debian-7.2.0-amd64-VitualTools /path/to/debian-7.2.0.box

Create a new Vagrantfile

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # All Vagrant configuration is done here. The most common configuration
  # options are documented and commented below. For a complete reference,
  # please see the online documentation at vagrantup.com.

  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "Debian-7.2.0-amd64-VitualTools"

  # The url from where the 'config.vm.box' box will be fetched if it
  # doesn't already exist on the user's system.
  config.vm.box_url = "https://dl.dropboxusercontent.com/u/197673519/debian-7.2.0.box"

  # Create a forwarded port mapping which allows access to a specific port
  # within the machine from a port on the host machine. In the example below,
  # accessing "localhost:8080" will access port 80 on the guest machine.
  config.vm.network :forwarded_port, guest: 80, host: 8080
  config.vm.network :forwarded_port, guest: 8000, host: 8001
end

We should now be able to start and stop our vagrant instance with the commands:

$ vagrant up

$ vagrant down

Fabric and Cuisine

We will want to install both Fabric and Cuisine.

Fabric is a Python library and command-line tool for the use of SSH for deployment and system administration which we will combine with Cuisine’s higher level functions to prepare a Vagrant VM for app deployment. Here is the quick hit list to get us going:

$ mkdir parent_directory && cd parent_directory

Setup virtualenv:

$ virtualenv venv

Activate the environment:

$ source venv/bin/activate

Install Fabric

$ pip install fabric

Install Cuisine

$ pip install cuisine

First steps

We are now ready to configure our Vagrant instance. First thing we should do is set up our first recipe (if you want to call it that).

# fabfile.py
import cuisine
from fabric.api import *

def vagrant():
    "Using Vagrant for development testing."
    env.user = 'vagrant'
    env.hosts = ['127.0.0.1:2222']

    # Check vagrant machine status and power it up if off
    status = cuisine.run_local('vagrant status | grep default')
    if 'running' in status:
        puts('Vagrant already up.')
    else:
        puts("Vagrant VM powered off so starting...")
        cuisine.run_local('vagrant up')
        puts("Vagrant up!")

    # retrieve the IdentifyFile for vagrant
    result = cuisine.run_local('vagrant ssh-config | grep IdentityFile')
    env.key_filename = result.lstrip('  IdentityFile').strip() # parse IdentifyFile

def update():
    "Updates remote package manager."
    puts("Updating package list.")
    cuisine.package_update()

def setup():
    "Installs pre-requisite packages."
    update()

    puts("Installing pre-requisite packages.")
    pre_requisites = ['build-essential', 'gettext']

    for package in pre_requisites:
        cuisine.package_ensure(package)

We can now start preparing our new Vagrant box with the command:

$ fab vagrant setup

Re-organise our recipe

We are on the right track, but let’s make it a little more modular. We are going to split up our fabfile.py into a Python package. We will create a new directory called fabfile and shuffle our code a bit. The resulting structure will look like this:

/parent_directory
    /fabfile
        /__init__.py
        /pre_requisites.py
    /venv

The split up fabfile.py now looks like:

#/fabfile/__init.py___
import cuisine
from fabric.api import *
from . import pre_requisites

env.roledefs = {
    'vagrant': ['127.0.0.1:2222']
}

def update():
    "Updates remote package manager."
    puts("Updating package list.")
    cuisine.package_update()

@roles("vagrant")
def configure():
    "Using Vagrant for development testing."
    env.user = 'vagrant'

    # Check vagrant machine status and power it up if off
    status = cuisine.run_local('vagrant status | grep default')
    if 'running' in status:
        puts('Vagrant already up.')
    else:
        puts("Vagrant VM powered off so starting...")
        cuisine.run_local('vagrant up')
        puts("Vagrant up!")

    # retrieve the IdentifyFile for vagrant
    result = cuisine.run_local('vagrant ssh-config | grep IdentityFile')
    env.key_filename = result.lstrip('  IdentityFile').strip() # parse IdentifyFile

    update()
    pre_requisites.install()

And a new pre_requisites.py file which will install all the non-Python stuff.

#/fabfile/pre_requisites.py
from fabric.api import puts
import cuisine

def install():
    "Installs pre-requisite packages."

    puts("Installing pre-requisite packages.")
    pre_requisites = ['build-essential', 'gettext', 'nginx']

    for package in pre_requisites:
        cuisine.package_ensure(package)

    puts("Starting Nginx...")
    cuisine.sudo('service nginx start')

Now our fab command looks a little different but it still accomplishes the same thing plus we install Nginx and start it.

To make sure Cuisine is doing what I think it is doing, I am going to first delete my Vagrant VM so I get a fresh install:

$ vagrant destroy

Now let’s have fab start over and see if it still works:

$ fab configure

This will update the package manager and then install the pre-requisite packages for the server as well as start Nginx. We haven’t done anything really new; we have just re-organised how we will keep track of our recipes. Next up is a recipe to install all the Python related packages will will need.

#/fabfile/python.py
from fabric.api import puts
import cuisine

def install():
    "Installs Python related packages."

    puts("Installing Python related packages.")
    packages = ['python', 'python-dev', 'python-setuptools', 'python-pip', \
                'python-virtualenv']

    for package in packages:
        cuisine.package_ensure(package)

To use this new recipe we need to add it to the imports in __init__.py
from . import pre_requisites, python and append python.install() to the configure() function:

...
def configure():

...
    update()
    pre_requisites.install()
    python.install()    

Re-run the fab command and the Python dependencies will be installed.

$ fab configure

Everything looks good. We could create a Flask app if we want, but first let’s configure Nginx so it can forward any requests to a hypothetical web app:

#/fabfile/nginx.py
import cuisine
from fabric.utils import puts
import sys

def configure_nginx():
    if cuisine.file_exists('/usr/sbin/nginx'):
        pass
    else:
        puts("Nginx is not installed!")
        cuisine.package_ensure('nginx')
        cuisine.sudo('service nginx start')

    puts('Configuring Nginx web server')

    config_template = cuisine.text_strip_margin('''
    |
    |server {
    |   server_name localhost;
    |
    |   access_log /var/log/nginx/mailer_access.log;
    |   error_log /var/log/nginx/mailer_error.log;
    |
    |   location / {
    |       proxy_pass http://127.0.0.1:5000/;
    |       proxy_redirect off;
    |       proxy_set_header Host $host;
    |       proxy_set_header   X-Real-IP        $remote_addr;
    |       proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
    |   }
    |
    |}
    |
    ''')

    cuisine.file_write('/etc/nginx/sites-available/mailer.conf', \
                       config_template, sudo=True)

    if cuisine.file_exists('/etc/nginx/sites-enabled/default'):
        cuisine.sudo('rm /etc/nginx/sites-enabled/default')

    if not cuisine.file_exists('/etc/nginx/sites-enabled/mailer.conf'):
        cuisine.sudo('ln -s /etc/nginx/sites-available/mailer.conf \
                     /etc/nginx/sites-enabled/mailer.conf')

    cuisine.sudo('service nginx reload')

Just like above we will add the Nginx recipe to __init.py__ and call configure_nginx(). To explain what’s happening above, we are creating a new website configuration. Nginx will listen on localhost:80 and pass requests for / to localhost:5000 (the default port of Flask).

To make sure everything worked we can create a quick Hello World Flask app:

$ vagrant ssh

vagrant@debian-7:~$ mkdir test_app && cd test_app

vagrant@debian-7:~$ virtualenv venv && source venv/bin/activate

vagrant@debian-7:~$ pip install Flask

vagrant@debian-7:~$ cat << EOF > hello.py

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello World!'

if __name__ == '__main__':
    app.run(host='0.0.0.0')

EOF

If you load up http://localhost:8080/ in your web browser, on the host machine, you should see the words, Hello World.

Conculsion

We can now provision a Vagrant box with one command giving us a box which can serve a basic Flask application. The next step would be to use Fabric deploy a more complex Flask app. Stay tuned.

  1. drwlrsn posted this
How one shouldn't name a blog while listening to Cypress Hill

view archive