Blog post:

Django deployment using Stackato

As mentioned in the previous post, Stackato has its roots in CloudFoundry and ActiveState claims that it’s backwards compatible with Cloudfoundry’s API. Stackato is unique in that it’s not a publicly hosted offering (like Heroku or Dotcloud) but rather a PaaS-in-a-box that you install on your own servers or on a IaaS provider such as Amazon EC2 or HP Cloud. Here is a diagram that shows architecturally how Stackato is put together.

To try it out, you have 3 options.

  1. Deploy to ActiveState’s Stackato sandbox.
  2. Download a VirtualBox image and run it on your local machine (what they call a “microcloud”)
  3. Launch an EC2 instance using the Stackato AMI

In this post, I’ll only cover the first option but it’s pretty easy to test with the microcloud or EC2 instance if you want to have your own environment to play with, and see the management backend of Stackato.

You can apply for sandbox access to ActiveState’s Stackato sandbox which is usually granted within a few hours after applying. You get the Stackato Sandbox Access username and password on your ActiveState account page.

Installing the Stackato commmand line tool

Next you need to download the Stackato command-line tool, a small binary client written in TCL (if you’re curious, you can look at the source in their Github account). The Stackato docs have a whole section ondeploying Django that will be useful to reference as we walk through this example.

For convenience, put the client in your /usr/local/bin directory so that it’s available from anywhere:

$ unzip stackato-1.4.5-macosx10.5-i386-x86_64.zip
$ sudo cp stackato-1.4.5-macosx10.5-i386-x86_64 /usr/local/bin
$ sudo ln -s /usr/local/bin/stackato-1.4.5-macosx10.5-i386-x86_64 /usr/local/bin/stackato

Mezzanine revisited

Mezzanine is the most popular CMS/blogging add-on for Django, and we’re going to use it as the example Django project to deploy to Stackato, and all subsequent PaaS providers in this blog article series. In this postwe described how to get a stock Mezzanine site set up and running locally. You can see the this Mezzanine project template in the master branch, and the Stackato version that we’re going to make below in theStackato branch.

Configure WSGI

Stackato uses uWSGI by default, so we need to create a wsgi.py file that uWSGI will use to load Django. Put this file in the paasbakeoff directory:

1
2
3
4
5
6
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mywebsite.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Create requirements.txt file

You can see that the ‘mezzanine-website’ command has made a skeleton Mezzanine project for us, including a requirements directory, which contains a project.txt. Since Stackato requires the requirements.txt file to be in the top level directory, let’s make a requirements.txt file in the root, which points to this project.txt file:

$ echo "-r mywebsite/requirements/project.txt" > requirements.txt

Target and login to the Stackato endpoint

Now we’re going to try to push the app to the Stackato’s sandbox. First we need to tell Stackato what API endpoint to use. In this case it’s api.stacka.to, but if you’re using your own EC2 or microcloud then that domain name instead.

$ stackato target api.stacka.to
$ stackato login --email user@domain.com
Attempting login to [https://api.stacka.to]
Password: *********
Successfully logged into [https://api.stacka.to]

Deploy to the Stackato sandbox

Now that we’ve successfully logged in, let’s deploy the app to the Stackato sandbox with the ‘push’ command. We’ll answer a bunch of questions the first time, and at the end it will prompt us to save this configuration as a stackato.yml file:

$ stackato push
Would you like to deploy from the current directory ?  [Yn]:
Would you like to use 'paasbakeoff' as application name ?  [Yn]:
Detected a Python Application, is this correct ?  [Yn]:
Framework:       python
Runtime:         <framework-specific default>
Application Deployed URL [paasbakeoff.stacka.to]:
Application Url: paasbakeoff.stacka.to
Enter Memory Reservation [128M]:
Creating Application [paasbakeoff]: OK
Create services to bind to 'paasbakeoff' ?  [yN]: y
What kind of service ?
1. filesystem
2. memcached
3. mongodb
4. mysql
5. postgresql
6. rabbitmq
7. redis
Choose: 5
Specify the name of the service [postgresql-cf691]:
Creating Service: OK
Binding Service: OK
Create another ?  [yN]:
Would you like to save this configuration? [yN]: y
Uploading Application [paasbakeoff]:
  Checking for bad links: 29 OK
  Copying to temp space: 28 OK
  Checking for available resources: 44022 OK
  Packing application: OK
  Uploading (20K): 100% OK
Push Status: OK
Staging Application [paasbakeoff]:
-----> Installing dependencies using pip
...
InstallationError: Command /opt/ActivePython-2.7/bin/python -c 'import setuptools;__file__='/staging/staged/build/pillow/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))' install --single-version-externally-managed --record /tmp/pip-v9RVEi-record/install-record.txt --user failed with error code 1 in /staging/staged/build/pillow

This will fail with this error:

1
2
3
4
libImaging/JpegDecode.c: In function ‘ImagingJpegDecode’:
libImaging/JpegDecode.c:136:2: error: ‘JPEG_LIB_VERSION’ undeclared (first use in this function)
libImaging/JpegDecode.c:136:2: note: each undeclared identifier is reported only once for each function it appears in
error: command 'gcc' failed with exit status 1

The solution is to add ‘pillow’ as a PyPm dependency in the stackato.yml file. This is described in the stackato.yml reference in the Stackato docs (http://docs.stackato.com/reference/stackatoyml.html#language-modules).

Defining package dependencies

The auto-generated stackato.yml file looks like this:

1
2
3
4
5
6
7
name: paasbakeoff
instances: 1
framework:
  type: python
mem: 128
services:
  postgresql-cf691: postgresql

We’re going to add a requirements section to include pillow as a dependency, and while we’re at it, since we chose PostgreSQL as our database above, we also need to add psycopg2 as a requirement (if we were using MySQL, then we would add MySQL-python instead):

1
2
3
4
requirements:
  pypm:
    - pillow
    - psycopg2

If we wanted to install psycopg2 using pip, we could have added a pip requirements section:

1
2
3
4
5
requirements:
  pypm:
    - pillow
  pip:
    - psycopg2

Or we could have added it to our requirements.txt file with:

$ echo "psycopg2" >> requirements.txt

But using ActiveState’s PyPm repository is faster than using pip, since the PyPm packages are already compiled.

Once you’ve pushed the app the first time, subsequent pushes must be done with the ‘update’ command:

$ stackato update -n
Updating application 'paasbakeoff'...
Application Url: paasbakeoff.stacka.to
Uploading Application [paasbakeoff]:
  Checking for bad links: 29 OK
  Copying to temp space: 28 OK
  Checking for available resources: 44056 OK
  Packing application: OK
  Uploading (20K): 100% OK
Push Status: OK
Note that [paasbakeoff] was not automatically started because it was STOPPED before the update.
You can start it manually using `stackato start paasbakeoff`

We won’t start it yet, because there’s still something we need to do in order for Mezzanine to recognize the PostgreSQL database that Stackato has created for us. Modify the settings.py file, to include these imports at the top of the file:

1
2
import os
import urlparse

And replace the ‘DATABASES’ section with the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#############
# DATABASES #
#############
DATABASES = {}
if 'DATABASE_URL' in os.environ:
    url = urlparse.urlparse(os.environ['DATABASE_URL'])
    DATABASES['default'] = {
        'NAME': url.path[1:],
        'USER': url.username,
        'PASSWORD': url.password,
        'HOST': url.hostname,
        'PORT': url.port,
        }
    if url.scheme == 'postgres':
        DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
    elif url.scheme == 'mysql':
        DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
else:
    DATABASES['default'] = {
        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'dev.db',                      # Or path to database file if using sqlite3.
        'USER': '',                      # Not used with sqlite3.
        'PASSWORD': '',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
        }

Run the stackato update command again:

$ stackato update -n
...
Successfully installed Mezzanine dj-database-url django filebrowser-safe grappelli-safe bleach pytz html5lib
Cleaning up...
OK
Starting Application [paasbakeoff]: OK
http://paasbakeoff.stacka.to/ deployed to Stackato

If it doesn’t start up, you can start it with:

$ stackato start

If there are any problems, you can look at the logs with:

$ stackato logs

Even though it looks like everything worked properly, there is a problem because Stackato is using local_settings.py which defines SQLite as the database, which will override our special Stackato database settings. We need to make sure that .gitignore has this file included, and it doesn’t hurt to add an ignores: section in stackato.yml as well:

1
2
3
4
ignores:
  - .git
  - mywebsite/dev.db
  - mywebsite/local_settings.py

Run stackato update -n after you make this change.

Running manage.py syncdb

The database is not set up yet because we haven’t run manage.py syncdb. (Note: in Django 1.4, the manage.py file is one directory above where the settings.py file is, so for the following command you would drop the ‘mywebsite’ prefix if you’ve created a new Django project using the django-admin.py startproject command):

$ stackato run python mywebsite/manage.py syncdb --noinput
Creating tables ...
...
Creating table django_comments
Creating table django_comment_flags
Creating default account (username: admin / password: default) ...
Creating default Site 127.0.0.1:8000 ...
Creating initial content (About page, Blog, Contact form, Gallery) ...
Installed 18 object(s) from 3 fixture(s)
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Serving up static assets

When we go to http://paasbakeoff.stacka.to/ we can see that the site is up, but none of the images or CSS are loading properly. This is because we need to collect all the static files using the collectstatic command:

$ stackato run python mywebsite/manage.py collectstatic --noinput
...
Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/rtl.css'
Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/ie.css'
Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/forms.css'
554 static files copied.

You can make sure that the files were copied to the right place with this command:

$ stackato files app/mywebsite/static
admin/                                       -
css/                                         -
filebrowser/                                 -
grappelli/                                   -
img/                                         -
js/                                          -
media/                                       -
mezzanine/                                   -
robots.txt                                 24B
test/                                        -

And then we need to tell Stackato to serve up these static assets using uWSGI. This is easy to setup, by a few lines to our stackato.yml file:

1
2
3
4
5
6
framework:
  type: python
  home-dir: app
processes:
  web: $STACKATO_UWSGI --static-map /static=$HOME/mywebsite/static

Run stackato update -n after you make this change.

Read more about serving static files using uWSGI

Run syncdb and collectstatic commands on every deploy

It would be nice if the syncdb and collectstatic commands would be run every time we do a deploy. We can add these to a post-staging hook in the stackato.yml file:

1
2
3
4
hooks:
  post-staging:
    - python mywebsite/manage.py syncdb --noinput
    - python mywebsite/manage.py collectstatic --noinput

Note: it’s important to put the –noinput, or else the command will halt waiting for user input.

South migrations

If we were using South for database schema migrations (which is highly recommended), then we could add a manage.py migrate command here as well:

1
2
3
4
5
hooks:
  post-staging:
    - python mywebsite/manage.py syncdb --noinput
    - python mywebsite/manage.py collectstatic --noinput
    - python mywebsite/manage.py migrate --noinput

Just make sure that you add South to the PyPm requirements in stackato.yml, or add it to your requirements.txt file:

1
2
3
4
5
requirements:
  pypm:
    - pillow
    - psycopg2
    - south

Persisted filesystem for file uploads

Mezzanine, like many other Django-based CMSes lets users upload files (images, PDFs, etc.) which need to be persisted on the server’s filesystem. By default, the filesystem is ephemeral meaning, that it will get wiped out on each deploy. Thankfully, Stackato provides a way to persist these files across deploys. You need to add a filesystem service to the services section of stackato.yml:

1
2
3
services:
  postgresql-cf691: postgresql
  filesystem-paasbakeoff: filesystem

In then in settings.py, we need to tell Django to use this as the MEDIA_ROOT:

1
2
3
4
STACKATO = 'VCAP_APPLICATION' in os.environ
if STACKATO:
    MEDIA_ROOT = os.environ['STACKATO_FILESYSTEM']

Run stackato update -n after you make this change.

Read more about the persistent filesystem support on Stackato here.

Caching with memcache

One of the supported services of Stackato is memcached, and to provision a new memcached instance, you simply add it to the services section of stackato.yml, and add python-memcached as a dependency in the pypm requirements:

1
2
3
4
5
6
services:
  postgresql-cf691: postgresql
  memcached-paasbakeoff: memcached
requirements:
  pypm:
    - python-memcached

And then you need to add the following to your settings.py file:

1
2
3
4
5
6
7
if 'MEMCACHE_URL' in os.environ:
    CACHES = {
        'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': os.getenv('MEMCACHE_URL'),
        }
    }

And make sure that these are in your middleware classes (should already be there in default Mezzanine):

1
2
3
4
5
MIDDLEWARE_CLASSES = (
    "mezzanine.core.middleware.UpdateCacheMiddleware",
    ...
    "mezzanine.core.middleware.FetchFromCacheMiddleware",
)

Run stackato update -n after you make this change.

Read more about memcached support on Stackato.

The benefits of using Linux Containers

Stackato uses Linux Containers (LXC), almost like a lightweight virtual machine, to isolate each app from the other apps running on the server. This is not only more secure but it prevents one rogue app from sucking up all the memory on the machine, because each app is memory-bound.

SSHing into the container

Another benefit of using LXC is that it’s possible to SSH into the container and have full control without the risk of affecting other apps:

$ stackato ssh

Defining OS dependencies

Since each app is essentially running in its own Linux box, you can install OS-level dependencies with apt-get or yum. You can either do this by SSHing into the container, or to persist the changes across deploys, it’s best to define these dependencies in the stackato.yml file.

For example, if we wanted to define python-lxml as a OS level package to be installed, we could add it as follows:

1
2
3
4
requirements:
  running:
    ubuntu:
      - python-lxml

Read more about OS dependencies on Stackato

Wrap up

That concludes the first article in this PaaS deployment for Django series! I hope you enjoyed learning how to get your Django app deployed quickly using Stackato. For the next article, we’ll walk through how to deploy Mezzanine with Dotcloud.

If you find an errors in the examples, or have feedback, I welcome you to use the feedback button on the right of the page, or the live chat button on the bottom of the page. You can also submit an issue to the Github issue tracker, and we’ll fix it right away.

Don’t miss the next article!

If you’d like to be notified when new articles in this series are posted, sign up for the SaaS Developers Kitannouncement list, and we’ll shoot you an email as soon a new post is available.

 

End of post.