A journey through time and PaaS – introducing django-deployer
It seems that the topic of Django deployment never seems to go out of style. It’s a recurring theme for Django user groups across the country. In this post, I want to introduce a new project that I’m working on called django-deployer. But first, a brief history of deployment discussions over the last 3 years:
Python deployment and hosting open space (DjangoCon 2010)
At DjangoCon 2010, I organized an open space to discuss Python deployment and hosting issues which was surpringly well-attended. It became obvious then that deployment was a huge pain point for everyone from small two person dev shops, all the way up to larger companies like Redhat and Mozilla. That was also the year that I gave a 5 min lightning talk about DjangoZoom, the pivotal moment that set the stage for what was to come next. (if you want to watch me sweating on stage watch this video at 50:40)
Django deployment bonanza (San Francisco and Boston Django meetups)
A couple months later, I attended the Django Deployment Bonanza meetup in San Francisco and met Dave and Sameer, the founders of the now defunct Djangy PaaS. Just one month later, the Boston Django meetup got a special guest speaker when Jacob Kaplan Moss gave a talk on DevOps.
Python deployment and hosting open space (PyCon 2011)
At PyCon 2011 in Atlanta, I organized another open space on Python deployment and hosting which was attended by Mitch Garnaat (author of Boto and now works at Amazon), Graham Dumpleton (author of mod_wsgi and now works at New Relic), Brian Rosner (steward of Gondor PaaS), Andrew Godwin, co-founder of now defunct ep.io (now works at Lanyrd) and folks from Excite, Opscode and Disqus.
Panel: Creating a better deployment story (PyCon 2012)
At the 2012 PyCon Web Summit, I sat on a panel “Creating a better deployment story” with Ian Bicking, Jim Fulton, Kenneth Reitz and Tarek Ziade moderated by Jacob Kaplan Moss. See some of the questions that were voted on in advance of this panel. The point I tried to make on the panel is that Python developers would be better served if we as a community were to put forth a standard for defining the deployment requirements for a Python application. This standard format could then be consumed by all the PaaS providers and translated into their specific way of doing things.
But the scope of the discussion changed when it diverged into the realm of Python packaging. Packaging is also an important topic, and peripherally related to deployment, but it also happens to be one that people feel very strongly about which meant that we spent most of the panel talking about packaging.
django-deployer is born at DjangoCon sprint (DjangoCon 2012)
The photo is from a dinner at DjangoCon 2012 where I once again tried to get everyone together to discuss the idea of a PaaS-agnostic definition for deployment requirements. Andrew Godwin (formerly ep.io), Kenneth Reitz (now at Heroku), Ken Cochrane and Gabriel Grant (formerly Dotcloud) and Jacob Kaplan Moss. I think Diane Mueller was also at the table (formerly ActiveState and now Redhat OpenShift).
At that same conference, I proposed to sprint on a universal deploy tool for Django. I pitched the idea and managed to attract a few folks to work with me on it: John Debs, Kevin Daum and Gabriel Grant. We only sprinted for a couple days, but got a working prototype pushed to Github that could create all the files needed to deploy your Django app to Dotcloud or Stackato.
Snakes in the Cloud (East Bay and DC Django meetup groups)
It’s now 2013, and the interest in deployment remains a hot topic! On February 12, on opposite coasts, there were two talks happening on the same night, both about Django deployment! Aleksandr Vladimirsky was talking about deploying Django-based Mezzanine to Heroku and S3 and at the Django-District meetup in DC, they were talking about Django & Deployment. By the way, if you’re in the DC area, next week the Django District group is hosting a hands-on workshop on deploying Django to Amazon Web Services.
django-deployer revisited at PyCon sprint (PyCon 2013)
Like many sprint-born projects, django-deployer didn’t see much action for a few months, but at the most recent PyCon, I was giving a talk about deploying Python apps to PaaS providers, so I decided to see if anyone wanted to sprint on django-deployer with me. As it turns out, there were others interested in deployment (no surprise there), so we grabbed a sprint room and started working.
Colin Wu from Taiwan had experience working with Google App Engine (GAE), so he volunteered to add support for GAE. Kevin Grinberg, Kyle Kelley and Wes Winham (author of Neckbeard – best tool name ever!) also provide valuable input. I created a “PaaS matrix” Google Spreadsheet that was essentially a condensed form of my PaaS bakeoff post, to make it easier to extract the code snippets we needed for each provider. (The PaaS matrix spreadsheet has since been expanded into the PaaS Cheatsheet)
django-deployer now provides the following:
- Fabric script that asks questions about your Django app (settings file, requirements, database, etc.)
- The answers to these questions are stored in deploy.yml file
- The script then reads this file as input and outputs config files for a PaaS provider of your choice (currently Dotcloud, Stackato and Google App Engine)
You then use the native clients for those PaaS providers to do the actual deployment. The nice thing is that all the information about your app is captured in the deploy.yml, so if you want to experiment with different PaaS providers, you can simply re-run the Fabric script, and it will create new config files for the other provider. It recognizes the existing deploy.yml, and doesn’t ask you the same questions again.
Note: If you’re going to try various providers, I suggest that you make a new branch for each provider, as sometimes the config files will have the same names, and there’s a risk that you might unintentially overwrite one of those files when you re-run the script.
The code is on Github if you want to try it out on your project. I’d love to hear your comments and suggestions. And if you prefer a different PaaS provider than the ones already supported, it would be awesome if you contributed code to support additional providers. I consider this an early working prototype, and there is still a lot of work to do.