12 Tangents Later, I Publish a Django Site

Last week, I deployed my first live Django app. Time from start to finish: three years.

Cue the sound of snickers and a thousand eye-rolls. Go ahead. But I confess: From the moment I said, “I want to build something using Django” to the moment I restarted Apache on my WebFaction server and watched the site load for real in my browser, 36 months passed through the hourglass of time.

You see, I got diverted along the way. I’ll tell you why. But first, two things:

1. Learning is wonderful, thrilling, maddening and rewarding. If you’re a journalist and want to see new worlds, let me encourage you to take a journey into code.

2. The site is right here and the code is here. It falls way short in the Awesome Dept., and it will not save journalism. But that’s not why I built it, really.

* * *

The tale began March 2009 in Indianapolis at the Investigative Reporters and Editors Computer-Assisted Reporting conference. That’s the annual data journalism hoedown that draws investigative journalists, app coders and academics for a couple of days of nerdish talk about finding and telling stories with data.

I fit right in, with a standard toolkit of CAR skills: data analysis with Excel and Access, statistics with SPSS, mapping with ArcView. I had coding chops too: I’d built interactive apps using C# and Microsoft’s .NET framework to pull XML from SQL Server.

Still, honesty told me that my skills had stagnated.

It had been a few years since I’d learned something. Even as I cranked out stories and projects, some that won awards, I’d become comfortable in a rut.

The more I thought about it, the more it nagged me. So, when I registered for the conference, I signed up for the “Django Mini Bootcamp.” I’d heard a lot about how journalists were using Django to build interactive apps, and I wanted new skills.

At the conference, we dove in. Three smart journo-coders taught: Derek Willis, Chase Davis and Matt Waite, whose first Django site, Politifact, would win a Pulitzer Prize a month later. We even had a visit from the Django creator himself, Adrian Holovaty.

Most of what they said went by fast. All I knew then about Python, the language Django’s built on, was that it had some strict indentation rules. But I kept up. By the end, we’d wired up a demo site, and I had a rough idea of how the model-view-template pieces fit.

I was encouraged. Derek, Chase and Matt kept saying how quickly we could learn enough Django to build a site. I figured I could get something up and running in no time. Django is, after all, “the web framework for perfectionists with deadlines.”

* * *

I am indeed a perfectionist, but I did not have a deadline. And that explains 36 months. The more I set out to build a Django app, the more I kept discovering other technologies that, though not strictly prerequisites for Django building, became awfully valuable to me.

Thus, I took some tangents:

1. WordPress: I liked the idea of having a site of my own, so I registered a domain. For hosting, several people recommended WebFaction. I signed up and took a look at the WebFaction control panel. Right away, WordPress called to me like a shiny object visible beneath the surface of a pond. In minutes I had a stock WordPress site running, and wow … there are all these plugins available for things like caching … and I can hook up Google Analytics … and … well, I was down the trail. I decided I didn’t like any of the stock WordPress themes, so I coded one myself, diving into PHP. A few months later I threw out that theme and built another one. All that led to a lot of quality time learning …

2. HTML & CSS: I learned about <div> elements and valid CSS and hex colors. The difference between elements with “id” and “class” tags. Meta descriptions and keywords and making things SEO friendly. The CSS box model. I could go on, but …

With all that and writing posts, I chewed through a year. At the next CAR conference, I felt a little guilty. I told people I REALLY wanted to do some Django, but I’d learned enough by now to realize most Django developers didn’t work in Windows. That’s when journalist Matt Wynn recommended I install …

3. Linux: Or, more specifically, the Ubuntu distribution. Suddenly, I was in an alternate operating system universe. New releases every six months? Wow. I had to get to know some basics, such as …

4. Bash and vim: A lot of Linux happens at the command line, so I explored the Bash shell and commands such as sudo and apt-get and repositories and other Linux staples. And then there was vim, a text editor coders rave about even though it’s filled with arcane commands. Soon enough, though, I was ready to build me some Django, so I bought …

5. “Practical Django Projects” by James Bennett: Bennett is the Django project release manager, and his book is one of the few on the topic. I built the polls app from the Django tutorial and then dove into the book. A mere 250 pages, it walks through code for three sites: a simple CMS, a fully featured blog, and a code-sharing site. Night by night, hour by hour, I went through every single page and built all three sites on my local machine. Every. Single. Page. But not straight through, because along the way I also got interested in learning about …

6. PostgreSQL: Django supports the simple database manager SQLite, but as a long-time SQL Server guy I was much more interested in checking out Postgres and its enormous feature set. So I did, learning the quirks of its SQL syntax and how to use pgAdmin for queries. But I spent even more time on …

7. Python: Diving into Django without knowing Python first was a handicap. I didn’t know enough to know when something in Bennett’s book was a Django construct or something native to Python. So, when my soon-to-be colleague Chris Schnaars launched a Google group for journalists to study Python by reading “Learning Python” by Mark Lutz, I jumped at the chance. And the more I understood Python, the more I wanted to unleash it for my journalistic good. So I moved on to …

8. Web scraping: Starting with a tutorial by Ben Welsh, I began to teach myself how to call a URL, parse the response, use the BeautifulSoup library to pull elements out of HTML, and write the results to a CSV. I still had no Django site, but I was now using Python in my daily work. Which led to more Python …

9. Python packages and virtualenvs: To parse an XML document, I learned lxml. To connect to databases, pyodbc. To output to JSON, the json library. And to keep code tidy amid constant upgrades, virtual environments.

By now, the Python code in Bennett’s book made more sense, and when I wrapped up the final exercise, I was ready to make something in Django from scratch. My idea: a sandbox site to serve as a test-bed for trying various aspects of Django. One big piece would organize inspiring words from artists. Work commenced. Still …

Bennett’s book concludes with two chapters about smart development techniques. One is the use of source control to track changes. As a .NET user, I had experience with Visual Source Safe, but for my own projects I decided to learn about …

10. Git and Github: I set up a Github account, uploaded a Python scraper and began to explore version control from the command line. The initial commit of my Django sandbox site went to Github on Jan. 5 of 2012. My code progressed, but I still needed some …

11. JavaScript and jQuery: Django has a set of form classes for building templates that take user input, but I wanted custom validation on the fields. I also wanted a modal window to pop when users selected certain information. Cue up a couple of weeks to learn enough JavaScript and jQuery to do that.

Soon enough, I had a site that I felt would be seaworthy enough to make public. So I went back to where I began …

12. WebFaction: Every web host has its quirks, so I spent time reading WF’s documentation. I wanted to know how to take the site I’d built locally and make it work publicly, and thankfully I learned I wouldn’t have to make many mods. This turned out to be the easiest and least time consuming of all the tangents I took. Those tasks done, I restarted Apache on my server and watched my site load.

Three years after my bootcamp in Indianapolis.

* * *

I realize that some of the items I list aren’t tangents per se but necessary steps. If I’d had a deadline — if my company had been using Django at the time — perhaps this experience would have been compressed. But this was an entirely self-motivated exercise, and aside from the Python scripting I began applying to my daily data journalism, all this learning happened at night and on weekends. My family deserves a lot of thanks for letting dad stare at code for many hours.

My new baby Django site won’t save journalism, win a Pulitzer, or even get much attention. But it will represent a lesson for me and, I hope, for other journalists: take some passion, add some vision and perseverance, and you’ll be surprised what you can learn.

3 Responses to “12 Tangents Later, I Publish a Django Site”

  1. The first time I met you was when I sat next to you in the front row of that Django boot camp. (One of the best career decisions I ever made — not because I became a Django expert, but because you probably had a lot to do with me getting hired at USA TODAY.)

    I built my first Django app in three weeks (it’s now in perpetual maintenance mode). I found out that running a Django app on my Mac was one thing — deploying it on an actual public domain was something else entirely. So I took some of the same tangents, and definitely recommend Ben Welsh’s Palewire Python scraping tutorial and Webfaction.

  2. Anthony says:

    Three weeks! That’s probably when I was first getting diverted by WordPress.

    One thing I left out of my tangents was a Sept. 11-related app I began building with Django for work that was later scrapped. Katharine Jarmul helped me a lot at the time, and she deserves credit for getting me excited about Python.

  3. [...] required to solve a problem in code. That’s why I feel learning to code is more about a journey rather than any particular destination (this is why deadlines are really just harshly worded suggestions and rightly ignored). All in [...]

Leave a Reply