In 2011, a year when consumers unboxed millions of e-readers, fiction dominated even more of USA TODAY’s Best-Selling Books list. Colleague Carol Memmott and I reported today that 78% of the titles in the weekly book lists last year were fiction, up from 67% in 2007. The finding is one of several covered in our annual look at trends off the book list:

“People are interested in escape,” says Carol Fitzgerald of the Book Report Network, websites for book discussions. “In a number of pages, the story will open, evolve and close, and a lot of what’s going on in the world today is not like that. You’ve got this encapsulated escape that you can enjoy.”

We’ve posted the 100 top-selling titles of 2011 in a handy data table that includes the annual lists back to 2007.

Jan 11, 2012 | Journalism
No Comments »

The feeling came a few weeks ago as I drove along a back road near the Potomac River. I was in the lowlands, about to cross from Virginia to Maryland, driving alone during a day in which I’d purposely disconnected from email, Twitter and most things digital.

I think we see things differently on those days.

My car rounded a bend, and through the trees I could see the river. The scene was perfection: bare trees arrayed on a grassy plain, standing watch next to the Potomac. If I’d shot a photo, it would have brushed up against Ansel Adams in intent if not quality. It took my breath, and I gave thanks.

Soon I was on a bridge crossing the river and then into Maryland. But the scene stayed in mind as I drove toward my destination, the road now winding through rustic small towns that seemed to take me even farther from the office.

I’ve thought back on those minutes often as 2011 disappeared into time past. I’ve thought how I need many more of those minutes.

Read more » »

Jan 2, 2012 | Life
4 Comments »

So, you’re the 67-year-old editor of a small-town newspaper who also happens to do the books for a local businessman.

The local businessman’s not just your boss. He’s also the owner/landlord of your newspaper’s office, your residence, your son’s residence and your daughter’s business. You live in one of those in-grown places that dot America, a place where everyone whispers everyone’s business.

One day, you’re arrested. The charge: embezzling $9,000 from this businessman-boss-landlord.

The arrest happens in the middle of the day. Somehow, the local police chief decides to give you a perp walk in handcuffs down a main street of your little town, where everyone knows you and you know everyone. And, somehow, a freelance photographer just happens to be there, takes photos of you perp-walking, and sells them to a rival weekly newspaper, which of course publishes them.

You, the newspaper editor, say it’s all a mistake. Of course she didn’t steal anything … it was an accident!

The town’s in an uproar. Scandal! And on top of it a perp walk right in town for a 67-year-old lady!

Read more » »

Nov 26, 2011 | Journalism
No Comments »

At the start of 2011, I simplified the first WordPress theme I’d built for this site and turned it into something far more minimalistic. I went from two sidebars to one, lost the bulky header and turned from color to black and white. Part of this was a desire for simplicity; part was my reaction to my lack of design sense. Color is not my strong suit, and I shouldn’t be caught trying to pretend.

Since then, I’ve made a few tweaks, but one thing I hadn’t done all year was post the theme — which I call Goshen — for anyone to use. Today I fixed that and pushed the files up to their own repository on Github. You can download the files and hack away. (In your WordPress install, under /wp-content/themes/, create a folder called Goshen and unzip the files there; then you can activate the theme via the dashboard.)

I’ll continue to tweak when I have time. I can’t say enough about how much WordPress theme hacking has taught me about HTML, CSS, templates and web design. If you want to start from scratch, I recommend this excellent tutorial. You’ll discover that WordPress themes have only a few moving parts. Mastering them will let you make your site exactly what you want it to be.

Nov 18, 2011 | WordPress
No Comments »

Getting my flu shot this week reminded me about weekly surveillance data the Centers for Disease Control and Prevention provides on flu prevalence across the nation. I’d been planning to do some Python training for my team at work, so it seemed like a natural to write a quick Python scraper that grabs the main table on the site and turns it into a delimited text file.

So I did, and I’m sharing. You can grab the code for the CDC-flu-scraper on Github.

The code uses the Mechanize and BeautifulSoup modules for web browsing and html parsing, respectively. Much of what I demonstrate here I started learning via Ben Welsh’s fine tutorial on web scraping.

We’re still early in flu season, but if you watch this data each week you’ll see the activity pick up quickly.

Update 10/22/2011: Ben Welsh has lent some contributions to this scraper, adding JSON output and turning it into a function. Benefits of social coding 101 …

Oct 21, 2011 | Programming, Python
No Comments »

An all-wise journalist once told me that “everything is easier in Linux,” and after working with it for a few years I’d have to agree — especially when it comes to software setup for data journalism. But …

Many newsroom types spend the day in Windows without the option of Ubuntu or another Linux OS. I’ve been planning some training around Python soon, so I compiled this quick setup guide as a reference. I hope you find it helpful.

Set up Python

Get started:

1. Visit the official Python download page and grab the Windows installer. Choose the 32-bit or 64-bit version, depending on your version of Windows 7 (right-click the Computer icon on your desktop and select Properties to find out which one you have). Note: Python currently exists in two versions, the older 2.x series and newer 3.x series (for a discussion of the differences, see this). This tutorial focuses on the 2.x series.

2. Run the installer and accept all the default settings, including the “C:\Python27″ directory it creates.

Read more » »

Oct 15, 2011 | News technology, Programming, Python
3 Comments »

If you’ve ever stared into the abyss of a big, uncooperative comma-delimited text file, it won’t take long to appreciate the value and potential of csvkit.

csvkit is a Python-based Swiss Army knife of utilities for dealing with, as its documentation says, “the king of tabular file formats.” It lets you examine, fix, slice, transform and otherwise master text-based data files (and not only the comma-delimited variety, as its name implies, but tab-delimited and fixed-width as well). Christopher Groskopf, lead developer on the Knight News Challenge-winning Panda project and recently a member of the Chicago Tribune’s news apps team, is the primary coder and architect, but the code’s hosted on Github and has a growing list of contributors.

As of version 0.3.0, csvkit comprises 11 utilities. The documentation describes them well, so rather than rehash it, here are highlights of three of the utilities I found interesting during a recent test drive:

Read more » »

Sep 11, 2011 | News technology, Programming, Python, Tools
No Comments »

I was looking at my watch because the meeting was scheduled for an hour, and the hour was nearly over.

We were in a second-floor conference room in the USA TODAY building in McLean, Va. That side of our glass-enclosed HQ faces the intersection of the Dulles Toll Road and the Capital Beltway, and for the last few years we’ve been front-row-center to the construction of new HOT lanes for the Beltway and the work going on for the new Metro Silver Line.

Loud noises are not uncommon.

At 1:50 p.m. I checked the time. I have a bad habit of frequently and obviously looking at my watch, which implies that I am bored or inpatient. I’m not; I just like to know what time it is. I’ve always been a clock-watcher. I’m always on time. So, I looked, mentally noting that I had a free hour until my next meeting at 3.

A moment later, the floor began to vibrate. There was a sound, rumbling, like the bulldozers and cranes that had been outside for months, but somehow different.

“Is that a crane coming toward the building?”

I stood to push back the shade and look out the window. I never got that far. The room began shaking from side to side, and people in the next room started exclaiming.

Earthquake, I thought. I dove under the conference table and lay on my side while the room pulsed.

Part of me was in disbelief. They always said earthquakes don’t happen here.

And then it was over, and someone said, “Let’s get out of here!” And then we were outside, everyone trying to make a call on a cell phone and no one getting through.

Aug 27, 2011 | Life
No Comments »

With the 100-degree heat broiling the East Coast this weekend, I decided to stay inside and make some design and performance tweaks to my site. I added Google +1 buttons to posts and the index page, and I also tweaked some of the settings in my plugins.

Speaking of those, here’s what I’ve been using to make life easier:

Akismet: Gets rid of a ton of comment spam for various Russian “services” so I can spend my time doing other things. You’ll need to sign up for an API key, but otherwise it’s simple and effective.

Contact Form 7: After trying a few contact plugins, I settled on Contact Form 7 and have had great results. It powers my Contacts page, which I prefer to use instead of posting an email address. For spam filtering, I implemented the quiz feature, but the plugin also supports CAPTCHA. I rarely get spam.

Google XML Sitemaps: Generates a sitemap.xml file that Google and other search engines use to index the site. Lets me include or exclude content and control how often to update the file.

Read more » »

Jul 24, 2011 | Programming, WordPress
No Comments »

The USA TODAY Best-Selling Books list has a new look and added interactivity, part of a relaunch of books coverage. It’s been a fun project that has been on my front burner for about three months.

I get to work with all kinds of data at USA TODAY, but the book list has been a constant. When I arrived at USAT in 1997, one of the first projects I took on was to build and analyze an archive of the list to mark its fifth anniversary. Since then, as that archive grew to hold nearly 18 years of data, we’ve used it to anchor stories about authors and trends in publishing. We’re awfully proud of the list, and people in the publishing industry tell us it’s one of the most accurate accounts of Americans’ weekly reading habits.

Last year, we opened the archives up to developers via a Best-Selling Books API. This year, giving the list itself a facelift was the next logical step.

We were fortunate to assemble a crack team of designers, developers and product managers who, in a short time, conceptualized, designed, redesigned, and coded an entirely new collection of book-related pages for our site. What’s new:

Read more » »

Jul 1, 2011 | Analysis, Journalism
No Comments »