Lessons From a Census Factory

After two months of processing Census data and writing about it here, I’m ready for a nice break. But before I go off to explore other topics, I thought I’d wrap this episode of Census 2010 with a look at how my teammates and I processed the data. My deepest thanks to my colleagues for doing such a great job. And many thanks to the journalists across the U.S. who offered encouragement as we shared our work with the journalism community.

*   *   *   *

On a Thursday afternoon in the first week of February, three of us from our newsroom’s database team gathered at my computer and tried our best to subdue the butterflies swarming in our stomachs. What we were about to do, we hoped, would not only help us cover the year’s biggest demographic story but also help journalists across the country do the same.

That’s because weeks earlier, somewhere in the midst of poring through Census technical manuals and writing a few thousand lines of SAS code, we’d had a bright idea:

Let’s share this.

Really?
Continue…

Census 2010 State Stories: Week 8

The eighth and final (phew!) week of Census 2010 P.L. 94 redistricting data releases brought data nerds back to east coast states — including one of the largest, New York. Here’s my final roundup of interesting stories and data applications made by journalists for this round of the Census:

District of Columbia: With 39,000 fewer black people since 2000, the nation’s capital is on the verge of seeing blacks lose majority status there, The Washington Post wrote. Its story explained:

The demographic change is the result of almost 15 years of gentrification that has transformed large swaths of Washington, especially downtown. As housing prices soared, white professionals priced out of neighborhoods such as Dupont Circle began migrating to predominantly black areas such as Petworth and Brookland.

The Post offered a ward-by-ward graphic explaining the city’s population changes, and its interactive map was updated to include D.C. along with Maryland and Virginia.

Maine: The state, which is 94% white, lost population in its north and eastern counties, The Bangor Daily News reported. On that page, note the BDN’s use of a Census Bureau-provided interactive map — one of many cases where news orgs picked up a government-issued graphic.
Continue…

Census 2010 State Stories: Week 7

This week’s release of nine states’ worth of Census data took us from corner to corner of the U.S. — from Alaska to Florida — with a bunch of upper Midwest states thrown in. Only eight states plus Washington, D.C., are left.

My USA TODAY colleague Paul Overberg and I continued pulling each state’s data for our interactive map and state profile pages, and our shop continued to write at least one story about each state. This week, reporter Dennis Cauchon’s story on North Dakota’s population boom was picked up by the Drudge Report and became our site’s top story for a day and a half. Who’d have thought?

Here’s a rundown of interesting stories and interactives:

Smart story: Rob Chaney of Montana’s The Missoulian wrote about Huson, one of 85 new “places” designated by the Census Bureau in the 2010 count. Shows what you can do if you can think non-numbers about a numbers story. Don’t miss the final quote.
Continue…

Census 2010 State Stories: Week 6

Week 6 in the Census 2010 redistricting data rollout included some of the nation’s most populous states — California, Ohio and Pennsylvania among them — and one of the deepest selections of stories and news apps yet.

Highlights:

Arizona: The state’s 46% increase in Hispanic residents in the last decade was a prime mover in its growth, The Arizona Republic reported. The New York Times’ story says that Arizona’s Hispanic growth was slower than expected, however, and some activists suspect an undercount.
Continue…

Census 2010 State Stories: Week 5

This week’s release of Census 2010 redistricting data for Delaware, Kansas, Nebraska, North Carolina and Wyoming brought the number of states out so far to 26. Next week, biggies California, Arizona, Ohio and Pennsylvania are among seven states due. So, if you’re looking for national stories, you’ll soon have more than enough of a national data set to mine.

On to this week’s highlights. USA TODAY added stories on each state released in Week 5, and we updated our interactive map and data profile pages. A quick take on our stories:

Delaware: Mike Chalmers of The News Journal in Wilmington wrote that the state’s two smaller southern counties grew much faster than its more-populous northern county. Asians, he wrote, were the state’s fastest growing racial group, up 75.6%. (Also see Chalmers’ lengthier analysis at DelawareOnline.)
Continue…

Census 2010 State Stories: Week 4

The week was the busiest so far in the rollout of 2010 Census P.L. 94 data, with the bureau releasing data for eight states. That made for intense times for me and my USA TODAY colleagues — we had to process the files while attending the 2011 Investigative Reporters and Editors computer-assisted reporting conference in Raleigh, N.C. (Thanks to IRE for getting us a quiet room to work.)

For our part, we had stories on Alabama, Colorado, Hawaii, Missouri, Nevada, Oregon, Utah and Washington. We also continued to update data-driven profile pages of each state and an interactive map. And we’re spreading the Census love by sharing the data with IRE members.

Other work I noticed, in no particular order:
Continue…

Census 2010 State Stories: Week 3

Last week’s Census 2010 redistricting data releases included two of the most populous states — Texas and Illinois — along with Oklahoma and South Dakota. Highlights in stories and apps:

The Chicago Tribune’s news apps team launched an interactive map and print graphic that show a dramatic increase in the city’s downtown population in the last 10 years, even while many of the surrounding neighborhoods lost population. As a Tribune story explained:

Hardest hit were the South and West sides, where thousands of African-Americans abandoned neighborhoods beset by crime, foreclosures, bad schools and economic squalor.

Continue…

Census 2010 State Stories: Week 2

This week saw the Census Bureau post 2010 redistricting data from five more states — Arkansas, Indiana, Iowa, Maryland and Vermont — bringing the total so far to nine. As time allows, and because I’ve spent the last two months prepping code for this, I’m chronicling stories and graphics that catch my eye. This week, by state:

Arkansas: My colleague Rick Jervis’ story noted the growth in northwestern Arkansas fueled by employers Wal-Mart and Tysons Foods. This will lead to fairly substantial redistricting:

… Arkansas’ representatives soon will answer to whole new neighborhoods of voters, says Thomas Paradise, a University of Arkansas professor of geosciences. “It’s not hard on the population. It’s going to be hard for the congressmen,” Paradise says. “They’re going to have a radically different constituent.”

– The Arkansas Democrat-Gazette also noted the shift northwest and reports another mayor planning to challenge the count with his city losing 6,000 people. Both stories available to subscribers only.

Indiana: The Indianapolis Star’s Tim Evans (writing for USAT) wrote that the state became more Hispanic and suburban in the last 10 years. Still:

“While Indiana’s racial and ethnic makeup has shifted, the state remains less diverse than the nation,” says Matt Kinghorn, a demographer with the Indiana Business Research Center at Indiana University’s Kelley School of Business. “Compared to the most recent population estimates for the nation, the share of Indiana’s population that is white is well above the U.S. mark of 79.6%.”

– The Star’s graphics team also launched an interactive map that loads data from other states (zoom out to see).

Iowa: The Des Moines Register highlighted the state’s population shift from rural to urban:

Iowa State University economist Liesel Eathington said those population trends reflect a pattern that’s become common throughout the Midwest. One factor is that mechanized agriculture requires increasingly fewer farmers to till ever-larger tracts of land.

(By the way, check out the modal pop-up graphics on that page. Great work, but the one that shows all 99 counties makes me glad I don’t have to memorize their names for a geography bee — I guess the folks who divided up the state’s geography liked things uniform.)

– The Register also launched an interactive map with a bonus: population histories for each county. Really shows the dramatic rise of Dallas County, west of Des Moines.
Continue…

And We’re Off: Early Census 2010 State Stories

Four states received Census 2010 P.L. 94 redistricting data last Thursday, and just like that Census reporting season was off and running. Our newsroom quickly tackled stories on trends in Louisiana, Mississippi, New Jersey and Virginia and posted an interactive state/county map on our Census page that we’ll keep updating.

I thought it would be fun, at least as time allows, to chronicle here some of the Census work from other newsrooms that catches my eye. So, here are a few stories and visualizations from week 1:

Stories

Given the geography-specific nature of the data, most stories focused on states or cities. Several examined post-Katrina New Orleans, using Census data to measure the effect of the hurricane on demographics:

The Washington Post pointed to the political effect of the city losing 29% of its population between 2000 and 2010:

The city will also probably lose a voice in Washington: Louisiana will end up with six congressional seats instead of seven because of the lost population, and state legislators are expected to eliminate one of the city’s three congressional districts.

The Times-Picayune reported that the broader New Orleans’ regional population drop was less, down 11% in the seven-parish metro area. Its story, as did others, pointed out that New Orleans after the hurricane was more white and Hispanic than before:

Black residents comprised 60 percent of city residents last year, compared with 67 percent in 2000, the data show. Meanwhile, the proportion of white residents grew from 28 percent to 33 percent. The city’s proportion of Hispanic residents, who can be black or white, inched upward, from 3 percent in 2000 to 5.2 percent last year.

Elsewhere in Louisiana, The News-Star in Monroe said the mayor is planning to challenge Census data because his city’s population drop below 50,000 could mean a loss of federal funding:

“We are not going to panic,” [Jamie] Mayo said. “We are going to see what the process is, and we will pursue it. We are not pleased about going below 50,000. Our whole objective is to grow our city. To me, the ideal size would be 60,000 to 65,000.”

Also from my Gannett colleagues: In Mississippi, The Clarion-Ledger wrote about accelerated white flight from Jackson. In Virginia, The News-Leader in Staunton covered the aging population in the central Shenandoah Valley. In New Jersey, the state bureau covered how the population shift to the south could pose an issue for Democrats, and The Daily Record in Morristown covered a rise of vacant homes in the state to near 10%.

Finally, stepping back, The New York Times wrote about a continuation of the long-term trend toward an increasingly diverse America. Examining the data in the first four states, it noted a sharp drop in white youths and called it “a shift that demographers say creates a culture gap with far-reaching political and social consequences.”

Visualizations

I know how hard this work is, so hats off to all the developers and data journalists out there working on interactives.

– Our Juan Thomassie did a really nice front end to the Census storehouse that Paul Overberg and I built.

The Washington Post’s interactive map of Virginia lets readers zoom down to the block group level for population growth and race/ethnicity counts.

The New York Times has a detailed map of population changes in New Orleans.

I’m sure that’s just the tip of the iceberg, but the Super Bowl is calling. Add more in the comments. And remember: up to five states coming out this week — Arkansas, Indiana, Iowa, Maryland and Vermont.

Prep for Census 2010′s First Wave

In a few weeks, the first detailed results of the decennial U.S. Census will start pouring from Census headquarters in Suitland, Md., and a Panic Season will commence in unsuspecting newsrooms. What are these numbers? Where can I download them? Didn’t we just get new Census data? Can you tell me whether Census counts X or Y or Z?

On deadline, that’s a lot of potential headache. I know you want to avoid the pain, so take some advice from a guy who survived reporting on Census 2000: prep is everything.

Here are five steps you can take now:

1. Know your Census products: These days, “Census data” means more than it did a decade ago. The advent of the American Community Survey — a survey of about 3 million households each year that replaced the old Census long form — means we get annual estimates in between the full decennial counts. And the ACS comes in three flavors: single-year data plus three- and five-year aggregates, each providing different levels of geographic granularity.

The regular releases of ACS data make Census seem more routine these days, but the data coming out soon are different. These aren’t estimates from a sample — they’re the complete counts taken in spring 2010 via a short questionnaire sent to every household in America.

This first wave of Census 2010 data, coming state-by-state in February and March, are the Redistricting Data (P.L. 94-171) summary files. They’ll contain the basic counts of population by race for every state, county and place in America, all the way down to the smallest geographies, called blocks. As its name implies, these data will be used to redraw the boundaries of legislative, electoral and other districts in states — a process journalists will want to keep tabs on.

Later, in the summer, Summary File 1 will offer more detailed data on age, sex, households, families, and housing units — again from complete counts. Then, in the fall, we’ll see the next release of ACS data. Got all that?
Continue…

On the Beat: Census 2010 Coverage

The decennial Census has kept us busy this spring, especially as the government released the daily tally of participation. Today’s the last day to mail back forms, so it feels like time to take a breath and recap some of the work my excellent colleagues have done the last few weeks:

– Stories on states and counties beginning to top their response rates from 2000, on hurdles to participation and  Census jobs going unfilled.
Maps tracking participation and an interactive where you can search rates by state, county or locality.
– A neat interactive explaining how Congress is reapportioned after every Census.

More to come as Census workers fan out to households that didn’t respond. But, of course, this is nothing compared to the deluge we’ll face next spring when Census turns on the firehose of the actual data.

Tracking Census 2010 Participation

This week, my USA TODAY colleague Paul Overberg and I launched a simple database application to display the Census 2010 mail participation rates for states, counties and 27,000 cities and towns.

Through late April, the Census Bureau is updating the data each weekday. They’ve launched their own interactive map and offer the data in CSV or double-pipe-delimited format (a new one for us). We didn’t want to duplicate the bureau’s map, but we did want to offer something Census isn’t: the ability to quickly find and rank geographies.

Here’s more on how it came together:
Continue…

Notes from Pew’s Census 2010 Workshop

The Pew Research Center’s Census 2010 workshop Jan. 21 featured two panels to help journalists and analysts prep for the decennial count of America and the data dump to follow. Two of my USA TODAY colleagues and I sat in. Paul Overberg, a fellow database editor, led one of the panels.

Pew staff recorded the sessions and is sharing some of the material on its Census site. Here are some of my notes for those who couldn’t make it:

‘Conducting Census 2010′
Presented with the Washington Statistical Society and D.C.-American Association of Public Opinion Research

Robert Groves,  director, U.S. Census Bureau
Constance Citro, director, Committee on National Statistics
Jeffrey Passel, senior demographer, Pew Research Center
Joseph Salvo, director, Population Division, NYC Dept. of City Planning
Scott Keeter, director of survey research, Pew Research Center

Groves led with a broad description of the planning and operational aspects of the count:

Continue…

Adjusting for inflation: A beginner’s guide

When Daniel Craig hit theaters last year in Quantum of Solace, the 22nd film in the James Bond spy series, his ability to dispatch bad guys (and charming good looks, no doubt) helped it earn $168.4 million. That was enough to rank Solace among the top 10 grossing films of 2008.

But how did Solace fare against the rest of the Bond canon, which stretches back to 1963′s Dr. No? The answer depends on whether you adjust for inflation.

We all know that the price of a loaf of bread isn’t what it used to be. The cost of consumer goods tends to rise each year, except during downturns or various calamities. So, taking inflation (or deflation) into account is the only way to  meaningfully compare dollar amounts over time.

There are plenty of apps just for this. The Bureau of Labor Statistics offers one basic calculator, and there’s another at this site. They’re fine for a quick check, but I’d rather do my own calculations. A web app might not have the latest data. And if you’re adjusting more than a couple of amounts, using a spreadsheet will save time. Here’s an exercise from Bond-land:

Continue…

On the 2008 American Community Survey

This year’s release of the American Community Survey felt like the opening act for the big show to come. In about six months, every American household gets its decennial Census form, and the release of that data in late 2010 and spring 2011 will open new realms of analysis and visualization. Not to mention effort. Downloading, massaging and understanding reams of data on deadline needs to be experienced to be appreciated.

So, the 2008 version of the ACS was a good warm up. It saw the Census Bureau add questions on health insurance and the number of times a person had been married, among other tweaks. The release also came in the midst of the country’s economic woes, so journalists everywhere were on the lookout for how the data would reflect the recession.

We are fortunate at USA TODAY to have Paul Overberg on our team, one of the smartest Census analysts in the business. We began planning early and scrambled through a busy weekend to put together four stories, an interactive state map and assorted print graphics. Here’s what we did:

Immigrant population dipped last year, Census says
Driving habits alter during recession, Census reports
Housing is getting even less affordable
Census: 76% marry just once; new count for same-sex couples
State-by-state maps on foriegn born, other indicators