Excel (and other spreadsheets such as the open source Calc) make sorting data easy. In fact, I often make sorting my first step when “interviewing” data because it quickly reveals high and low values and often highlights some that may seem questionable.

Let’s work through a simple sort in Excel. I’ll be using Excel 2007, but older versions have similar functions. Start by downloading the file “sorting.xls” and saving it to your computer. Open it and follow along:

1. We have a table of Census data from the 2006-2008 American Community Survey. It shows the median age of the population for each of 79 school districts in Virginia plus the state itself.

We want to know which district has the oldest and youngest populations. Let’s sort it!

2. Click once on one cell anywhere in the table. This will help Excel auto-discover your table in the next step.

3. On the Excel ribbon, select the “Data” tab and click “Sort.”

4. Two things happened. One, your entire table was selected (or highlighted). Two, a dialog box popped up to offer sorting options. Check off “My data has headers.” That will prevent your header row from getting sorted with the data, and it will add the three column names under the “Sort by” drop down.

5. Under “Sort by,” select “Median.” Under “Order,” select “Largest to Smallest.”

6. Click “OK.” Excel sorts your table, ranking the districts by median age — from highest to lowest. Your first few rows should look like this:

Now, we can do a quick scan and look for patterns. For example, several of the “oldest” counties are in southern Virginia, far away from the Northern Virginia economic engine. Meanwhile, the district with the lowest age is Harrisonburg City Public Schools — with a median age of a barely-legal 22.8. Could the fact that the city hosts two universities have something to do with that?

Good fodder for reporting, all made possible by a simple Excel sort.

A couple of tips and cautions:

– A good general practice is to work on a copy of your original data. Because things happen.

– Excel does best at sorting when your table has a header row and is not contiguous to any unrelated data, such as footnotes. Insert blank rows and columns between the data you want to sort and any information you want to keep separate.

– I recommend selecting only one cell in your table before selecting the “Sort” button. If you grab more than one, Excel may attempt to sort only those cells rather than the whole table. The 2007 version asks if you want to expand the selection, but older versions sometimes do not. This creates the possibility that only some of your data would get sorted, which is a nightmare. Always make sure your entire table gets selected!

– You can sort by more than one field. In Excel 2007, click “Add level” in the sort dialog.

Questions? Tips of your own? Add them below …

I’m giving a talk this week at the IRE Computer Assisted Reporting conference on how we acquired and analyzed the federal data that helped fuel the story.

Major stories in the series include:

– Schools in the dark about tainted lunches
– Why a recall of tainted beef didn’t include school lunches
– Fast-food standards for meat top those for school lunches
– 26,500 school cafeterias lack required inspections

Our series spurred congressional calls for reforms to USDA policies, and in February the agency announced tighter requirements on companies that supply food to the National School Lunch Program, including stricter testing of meat.

Here was my advice:

Focus on a few: Instead of holding building-wide Excel classes or database journalism seminars, start with just one or two reporters who show a combination of interest and decent technical smarts. That lets you go deep on a couple of beats rather than spread yourself thin. Also, success breeds success. Watching a few reporters land great stories will possibly spur interest from others.

Have the right goals: Goals like “publish one CAR story a week” miss the point. Better objectives are to have data-thinking ever present in the reporter’s mind, have the reporter well-versed in her beat’s data sources, and have the reporter develop basic data skills. From that, stories will flow.

Inventory data: Speaking of data sources, have each reporter you work with find out the sets of data local governments keep. File FOIA requests for table layouts and database schemas. Get the data, then study it. That will spur story ideas.

Crawl first, run later: All the hot talk in data journalism these days is on Web frameworks and visualizations, but there’s plenty of work for the beginner in the land of Excel and Access. Build those skills as a starting point.

Your thoughts? Add a comment below …

Also mentioned: The Washington Post’s investigation of the Metro Red Line crash and The New York Times’ series on toxic water.

From the item, by Mark Katches:

Just to be clear, this is by no means a comprehensive list. It represents only a small, informal survey about stories that some highly respected investigative journalists are buzzing about.

Indeed, every year I am amazed at the quality and depth of investigative reporting that American newsrooms continue to produce even as the industry fights hard times. It’s an honor to be mentioned in that company.

Let’s say six kids go on a field trip, ages 10, 11, 10, 9, 13 and 12. It’s easy to add the ages and divide by six to get the group’s average age:

(10 + 11 + 10 + 9 + 13 + 12) / 6 = 10.8

Because all the ages are close, the average of 10.8 gives us a good picture of the group as a whole. But averages are less helpful when the values are skewed toward one end or if they include outliers.

For example, what if we add a much older chaperone to our field trip? With ages of 10, 11, 10, 9, 13, 12 and 46, the average age of the group rises considerably:

(10 + 11 + 10 + 9 + 13 + 12 + 46) / 7 = 15.9

Now the mean is not an accurate representation. The outlier skews the average, and no journalist should feel comfortable reporting it.

This is where calculating a median is handy. The median is the midpoint in an ordered list of values — the point at which half the values are higher and half lower. If the median household income in East Middletownburg is $50,000, then half the households earn more and half less.

Using our field trip, we order the ages from lowest to highest:

9, 10, 10, 11, 12, 13, 46

The middle value is 11, and that’s the median. Half the values are higher, and half lower. If there had been an even number of values, we’d average the two middle values to find the median. For larger sets of numbers, you can use the MEDIAN function in Microsoft Excel.

Given this group, the median of 11 is a much better representation of the typical age than the average of 15.9. That’s what makes median such a useful statistical measure. Scan financial news, and you’ll see medians reported frequently. Reports on housing prices often use medians because a few sales of McMansions in a zip code that’s otherwise modest can make averages useless. Same for sports player salaries where one or two superstars can skew results.

A good test: calculate the average and the median for a group of values. If they’re close, then the group is probably normally distributed (the familiar bell curve), and the average is useful. If they’re far apart, then the values are not normally distributed and the median is the better representation.

In today’s story, reporters Blake Morrison and Peter Eisler worked with me to examine data on the number of schools in each state that met a federal requirement to have two cafeteria inspections annually. We found that in eight states, more than half of schools reporting failed to meet that standard in 2006-07 and 2007-08 school years.

Meanwhile, the series continues to draw attention on Capitol Hill. This week, Sen. Kirsten Gillibrand, D-N.Y., called on the federal government to increase its standards for meat used in school lunches and to cut contracts with companies that repeatedly did not meet standards.

We also write about “spent hens” — birds that are past their egg-laying prime. Tough and stringy, these old birds typically are turned into pet food or compost. But egg producers struggling to find a market for all of them have had help from the federal government:

From 2001 though the first half of 2009, USA TODAY found, the government spent more than $145 million on spent-hen meat for schools — a total of more than 77 million pounds served in chicken patties and salads. Since 2007, 13.6 million pounds were purchased.

Both stories were heavily informed by analysis of data sets obtained from the USDA under the Freedom of Information Act. They included hundreds of thousands of orders from a federal inventory system and about 150,000 results of microbial tests of beef destined for school lunches.

Prompted in part by our series but also by last week’s recall of beef by a company we identified in Part Two of our series, one lawmaker has called for the government to investigate a supplier to the school lunch program. From another story of ours today:

Rep. Rosa DeLauro, D-Conn., called on the U.S. Department of Agriculture to “undertake a comprehensive examination” of the facility, Beef Packers, to “identify and correct any major problems” before it produces more beef.

To see all the stories in our series thus far, click here.

When the firm, Beef Packers, recalled beef last summer because of a salmonella outbreak in 11 states, the government decided not to recall beef made for school lunches that the company made around the same time:

The recall, announced by the government Aug. 6, covered only ground beef sent to certain retailers. In the days after it was announced, government and company spokesmen said meat sent to schools was not included. Documents obtained by USA TODAY through the Freedom of Information Act reveal a more complicated story — one that raises questions about whether the government took adequate steps to ensure that meat it bought for schoolchildren during the same period was safe.

To get at the story, we filed FOIA requests for several government data sets. They included the results of hundreds of thousands of microbial tests conducted by the USDA as well as a dump from an inventory system the government uses to track orders for the school lunch program.

Update, 12/7/2009: Morrison and Eisler report that Beef Packers issued its second recall this year for beef tainted with salmonella.

One of the handiest resources I’ve found — and recommended just this week to a roomful of colleagues — is Sarah Cohen’s “Numbers in the Newsroom.” It’s a 108-page guide that covers the basics on percent change, rates, graphics, probability and much more. Cohen is a Pulitzer-winning former Washington Post staffer and one-time training director for Investigative Reporters and Editors. She’s now at Duke University, where she is the Knight Professor of the Practice of Journalism and Public Policy.

The book is a few years old, but its lessons are timeless. You can pick it up through IRE’s online store.

Have your own math book recommendations? List them below …

Among the key findings, straight from today’s lead story:

– At least 25 college head football coaches are making $2 million or more this season, slightly more than double the number two years ago.

– The average pay for a head coach in the NCAA’s top-level, 120-school Football Bowl Subdivision is up 28% in that time and up 46% in three years, to $1.36 million.

– Our first look at the salaries of assistant coaches finds many approaching and even exceeding presidents’ compensation and most eclipsing that of full professors.