In case you missed it, Buffalo NY was just ranked #1 in Travel+ Leisure's America's Favorite Places survey.
We love Buffalo. John went to high school there, which is also where he met Mrs. Everydata. The city has come a long way in recent years, thanks to Canalside, the Medical Campus and other developments.
But is Buffalo really #1? It depends how you look at the data.
- The rankings were based on publicly available online survey, which meant that anyone with Internet access could cast a vote. Presumably, this also means that all votes are equal. A vote from the top travel writer in the country would count just as much as a vote from your Uncle Joe.
- Cities were defined as "governed bodies with a population over 100,000" according to Travel+Leisure. So if your favorite city was smaller than that, it wasn't eligible.
- Voters could rank cities based on more than 65 categories, and then "each entry was ranked according to an average score." As you know, an average can hide variation in the data. For example, if everyone ranked Buffalo highly on 60 out of 65 categories, and poorly on the other 5, Buffalo would still likely have a very high average score—a score that effectively hides the low scores in 5 categories.
- Another question to ask: were all categories treated the same in the average, or were some categories weighted more heavily than others?
This all comes back to something we stress in our book: Are you asking the right question to get the answer you want? In this case, are you asking which city is ranked the highest by travel experts? Or are you asking which city is the most popular with people who answer online surveys?