Yelp Theory

Which people and why?

Which people and why?

I first linked to Yelp in June 2008, according to a search of “yelp” in my Gmail account.*  I apparently jumped on the bandwagon when many did:  2008 was a banner year for Yelp, the first year Yelp received more pageviews than Citysearch.  Since 2008, I, like so many others, have come to rely on Yelp as my primary source for deciding where to eat and drink.  Let me share some Yelp tips I’ve developed over the past few years.

*In case you’re wondering, this is where I linked, suggesting a decent pizza place near Madison Square Garden, where, aside from Koreatown, good food is hard to come by.

To start, never trust the star rating of a Mexican restaurant that serves margaritas.  Too often, the margarita-serving Mexican restaurant gets most of its four or five-star ratings from patrons whose number one concern is the quality, size, and cost of the restaurant’s margarita.  Too often, these patrons couldn’t care less about the food.  Or their tipsy post-margarita state makes their food assessments (of everything but the chips and salsa) unreliable.  Not all margarita-serving Mexican restaurants, of course, suffer from an inflated rating.  Run a search of the reviews:  if, say, only one-out-of-six reviews mentions the revered alcoholic drink, then the risk of inflation is mitigated.

This sad tale leads me to a broader point, one apparently shared by Pete Wells, of Guy Fieri-bashing fameread the individual reviews for common praises and complaints.  The Yelp rating is simple to a fault:  one-to-five stars.  One dimension with little guidance on what exactly should inform your rating.  When all of these individual ratings—based on an array of different criteria—are combined into one aggregate rating, we can’t possibly divine the meaning of the collective rating.  This makes perusing through the ratings particularly important.  Sometimes such perusing will reveal patterns similar to the margarita phenomenon:  the Chinese restaurant praised for its egg rolls or the Spanish restaurant celebrated for its sangria.

And sometimes such scanning reveals other types of patterns.  Yelp users have different tastes and preferences.  Parse reviews for characteristics you care about that you suspect most Yelp users don’t care as much about or value differently from you.   For me, these characteristics are things like the number of outlets in a coffee shop* or the clientele of a particular bar.  Another big characteristic for me is ambience:  If I’m, for example, trying to find the best taco in Denver, I could care less about the dinginess of the restaurant.  But such dinginess may take a half-star off a restaurant’s rating.

*Looking at the Yelp pictures in addition to reviews is particularly useful for this inquiry.

Pay attention to what meal reviewers are discussing.  A restaurant may be great for lunch but not so great for dinner, or vice-versa.  As Wells implies in his post, brunch reviews tend to inflate ratings.  Also pay attention to which foods people are discussing.  A restaurant may be have five-star quality soups, but have only three-star quality sandwiches.  For a soup-lover, the Yelp rating will undervalue the quality of the restaurant.

Let’s move to restaurants that tend to be systematically rated too highly. For one, be especially skeptical of reviews of obscure ethnic restaurants.  By this, I mean restaurants like Bosnian or Sri Lankan that are exceedingly rare in most cities.  Some patrons eat at obscure ethnic restaurants because of their obscurity; these patrons sometimes value such obscurity for its own sake.  And they often don’t have a basis for comparison since this may well be their first restaurant experience of this kind.*

*The inverse supports a broader principle.  You should trust a restaurant’s rating more when that particular type of restaurant is common in the city where it’s located.  I’m much more likely to trust the Yelp rating of a Polish restaurant in Chicago than in Denver, or even New York, for that matter.  

Be skeptical, too, of mom-and-pop restaurant ratings.  It’s much more difficult to write a negative review when the adorable husband and wife who run the restaurant came up to your table and charmed your party.

Take note of the number of ratings.  First, the obvious reason:  a rating means next-to-nothing for a restaurant with only three reviews.  But a restaurant with 724 reviews could also be problematic because of the high number of reviews.  Restaurant quality changes over time.  If a once-good restaurant with 724 reviews has recently taken a turn for the worse—perhaps because of a new chef on board—even a healthy number of recent two or three-star reviews may not alter its high rating.  So, pay particular attention to recent reviews when a restaurant has a ton of reviews.

What, if anything, is the aggregated Yelp rating, by itself, useful for?  It can narrow down the field.  Generally, to have an aggregate rating of three stars or lower, a restaurant has to do something that a lot of people—most likely including you—don’t like. Usually either bad food or bad service.  If I’m running a quick Yelp search, I’ll limit my search to restaurants with ratings of four or more stars.  That will surely knock out some meritorious 3.5 star restaurants, but it’ll surely also knock out all of the very worst restaurants.  After this sifting, look to the content of the individual reviews.

Let me finish with two caveats:

First, Yelp doesn’t include every bar and eatery.  Even in big cities, Yelp misses many food trucks and stands as well as some smaller eateries.  Yelp, for example, doesn’t list* every taqueria in the Pilsen neighborhood of Chicago.  And, in smaller cities and towns, Yelp isn’t always as exhaustive as it is in big cities.  How to find new restaurants in bars in such situation? You can, of course, consult other websites.  Or you can follow some of Tyler Cowen’s advice on choosing restaurants.  Or just use your standard techniques: word-of-mouth, restaurant reviews, etc.

*You can, of course, add a business if it’s unlisted.

Second, you might have different preferences than me.  If you’re looking for good margaritas, the Yelp rating of margarita-serving Mexican restaurants may be perfectly rated for your tastes.  Or the charm from the adorable husband and wife restaurant owners might add an extra star to you.

Bon appétit?


5 responses to this post.

  1. Posted by Alex on December 18, 2012 at 9:22 AM

    Excellent post, Josh! I’d just add a couple of things:
    -Use pictures for EVERYTHING, not just outlets. A book’s cover doesn’t tell you much about the book (it’s not like the author designed it!) but the ambience of a place and the appearances of its wares tell you a lot, at least about whom it’s trying to attract.
    -Yelp varies in accuracy heavily in different cities. It’s almost useless in DC, where some great places languish at three stars while terrible ones get four and a half.
    -The Yelp maps are almost always more accurate than Google’s.
    -The front page lists newly-opened places. Very useful once you’ve tapped out a city’s offerings.


  2. Posted by Chris on December 18, 2012 at 12:38 PM

    In sum: reductionistic quantitative analysis often is crude and misleading, particularly when human preferences are involved

    humanistic analysis ftw


    • Posted by Josh on December 19, 2012 at 12:55 PM

      Just as many posts support: reductionist humanstic analysis is crude and misleading, comprehensive quantitative/empirical analysis ftw.


  3. Posted by nick on December 19, 2012 at 12:51 PM

    “[W]e can’t possibly divine the meaning of the collective rating.”

    I think Levmore wants to beat you over the head with a stick.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: