HBS’s Luca: Yelp Paid Search Ads Really Work

Poets&Quants’ Professor of the Week Michael Luca of Harvard Business School. Photo by Stu Rosner Photography

How many times have you used Yelp to find the best dentists, nail salons, or restaurants in your neighborhood? And how many times have you scrolled past one of those sponsored ads on the top of the page and gone right to the highest rated businesses?

You’re not alone. But plenty of people click on the first listing they see and go directly to the advertiser’s page. They may even patronize the business and spend real money there.

Now, an academic study tries to measure the effectiveness of paid search ads for restaurants on Yelp. It finds that sponsored ads drive traffic to the restaurants’ home pages, increase leads and ratings, and generate a meaningful bump in revenues. But independent, rather than chain restaurants, that get the highest ratings from users tend to get the best results.

SEARCH-AD SPENDING HIT A TOTAL OF $53.5 BILLION LAST YEAR

The study was done by our latest Poets&Quants’ Professor of the Week, Michael Luca of Harvard Business School, along with Weijia (Daisy) Dai of Lehigh University and HBS doctoral candidate Hyunjin Kim. The latest draft of their working paper, “Effectiveness of Paid Search Advertising: Experimental Evidence,” was published last August.

Search-ad spending surged 22.5% in 2018 to a total of $53.5 billion, according to Magna, part of the Interpublic Group of Companies. That represented about one in every four ad dollars spent in the U.S. last year. Google has an estimated 75-80% of that market. Yelp reported total ad revenues of $907.5 million in 2018, and restaurants remain its most-reviewed category.

To test search advertising’s effectiveness, Luca and his co-authors randomly assigned the company’s standard ad package to more than 7,000 restaurants that had not previously advertised on Yelp. They sent out tens of millions of ad exposures for those restaurants over a three-month period and tracked metrics like page views of the restaurants’ Yelp page, requests for directions, phone calls from Yelp’s mobile page, and click-throughs to the restaurants’ own URLs from Yelp.

YELP SEARCH ADS GENERATED A 19% JUMP IN PAGEVIEWS FOR RESTAURANTS

They found Yelp’s search ads generated a 19% increase in page views on the restaurant’s Yelp page, although traffic from other leads rose a smaller 7-14%, “suggesting that consumers drawn to the page by search ads are less likely to make a purchase than consumers visiting the page without seeing the search ads,” Luca and his colleagues write.

The researchers then tested whether paid search ads resulted in more foot traffic to the restaurants. They used reviews as a proxy since it’s unlikely people would review a restaurant they hadn’t visited. And indeed, “consistent with this hypothesis, we find that advertising leads to a 5.1% increase in the number of reviews left for a restaurant,” the researchers write. In what they call a “back-of-the-envelope calculation,” they estimated the ads led to a 6.2% increase in restaurant revenue “and would produce a positive return to advertising, on average, within our sample,” they conclude.

Unfortunately, that increase was not sustained after the three-month ad programs ended. “The effects drop to zero immediately after the treatment ends, suggesting that search advertising temporarily raises the awareness of the advertised businesses,” the authors write. While some of the customers may stick around, that suggests a short-term ad program only attracts customers while it is running. So, restaurants may be better off running ads for long periods of time or not advertising at all.

YELP REVIEWS REALLY MATTER

Finally, they found the impact of paid search ads was greater for independent restaurants and those newer to Yelp, confirming that the ads provided new information for potential customers. And reviews really matter: Ads for restaurants that got ratings of 4.5-5 stars had a click-through rate almost one standard deviation higher than those for restaurants that were rated three stars. Higher-rated restaurants also had a 40% less drop-off in conversion rates from page views to leads than did lower-rated restaurants.

Luca, Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School, has been researching online platforms and marketplaces for some time. He’s now focusing on ways data from those platforms can inform managerial and policy decisions. Previous studies he did have had real-world impacts. An experiment he ran with Ben Edelman and doctoral student Dan Svirsky (who are now at Microsoft and Uber, respectively) got national attention in 2016 when it uncovered widespread discrimination against African-American guests on Airbnb, prompting the company to make changes in its platforms. A 2017 study he co-authored with HBS colleague Deepak Malhotra and doctoral student Christopher Poliquin found waiting periods in 17 states for handgun purchases prevent about 750 gun deaths each year in the U.S.

Luca teaches the elective From Data to Decisions: The Role of Experiments and an immersive UK-based elective field course called Behavioral Insights, in which students work on live projects aimed at using insights from behavioral economics for the social good. He got his BS in mathematics and economics from the University at Albany and earned a PhD in economics from Boston University. He has taught at HBS since 2011.