Ratings help consumers make better choices by learning from the experiences of other consumers. As such, the informativeness of ratings is an important consideration for platform design. However, over time, ratings may inflate and become less informative. Imagine a platform in which a majority of listings have identical ratings. Such a rating system will be ineffective in helping users choose between listings. In this project, we examine the consequences of rating inflation on user purchases, trial, and the concentration of sales in the context of a food delivery platform. We conduct a quasi-experiment that induced rating inflation in a geographical neighborhood and we compare the trends of purchases, trials, and sales concentration in that neighborhood with other control neighborhoods. We find that while rating inflation increases purchases, it decreases trial among users. It also leads to a concentration of sales towards popular restaurants. These findings have implications for platforms that want to increase customer lifetime value by encouraging trial and limiting excess market concentration on their platforms.
Description | N |
---|---|
Number of users | 198044 |
Number of restaurants | 2244 |
Number of transactions | 1510739 |
Number of full pre-experiment weeks | 9 |
Number of full post-experiment weeks | 4 |
Variable | Description | Mean | St.Dev. | Min | Max |
---|---|---|---|---|---|
n_purchase | Purchases by user | 0.64 | 1.23 | 0.0 | 29.0 |
n_trial | New restaurants tried by user | 0.92 | 1.00 | 0.0 | 14.0 |
sales | Number of transactions per restaurant | 67.28 | 99.87 | 0.0 | 1777.0 |
pre_rating | Average restaurant rating pre-experiment | 3.53 | 0.28 | 2.1 | 4.4 |
Figure 1: Illustrative diagram of the delivery platform’s mobile app. The restaurant rating is displayed in the bottom center of each listing. To the left of the rating is the price range of the restaurant, and to its right is the estimated time to delviery based on the user’s location.
Figure 2: Timeline for rating changes. Original inflated ratings were changed to deflated ratings 14 months before the observation period. First week’s data is dropped for being incomplete. Ratings were inflated for the treated region in week 11 of the observation period. During the experiment, inflated ratings were displayed for the Treated neighborhood while deflated ratings continued to be displayed for the Control neighborhoods.
Figure 3: Rating distribution before and after the experiment for Treated and Control groups. Treated group experiences rating inflation while the Control group does not
Figure 4: User Activity Levels. Top row shows the proportion of users who make no purchase, 1 purchase, 2 purchases, or 3 or more purchases each week. Bottom row shows the proportion of users who have no trial, 1 trial, or 2 or more trials each week. The experiment occurs in week 11.
Figure 7: Average Weekly Purchases by Group
Figure 8: Average Weekly Trial by Group