Dellarocas, C., Sanctioning Reputation Mechanisms in Online Trading Environments with Moral Hazard (July 2004). MIT Sloan Working Paper No. 4297-03.

Discussion leader/summarizer: CaRichardson

Research Question

How can reputation systems be designed to maximize auction market efficiency?

Key Points/Claims/Methods

This is an economic analysis of the effect of various reputation system designs on the efficiency of an auction market. Results presented in this paper include a number of interesting suggestions for optimizing efficiency of auction markets by using appropriately designed reputation systems. There is also one somewhat counterintuitive result that will be discussed further below.

In the introduction section, Dellarocus makes a distinction between two types of reputation systems, those that function by a signaling mechanism and those that function by a sanctioning mechanism. Reputation systems that function by a signaling mechanism generally rate products that have a fixed, static value or quality. Amazon.com’s rating system is an example of a signaling system. The primary purpose of a signaling mechanism is to give future buyers information about the quality of a product. In contrast, sanctioning reputation systems function by imposing a penalty on sellers who cheat. The assumption is that all sellers are equally able to provide high quality service (which may mean selling a low quality product, but with full disclosure and therefore at an appropriately low price) and that sellers are rational actors who will cheat if it is more profitable than being honest. The fundamental property of the transaction that distinguishes signaling systems from sanctioning systems, is that in sanctioning systems both the bid price and the quality of the product or service will change in response to feedback and penalties while in a signaling system only the bid price will change while the quality is fixed and cannot respond to penalties. The analysis in this paper focuses only on sanctioning reputation systems with only an occasional mention of signaling reputation systems mostly to point out where they differ from sanctioning systems.

There are two main findings in the paper. The first seems somewhat obvious: The probability that a honest seller might receive an unfair rating from a buyer limits the efficiency of the market.

Here is the counter-intuitive result: Compared to a sanctioning reputation system that presents all feedback ever given to a particular seller, reputation systems that truncate feedback reports (i.e. only deliver recent feedback results) to potential buyers, do not result in less efficient markets. Dellarocas further points out that this result is not true for signaling reputation systems. This result seems counter-intuitive because I would guess that more information (ie a complete, untruncated report) would be better at preventing any episode of cheating. The importance of more proximal (recent) feedback in maximizing market efficiency is thus a function of the feedback’s ability to improve the level of service (or effort) provided by the seller in future transactions. Shorter periods of feedback available to potential buyers would actually increase the reward to sellers who previously provided poor quality service but subsequently decide to clean up their act. The shorter the period of feedback available, the faster their reputations will roll over into a clean slate.

PaulResnick: Here's how to make this result seem more intuitive. Suppose I'm a baseball player and it's September, most of the way through the season. If you tell me that my salary for the next month depends on how well I hit that month, I have a stronger incentive to play well in September than if you tell me my salary for September depends on my batting average for the whole year. More generally, the incentives for doing well in my next action are higher if *only* that action will be visible than if it will be averaged with the rest of my history. It's true that my bad behavior will be more quickly forgotten when there's a short history window, but it will also be more severely punished while the window is open. For signaling, on the other hand, the longer history window gives a more accurate indication of long-term average quality.

Critique

I think the model and the associated results in this paper provide an interesting way of framing the problem. However, as with all economic analyses, the questionable assumptions in this paper are rampant. For example, the assumption that buyers have no incentive to report falsely negative evaluations is questionable. Dellarocas further assumes that that a buyer only buys one item from a particular seller and never returns to purchase a second item from the same seller. These assumptions are related because the prospect of future transactions with a seller could influence a buyer to give worse feedback in an attempt to lower the cost of transacting with that seller in the future. Using the eBay example, there are likely to be small markets where there are relatively few sellers and buyers would therefore be likely to make multiple purchases from the same seller. Thus, providing false negative feedback on a transaction might lower the price of a future transaction by scaring away other potential buyers. Another example of why this assumption might fail is that sellers are not only sellers. Sellers can buy. A seller could buy an inexpensive item from a competing seller just to have the opportunity to leave negative feedback, thus driving other prospective buyers from that seller and increasing the likelihood that the prospective buyer would buy from him.

Another thing that seems not quite correct is the strong distinction between signaling and sanctioning systems. It seems to me that all sanctioning systems depend on the signaling mechanism to impose a penalty on the cheating seller. Thus it seems like most reputation systems have elements of both signaling and sanctioning. (PaulResnick: agreed.)

Finally, the statement on page 6 that “In professional services (medical consultations,…) there are well defined standards of high quality service an the uncertainty is focused on whether the provider will adhere to those standards or try to cut corners.” is a gross oversimplification of health care quality. In fact there is no agreed upon standard quality standard for most medical practices and subtle factors like patient preference, co-morbidities etc make most medical encounters much more of an art than a science.

Connections with other readings, ideas, etc.

I think Slashdot would be an example of a sanctioning reputation system. (PaulResnick: mostly, though here, too, there are underlying quality differences; some people just can't write posts that would be interesting to other Slashdot readers.)

DerekHansen: it is interesting to hear people justify Slashdot's reputation system by appeals to both of these distinct ideas. Some people say that it helps to overcome information overload (i.e., as a signal of decent content), while other people say it is to prevent deviant behavior (i.e., it acts as a sanctioning mechanism). I agree with Caroline and Paul's comments that suggest that in practice most reputation systems have aspects of both. However, I do think it can be useful to make the distinction between the two.

AycaObekci: Art communities such as photoSIG, artSIG, flickr (?) provide an interesting form of signaling reputation. As the art object is directly associated with/ produced by the member, the rating of the member is both an indication of the quality of the artwork and an indication of the skill of its creator. At first glance, I though the reputation of the member in this case would be a sanctioning reputation as well a signaling one, because the rating doesn't only comment on the product but also the person associated with the product. Soon I disagreed with myself that a sanctioning system should assume that each player has the same ability to get a positive rating initially (such as the ability to ship the product on time). Art communities reputation systems couldn't be "sanctioning", because it can't be assumed that each player begins with the same advantages. Some members are more talented/skillful than the others. (PaulResnick: agreed.)