Hello, I am Yong Kim, a second year HCI student.

yongskim@umich.edu

My Blog∞

---

[Design Assignment]

Elance and Feedback System Overview
This paper will discuss a successful design feature, Feedback system, of Elance community∞. Elance is an online marketplace that matches self-employed freelancers (service providers) and service buyers.

Elance allows service buyers to post descriptions of their projects free of charge then receive proposals from a pool of professional service providers. The service buyers then determine which bidder is best for the job by reviewing feedback from previous buyers who have worked with that service provider, bid amounts, online portfolios and work experience. Once the project starts, they can use a private message board to exchange ideas, archive their email exchange, and store files.

Feedback system, as a reputation system, shows a history of how a service provider or a buyer has been evaluated by others. Buyers and service providers have overall feedback statistics as well as individual feedback reviews for specific projects. Buyers rate and leave comments about service providers, while service providers leave only comments for buyers after completing a project. The feedback rating is based on a 5 likert scale for criteria such as responsiveness, professionalism, subject matter expertise, adherence to schedule, and adherence to cost. Buyers also indicate whether they would work with the service provider again if they had a similar project need.

Elance has substantially grown since its foundation in 1999. According to Alexa∞, an Internet traffic-ranking research site, Elance is ranked 879th in terms of quarterly traffic rank, and the number of visitors has increased by 39% from the last quarter. Also, hundreds of new projects are posted every week. Although there are other factors that contribute to the noticeable success of the community, Feedback system is unarguably the backbone of the community as it promotes trust and cooperation between strangers. Relevant social and economic theories and empirical findings that may explain the success of Feedback system will be discussed in the following sections.

Trust Development on the Net: The Effects of Feedback (Reputation) System
In Elance, the capability of service providers to build trust relationships with the service buyers is critical as trust is essential to create the willingness to take a risk of starting collaboration. In the absence of long-term fact-to-face (FTF) relationships, Feedback system as a reputation system, elicits trust and facilitates cooperation. A number of research studies show that online reputation mechanisms are a proven solution for providing significant stability in otherwise very risky trading environments (Dellarocas, 2003). In general, reputation systems provide information on whom to trust, promote reliable behavior, and discourage those with little skill or dishonesty (Resnick et al., 2000).

However, the success of online reputation systems depends on the continued voluntary contribution of feedback by a sufficient number of community members. A minimum scale is required in order for the mechanisms to effectively work (Bakos and Dellarocas, 2003). Experimental studies show that people are more likely to contribute to public goods (i.e. voluntary online feedback contribution) than economic theorists predicted (Ledyard, 1995). Why do people, in spite of the cost, leave comments for others in the absence of direct reward? What theories could explain the voluntary feedback contribution in Elance?

Social learning theory and social identity theory
Sproull et al. (2003) provide a good insight on why people voluntarily make feedback contribution. They claim that social learning and social identification theories provide a good explanation for how pro-social behavior emerges and is maintained in online communities.

Social learning theory suggests that members, especially new ones, learn behavior that is visible and socially reinforced. Further, visible peer recognition of help and contribution reinforces learning. The entry barrier of a small contribution, that is leaving a comment and rating in Elance, is relatively low in online communities, requiring just a few clicks, and it helps social learning.

Social identity theory helps explain why people substantially and voluntarily contribute to their communities over time. People enhance self-esteem and other utilities by identifying with a group. They then behave in a way that's consistent with the group norms, showing (or emulating) prototypical properties. It is empirically found that experienced users at eBay tend to rate more frequently (Dellarocas, 2003), indicating that more experienced members tend to conform to the norms. In Elance, some experienced service providers are more likely to form bonds with other service buyers (and vice versa) given the number of projects done with the same buyers. Elance new members may also form a social identification with the "core group" of members – the most popular providers on high demand.

[JudeYew: I am not too clear about this aspect of elance. Are the "service providers" able to see each others' ratings? You draw a direct comparison between eBay and elance's reputation/feedback mechanism. I think that the comparison might not be so clear cut. It would seem that in eBay, the members are rating on each other, it's only whether they play the role of the buyer or the seller. In elance, the members ("service providers"?) are rated on by the "service buyers". The difference in the dynamics between eBay and eLance would seem to stem from this distinction. What do you think?]
[PaulResnick: I'm not sure I follow Jude's question. It seems like in both cases, there are two parties to a transaction, and each can rate the other.]

The two theories suggest that Elance members might lower the cost of rating through learning and increase a sense of attribution to the community over time. Although any service providers can list their profiles for free, only subscribing providers can bid on projects. It ensures that buyers are receiving bids from motivated service providers, thus having better experience. This better experience may increase their sense of belonging to the community and make them leave feedbacks more often as a result.

PaulResnick: The social identity theory is a good one. Can you say how the giving of feedback is visible and socially reinforced? Can you check to see whether people are more likely to give feedback as they get more experienced with the system?

Public Goods: Altruism? Reciprocity? Selfishness?
Another approach to explaining voluntary online feedback contribution is public goods. Online feedback is considered to be a public good. Everyone in the community gets the benefit from it (with the assumption of honest reporting), regardless of the amount of their contribution, without competing for the utility. Kollock and Smith (1996) discuss the two classic social dilemmas in public goods: “the tragedy of the commons” and “free-riding” in the study of cooperation in Usenet groups. From this standpoint, Elance feedback system seems to be reasonably working although not perfectly. Elance users leave one feedback per 2 to 2.5 projects on average.

[JudeYew: I am not too clear here. Are you saying that the social dilemmas of "the tragedy of the commons" and "free-riding" are absent from eLance? What would some of the reasons why this is so?]
[PaulResnick: the public good here is the feedback itself. He's saying that there is some "free-riding", but there is also plenty of contribution. Yong: I think you should describe it as a public good, not as a common pool resource.]

There has been a great deal of discussion on the motivations behind the contribution to public goods. Sociologists, political scientists, and social psychologists argue that societies are inherently cooperative due to developed social norms and altruism whereas economists discuss contribution to public goods with the hypothesis of selfish motives (Ledyard, 1995). Dellarocas et al. (2004) empirically and analytically prove that the expectation of reciprocal behavior from partners enhances reputation system contribution from self-interested eBay buyers and sellers. They all seem applicable in Elance situation. In addition to these theories and findings, the log analysis on Elance feedback reveals another possible explanation for the motivation. Elance members leave comments after their projects complete. Compared to eBay members who rarely involve personal interaction in their transactions, Elance members constantly interact with each other once the project begins, using various communication channels, possibly including FTF. Since their feedbacks result from their experiece of working together over time, the comments tend to be more specific, more personal, and more genuine than those at eBay. Therefore, the relationships developed over time while the service buyers and providers worked on projects may strengthen the incentive to leave comments. At this moment, it has not been empirically studied what motives are more salient than others among Elance members, but it is important to have a deeper understanding of the psychological factors that drive the contribution.

Single online identity
In online communities, it is easy to change online identity because people choose pseudonyms when registering. The volatility of online community hinders the effectiveness of reputation systems. Friedman and Resnick (2001) suggest ways to deal with the social cost of cheap pseudonyms.

In Elance, unlike eBay, service providers are selected based not only on bid price and feedback by others, but also on their skills, portfolios, credentials, and prior work experience. Service buyers, therefore, are more likely to reveal themselves on these multi-dimensions in their effort of making their profiles more reliable. Moreover, service providers can register as Select service providers by agreeing to get credentials verified by the online identity verification service agencies partnered with Elance. On the other hand, buyers who post a project are asked to provide a valid credit card for identity verification. Although this membership is optional, more and more buyers require service providers to get Select membership in order to bid on their projects. As a result, it is not only very difficult but also costly to change identities in Elance, and such rigidity seems to increse the effectivness of Feedback system, promote trust between members and make the community safer.

References
Bakos, T. & Dellacocas, C. Cooperation without Enforcement? A Comparative Analysis of Litigation and Online Reputation as Quality Assurance Mechanisms. Proceedings of the 23rd International Conference on Information Systems, Barcelona, Spain, 2003.

Dellarocas, C. The Digitization of Word-of-Mouth: Promise and Challenges of Online Feedback Mechanisms. Management Science, October, 2003.

Dellarocas, C., Fang, M., & Wood, C. Self-Interest, Reciprocity, and Participation in Online Reputation Systems, Working Paper, MIT, 2004.

Friedman, E. & Resnick, P. The Social Cost of Cheap Pseudonyms. Journal of Economics and Management Strategy, 10(2), 2001. 173-199..

Kollock, P., & Smith, M. Managing the Virtual Commons. In Computer-Mediated Communication: Linguistic, Social, and Cross-Cultural Perspectives, edited by Susan Herring. Amsterdam: John Benjamins. 1996. 109-128.

Ledyard, J. Public goods: A survey of experimental research. In J. H. Kagel & A. Roth (Eds.), The handbook of experimental economics. Princeton, NJ: Princeton University Press. 1995.

Resnick, P., Zeckhauser, R., Friedman, E., & Kuwabara, K. Reputation Systems. Communications of the ACM, 43(12), December 2000, 45-48

Resnick, P., Zeckhauser, R., Swanson, J., & Lockwood, K. The Value of Reputation on eBay: A Controlled Experiment. Working paper originally presented at the ESA conference, June 2002.

Sproull, L., Conley, C., & Moon, J. Y. (in press). Pro-social behavior on the net. In Y. Amichai-Hamburger (Ed.), The social net: The social psychology of the Internet. New York: Oxford University Press. 2004.