Preece, J. and Maloney-Krichmar, D. (2003) “Online Communities.” In J. Jacko and A. Sears, A. (Eds.) Handbook of Human-Computer Interaction, Lawrence Erlbaum Associates Inc. Publishers. Mahwah: NJ. 596-620. (Only sections 5 and 6).

Discussion leader/summarizer: Laurie Buis

Research Question:
The primary research question that this article is trying to address asks "What are the best methods for studying online communities".

(DerekHansen: Our discussion of this reading highlighted the fact that Preece does not cover all of the methods that are relevant to studying online communities. For a more complete list see ResearchMethods)

(LaurieBuis: In class, we discussed the following methods:)
ethnography
data logging
surveys
experiments (field, lab, simulation, mathematical modeling, agent based modeling)
content analysis
social network analysis

Key Points/Claims/Methods
Section 5- Research Techniques:




Section 6 lists several proposed topics for research relevant to online communities.

AycaObekci
In section 3, the writers discuss three design issues key to successful online communities: Sociability, usability and evaluating success.
3.1. Supporting Sociability and Designing Usability
Sociability, in the context of the text, is about defining the community's purpose, taking into account the sociability and usability needs the individual people as well as the whole community, and deciding on the policies of the community early.
Usability requirements are similar to other software such as consistency, efficiency and being easy to learn and remember. Dialog and social support, information display, navigation and access are emphasized as the main usability features for online communities.
3.2.In section 3.2, the relations between usability and sociability are described in Table 3, by answering some user questions in terms of both usability and sociability. I find this table useful for practitioners, and I plan to use some of the suggestions in evaluating the design of the community I selected.
3.3. Determinants of Success: In this section, indicators of good sociability (such as the number of participants in the community, the number of lurkers, the number of messages per participant...) and good usability (such as speed of learning the interface, retention, productivity, error rate and user satisfaction) are noted. Table 4 provides useful examples determinants of success for these two areas.

Critique
Section 5 does a nice job explaining the major methods used to study online communities. I would like to have seen more discussion on the pros and cons of each method as well as practical solutions for the problems associated with each method. I would also have like more discussion on how to conduct interviews in an online community as well as a detailed approach for introducing yourself to a community.


Connections with other readings, ideas, etc.
I think that this was a pretty good companion article to the other assigned readings for this week. The Kraut article did a very nice job of introducing several of the more indepth concerns with conducting research in an online environment such as bias, control, human subjects, concerns with online research, and consent. In addition, the Eysenbach article nicely discussed the ethical concerns. But, the problem with those articles is that they did not effectively explain the practical methods employed in the investigation of online communities. Again, this made a very nice block of reading, though I wonder if other articles would discuss the methods more thoroughly. In the future, I would consider having the 684 class read these sections of this article as well. I think that there is some good, practical information in here that could be effectively employed in the design and evaluation of online communities, and would therefore be of benefit to them.

CaRichardson
I'm not sure why the authors jump from experiment to lab study as described in method number 4. It is possible to design an experiment that has a significant amount of ecological validity. I would think that this would be particularly true of a study of an Internet based intervention. In medicine we make a distinction between efficacy(highly controlled, best case does it work) and effectiveness (less control, more real world testing, delivery methods) research but either type of research can be conducted using an experimental design.

XiaomuZhou
This week’s readings bring the important issue in qualitative research on Internet communities: Ethical concerns. However, after I read them, I am still not confident to find a way to deal with the data that involves privacy and confidentiality. (So I suggest we need to discuss more on class about this issue.) Preece (2003, Online Communities) raises the questions to researchers: how much and how often should inform the community about their study and how much information should be revealed about the data sources. Preece uses two examples, Nancy Baym and David Silver’s researches, to explain how other researchers did in the same situation. However, to me, those examples are just special cases. I would like to know whether there are some specific and normal rules we should follow when we start to do our projects for this class. Most of us will soon be “lurkers” of the eCommunities we have targeted, and some of us might have to participate some sort of activities in order to get more information and the trust of the studied communities. However, if our papers are not aiming to get published, should we have to inform the communities we are lurking them?
(PaulResnick: yes we'll discuss in class. Even if you are not aiming to get published, the same ethical principles still apply. If members have a reasonable expectation of privacy, then you have to inform them and get consent even for lurking.)

Data logging provides huge amount of information to researchers to examine interaction of online communities, and a lot of researchers have done that without disturbing the community and published papers based on the log data analysis. However, it is not clear to me what “promises” I should give to the owner of the websites to use their data and how to use those data? Eysenbach (2005, Ethical issues) states “Participants should therefore always be approached to give their explicit consent to be quoted verbatim,” given that Google has powerful indexing and searching. However, this suggestion has been proved to be very difficult to realize because of the reasons such as no response from participants and changing contact information. Then, what is the feasible solution to it?
(PaulResnick: If it's not a "public" post, don't quote it unless the writer gives you an affirmative response.)

Kraut&Olson (2003, Psychological Research Online) calls for understanding the nature of human subjects risks in online research and provides possible solutions, of which one is to consult IRBs. Given my experience of previous research projects with the IRB in UM, I doubt it would work in the near future. One faculty mentioned that no one on IRB (of UM) can really understand the nature of ethnographic research, which is supposed to be a popular approach for understanding the dynamics of online communities.

YongKim
Most of laboratory experiments are, by and large, conducted in strongly controlled, thus artificial settings over a short period of time. In contrast, Field studies provide richer information on work group behaviors over time, but it is hard to control and manipulate key conditions (McGrath and Arrow, 1996, p. 114). As explained at Section 5.1, field studies including ethnography allow extensive observation leading to rich qualitative data, but lack in systematic measurements leading to quantitative analyses and inferential implications. Furthermore, it is very hard to maintain conditions like demographics of users over extended period of time.

In a response to the limitations of these two types of studies (lab experiments / field studies), experimental simulation studies have become popular to some considerable degree. Judy Olson addressed in a STIET seminar (Sep 2004), while introducing the ShapeFactory game, that an experimental simulation allows her research team to “study larger groups interacting over time with more realistic, but controlled settings.” This methodology is novel in that the traditional laboratory research on work groups using technology have constraints in the number of participants and time, which provide little information on “how work groups develop, function, and change over time” (McGrath and Arrow, 1996, p. 107). Whereas, the Shape Factory experimental simulation makes it possible to control variables such as location, communication tools, group history while studying the group behaviors (in a broad sense, as a part of an online community), particularly group collaboration in this study, on longitudinal time scale. Therefore, it gives us “more inferential power than a field study (but less than a lab experiment); while it gives us more realism than a lab experiment (but less than a field study)” (McGrath and Arrow, 1996, p. 115).

McGrath, J.E., and Arrow, H. (1996). “Introduction: the JEMCO 2 study of time, technology, and groups,” Computer Supported Cooperative Work, 4(2), 107-126

CaRichardson
Response to YongKim above: I have not read or heard of the ShapeFactory before and I am at a bit of a disadvantage not knowing the SI lingo, but it seems to me that there is a continuum between the three types of studies (field / simulation / lab) and that particularly when the Internet is involved, the concept of a laboratory starts to have very blurry boundaries. Maybe we could add the McGrath/Arrow article above to the optional readings list for this week (or some future week).