Seminar: Collective Intelligence, Human Computation and Crowdsourcing

The Internet has enabled the design of processes that include large numbers of human actors as parts of an over all computational process. Applications such as Wikipedia, Mechanical Turk, Innocentive, GalaxyZoo, or TopCoder have illustrated the potential in employing the crowd as a computational unit. As a consequence, the exploration of approaches to include large humans into computational processes has been met with increased interest both in research and practice. The goal of this class is to explore the foundations and applications of human computation, crowd computing, and collective intelligence applications. In this class, we will both look at theory by discussing the latest literature and engage in a practical project.

(1) The course materials of the theory (discussion) sessions will include of current and seminal research papers in the field of human computation, collective intelligence and crowdsourcing and are discussed in class.
(2) In the practical sessions teams of students will design, develop and evaluate new applications interweaving both human and machine computation.

The literature for the discussion sections will be made available at the kick-off meeting. Below you find a a video, some journal articles and a list of papers that may serve as an introduction to human computation and helps you to evaluate if you are interested in the course:

Video on Collective Intelligence: Davos 2010 - IdeasLab with MIT - Thomas Malone
CrowdLang Project highlighted in UZH Magazin(Abraham Bernstein and Patrick Minder)
Interview with Abraham Bernstein about Crowdsourcing

Programming the Global Brain(Abraham Bernstein, Mark Klein, Thomas Malone)
The Collective Intelligence Genome(Thomas Malone, Robert Laubacher, Chrysanthos Dellarocas)
Programming the Global Brain(Abraham Bernstein, Mark Klein, Thomas Malone)
Soylent: a word processor with a crowd inside(Michael Bernstein, Greg Little et al.)
Designing Games with a Purpose(Luis von Ahn et al.)

Schedule:

Active participation in discussion sessions and the collaboration on team projects. Each team (3 - 4 persons) will have to submit a final report of 8 pages and give a final presentation.

Assignment 1 (Due: Friday 16.03.2012, 12:00 (Email to Patrick Minder)

Meet up in teams of 3 - 4 persons and brainstorm about ideas for the team project. You have to hand in 4 - 5 ideas (described by 3 - 4 sentences)

Additionally, read the following papers for the next discussion sessions:
Malone, T. W., Laubacher, R., Dellarocas, C. (2010) The Collective Intelligence Genome, Sloan Management Review, Spring 2010, 51, 3, 21-31.
Quinn, A. J., Bederson, B. B. (2011) Human Computation: A Survey and Taxonomy of a Growing Field, CHI 2011, May 7–12, 2011, Vancouver, BC, Canada.
Abraham Bernstein (2000), How can cooperative work tools support dynamic group processes? Bridging the specificity frontier, In Proceedings of the Intern. Conf. on Computer Supported Cooperative Work, 2000.
Abraham Bernstein, Mark Klein, Thomas Malone, Programming the Global Brain, Communications of the ACM, Vol. 55 (5), 2012. (Journal Article)
Wolfers, Justin, and Eric Zitzewitz. Prediction Markets (2004)

Assignment 2 (Due: Sunday 01.04, 23:59 (Email to Patrick Minder)

Define the group project more precisely. Therefore, each group has to submit: a project describtion (1-2 pages) including the idea of the project, a short overview about related work, a development plan for a prototype and an evaluation strategy. Additionally you should provide one slide (PDF) for your 3 minute presentation. Note: You should especially try to answer the following questions:

• Why should we care (which problem do you try to solve)?
• What is you solution?
• How can you evaluate whether you succeeded?

Additionally, read the following papers for the next discussion sessions:
Financial Incentives and the Performance of Crowds. Mason, W., and Watts, D. J. HCOMP 2009
Soylent: a word processor with a crowd inside. Michael S. Bernstein , Greg Little , Robert C. Miller , Björn Hartmann , Mark S. Ackerman , David R. Karger , David Crowell , Katrina Panovich. UIST 2010. Management Review, Spring 2010, 51, 3, 21-31
Human Computation Tasks with Global Constraints Haoqi Zhang, Edith Law, Krzysztof Gajos, Eric Horvitz, Rob Miller, and David Parkes. In CHI '12 (to appear), Austin, 2012.
Labeling Images with a Computer Game, Von Ahn L, Computer, 2006
CrowdLang - Working Paper, Patrick Minder and Abraham Bernstein, 2012