Crowd-Powered Systems

Spring 2015 :: ECE 69500 :: Purdue University

This is an archived site from Spring 2015 ()


Week 1: 1/13
Foundations, terminology 20150115-overview.pptx
Warm-up #0: self-introduction in HTML, due 1/18 (Sun)
Week 2: 1/20
Crowd-powered systems
Warm-up #1: post a HIT via AMT web site, due 1/25 (Sun)
Week 3: 1/27
Interface design: forms
Instead of a normal response, please post a screenshot of an example from the web, and analyze it in the context of the respective reading.
Warm-up #2: design and implement task UI, due 2/1 (Sun)
Interface design: instructions
Post a screenshot of a task interface—from AMT or any other online “task”—and analyze it how well it follows the article's guidelines.
Week 4: 2/3
Open problems
Project: declare groups, due 2/4 (Wed)
Warm-up #3: implement server backend + add UI tracking, due 2/8 (Sun)
Hierarchical task decomposition
Week 5: 2/10
Warm-up #4: peer testing/feedback + improve UI, due 2/15 (Sun)
Week 6: 2/17
Project: proposal, due 2/18 (Wed)
Week 7: 2/24
Warm-up #5: server push, due 3/1 (Sun)
Security 20150303-security.pptx
Week 8: 3/3
Project: v0.1, due 3/4 (Wed)
Week 9: 3/10
Modeling the workforce
Warm-up #6: coding with AMT, due 3/15 (Sun)
Project: email check-in, due 3/15 (Sun)
Week 10: 3/24
Project: v0.5, due 3/25 (Wed)
Week 11: 3/31
Ethics and fair labor
Week 12: 4/7
Beyond AMT
Project: email check-in, due 4/12 (Sun)
Beyond AMT
Week 13: 4/14
Games 20150416-games.pptx
Week 14: 4/21
Machine Learning 20150421-machine_learning.pptx
Project: v1.0, due 4/22 (Wed)
Active Learning 20150423-active_learning.pptx
Week 15: 4/28
Human-computer interaction 20150428-hci.pptx
Project: web report, due 4/29 (Wed)
Assignments and readings may be changed without notice up to 1 week before the due date.


How to read papers for this course

Many people find it easier to read papers if they have a purpose in mind.  As you read each paper, you might find it helpful to focus on a few questions:

  1. What was the contribution type? (examples)
  2. What do the authors claim as their key contributions?
  3. What strategies, methods, and technologies were used?
  4. What generalizable knowledge does the work contribute?  What research questions does it address?  How will this benefit other researchers?
  5. Do you find the conclusions convincing?  Are the results well-supported by data obtained with sound methods?
  6. What aspects of the work you find strongest? … and weakest?
  7. What would be a natural next step for the work?

Note: These questions are included only to help guide your reading. They won't apply to all of the readings.  Your responses may or may not include your answers to any of these questions.

How to write your response

A good response will clearly express a well-founded opinion about the work.  The requires understanding the paper and thinking about what you liked/disliked about it.  Your response will show that you read and understood the paper, without directly summarizing it.  This might take some practice.

As a starting point, look at the discussion think through your answers to questions #5, #6, and #7 from above.  Next, look at the discussion so far (if any).  Do you agree? … or disagree?  Think of 2-3 insights, including 1-2 points about the paper as a whole, and 1-2 points about specific details (i.e., methods, analysis, etc.).  Try to avoid duplicating existing comments.

Reading responses must be posted to the corresponding discussion thread by 10:00 AM the day of the class it is listed with.  There should already be a post for each assigned reading.  Add a "Follow-up comment" to the post.  There is no hard requirement on length, but 2-3 paragraphs with 100-200 words (total) is probably a good target.  2 points will be reserved for especially insightful comments.  I expect that most will receive 1 point.

Company profiles

Find a company related to human computation or crowd labor. Give a 5-10 minute presentation in class about it. Send me the company's name, URL(s), and a short (2-3 sentences) summary by email by the night before your presentation. I will post your summary to the course web site.

Sign up here

Companies focused on crowdsourcing and/or human computation:

For more, search CrunchBase for terms such as “crowdsourcing”, “freelancing”, “task management”, “collaboration”, and “outsourcing”. Then read the description, look at the company's web site, or search around to determine if their operations actually the bill.

Major products driven by crowdsourcing and/or human computation:

* Big companies tend to keep internal processes and architectures private, so this list is undoubtedly incomplete.

Patents held by big companies: