Warm-up #0: self-introduction in HTML, due 1/18 (Sun)
Warm-up #1: post a HIT via AMT web site, due 1/25 (Sun)
Interface design: forms
Instead of a normal response, please post a screenshot of an example from the web, and analyze it in the context of the respective reading.
Warm-up #2: design and implement task UI, due 2/1 (Sun)
Interface design: instructions
Post a screenshot of a task interface—from AMT or any other online “task”—and analyze it how well it follows the article's guidelines.
Project: declare groups, due 2/4 (Wed)
Warm-up #3: implement server backend + add UI tracking, due 2/8 (Sun)
Hierarchical task decomposition
Warm-up #4: peer testing/feedback + improve UI, due 2/15 (Sun)
Project: proposal, due 2/18 (Wed)
Warm-up #5: server push, due 3/1 (Sun)
Project: v0.1, due 3/4 (Wed)
Modeling the workforce
Warm-up #6: coding with AMT, due 3/15 (Sun)
Project: email check-in, due 3/15 (Sun)
Project: v0.5, due 3/25 (Wed)
Ethics and fair labor
Project: email check-in, due 4/12 (Sun)
Project: v1.0, due 4/22 (Wed)
Project: web report, due 4/29 (Wed)
How to read papers for this course
Many people find it easier to read papers if they have a purpose in mind. As you read each paper, you might find it helpful to focus on a few questions:
- What was the contribution type? (examples)
- What do the authors claim as their key contributions?
- What strategies, methods, and technologies were used?
- What generalizable knowledge does the work contribute? What research questions does it address? How will this benefit other researchers?
- Do you find the conclusions convincing? Are the results well-supported by data obtained with sound methods?
- What aspects of the work you find strongest? … and weakest?
- What would be a natural next step for the work?
Note: These questions are included only to help guide your reading. They won't apply to all of the readings. Your responses may or may not include your answers to any of these questions.
How to write your response
A good response will clearly express a well-founded opinion about the work. The requires understanding the paper and thinking about what you liked/disliked about it. Your response will show that you read and understood the paper, without directly summarizing it. This might take some practice.
As a starting point, look at the discussion think through your answers to questions #5, #6, and #7 from above. Next, look at the discussion so far (if any). Do you agree? … or disagree? Think of 2-3 insights, including 1-2 points about the paper as a whole, and 1-2 points about specific details (i.e., methods, analysis, etc.). Try to avoid duplicating existing comments.
Reading responses must be posted to the corresponding discussion thread by 10:00 AM the day of the class it is listed with. There should already be a post for each assigned reading. Add a "Follow-up comment" to the post. There is no hard requirement on length, but 2-3 paragraphs with 100-200 words (total) is probably a good target. 2 points will be reserved for especially insightful comments. I expect that most will receive 1 point.
Find a company related to human computation or crowd labor. Give a 5-10 minute presentation in class about it. Send me the company's name, URL(s), and a short (2-3 sentences) summary by email by the night before your presentation. I will post your summary to the course web site.
Companies focused on crowdsourcing and/or human computation:
- Mechanical Turk - microtask
- CrowdFlower - microtask
- Appen - microtask
- ClickWorker - microtask
- CloudFactory - microtask
- CrowdSource - microtask
- LeadGenius - microtask, lead generation
- Lionbridge - microtask
- MicroTask - microtask
- Samasource - microtask, workforce extension
- oDesk - freelancing
- Elance - freelancing
- Freelancer - freelancing
- Scripted - freelancing, writing
- LiveOps - customer service, call center
- Needle - customer service
- Anydoor dba Conyak - translation
- Babelverse - translation of live conversation
- Gengo - translation
- Lingohub - translation of software, localization
- Lingotek - translation
- SpeakLike - translation of social media and web sites
- Transfluent - translation of social media and web site
- Transfluent - translation of social media, games, etc
- VerbalizeIt - translation of live conversation
- Viki - fan translation of tv shows, movies, etc
- InnoCentive - ideation with bounties
- Be-Novative - ideation
- Cambrian House - ideation
- Chaordix - ideation
- eYeka - ideation
- IdeaScale - ideation
- Kluster - ideation
- Polisofia - ideation, Spanish
- Springleap - advertising
- Tongal - advertising?
- Zoopa - advertising
- Adtriboo - advertising, graphic design, video production
- 99designs - graphic design
- DesignCrowd - graphic design
- Minted - graphic design
- Crowdzu - graphic design
- Qukku - video production
- VoiceBunny - voice recording
- FanFootage - live video
- Snapwire - photography
- TapShield - public safety
- Zeef - information retrieval
- ImageBrief - photography
- Kanga - local delivery
- Mobee - retail analytics
- Roamler - retail analytics
- Locu - marketing, restaurant data
- Crowdmark - grading for schools
- RecruitLoop - recruiting
- RecruitiFi - recruiting
- Crowdcurity - security audits, software testing
- Darjeelin - travel planning
Major products driven by crowdsourcing and/or human computation:
- Facebook and Twitter - content moderation
- Google Translate - driven by Translate Communities
- Google Maps - via MapMaker and Waze (acquired for $1.3 billion)
- Google Books - via reCAPTCHA
(university project acquired by Google)
- Amazon - Mechanical Turk
- Microsoft Bing - via Appen
- Microsoft Translator - via Lionbridge, Clickworker, and Appen Butler Hill
- Apple Maps - via Locationary and Hopstop
- US Postal Service - reading handwritten addresses
Patents held by big companies: