You will work in groups of 2-3 to design, implement, and evaluate your own crowd-powered system. It should be something within the realm of “human computation” and/or “crowdsourcing”.
See the course schedule.
The course readings and discussions should hopefully give you plenty of open research problems and interesting application ideas. In addition, the following may give you some inspiration:
- project ideas from Crowdsourcing & Human Computation course taught by Prof. Chris Callison-Burch at Penn.
- Follow the Crowd blog
- program from the recent CHI, CSCW, UIST, and HCOMP conferences
Human subjects research
If you are doing something that you might want to publish as “research”, you must obtain approval from the Purdue IRB. This typically takes a few weeks. You can submit your application early, even before you are finished implementing the system. Approval is not needed for a purely educational exercise. See me and check the Purdue IRB web site—especially their Determination of Human Subjects Research Worksheet—for more information about this.
Each student will have a budget of at least $50 to spend on paid crowd labor. You can pool your allocation with your group. If you need more, let me know and I can most likely accomodate.
|2/3||declare groups||Send your group declaration by email with the subject line "project milestone 1 group declarations [ece695cps]".|
|2/15||proposal||Send your group declaration by email with the subject line "project milestone 2 proposal [ece695cps]".|
|3/5||introduction||Create an HTML file with a 600- to 1200-word description of your project plan. If your project is research-oriented, the description should include (a) title, (b) introduction, (c) ≈5-10 references, (d) research question, (e) what you might build as far as you know, (f) how you will test, and (g) how you will recruit participants. For system design projects, it should include (a) title, (b) introduction, (c) ≈5-10 references, (d) description of the target users and user goals this would serve, (e) how you will study the target users, (f) what you might build as far as you know, (g) how you will evaluate its success, and (h) how you will recruit participants. One of the group members should deposit the ZIP file under /d/g/695/$USER on the server and send me email with the SHA1 has (like in warmup #3). Use the subject line “project milestone 3 introduction [ece695cps]”.|
|3/12||v0.1||Create an interface prototype or a detailed mock-up or wireframe and an updated description of your design. The only expectation is that you show evidence that you've started and made some progress beyond the proposal last week. Send your proposal by email with the subject line "project milestone 4 prototype v0.1 [ece695cps]". If appropriate, you can upload the files to the server and email just the path and SHA1 checksum, like before.|
|3/26||email check-in||Send an email status report with the subject line "project milestone 5 prototype v0.1 [ece695cps]".|
|4/5||v0.5||At this point, you should have something working and running on a server. Make an appointment to show me your progress and discuss your path forward. In addition, write up the first half of your final report (int). You should introduce your project and why it is needed. Feel free to write as though it is all done now, and just use the <s> tag to cross out anything that is not currently correct. Include at least 15 references to related work in the manner described below. If appropriate, you can upload the files to the server and email just the path and SHA1 checksum, like before. Use the subject line “project milestone 6 code and data [ece695cps]”.|
|4/12||email check-in||Send an email status report with the subject line "project milestone 7 prototype v0.1 [ece695cps]".|
|4/19||v1.0||Development and evaluation are now complete. Deposit your your code, instruction for running it, evaluation data, and a summary of your accomplishments on the server like before, and send an email with the SHA1 checksum with the subject line “project milestone 8 code and data [ece695cps]”. At this point, the only thing left is to create your project page and video.|
I hope to have a dedicated web server available for all to use in mid- to late-February. That server will provide more flexibility than most other options on campus.
Project page (final product)
For your final report, you will create a project page (≈1500-3000 words), including a video (≈3-5 minutes) demonstrating and explaining your system.
Content. You should include all of the same information that would be found in an academic paper
- Problem you set out to solve
- Existing solutions and other related work
- Your design (including the process that led you to it)
- Your implementation
- Your evaluation method
- Your results
- Acknowledgements (including outside help or code not mentioned elsewhere)
Writing. Write in the style and tone of an extended blog post. Start with a summary that explains at a high level what you did, in a way that flows smoothly into the rest of your article. Although this will include all of the same information that would be in an academic paper, it should be much easier to read. Your top priority should be to clearly understood and pique your readers' interest.
Formatting. Everything should be neatly formatted. Templates are okay if they are clean and low-key (and acknowledged).
References. Include at least 15 references to related work as hyperlinks within the text. For academic articles, use a DOI link (e.g., “http://dx.doi.org/10.1145/368481.368507”) and include the full citation inside the title attribute using a consistent reference format. For non-academic references, try to choose permalinks or main page URLs that are less likely to go away, and include the title and author (if applicable) in the title attribute.
Video. Explain your project and give a demo of your system. A narrated screencast is fine. Aim for about 3-5 minutes.
Visibility. Project pages and videos will be posted to the course web page unless any team members request that they be omitted or anonymized. I hope these project pages will be something you can refer to as you move along in your endeavors, as well as a resource for future students. However, that is entirely optional.
Projects will be graded based on the quality of the design (including the process that led you to it), the novelty and boldness (challenge) of the idea, your success at achieving what you set out to do, the quality of your final product (web page and video demo), and the overall effort.