Situated Crowdsourcing

One of the limitations of traditional crowdsourcing environments such as Amazon Mechanical Turk is the lack of a domain-specific workforce. In addition, recruiting workers from a tightly geo-fenced area is difficult.

We have started to explore situated crowdsourcing to alleviate these issues, and to tap into peoples’ free time for something beneficial. Years of studies with our multipurpose displays have revealed that people are more than willing to donate their free time for science and common good, if only given the chance.

Situated crowdsourcing simply refers to the process of breaking a large task to smaller pieces, and then offering the subtasks for the public to do using situated technology installations. We have used public displays and dedicated kiosks for situated crowdsourcing projects.

People have varying motivations to participate. In the Bazaar study, we actually paid people to do tasks and demonstrated that workers in a distributed crowdwork platform react to price changes just like in any other market. We were able to motivate people to work during different hours of day, or to make them move to locations that yielded higher rewards for their tasks. Bazaar was deployed at the University of Oulu campus, and during just a few weeks of deployment it was used to complete over 70 000 tasks. Moreover, the work quality was in fact very high.

Bazaar: the world's first situated crowdsourcing market that follows a market model.
Bazaar: the world’s first situated crowdsourcing market that follows a market model (click to enlarge).

Gamification is also a great way to make people do something useful. It is possible to develop an important service that masquerades as a game. In one project, we constructed word dictionaries relevant to the space where the game was located. Users who played Game of Words first contributes words to the system and then ranked other players’ contributions.

A dictionary related to a particular space can be leveraged in the one and only killer application of today’s public displays: advertising. Better knowledge of the mental models and language that people associate with a space are absolutely essential to facilitating strong targeted advertising instead of generic brand broadcasts. While digital signage on public displays still lags behind the rich targeting options of its online counterpart, there is no doubt that in the future we will see more context-aware ads on public displays.

Finally, situated crowdsourcing can be used to study public opinion about location-dependent issues. For example, in our case we collected civic feedback in downtown Oulu about a very controversial and disruptive renovation project. Situated means of communicating with citizens comes in handy in cases like this, as it is a known fact that people lose interest in public commentary the more distance there is between them and an obstruction. Feedback needs to be collected on-site, and the expensive process of hiring dedicated personnel for this purpose does not always produce truthful feedback.

Related publications

Hosio S, Goncalves J, Lehdonvirta V, Ferreira D & Kostakos V. (2014)
Situated Crowdsourcing Using a Market Model
Proc. The ACM Symposium on User Interface Software and Technology (UIST), 2014, Hawaii, US

Hosio S, Goncalves J, Kostakos V & Riekki J. (2015)
Crowdsourcing Public Opinion using Urban Pervasive Technologies: Lessons from Real-Life Experiments in Oulu
Policy & Internet 7 (2), 2015

Goncalves J, Hosio S, Ferreira D & Kostakos V. (2014)
Game of Words: Tagging Places through Crowdsourcing on Public Displays.
Proc. ACM Conference on Designing Interactive Systems (DIS), 2014, Vancouver, Canada

Goncalves J, Hosio S, Liu Y, Kostakos V. (2016)
Worker Performance in a Situated Crowdsourcing Market.
Interacting with Computers, online first 2015