Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing - Human-Computer Interaction – INTERACT 2021
Conference Papers Year : 2021

Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing

Md. Rabiul Islam
  • Function : Author
  • PersonId : 1280655
Shun Nawa
  • Function : Author
  • PersonId : 1280656
Andrew Vargo
  • Function : Author
  • PersonId : 1280657
Motoi Iwata
  • Function : Author
  • PersonId : 1280658
Koichi Kise
  • Function : Author
  • PersonId : 1280660

Abstract

A significant challenge for creating efficient and fair crowdsourcing platforms is in rapid assessment of the quality of crowdwork. If a crowdworker lacks the skill, motivation, or understanding to provide adequate quality task completion, this reduces the efficacy of a platform. While this would seem like only a problem for task providers, the reality is that the burden of this problem is increasingly leveraged on crowdworkers. For example, task providers may not pay crowdworkers for their work after the evaluation of the task results has been completed. In this paper, we propose methods for quickly evaluating the quality of crowdwork using eye gaze information by estimating the correct answer rate. We find that the method with features generated by self-supervised learning (SSL) provides the most efficient result with a mean absolute error of 0.09. The results exhibit the potential of using eye gaze information to facilitate adaptive personalized crowdsourcing platforms.
Fichier principal
Vignette du fichier
520516_1_En_8_Chapter.pdf (1.79 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04196869 , version 1 (05-09-2023)

Licence

Identifiers

Cite

Md. Rabiul Islam, Shun Nawa, Andrew Vargo, Motoi Iwata, Masaki Matsubara, et al.. Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.104-113, ⟨10.1007/978-3-030-85616-8_8⟩. ⟨hal-04196869⟩
19 View
10 Download

Altmetric

Share

More