Designing a text-based adventure game to train an AI in reading handwriting
In collaboration with STABILO International GmbH, we developed a text-based adventure for the STABILO DigiPen. STABILO Dungeon is a gamified crowdsourcing approach that aims to engage people in writing words with the IoT device Digipen, a classic ballpoint pen equipped with sensors to detect movement and pressure of a person’s handwriting. The aim of this project is to collect handwriting data for further developing the Digipen’s machine learning algorithms.
The core game mechanism of STABILO Dungeon is primarily driven by a narrative: At predefined points in the game, the player controlling the protagonist can decide the story’s progression by writing down an actionable activity on standard paper. The possible activities are a set of pre-defined actions in the game, which makes it easy for the DigiPen to recognise the handwritten word and thus to further train the handwriting recognition.
While the benefits and potential of gamification have been consistently demonstrated in various research, it is crucial to acknowledge that gamification approaches are not always effective and vary in their success, possibly because standard gamification concepts such as one-size-fits-all (OSFA) approaches commonly lack the consideration of the diversity of users. However, with adaptive gamification, this issue can be addressed by adapting the system to the individual user’s preferences.
Between January and May 2022, the experiment was presented on two consecutive weekends at JOSEPHS, an open-innovation laboratory in the city-centre of Nuremberg, as well as at the “Long Night of the Sciences” in Nuremberg. In total, the study resulted in a collection of 8,563 crowdsourced handwritten words. 135 participants completed the experiment by playing the game as well as filling out two questionnaires which measured the players’ enjoyment and intention for future play, amongst others.
The public-service radio and television broadcaster Bayerischer Rundfunk released a short video in German showcasing the experiment as well as the DigiPen as part of the Frankenschau at 28.02.2022.
For more details, see: https://www.br.de/mediathek/video/erste-testlaeufe-intelligenter-stift-digitalisiert-handschrift-av:621d14290b569700085cde93
We thank Jens Barth, Tim Hamann and Peter Kaempf of STABILO International GmbH for the exciting collaboration.
The study was presented as a conference paper at the Hawaii International Conference on System Sciences 2023 in January 2023. Please see the paper for full details: