Kai Kunze

In this work, we propose Frisson Waves, an exploratory real-time system to detect, trigger and share frisson in a wave-like pattern over audience members during music performances. The system consists of a physiological sensing wristband for detecting frisson and a thermo-haptic neckband for inducing frisson.

Frisson is a feeling as well as an experience of physical reactions such as shivers, tingling skin, and goosebumps. In this work, we propose Frisson Waves, an exploratory real-time system to detect, trigger and share frisson in a wave-like pattern over audience members during music performances. The system consists of a physiological sensing wristband for detecting frisson and a thermo-haptic neckband for inducing frisson. This research explores the framework for accessing, triggering and sharing frisson. This research aims to enhance live performances experience of using entrainment through facilitating interpersonal transmission of embodied sensation, frisson. We present background, concept design and system development. https://dl.acm.org/doi/pdf/10.1145/3476122.3484847

With recent development in capture technology, preserving one's’ daily experiences and one's’ knowledge becomes richer and more comprehensive. Furthermore, new recording technologies beyond simple audio/video recordings become available: 360° videos, tactile recorders and even odor recorders are becoming available. . The new recording technology and the massive amounts of data require new means for selecting, displaying and sharing experiences.

Sharing experiences and knowledge have always been essential for human development. They enable skill transfers and empathy. Over history, mankind developed from oral traditions to cultures of writing. With the ongoing digital revolution, the hurdles to share knowledge and experiences vanish. Already today it is, for example, technically feasible to take and store 24/7 video recordings of one's’ life. While this example creates massive collections of data, it makes it even more challenging to share experiences and knowledge with others in meaningful ways. A recurring theme in science fiction literature is the download of the abilities of another human to one's mind. Although current cognitive science and neuroscience strongly suggest that this is impossible, as our minds are embodied; we believe that skill transfer and effective learning will accelerate tremendously given recent technological trends; just to name a few of the enabling technologies, human augmentation using virtual/augmented reality, new sensing modalities (e.g. affective computing) and actuation (e.g. haptics), advances in immersive storytelling (increasing empathy, immersion, communication) etc. The talk starts with sensing and actuation technology, giving an overview about them and discussing how they can be used. I’m discussing several novel upcoming sensing modalities for VR and AR, first of all eye movement analysis for interaction and activity recognition, introducing the pupil eye tracker (open source eye tracker from pupil labs), affective wear (one of our research to track facial expressions on affordable smart glasses) to J!NS MEME (EOG glasses that can detect how much you are reading and how attentive you are). In the next part of the talk I go into details about actuation.Here I especially discuss haptics. From the TECHTILE Toolkit (a rapid prototyping haptic toolkit from two of my colleagues Kouta Minamizawa and Masashi Nakatani) to the REZ Infinite Haptic Suit. In the end, I give an outlook on projects that push the limits for experience sharing and skill transfer: the Swiss Cybathlon and the Japanese Super Human Sports Society. I’m a researcher in the wearable computing, AR and VR field organizing a Dagstuhl Seminar on a similar topic, I’m also a founding member of the Japanese Super Human Sports Society.

The talk gives an overview about the emerging field of smart glasses and how they can be used to augment our mind (e.g. how to improve our brain with technology). The talk will focus mostly on how to quantify cognitive tasks in real world environments. I also present a first application scenarios on how to use smart eyewear (e.g. google glass or JINS MEME) for short term memory augmentation and cognitive activity recognition.

Considering the last centuries, major scientific breakthroughs aimed at overcoming our pyhsical limitations (faster transportation, higher buildings, longer, more comfortable lifes). Yet, I believe the coming big scientific breakthroughs will focus on overcoming our cognitive limitations. Smart glasses can play a vital role in 1. understanding our cognitive actions and limitations by quantifying them 2. helping us design interventions to improve our mind. The talk will focus mostly on the first point, what kind of cognitve tasks can we track already with the smart glasses that are available in the market and what will happen in the near future. I will discuss application examples for Google Glass and J!NS MEME. J!NS MEME is the first consumer level device measuring eye movements using electrodes also called Electrooculography (EOG). The MEME glasses not a general computing platform. They can only stream sensor data to a computer (e.g. smart phone, laptop, desktop) using Bluetooth LE. Sensor data includes vertical and horizontal EOG channels and accelerometer + gyroscope data. The runtime of the device is 8 hours enabling long term recording and, more important, long term real-time streaming of eye and head movement. They are unobtrusive and look mostly like normal glasses. For Google Glass I present an open sensor-logging platform (including the infrared sensor to count eye blinks) and a fast interface to do lifelogging. We will discuss which eye movements correlate with brain functions and how this fact can be used to estimate the cognitive task a user is performing, from fatigue detection, over reading segmentation to cognitive workload and the advances to track attention and concentration. Challenges discussed in the talk include how to get ground truth and how to evaluate performance in general.