User:Sasalz/sandbox

= Sensory Augmentation =

The sensory organs of humans constitute the various ways we have of perceiving the world. Thus, losing any sensory function is a terrifying prospect. Sensory substitution devices offer the opportunity to replace a lost sense in full or in part by exploiting another sensory modality. Already in 1969, Bach-Y-Rita and colleagues showed that the brain is plastic enough for blind people to learn how to use a tactile substitution system to perceive visual inputs. More formally, sensory substitution tries to replace a missing sense by delivering some or all of the information usually gathered by one sense to another sense.

While the possibility of replacing a sense is potentially transformative to those who have lost a sense, it also gives rise to new questions about perception in general. If replacing a sense is possible, could we give ourselves a whole new sense or could we enhance any of our existing once? Sensory augmentation follows up on these questions and tries to develop devices that produce novel sensory inputs. To achieve this, one has to essentially do the same as in sensory substitution: deliver sensory information regarding a novel input via a sense that does not usually deliver that information.

The possibilities of the senses we could give humans is enormous, especially when considering the many senses of nonhuman animals. Pigeons for example can detect magnetic fields and their properties; many fish have an electric sense allowing them to detect electric fields; or we could imagine having echolocation as for example bats do. The usefulness of such capabilities, in contrast to the immediate need for sensory substitution devices, is an open question. Thus far, any augmentation device has mainly been developed to investigate scientific questions regarding perception.

The feelSpace belt
One of the most studied sensory augmentation device is the feelSpace belt, developed by Peter König in 2005, which tries to give its user a sense of magnetic north. The idea was originally developed as a means to investigate theories about the nature of conscious perception. However, by now the belt is commercially available as a navigation device which the company especially advertises as an enhancement tool for blind people.

The original belt used within the experiments, shown in the figure below, consists of a waist belt with 13-30 vibrating miniature motors attached to it. A simple control unit maps the orientation information from an electronic compass to the vibrators such that the element pointing north is activated. Thus, if a person facing north turns clockwise the next stimulation site will be located in a counter-clockwise location. Vibration frequency is tested to have an optimal tactile sensitivity at frequencies between 150 and 300 Hz. Experiments showed that it is best for the user to only activate a single element at a time. Furthermore, to ensure comfort the signal needs to be as accurate, fast and reliable as possible. This requires the hardware to sustain sudden changes in motion velocity and direction that are associated with the kinetics of everyday human motion.

The compass has a tilt range of 360° over all three motion axes. To ensure a stable and reliable signal the compass utilizes a gyroscope, accelerometers and magnetic sensors. Orientation is hereby measured of the trunk around the body’s main axis. The control unit is mounted on a circuit board which allows additional components to interface with the power supply and the vibrators. This interface allowed the researchers to simulate navigation tasks within virtual reality, as well as to test trained users with wrong sensory stimulations. In its initial form, the feelSpace belt used an array of standard 1.2V NiMH rechargeable batteries with which it could achieve continuous operation for more than 10 hours.

First experiments with the sensory augmentation device demonstrated that the signal provided by the belt improves behavioral performance, influences physiological reactions and leads to qualitative changes in perceptual effects. A follow-up study further tested the belt as a sensory enhancement tool for congenitally blind subjects. The results are consistent in that they again show an improved behavioral performance and induced perceptual effects.

The specifics of the described set up is based on the initial design which was used for experimental purposes only. Later versions continually improved in comfort and ability. Especially making the belt MRI compatible allowed for interesting investigations regarding theories of sensory experiences*. As already mentioned, the developers realized the potential of the feelSpace belt as an interesting new navigation device. The newest version, called naviBelt, allows to choose between 3 different navigation modes. The routing feature gives concrete directional information like any other navigation device, however, the directional instructions (left, right, …) are given via tactile stimulations. The other feature, the beeline mode, provides stimulations that should give the user a general sense of the direction of their destination. Lastly, the belt allows to select a compass feature which functions as described in the original set up. All of this is achieved by coupling the belt to a smartphone or computer.

Sensory Experience
While sensory substitution devices are often developed to aid a person’s sensory loss, sensory augmentation is mainly used as a tool to investigate the nature of perception in general. There are many different theories about perception in philosophy and psychology. Many scientist hope that experiments with substitution and augmentation devices will shed light on some of the fundamental questions in this field. One such theory that is extensively studied using the feelSpace belt, is the theory of sensorimotor contingencies (SMCs) of conscious perception.

The theory of SMC was developed within the broader framework of embodied cognition, which defines cognition as embodied action. This paradigm emphasizes that any cognitive process includes mind, body, and environment. As a consequence, embodied cognition can be understood as an active and multisensory probing of the environment. The sensory-motor theory, first formulated by O’Regan and Nöe, more specifically proposes that actions and associated sensory information are coupled through systematic relations, i.e. sensorimotor contingencies. These rules and regularities have to be actively learned and concomitantly shape how we perceive the world. Furthermore, the structure of SMCs are hypothesized to be specific to the different sensory modalities and in turn, could explain the qualitative difference in the experience of our different senses. Such SMCs are called modality related SMCs are different from so called object related SMCs. The latter support the evidence of multisensory changes related to actions towards objects.

Experimental evidence in the field of sensory substitution supports the theory of SMCs. The theory states that the learning process of new percept strongly depends on the subjects’ ability to actively explore and manipulate the sensory modality. This realization was used within the Tactile Visual Sensory Substitution device (TCSS). After allowing the users to control the cameras, such as changing the angle or zoom in and out of objects, subjects reported a significant improvement in the richness of their perception. The advances within sensory substitution devices, and their support of the theory of SMC gave rise to the idea of sensory augmentation. The theory explicitly states that sensorimotor contingencies are learnt. Thus, the immediate question that follows is whether humans can learn new SMCs about sensory information’s that are not natural to them.

An extensive study addressed this question using the feelSpace belt since it provides directional information (about the true north) for which humans do not have a natural sensory modality. In the study, subjects sleep was monitored using EEG recordings before, during and after the 7 weeks of training. The hypothesis is that the development of a SMC of a new sensory input requires active procedural learning which is typically reflected by an increase of REM sleep. Furthermore, they explored whether the learning process induces an observable change in the neural activity of the sensory and the motor cortex. The data from this study support that the sensory augmentation with the feelSpace belt leads to procedural learning and an increased involvement of motor areas that are known to be involved in navigation tasks. Finally, they tested two other consequences of the SMC theory: once a new sensory input has been mastered it can be actively used and it will change the persons’ perception of the world. Subjects report a change in their perception of space, especially after starting to more and more trust the navigational device. However, neither the control nor the belt-wearing subjects showed an increase in performance using the belt during a complex navigation task.

In summary, sensory augmentation as well as sensory substitution support the theory of sensorimotor contingencies. Moreover, the results of the experiments with the feelSpace belt provide evidence for a causal role of SMCs in perceptual awareness. For the interested reader, a much more extensive overview of different theories of perception and what experimental evidence from sensory substitution can tell us about these theories can be found in the paper “''Sensory Substitution and Augmentation. An introduction.”'' by Fiona Macpherson.