Introduction

Abstract

Proposal

Letters of
Support

Outreach

KC-135A

Life Sciences
Report

Pictures

This report was published in the "NASA Summary Report 2000: KC-135 and other Microgravity Simulations" in August 2000. It concerns the experiment we flew in August 1999, but holds relevance to our current project.

Principal Investigators:
Joachim Deguara
Ryan Traylor
Adrian Lim
Jennifer Glassley
Ryan Casteel

Co-Investigator:
Dr. Hong Tan

Goal
The goals of our proposed experiment are (1) to examine if the illusion of sensory saltation is robust under altered-g conditions, and (2) to gain some insight into whether this illusion interacts with the vestibular sensory system. Specifically, the perceived direction of various directional signals, as well as the perceived strengths of such signals, will be documented.

Objective
This project proposes the use of a tactile feedback system to enhance spatial awareness. The proposed system will utilize a phenomenon called sensory saltation to simulate the feeling of someone drawing directional lines on the user's back. The system consists of an array of 3 x 3 vibrators attached to a vest (on the side touching the user's back). Our objective is to examine how the sense of touch can be engaged in a natural and intuitive manner to allow for the correct perception of the position, motion and acceleration of one's body, an aircraft, or spacecraft.

Introduction
In aviation, spatial disorientation (SD) is the incorrect perception of attitude, altitude, or motion of one's own aircraft relative to the earth or other significant objects. It is a tri-service aviation problem that annually costs the Department of Defense in excess of $300 million in lost aircraft. Spatial disorientation is the number one cause of pilot related mishaps in the Navy and the Air Force. The typical SD mishap occurs when the visual system is compromised by temporary distractions, increased workload, reduced visibility, and most commonly, g-lock [1]. G-lock occurs when the pilot undergoes a high-g maneuver and temporarily blacks out behind the stick. Frequently, after pilots recover from the distraction, they rely on instinct rather than the instrument panel to fly the aircraft. Often times, the direction the pilot thinks he or she is traveling is much different from the actual direction. Additionally, the role of vision on orientation in zero-g has been a major concern for NASA astronauts. Significant work in visual reorientation illusions has been performed by NASA's Neurolab, specifically Dr. Charles Oman, aboard several Spacelab missions during the last 15 years [2].

This project proposes a new approach to examining this problem, namely how the sense of touch can be engaged in a natural and intuitive manner to allow for the correct perception of position, motion and acceleration of one's body, an aircraft, or a spacecraft. The proposed system will utilize a phenomenon call sensory saltation to simulate the feeling of someone drawing directional lines on the user's back. If an astronaut or pilot wears such a system, they may suffer less frequently from spatial disorientation.

The idea of utilizing the sense of touch to replace vision or audition (i.e., sensory substitution) is not new. In fact, numerous devices have been developed for persons with visual or auditory impairments (e.g., the Optacon, a reading-aid for the blind, and Tactaid VII, a hearing aid for the deaf). It is conceivable that such devices can be employed under conditions where visual / auditory sensory channels are overloaded or information received via visual / auditory channels are distorted. However, devices like the Optacon require extensive user training and high level of concentration during its use. In addition, it requires the use of the fingertip, which would interfere with many manual tasks. In contrast, the tactual display system we propose to use has many advantages. It uses the part of the body that is not usually engaged by other tasks (i.e., the back). It requires no user training. It delivers directional information that is easy to interpret (i.e., in the coordinate frame of the user's body). The importance of solving the spatial disorientation problem can be derived from its history.

History of Spatial Disorientation
Spatial disorientation and situational awareness (SA) issues were recognized when humans began flying more sophisticated aircraft, particularly during the Vietnam War. Early solutions to the SA/SD problem focused on better visual displays. Early medical research proved that SA and SD were directly influenced by the interrelationships of vision, vestibular (inner ear), and somatosensory (skin, joint, muscle) sensors [3].

As aviation advanced, spatial disorientation became more of a problem. The Navy reports that from 1980-89, disorientation was listed as the definite cause in accidents that resulted in loss of 38 lives and 32 aircraft. During Desert Storm, four out of eight single pilot aircraft and three out of six helicopter non-combat mishaps were due to spatial disorientation [8]. General Rufus DeHart, a Command Surgeon in the USAF Tactical Air Command, has reported that "the most significant human-factors (HF) problem facing the TAF today is spatial disorientation (SD), followed by high-G loss of consciousness. Of all HF mishaps, 30% in the F-16 and 19% in the F-15 and F-4 are due to SD" [4].

Currently, the U.S. military loses around 20 aircraft and 20 officers per year as a result of spatial disorientation mishaps. Additionally, the Federal Aviation has reported that SD is a cause or factor in 16% of fatal general aviation accidents. Other countries have had similar problems. The Royal Air Force reports that 15% of its helicopter accidents and 33% of its helicopter fatalities result from SD. The Dutch military has lost nearly 10 aircraft in the last 10 years from SD-related mishaps. Canada has lost six CF-18's because of spatial disorientation [3].

Spatial disorientation has been a continuing problem for NASA as well. Around 1973, astronaut Owen Garriott explained to Dr. Charles Oman's research team of his experience with space sickness on Skylab. In smaller spacecraft, space sickness had not been a problem since the astronauts had very little room to move about. The new space shuttle would allow the astronauts to move about and possibly experience conflicting cues from their eyes, ears, and joints; hence, space sickness became an immediate concern. Dr. Oman and several colleagues wrote a successful proposal to develop a set of vestibular experiments for the Spacelab module. While waiting for the completion of Spacelab, they tried the experiments aboard the KC-135A. They learned about visual reorientation illusions, and begin to train the astronauts to be ready for them in orbit. Soon after they began flying in 1981, space sickness quickly became a well-publicized problem. Dr. Oman was able to fly experiments aboard several Spacelab missions during the last 15 years and has found some very interesting results. Specifically, "crewmembers became more dependent on visual and tactile cues to their self-rotation [2]." We propose to examine spatial disorientation from a different perspective.

Sensory Saltation
The "sensory saltation" phenomenon was discovered in the1970's in the Cutaneous Research Laboratory at Princeton University (the word "saltation" is Latin for "jumping"). In an initial setup that led to the discovery of this phenomenon, three mechanical stimulators were placed with equal distance on the forearm (see Figure 1). Three brief pulses were delivered to the first stimulator closest to the wrist, followed by three more at the middle stimulator, followed by another three at the stimulator farthest from the wrist. Instead of feeling the successive taps localized at the three stimulator sites, the observer is under the impression that the pulses seem to be distributed with more or less uniform spacing from the site of the first stimulator to that of the third. The sensation is characteristically described as if a tiny rabbit was hopping up the arm from wrist to elbow; hence the nickname "cutaneous rabbit".

We have constructed a 3-by-3-stimulator array that allows the simulation of "rabbit" paths in many directions. When this display is placed on the back, it simulates the sensation of someone drawing a directional line on the user's back. One might wonder whether there is enough spatial resolution in the back of the torso. Although it is true that the back is much poorer in spatial resolution as compared to, say the fingertip, it is well compensated by the much larger area that the tactual display can cover. Furthermore, the perceived taps in-between actual stimulators are perceptual illusions (see open circles in Figure 2); thus their spatial resolution may not necessarily be limited by the actual receptor densities in the skin of the back. Another important advantage of our tactual directional display, as compared to visual displays on an instrument panel, is that the user does not have to look for it - s/he simply feels it.

Research on the sensory saltation phenomenon has concentrated on the quality of lines that can be perceived [5]. Although some researchers have speculated on the neural mechanism of this phenomenon, it is still not well understood why and how this illusion happens.

Applications
Tactile technology can be expanded beyond solving SA/SD problems into areas such as navigation, communication, alarms and indicators, and training and simulation.

Tactile technology can be used to reduce mission failure, aircraft loss and pilot loss due to pilot disorientation, and to enhance pilot performance by simplifying the flight task. Currently the only accurate sensory information available to pilots concerning their attitude and motion is visual interpretation of instruments or outside reference to the horizon. By integrating the tactile directional display with existing systems, pilots can be steered in the right direction when they fall off course. Ideally, the pilot could maneuver the aircraft using tactile displays in the complete absence of visual cues.

Tactile technology utilizing sensory saltation can enhance EVA safety and effectiveness. Having a correct perception of their own position and motion will allow astronauts to work even more productively and confidently in space.

For military Special Operations, tactile displays provide the advantages of low signature, a silent form of communication, reduction of information overload, a backup to other senses, a good representation of 3-D space, and the utilization of an otherwise unused sense. For example, if a team is attacking target X at night, the leader can give silent commands for attack strategy: a pulse going around the operator's back could mean to "surround the building." Also, if a team is performing a High Altitude Low Opening parachute insertion in the pitch black of night, a system integrated with a Global Positioning System could allow all operators too easily find the on-ground rendezvous point [1].

Application of a tactile system for the blind is evident. With a GPS, the system could help guide the user through unfamiliar territory, and perhaps one day to even drive a car. Even for the non-blind, a wearable tactile display could guide the user through an unfamiliar building to find a room or help a tourist navigate through an unfamiliar city [9].

Methods and Materials

Subjects
This experiment will consist of one team member serving as a test subject and one team member running the experiment and collecting data. The test subject will put on the vest containing the tactual directional display and its associated electronics (see Figure 3). The experimenter will set up the experimental conditions, and document the experiment with a video camera and data collection sheets.

Instruments
The hardware used to generate the directional signals can be described by four functional blocks. These blocks are the control box, signal generator, vibrator driver circuit, and display as shown in Figure 4. The control box utilizes a keypad to input the desired directional signal. Inputs are decoded by the signal generator and the resulting on/off pattern is supplied to the vibrator driver circuit. This circuit accomplishes three tasks. First, it supplies a 220 Hz sinusoidal signal, and then it acts as a power amplifier to produce oscillations in the vibrators. Finally, a tactile display is implemented with a 3x3 array of vibrators attached to a vest. All of this hardware is enclosed in a box of length 11.02 inches, width of 7.87 inches, and depth of 2.95 inches (See Figure 5).

The next module in the hardware is the signal generation module performed by a microcontroller. The specific microcontroller chosen for this task is the PIC16C84 8-bit CMOS microcontroller with EEPROM memory (Microchip Inc., AZ). Ease of programming and its 13 I/O lines make this chip an ideal choice. The PIC16C84 (PIC) is programmed to read a four bit number supplied by the control box and convert it into a directional pattern signal employing nine output pins. An interrupt is generated using the Data Available pin from the control box. The interrupt signal is inverted and then supplied to the PIC's RESET pin. An inverter is necessary because the PIC has an active low RESET and the interrupt signal is active high. Upon reset, the PIC executes routines which select the appropriate pattern chosen by the keypad and applies the corresponding signals to the nine output pins. The schematic of the microcontroller portion of the hardware is shown in Figure 7 along with the pin assignments of the pattern array located on the vest. The PIC is able to control which outputs are turned on for specified amounts of time and what delay should be inserted between each signal. A carefully orchestrated sequence of pulses and delays directed by the precise timing of the PIC creates the sensation of a line being drawn on the user's back.

Vibrator Driver Circuit
Since the PIC can only supply gated high or low pulses, the output pins cannot be used directly to supply the 220 Hz sinusoidal wave to the vibrators. An intermediate device, the vibrator driver circuit, is needed to accomplish this task. The driver circuit's main function is to supply an amplified oscillating signal to the vibrators when prompted by the PIC to do so. The circuit consists mainly of a power supply, a 220 Hz oscillator, and nine 16-Watt bridge amplifiers. When the driver circuit receives a high signal from the microcontroller, it responds by supplying an amplified 220 Hz oscillating signal to the corresponding vibrator. A schematic of the bridge amplifier is shown in Figure 8.
 

Tactile Display
The final piece of hardware is the tactile display consisting of a collection of nine vibrators attached to the side of the vest touching the user's back. The vibrators are placed on a regular 3x3 grid with an 8 cm regular spacing between adjacent vibrators (see right panel of Figure 7). The vibrators are made of flat speakers, four inches in diameter, modified for this application (Audiological Engineering Corp., MA). The driver circuit's outputs also control a small array of LED's set up in the same arrangement as the vibrators. This secondary display is used by the experimenter to visually verify the directional pattern sent to the vibrator array.
 
 

Procedure
The experimenter will choose one of four signals - up, down, right, or left - by selecting one of the four buttons on a keypad mounted on the vest. The ordering of the four signals will be randomized. The other team member (test subject) will not be informed as to which of the four signals was selected. The test subject then will experience the directional signals delivered to his/her back by the tactual directional display mounted on the side of the vest towards his/her back. The subject will verbally report the sensations to the experimenter. The results will be documented on video camera and on data collection sheets by the experimenter.

The subject will be asked to experience and report his/her perception of the directional signal relative to his/her torso. For example, the sensation of something "crawling up the spine" will be reported as "up". The sensation of something "going down the spine" will be reported as "down". Reports of "left" and "right" signals will also be based on the orientation of one's own torso. In order to maximize the duration of data collection during the flight, all team members will have an opportunity to experience the four signals and their associated directions before the flight.

The selected signal is repeatedly delivered to the subject's back while his/her body is positioned at different orientations relative to the aircraft. These orientations range from lying down, sitting, floating in any direction, and free-floating with the subject's eyes closed. This test will include all four ranges of gravity: two times normal gravity, normal gravity, zero gravity, and the transition periods of gravity.

Once all possible cases are examined, the experimenter will select another signal at random, and the experiment will resume. This will continue until all possible cases are examined. If time remains and both team members are capable, the roles will be reversed, and another set of data will be collected.
 
 

Results
Data was collected on forty-one (41) trials during two flights. During the periods of microgravity, the signals felt considerably weaker to the test subjects as compared to the sensations felt during normal 1-g conditions. User success rate at determining the correct direction of the signal sent was approximately 44%. Data acquisition proved to be a challenging task during the zero-gravity period and due to the unexpected orientation of both the subject and the investigator.

Discussion
Since the signal felt weaker to the test subjects during the periods of zero-gravity as compared with the sensation in a 1-g environment, each of the test subjects had to think before responding about the perceived signal direction; thus they were unable to rely on a strong sense of feeling. In over half of the cases, the subject perceived the signal as going in the wrong direction. Only more time in zero gravity will be able to tell us whether or not there is a pattern as to the direction of signal sent or the body orientation or body motion of the user when he perceived the signal incorrectly. What actually caused the signal to feel weaker during the periods of zero gravity is still being investigated. By preliminary observation, probable causes of weak signals being felt could be due to the weightlessness of the vest, distractions experienced by the test subject, and lack of experience in a zero-gravity environment.

Conclusion
To make the signal feel stronger and more apparent to the user, we may use different actuators, change the time between pulses, or increase the size of the vibrator array. The phenomenon of sensory saltation has yet to perform to a standard required for astronaut EVA use or military aviation.

To accurately test the phenomenon of sensory saltation in a zero-gravity environment, more time in zero-gravity and more test subjects are needed. Additionally, a better data acquisition system is needed in order to handle the possibility of the data recorder getting sick during the flight. To prepare for this contingency as well as to double the amount of data and number of users, both flight crew members for a particular flight could wear vests and send each other directional signals via a remote control. They could each record the direction of the signal felt by entering a code onto a wearable number pad. This would solve the problem of the data recorder trying to change the signal on the user's back after the beginning of the zero-gravity period and also trying to record the results on data sheets. Wireless microphones could be worn in order to record real-time results in front of our own video camera.

The NASA Reduced Gravity Student Flight Opportunities Program provided a unique opportunity for us to observe whether the illusion of sensory saltation was robust under altered-g conditions and thus allowed us to gain some insight into whether this sensory illusion interacts with the visual system, other components of the tactual system (e.g., kinesthesis), and the vestibular sensory system.
 
 

References  
[1] Naval Aerospace Medical Research Laboratory

[5] Cholewiak, RW.

                    Exploring the conditions that generate a good vibrotactile line, presented at the
                    Psychonomic Society Meetings, Los Angeles, CA, 1995

[6] Geldard, F.A.,