Abstract
An innovative 3D joystick was developed to enable quadriplegics due to spinal cord injuries (SCIs) to more independently and efficiently operate a robotic arm as an assistive device. The 3D joystick was compared to two different manual input modalities, a keyboard control and a traditional joystick, in performing experimental robotic arm tasks by both subjects without disabilities and those with upper extremity mobility impairments. Fitts’s Law targeting and practical pouring tests were conducted to compare the performance and accuracy of the proposed 3D joystick. The Fitts’s law measurements showed that the 3D joystick had the best index of performance (IP), though it required an equivalent number of operations and errors as the standard robotic arm joystick. The pouring task demonstrated that the 3D joystick took significantly less task completion time and was more accurate than keyboard control. The 3D joystick also showed a decreased learning curve to the other modalities.
Methodology
The multimodal robotic arm user control systems consisted of three parts: a PC workstation, the different controller types (default joystick, keyboard or 3D joystick), and the actuated robotic arm (JACO) TM Robot Manipulator from Kinova Technology as shown in Fig. 1). The default controller for the JACO arm is a traditional joystick to control the movement of certain elements (i.e. arm, wrist) in two dimensions (see top of Fig. 2). Movement of the robotic arm in the 3rd dimension requires rotation of the joystick knob. This motion is extremely difficult or even impossible for individuals to perform with complete high-level (Cervical levels 1-8) SCIs.
Fig. 1. JACO robotic arm ready to grasp a water bottle. It can also be
mounted to a wheelchair.
Two alternative modalities were developed in this project to serve as superior user controllers for this robotic arm for quadriplegic users. The first alternative input method developed was through keyboard control (top of Fig. 2). Keyboards are widely used as a direct selection device for efficient and naturally intuitive operation. For keyboard operation, all the functions for robotic control were mapped to specific keystrokes (i.e up, down, left, right, forward, backward, change mode). Three keyboard input control modes were programmed: discrete, continuous and hybrid (a combination of discrete and continuous) modes. During discrete mode, the robotic arm moved in small increments
every time a key was pressed. During continuous mode, the arm would move continuously until stopped or another key to change directions was pressed. During hybrid mode, subjects could toggle between discrete and continuous modes at their discretion.
Fig.2. Subject with a SCI using the 3D joystick to perform the pouring task.
The other alternative control modality was a 3D joystick (Fig. 3) which was originally designed for haptic video game playing by Falcon Technology®. It was reprogrammed and adapted as a 3D joystick controller for the robotic arm. A handle developed for users with no finger gripping ability was positioned in the center of the joystick. The 3D joystick provides users a method of directed selection to control the robotic arm elements to move in 3D Euclidean space. The handle of 3D joystick was positioned at the center of the joystick as a home (or rest) position if not used by the user.
Fig.3.  3D joystick with adapted handle for quadriplegic users.
A force feedback control with a proportional and differential (PD) controller force the handle back to the center after each manipulation. The control diagram for 3D haptic joystick is shown in Fig. 4. A JACO API was used to fictionalize the haptic joystick to achieve 3D control of the robotic arm.
Fig.4. Â 3D joystick control diagram.