Typing and dragging a mouse are the most inefficient aspects of computing. Our minds race along at lightening-quick speed, creating, innovating, using the knowledge and insight we have gained throughout our lives. The computer calculates and decides and spits the results onto the screen, also with mind-boggling speed.
But our hands are the weak link, the one vestige of our animal selves that we've not been able to eliminate from this higher-minded pursuit. Typing and mousing slow the flow of information from brain to computer until that information moves at the relatively sluggish pace set by our hands. For those who, because of illness or injury, don't possess full control of their hands, these tasks are barriers to entry, nearly insurmountable obstacles between them and an increasingly important part of society.
Andrew Junker, an electrical engineer and neurophysiologist, has spent more than two decades searching for ways to bypass, or at least minimize, our manual involvement with computers. For the U.S. Air Force, he researched systems that allowed pilots to control aeronautical systems with subtle muscle activity, such as clenching jaws or furrowing brows.
After retiring, Junker began to explore the commercial possibilities of such technology. The product born from his research is Cyberlink, a device that translates facial muscle activity, eye movement and brain activity into on-screen cursor movements, mouse clicks and other computer functions.
All muscle activity generates predictable electrical impulses known as electromyographic (EMG) waves. Furrowing the brow, clenching the jaw, raising an eyebrow, even touching the roof of the mouth with the tongue — all produce distinct, repeatable EMG waves. Similarly, eye movement produces electrooculographic (EOG) waves, electrical impulses that can easily be categorized and associated with specific eye movements.
Even brain activity produces electrical signals. Known as electroencephalographic (EEG) waves, these impulses have long been used by medical personnel to determine the health and general activity levels of the brain. An intense or excited state produces a different wave pattern than relaxation. Wave patterns even vary amongst different thoughts and, if those thoughts can be repeated in the context of the same general state of the mind — relaxed, excited, bored, happy — then the resulting wave patterns can be identified and linked to that combination of thought and mental state.
Cyberlink gathers these brain-, muscle- and eye-generated charges from the user through three silver chloride coated sensors held in a cloth headband against the user's forehead. A wire carries the analogue signals from the headband to a modem-sized box in which they're digitized, amplified and separated. The altered signals then travel along a second wire and into the user's computer through the serial port.
Inside the computer, two types of software manage the processed EEG, EMG and EOG waves. The first is Cyberlink-specific in that it doesn't connect with other software on the computer. In this category are a musical composition program and games such as Tetris, Asteroids and billiards that teach users to control cursor movements and mouse clicks with facial muscle and eye movement.
The second type of software used by Junker's invention is the Cyberlink Actuated Tracker (CAT), a mouse driver that maps the electrical impulses generated by the user to certain mouse functions. Using the same technology that maps traditional mouse movements and clicks to on-screen actions, the CAT mouse driver allows Cyberlink to control any Windows software.
Becoming adept at moving the cursor and effecting clicks with face and eye movements requires practice, but it's generally an intuitive means of control.
"It usually only takes two or three minutes before a person can move the cursor around the screen a little," Junker says. "But to become good at it, they need a little practice. It's a retraining of motor skills."
Glancing upward moves the cursor up and purposefully relaxing this upward movement lowers the cursor on the screen. A quick look to the right moves the cursor in that direction, while bringing the eyes back to a relaxed position moves the cursor to the left. After each movement, users "click" to let the CAT driver know that they're finished moving. With a second click, they tell the driver that the next eye movement is intended to move the cursor again. These clicks are effected with slight eyebrow raises, one for a single click, two for a double-click and three to select and drag an item.
But the thoughts that produce EEG waves are not so easily steered. For EEG waves to be a reliable means of controlling a mouse, the user must be able to reproduce the same states of mind and the same thoughts on command. Anyone who has ever tried to meditate or otherwise harness the constant ramblings of the mind knows the difficulty of this task.
"Most people don't know how to control brain waves," says Junker, recognizing the practical limitations of this aspect of Cyberlink. "Most people who have used Cyberlink give up on this part of it and mainly end up using facial muscle activity and eye movement."
Consequently, Junker has switched his marketing focus, setting aside for now the idea that brain activity can be effectively mapped to computer control functions. He instead focuses on the accessibility that EMG and EOG waves can provide handicapped people.
Most people, even the severely handicapped, are capable of the small eye and face movements required to move the cursor and click the mouse. Since the Windows operating system dominates the retail software world, most functions are accessible through pointing and clicking. So, even though EMG and EOG waves can do little more than provide the functionality of a traditional mouse, they allow those who cannot use their hands to access much of the same computer software that the able-bodied use.
But, if Cyberlink can be tweaked to filter out the nearly inevitable noise of the human mind, the device could offer even more to users.
Cyberlink separates brain activity into 10 distinct frequencies, which Junker has termed Brainfingers. These Brainfingers allow the user to control 10 additional functions with 10 separate thoughts.
If Cyberlink could effectively cull those thoughts from the distractions swimming around them, these Brainfingers could be mapped to the functions most accessed by the user: the opening of a program, the selection of a function within a program, even the typing of a frequently used word or phrase.
And this typing assistance would be a welcomed feature, since entering text with Cyberlink is tedious. Individuals who cannot type on traditional keyboards or utilize voice recognition software often use a virtual keyboard, a graphical representation of a keyboard. With eye tracking software or, now, with Cyberlink, users move a cursor around the screen, selecting one letter at a time until a word is spelled out. Although Cyberlink presents another option for users to access virtual keyboards, it hasn't improved this cumbersome technique.
While institutions working with handicapped individuals have purchased nearly all of the $1,995 Cyberlink devices sold by Junker's company, Brain Actuated Technologies Inc. (BAT) of Yellow Springs, there might be other commercial applications for the product on the horizon. As computers continue to shrink in size and grow in capabilities, the traditional QWERTY keyboard will necessarily become obsolete. As evidenced by the cramped keyboards on many laptops and all handheld computers, a full QWERTY keyboard cannot be squeezed onto tiny computers.
One alternative is chorded keyboards, smaller keyboards with far fewer keys, each of which represents numerous functions. But, because each of these functions is accessed through a combination of key presses, chorded keyboards require a new style of typing and, consequently, might never be commercially accepted.
Yet another option, voice recognition software, has improved dramatically over the past several years. Dragon NaturallySpeaking from Dragon Systems Inc. (www.dragonsys.com) now boasts accuracy rates between 90 and 95 percent. Cursor movements and mouse clicks, however, are difficult with such software. Because Cyberlink excels at these very functions, the combination of Cyberlink and voice recognition software could provide a quick, effective means of entering text and controlling the on-screen activity of portable computers.
Cyberlink could also prove popular with video game enthusiasts. The product's melding of thought, subtle muscle activity and computers could connect game and gamer in a way joysticks never will. Virtual reality games could incorporate not only the movements but also the thoughts of players. And the financial, creative and intellectual resources of the multi-billion dollar computer gaming industry could eventually lead to a greater understanding of the mind-body-computer connection.
The current research into this connection is in its embryonic stage, from which future uses and advances appear limited to many. Alexander Graham Bell's primitive, nearly inaudible "electrical speech machine" sired our global telephone network. Similarly, the impractical and unreliable room-sized computers of the 1940s and 1950s were the precursor of today's powerful, inexpensive and compact personal computers.
Perhaps Cyberlink will some day be seen as a pioneering step in the evolution of human-computer interfaces.
To learn more about Cyberlink, visit BAT's Web site at www.brainfingers.com.
