Feeling Objects That Aren’t There
Imagine this. You wake up in the morning to the irritating buzz of your alarm. Instead of fumbling for a snooze button, you wave your hand in the air in the general direction of the clock. There, in mid-air, you find it: an invisible button. It’s an illusion you can feel, like a hologram for your fingers. One swipe at the button, and the alarm shuts off. You’re free to sleep for a few more minutes — even though you never touched the clock.
The science of touch is called haptics. Sriram Subramanian describes the floating alarm clock button as one example of how a new technology called “ultrahaptics” might be used. “It does seem a bit far-fetched,” admits this computer scientist at the University of Sussex in England. But, he quickly adds, such a device is possible. Researchers in his lab now create virtual, three-dimensional objects that people can feel.
The secret to their success — sound waves. Actually, it’s no secret. A growing number of researchers around the world are investigating how sound waves can be used to simulate touch. These sound waves are ultrasonic. That means they’re so high-pitched people can’t hear them. At the same time, they’re strong enough to put pressure on human skin and trigger the sensation of touch. Scientists can change the location and shape of a tactile (touch) illusion by adjusting the sound waves, focusing them on a particular spot.
An alarm clock with a levitating snooze button is just one example. Tom Carter, an engineer, joined with Subramanian to launch a company called Ultrahaptics. Carter imagines a future in which people use electronic devices with a wave of the hand. He and other researchers say the touchscreens and keyboards on current devices are limiting. They wonder: Why can’t we use the air around our devices as another way to interact?
Their research points to a whole new way to use electronics. Drivers might control phones or radios by twiddling their fingers in the air — while keeping their eyes on the road. Video gamers could feel the imaginary worlds that they already see and hear in their games.
Hiroyuki Shinoda, an engineer at the University of Tokyo in Japan, has been studying haptics for decades. In 2008, he became one of the first people to use ultrasonic waves to float virtual objects in midair. Since then, he has looked for ways for real and virtual objects to interact. He thinks that ultimately, the approach could help people connect with each other. For example, the technology might simulate the sensation of touching another person — like holding hands.
Subramanian says the idea of floating, three-dimensional illusions can inspire the imagination. Even though he developed the technology, he’s confident people will find other creative ways to use it. Fellow scientists, entrepreneurs and politicians flock to his lab. And immediately they become inspired.
“Everybody comes up with their own uses,” says Subramanian. “It’s amazing.”
Sounds and solids
Sound travels through the air as waves. But these waves are not like the ones that move up and down through water. A sound wave is an example of a longitudinal wave. It is made up of a series of compressions — places where the air is pressed together. To understand how a longitudinal wave travels, stretch out a spring. Give one end a quick push and pull, first toward and then away from the other end. A compressed group of coils will move down the spiral. In a sound wave, air particles bunch together like those coils.
Anyone who has been to a loud concert knows about the connection between sound waves and the feeling of touch. A low bass note doesn’t only reach the concert goers’ ears — it also vibrates their bodies. Subramanian says the experience of feeling such low notes inspired him to investigate sound waves.
The human body detects sound and touch in similar ways. Cells in the skin have nerve endings, called mechanoreceptors. They detect pressure, which triggers the release of signals to the brain. The inner ear also has mechanoreceptors. Called hair cells, they convert sound into electrical signals that travel along nerves to the brain.
Whether a sound is high or low depends on how many waves pass a single point during a given time. This measurement is called frequency. The higher the rate, the higher the frequency. Sound waves that make high notes have a higher frequency than do those that make low notes. An average person can hear sounds up to about 20,000 hertz, meaning 20,000 vibrations per second. (As people age, that upper limit drops. So children and teens generally can hear higher pitches than can older folks.) Ultrasonic waves are frequencies higher than those that the human ear can hear.
Many devices use ultrasonic frequencies. Some cars have parking sensors that send out ultrasonic waves and detect those that bounce back to identify obstacles. Medical ultrasound devices emit high-pitched sound waves to peer inside the body and “see” things, such as a growing fetus.
Feeling without touching
Physicists have been exploring the physical feeling of sound waves for more than 100 years. When sound waves hit the skin, their pressure triggers the mechanoreceptors. But scientists have only recently looked for ways to use that knowledge in electronic devices.
Subramanian began thinking about using sound waves to control devices a few years ago. He had been working with touchscreens, which always feel hard under the fingertips. He and his colleagues wondered if instead, the screens could communicate with users before someone even touched the device. For example, people might be able to start a program by waving their hands in front of the screen — not touching it. That led him to think about using ultrasonic waves to float objects in the air around the screen.
He started telling other people. “They laughed,” he recalls, saying “This is crazy. It’s not going to work.” But Subramanian’s team did not give up. “Other people never believed in our ambitions,” he says. “But they could not give us a good reason why it should fail.”
About five years ago, while he was at the University of Bristol in England, Subramanian began working with Carter. At the time, Carter was a college student looking for an interesting project.
Subramanian, Carter says, “had this crazy idea that you could feel things without touching them.” He asked Carter to build a grid of ultrasonic transducers. These are devices that send out high-frequency sound waves. His goal was to use those sound waves to push small objects.
After years of work, the researchers found a way to focus the ultrasound waves. Their device used 320 transducers connected to a computer. That setup allowed them to tune those waves precisely and create the illusion of an object floating in space. They debuted their first ultrahaptic device at a scientific meeting in 2013.
Since then, Subramanian has continued to push the science forward. Last October, he and his team showed how ultrasonic waves could be used to levitate, move and guide small objects. They called their invention a “tractor beam” — an idea made famous by science fiction. Those beams were supposed to use energy to capture objects, such as enemy space ships. The new acoustic tractor beam instead acts more like invisible tweezers.
Carter left graduate school to run the Ultrahaptics company. Next he wants to use the technology to simulate the feeling of touching different textures. “We can tailor the sound waves to any type of vibrations,” he says. At one frequency, the sound waves may feel like dry raindrops falling on your hand. At a higher frequency, they may feel like foam.
“How do you feel anything? You feel it by sliding your hand across the texture,” he explains. “Your skin is vibrating in a pattern as you drag it across.” The idea, he says, is that “if we can work out those vibrations, we can start to recreate complicated textures like rough or smooth wood, or metal.”
A personal touch
In Tokyo, Shinoda and his team recently unveiled a system called the HaptoClone. It uses a similar technology for communication. The system looks like two bulky boxes, each big enough to hold a basketball. One box contains a real object. The other displays the object’s reflection. Thanks to a series of mirrors between the two, the copy looks and moves identically to the original.
Shinoda and his team also installed a set of ultrasonic transducers. These allow the real object and its copy to “communicate” by touch. For example, if a person pushes on the real object, it moves. And so does the copy. That’s obvious — and would happen for any reflection! But here’s the interesting part. If someone reaches into the box and pushes on the reflection, their hand will truly feel it, because of the sound waves. And when they touch it, the copy will move — as will the original. Any action done to one side happens immediately to the other.
For example, imagine that one side contains a real ball. Someone can push on the reflected image — and thereby also shove the original ball out of its box. If two people each stuck their fingers into the box, they would get the sense they had actually touched each other — even though it had been sound waves creating that illusion.
“In the HaptoClone, real interactions between real objects can be realized,” says Shinoda. He thinks such a system might be most useful for people who want to connect with each other. “Physical contact between people is very important,” he notes. “Whether it’s simply shaking hands or stroking a person’s skin.”
Touching is a kind of nonverbal communication. He says it sends messages unlike anything people can say with images or words. He imagines a device like the HaptoClone may, for example, help children feel closer to a parent who is far away.
“My mission is to help people who have lost something,” he says.
He’s still fine-tuning the HaptoClone. Right now, the device is far too bulky to sell to people to keep in their houses. He’s working to make it smaller and easier to use.
Physicists may have first connected sound waves to feeling a century ago, but these new devices are truly cutting-edge. They’re also the result of hard work — often requiring years of research and testing.
Carter says his company, Ultrahaptics, began with an uphill battle. “We spent 18 months with our device not working, in various forms,” he says. But the struggle was worth it. In fact, he thinks the technology is only possible because of the hiccups he and his collaborators encountered along the way.
“You learn best by failing,” he says. “The fastest way to learn is to try to learn, and fail, and learn how to fail fast. If you don’t try to do something, you won’t fail, and you’ll never succeed.” n