Anyone who has played a game with a Nintendo Wii controller or felt their phone buzz in their hand when pressing an icon is familiar with haptic feedback: the use of touch to communicate with users. A research group at Illinois State is using that concept to introduce hands-on learning in a virtual reality setting.
By creating an immersive virtual reality (VR) experience, Assistant Professor of Technology Dr. Isaac Chang and engineering technology (ET) students Alex Diffor, Jordan Osborne, and Jake Weihe are investigating how the sense of touch affects learning.
Appears InDiffor, a junior double-majoring in engineering technology and graphic communication, felt the past year of virtual learning during the coronavirus (COVID-19) pandemic exposed a weak point in existing distanced-learning options.
“Not being in the classroom is a big setback for those of us who need to learn in that physical environment,” Diffor, of Normal, said. “VR gives us the chance to try to connect the dots in between conceptual skills and our hands, increasing the ability to learn outside the classroom.”
To test whether VR and haptic learning can effectively mimic a true hands-on experience, students participating in the research project will be tasked with building a chair, both in real life and in VR. Performance will be measured by data collected on speed and accuracy of assembly. Pre- and post-test surveys will provide insight into users’ individual experiences.
“In the simulation you’ll go through all the steps of how to build the chair in virtual reality by interacting with the pieces and being able to play around with them in the environment just as if we were at home, learning how to work a machine in the Turner (Hall) labs,” said Weihe, a senior engineering technology major from Bartlett. “After the simulation is done, we’ll give the participant the actual chair to see how well they remember the building steps and perform the procedure in real life.”
The haptic glove is rigged with sensors that will provide feedback linked to actions performed in a VR environment. 
Engineering technology majors Jake Weihe, Alex Diffor, and Jordan Osborne (left to right) are working with Dr. Isaac Chang to implement haptic feedback into VR learning experiences. 
VR headset and haptic glove
The research team is adapting the VR experience to include haptic feedback by having participants use a pair of gloves rigged with sensors. By referencing existing haptic gloves and utilizing open-source input, Osborne has already constructed a prototype of the glove.
“We have the five fingers and the palm,” said Osborne, a senior engineering technology major from Normal. “On each spot, we have these little black discs, which are vibrational motors, and those are going to be connected to a computer linked to the VR environment.”
For participants, this means holding a tool in the virtual landscape will offer a similar physical sensation as holding a tool in real life. This tool will allow participants to build up muscle memory and tool familiarity that can be applied to real laboratories.
“The assembly process isn’t really focused on the furniture building,” Chang said. “It’s replicating what an engineering technology student would encounter in a lab setting.”
The group’s research began in spring 2021. Last summer the team received a grant from the Office of Student Research’s Undergraduate Research Support Program (USRP), renamed the FIREbird Grant program, which provides funding of up to $3,000 to support faculty-mentored, independent student research projects.
The USRP grant, along with funding from the Department of Technology, enabled the team to make great strides in its research. The three students presented their preliminary research at The Association of Technology, Management, and Applied Engineering (ATMAE) Annual Conference held last November in Orlando, Florida.
“It was definitely nerve-wracking,” said Diffor, who presented on how haptic rendering can be used in procedural learning. “But the questions that got asked and the experience that we brought back, it made it count.”
The team hopes to begin testing the technology in the fall.
The group’s work has provided team members with a valuable experience they can build off in their future careers. Osborne, who plans to remain at Illinois State to pursue his master’s degree, is excited to apply what he has learned from participating in this research. “I now have this experience to lean back on because it’s given me an idea of what the process is going to be like,” Osborne said.
Weihe explained the challenge that conducting research presents.
“In classes we’re given a problem, and there’s almost always one or two ways to find a sure solution for that problem,” Weihe said. “With research it’s a very open-ended question where you’re given two unknowns. We don’t know how this problem happens, or what we can do to solve it, but it’s your task to do it, and that’s what happens in the real world.”
