“The skills gap in hands-on trades and emerging technology is real,” said Karthik Ramani, Donald W. Feddersen Distinguished Professor in Mechanical Engineering, and principal investigator for the project.  “Today, one of the best ways to transfer hands-on skills is through the traditional one-on-one apprenticeship model. However, it is costly and not scalable. You lose both the trainer and trainee. Also, older workers are retiring, and it’s difficult to recruit younger workers to replace them. And when they are hired, it is challenging to transfer those skills and knowledge to the next generation. The skills themselves are also changing, complicating the landscape. Skill-XR is all about boosting that skill transfer in a timely manner and enabling it to happen anywhere at any scale.”

The “X” in “Skill-XR” stands for a range of technologies that includes augmented reality (AR), virtual reality (VR), and extended reality (XR).  For example, a newly-hired factory worker may wear augmented reality glasses while being trained on a piece of equipment, and see graphics overlaid on the machine about how to operate the controls.  The instant feedback upon performing the task correctly ensures that workers are trained quickly, effectively, and safely.

“In this generation, YouTube videos have become stand-in educators,” said Ramani.  “People turn to YouTube to figure out how to tie a tie, change a tire, or cook a meal.  We want to push that concept into the next dimension, so that it’s no longer just a 2D video on a screen, but it’s an augmented reality experience, that actually responds to you and gives you feedback.”

Anyone can produce a YouTube video with today’s technology, but the same is not true of augmented reality.  Most AR experiences involve complex programming and other specialized skills that companies don’t have, and cannot afford to purchase.  Skill-XR will level that playing field, enabling anyone to generate an augmented reality experience, whether it’s in the factory, the classroom, or their own home.  It’s also designed to be platform-agnostic, able to be implemented in whatever new devices or mixed-reality technologies develop in the marketplace.

Scalability is also a big aspect to the project.  “It’s one thing to have a person with skills or knowledge, who shares that knowledge with a handful of others in a room,” said Ramani.  “Technology will allow that same person in the same room to teach those same hands-on skills to thousands, or even millions. As we’ve seen from the COVID-19 pandemic, remote learning is now a vital piece of our everyday lives, and Skill-XR will enable remote hands-on skilling to happen anywhere.”

While high levels of technology are involved, Ramani has built it upon the most foundational of human conditions: the desire to learn.  “As babies, we learn by doing,” he said.  “We all have eyes, ears, and hands.  We pick objects up and move things around. We copy what we see others doing. There’s no technology involved in these processes; we just naturally do it.  With Skill-XR, we’re enhancing that process, so that people can learn by doing, and not just watching.  That’s why learning scientists are such a big part of our team.”

The Skill-XR team includes co-principal investigators Alex Quinn, assistant professor of electrical and computer engineering at Purdue University; Thomas Reddick, associate professor of cognitive psychology at Purdue University; Niklas Elmqvist, professor of computer science at at University of Maryland; and Kylie Peppler, associate professor of informatics at University of California at Irvine.

“As skilled older manufacturing workers retire, their hands-on skills must be passed on to a new generation,” said Mark Lundstrom, Purdue’s Acting Dean of Engineering. “Professor Ramani’s team is addressing this critical challenge by making it possible for the first time for skilled workers to easily create augmented reality experiences that will allow younger workers to learn by ‘doing’ rather than by watching. The impact on the nation’s manufacturing workforce should be substantial.”

“All the companies we work with in the Discovery Park District and the Aerospace District are keenly focused on their workforce, and the ability to get their workforce fully productive as quickly and economically as possible," said Brian Edelman, President of the Purdue Research Foundation.  "I believe the Skill-XR technology is a game changer, both during the current pandemic and post-pandemic, as we all are learning how capable we are of working remotely.”

The building blocks for Skill-XR began in 2018, when Ramani’s team received a $2.5 million NSF grant to explore the “Future of Work at the Human-Technology Frontier.”  One of the ideas that emerged was Skill-LeARn, a concept for using augmented reality (AR) to educate workers.  This received Phase I funding from the NSF’s Convergence Accelerator, a new NSF capability designed to address national-scale societal challenges through use-inspired convergence research.  Ramani's team conducted several research projects under this umbrella, such as developing an AR “ghost” that imitates the actions of humans, and integrating AR content into student classrooms.  The Purdue Research Foundation’s Office of Technology Commercialization applied for several patents for technologies emerging from the project.  Now, Ramani’s team has received Phase II NSF Convergence Accelerator funding of $5 million over two years, to implement the project (now renamed Skill-XR) in the real world.

For a true convergence approach, Ramani has partnered with several Indiana manufacturing-focused companies, such as Wabash National, Kirby Risk, and Gaylor Electric, along with other large and small companies across the country.  The partners have been testing prototypes and determining how they can be best used on a real-world factory floor.  He’s also partnered with the Indiana Economic Development Corporation; and Ivy Tech, the community college system of Indiana, whose educational programs are often focused on specific job fields.  “Indiana is the perfect testbed for this platform,” said Ramani. “The economy of rural areas depends heavily on manufacturing, so when manufacturing succeeds, so does Indiana.”

“Every day, Indiana manufacturers across the state are delivering critical solutions and propelling innovative products that help power our world,” said David Roberts, Indiana Economic Development Corporation chief innovation officer. “As disruptive trends change the face of today’s manufacturing industry, we’re focused on supporting strategic initiatives like the Skill-XR technology to develop a diverse and skilled workforce in Indiana, equipping Hoosiers with the industry-focused skills and training needed for the future.”

Ramani sees potential for Skill-XR beyond the factory floor.  “The project also involves a variety of educational environments, such as the Strong National Museum of Play in Rochester, New York; and Indiana's own Rural Innovation Network,” said Ramani.  “Young children can learn in a hands-on manner, even while they play games.  College students can learn engineering concepts.  Even doctors and nurses can practice surgical techniques.  Our overarching goal is to extend the capabilities of all people, using technology.”

Note to journalists: Raw video and photos are available at this Google Drive.

Writer: Jared Pike, jaredpike@purdue.edu, 765-496-0374

Source: Karthik Ramani, ramani@purdue.edu, 765-494-5725