Isaac Arbelaez Venegas. Credit: Penn State. Creative Commons
Powering accessible, collaborative art
Leveraging knowledge gained through three majors and self-guided learning, Isaac Arbelaez Venegas helped develop software for a haptic glove to translate movements to virtual space
March 14, 2024
Editor’s note: A version of this article originally appeared on Penn State News.
By Jessica Hallman
UNIVERSITY PARK, Pa. — In a recent installment of “A Note from Neeli,” Penn State President Neeli Bendapudi visited the Center for Pedagogy in Arts and Design where she received a hands-on demonstration of “Together, Tacit,” an interdisciplinary research and creative collaboration between the College of Arts and Architecture and the College of Engineering. In the project, artists and engineers are building a haptic glove powered by virtual reality, allowing visually impaired and sighted individuals to work together to create a sculpture.
The concept is simple: A person with low or no vision wears a glove fitted with haptic sensors that vibrate with certain movements, helping them to “feel” their art as it is created. A sighted person wears a virtual reality headset, allowing them to view the sculptor’s movements in real time and guide them in their work. When the piece is finished, it can be 3D printed.
“Together, Tacit” has been in development since 2018, when Bonnie Collura, professor of art, set out to create a bridge to translate how sighted and blind or visually impaired communities assembled visual information. Numerous students have since contributed to advancing the project — including Isaac Arbelaez Venegas, a junior international student and Schreyer Honors Scholar pursuing three undergraduate degrees, in mechanical engineering, engineering science, and mathematics. He intends to graduate in May 2025.
Arbelaez Venegas talked to Penn State News about his experience working on this project and as a Penn State student.
Q: Why did you choose to come to Penn State?
Arbelaez Venegas: My older brother graduated from Penn State last year, so I was familiar with the University. We’re from Colombia. My parents of course would have preferred I'd stayed closer to home, but they were relieved that my brother and I would be together. I also chose Penn State because it was the best option financially and for research opportunities. I had checked out some of the labs when I visited, and they were very impressive.
Q: What made you want to get involved with research as an undergraduate?
Arbelaez Venegas: I have several conditions to meet as an international student, and some of the requirements had restrictions. But research didn’t have those barriers. Research was offered to me early in my college career. My first-year seminar in the College of Engineering showed students how they could get involved with research as early as their first semester at Penn State. I joined a group that semester, which was a great experience to understand the environment in which a research group works and what happens in different labs. I had the opportunity to be in multiple research groups and to explore different areas. Also, many professors open their doors and allow you to ask questions. I feel privileged that I’ve had that exposure and been able to explore different research areas. The opportunities here have really been unique.
Isaac Arbelaez Venegas observes Penn State President Neeli Bendapudi and local community member Laura Schaffer demonstrate the glove with haptic sensors that he helped to program. Credit: Penn State. Creative Commons
Q: Tell us about the “Together, Tacit” project, including how you got involved and your specific contributions.
Arbelaez Venegas: I first got involved through my capstone project, which is typically completed in a student’s senior year. I had the opportunity to do my capstone project as a sophomore because I had earned enough credits and was eligible. I was encouraged by a friend in Schreyer to take the honors capstone course with him. He was in another group working on “Together, Tacit,” and I was working in a group focused on a project for Lockheed Martin. He reached out to me and asked for my help with software and some of the coding. I had some programming skills, so I talked with Paul Mittan, professor of practice, the faculty mentor for the capstone course. He allowed me to be in both groups at the same time.
When I first started working on "Together, Tacit,” there was a group that was part of the mechanical side of building the glove. There were some communication issues that were occurring between what information the glove was receiving and how that information was translated on the software side. For example, there are flex sensors in the glove that are giving constant values that show how much output the person wearing it has, but there wasn't a strong connection between what they were doing and the software. So, I worked with my team to make that communication happen. It resulted in a working glove that not only allowed the individual wearing it to have those feelings and sensations, but now also communicated with the software. That was really important. As I continued to work on the project independently with Bonnie Collura beyond the initial capstone course, I was able to implement some other functions that I would say helped create a better and more personalized experience for the action of sculpting — for example, increasing the size or creating different resolutions for the sculptures themselves.
Q: You did a lot of programming for this project. But none of your majors are computer science. Where did you learn how to code? Why do you think you were chosen to contribute your programming skills to the project?
Arbelaez Venegas: I became really interested in technology because of a Guinness Book of World Records that my brother got as a gift when I was 9 or 10 years old. We were reading some of the science records, and I remember learning about the smallest guitar that was like 10 micrometers long. That really grabbed my attention and interest for what I could access through a computer. From there I started to learn coding through online resources and became much more interested in that side of computer science. I slowly started to dabble in some programming languages and apply them in different projects. And that led to my being able to contribute to “Together, Tacit.” In programming, once you know the base the only thing that changes is the syntax for each language. So, it was an easy transition for me. I think my classmates saw that I was passionate and knew that I would be able to help. It was a great collaboration — the team was at a point where it felt stuck, and they saw my passion for programming and coached me through what I needed to do to jump in and help tackle the problem.
I was able to see how my contributions on the back end might actually affect a human being and be translated into a project that was improving accessibility and therefore improving someone’s life. It was truly something that impacted me, at least in the way of knowing what kind of engineer I want to be in the future. From the beginning, Professor Collura allowed the group to feel the sculpting, and then we applied those experiences in virtual reality. You don’t really get that experience when you’re just coding or working on a computer. It allowed me to see a different perspective and identify areas I may have otherwise been overlooking.
In working with Laura Schaffer, a visually impaired woman from the local community who appeared in the video, we were able to get direct user input. She tried the advancements we implemented through different tests, and she gave us feedback on what she was feeling and what she thought could be better. It was a lot of communication that engineers normally wouldn’t have. Being able to have people like Bonnie and Laura test our project was really a wonderful experience because we got to really think in someone else’s shoes. They also gave insight into what I think collaborative and interdisciplinary work should be like — different perspectives feeding in to one project. The whole process has made me grow as an engineer and also as a person.
Q: What was it like to be able to personally share your work to the president of Penn State?
Arbelaez Venegas: It was really remarkable. Getting recognition and being able to show my work and home values through this video proves to me that I’m doing the right thing and that I’m on the right path. Being able to talk with President Bendapudi about my work shows me that all the sacrifices I’ve made — such as late nights studying for exams or certain opportunities I’ve turned down to focus on others — have paid off. And of course it made my parents back in Colombia incredibly proud.
Isaac Arbelaez Venegas had the opportunity to talk with Penn State President Neeli Bendapudi about his contributions to the "Together, Tacit" project. Credit: Penn State. Creative Commons
Q: What has motivated you in your work with “Together, Tacit?”
Arbelaez Venegas: In Colombia, I attended a conservatory and played piano. I remember being in a class focused on reading and writing music, and a particular percussionist who was also in that class. The first time I saw him he had a special notebook, which I later realized was in Braille. I was always impressed by him, because in the class the instructor would play music, and we would have to write it down — the notes, the key, the tempo, everything. I was probably 7 or 8 years old, and it was hard for me. But my visually impaired classmate was able to do it easily. It fascinated me. We became friends. In the end, I’ve never talked to him about how he perceived those sounds. But being able to create that connection, that bridge that I didn’t have back then, or maybe that interpretation that we would both be able to speak the same language would have been truly wonderful. That is a strong part of my motivation to be part of “Together, Tacit.” I’ve been able to see it slowly evolve into something where someone is truly able to communicate or see what the other person is feeling, through sculpting.
Q: In what other activities are you involved at Penn State?
Arbelaez Venegas: In 2022 I, along with my blood brother and a good friend, restarted the Penn State chapter of Sigma Lambda Beta, a Latino-based fraternity, after a few years of inactivity. Being far away from home, it was hard to find my place at Penn State. I was pretty sure that other students might be in similar situations — especially international students. So, we wanted to reopen a space where people would feel comfortable and find a family. Historically, the fraternity is Latino based, but we currently have members from the Middle East and Southeast Asia and from a variety of backgrounds.
Q: You’re an international student far from home who is involved in high-impact undergraduate research and pursuing three majors. How do you balance it all?
Arbelaez Venegas: I think a lot of it comes from discipline. At the conservatory in Colombia, we would wake up at 5 a.m. and go all day until 8 p.m. or later. I developed a desire for learning. I wished I had different things to learn all the time. And in engineering, there are so many topics to explore. I wish I had eternity to learn about them all. I developed a desire to learn and to try hard. And now that I have the opportunity to be pursuing my education in the United States, I want to use that desire to learn all I can here.
Isaac Arbelaez Venegas and his collaborators share their work with Penn State President Neeli Bendapudi in this "A Note from Neeli" video. Credit: Penn State
