https://giav2030.github.io/giav2030.github.io_AIVision/

The AI Vision project began as an assignment but is quickly transforming to a potential larger project. This first prototype looks to use Google Image and Text to Speech machine learning models to see, analyze and describe images for users with vision impediment. This sketch is using Gemini 2.5 flash model. It uses the webcam camera to get an image, turns what it sees to text and then uses the Text to Speech model to describe the image see. The model is prompted to prioritize potentially hazardous objects from the image to let the user know about them. I have in my interest to research further about people in this situation to develop a device or app that empathetically serves them best. I strongly believe in the power of technology to be used for positive purposes that increases people’s quality of life. As a Sound Artist and Musician, I am very sensitive to how sound has too many advantages in terms of providing and receiving information, that can make it low cost and easy to implement. This first test is about getting the connections and editing the prompting to one that serves them best.

I started working closely with my Machine Learning for the Web professor, Yining Shi. Asked for her advice towards this device/app. She already introduced me to other machine learning models that could suit better for this project.

On the other hand, I am aware of how much do people with visual impediment rely on their ears. Hence, the need of finding an easy, portable and hearable way to let them what the model sees through audio, without them using their headphones. Luckily, this piece of technology already exist in products like the Meta Glasses. These pieces of information is quickly turning the scope of the project in terms of how necessary is to build my own device, but maybe build a digital app/platform that allows the user to use their own glasses for it. I am still debating how this decision may affect the accessibility of the app and its usage. This could also be an opportunity to push myself into finding a device solution that makes it more accessible than the Meta Glasses per se by using related technology.