My idea is to use ml5 and runway API to create a little game where you’re controlling a character (in this case the ball) and running away from enemies (the arrows). I am using the soundClassifier and the speechCommands18w model to use the words up, down, left and right as the controller for the movement.
The rule is simple, you have to run away from the vehicle arrows and if they reach you for at least 1.5 seconds, you loose.
This game is inspired on the Nature of Code book that talks about attractors and autonomous agents.
Had some trouble to using the Runway API to my mainframe. My code began using credits really fast when testing and used the daily limit. Had to request for more and could not implement my idea in the end. The idea was to have the ball have a voice generated by runway so it reacts live depending on which state of the game you are, winning, about to loose or lost.
Still going to work on this sketch to have it looking better and exploring/achieving the functionality desired.
https://giav2030.github.io/giav2030.github.io-soundClassGame/