Twitch Plays Scribbler

In our SE 101 class, we were given the task of programming a robot. It boasted a multitude of sensors, but the execution was a tad lackluster. Sensors only worked some of the time and yielded inconsistent values. The camera operated at a maximum of 0.5 frames per second. Our task was, first, to make the tobot be able to autonomously navigate around obstacles, and then use this thing to make something and do some project management diagrams about the process (the project management aspect, it turns out, was what we were supposed to really get out of the course.)

The constraints actually managed to bring some pretty cool results out of the experience. The obstacle avoidance was challenging to get working consistently due to the faulty sensor data, but using calibration, some statistical analysis, and lots of heuristics, we got it working. We got it on video going around the simplest obstacle, a box perpendicular to it, but it also worked for other arbitrary orientations.

Next was the fun part: making whatever we wanted with it. We decided to try to not rely on the robot itself as much as we could due to the grievance the sensors caused us in our obstacle avoidance challenge. We eventually settled on Twitch Plays Scribbler.

If you haven't heard of Twitch Plays Pokemon, it's a constantly streaming video on Twitch.tv of Pokemon handheld games. Thousands of users type in commands into the chat simultaneously which are then executed in the game. The result is an eclectic and exciting and challenging new experience. We wanted to bring the fun and frustration of trying to accomplish tasks while working with lots of unknown strangers to the Scribbler Bot.

Twitch Plays Scribbler is a similar idea: everyone spams commands into a chat window and the robot executes them as fast as it can while sending results back to users via a video stream. This stream included both the 0.5 fps first-person camera (we were marked for actually using the robot's hardware) as well as an overhead webcam feed at a decent frame rate to make the experience actually feel responsive.

It even would sing the first line of "Hooked on a Feeling" if a user told it to.

The result was actually a surprisingly good learning experience. The robot itself had to be communicated with over Python, a language I had previously not had much experience in. The server was set up with Node.js and using Socket.io to connect with the clients. I did a lot of new things and I am proud of how it turned out so I would say it was a success.