Guest Post: Hacking a Voice Interface into a Microwave

13 Nov 2014 guest, post,, hackathon,, community

Wit.ai participated in a number of hackathons this fall and we saw tons of cool hacks using the Wit.ai API. One of our favorites was from Hack the North at the University of Waterloo in Ontario, Canada. Team Home Ease won our API prize for their voice controlled microwave and toaster oven project. Hackers Nick Mostowich, Mayank Gulati, Ram Sharma, and Austin Feight shared their experience of the hackathon and working with Wit.ai. Great work Home Ease!


Team Home Ease

As a group, we were really tired of people telling us that the Internet of Things is “coming soon.” We’ve seen demos of ten thousand dollar fridges at CES, seen Robert Downey Jr. use a fully automated garage in the Iron Man series, and seen glimpses of greatness with devices like Nest. Still, we don’t have the ability to buy smart devices off the shelf for a reasonable price.

So we decided to just build them ourselves.

Hackathons aren’t always the greatest place to build cutting edge hardware, since development iterations are much longer than most software projects. Most teams at hackathons hunker down three or four to a small table and code up a web app. Instead we thought of doing something different, something with hardware, and we knew it will be related to the Internet of Things.

So, we put together five tables together in the biggest room we could find in the beautiful E5 building at Waterloo. We covered the table with all the hardware we could find. Austin brought an entire tub full of parts, tools, and wires. Nick brought a full bag of goodies. Mayank helped one of the organizers for an hour and in return had access to every single microcontroller, actuator, and piece of hardware available for use at the hackathon. Ram drove Mayank home to pick up his own appliances, and we were off to the races. You could say this gave us a bit of a hometown advantage, but hey, you work with what you’ve got!

Toaster Oven

In total we had a waffle iron, a toaster, a toaster oven, and a microwave to work with. The goal was just to connect them to the internet so we could control them remotely and automatically. We felt that this would be a good starting point and a solid accomplishment for the weekend.

The waffle iron and toaster were simply too mechanical to bother with. Levers and springs don’t lend themselves easily to retrofitting, so we moved on to the toaster oven. We used spark.io as our microcontroller, and connected a servo directly to the potentiometer that controls the power level to the heating coils, giving us control over on/off and temperature. We ripped the timer right out; all our timing was done server side.

The microwave was a lot harder. Rather than control it mechanically, we decided to interface directly with it’s input controller. After ripping off the control panel, we realized that it was a relatively simple 2D input matrix. Whenever two of the thirteen input pins were set to high, a specific code was activated on the internal controller of the machine. For example pins 2 and 13 corresponded to “stop”. The only problem was that we had a maximum of 7 outputs from our relay. The breakthrough came with a relatively simple observation: we only needed enough buttons to turn the machine on and off and set the power level; timing was unnecessary as we could just start and stop the machine from the server. In the end we only hooked up start, stop, power level, 5, and 9. This was enough to control everything about the microwave a user would expect.

Thus we ended up with two kitchen appliances connected to a simple Node.js web backend. Things could be turned on and off at various power levels all via the internet. We had accomplished our goal with some time to spare. We thought, what would make this hack even better?

Toaster Oven

Enter Wit.ai.

We went to see Wit’s demo and were blown away. Jen was able to get simple voice control up and running in about fifteen minutes. Austin sat down with his phone and built a simple application to send audio to Wit and pipe the results back to our Node server. The intent/entity system made actually controlling the devices insanely easy. For our applications, the intent was the device to use and the entity was a recipe. For example, you could say “Microwave me popcorn” and the server would look up what the microwave recipe for popcorn was. It would then put the microwave on high for 4 minutes.

What really blew us away was how Wit.ai adapted to different types of input. There’s no need to be verbose and specific. You can say “I’m feeling like some pizza,” or “Cook me some mother fucking bacon!” and Wit just magically knows what you mean. It worked in noisy environments, and adding tons of recipes didn’t seem to slow it down noticeably. We were also able to validate the input messages through Wit.ai’s excellent web interface.

From our point of view it made our hack look really impressive with minimal amounts of work. Hands down Wit.ai is one of the best APIs we’ve ever worked with.

Team Home Ease and Team Wit