I was fortunate enough to attend the New York City (NYC) Kinect Hackathon held on June 21, 2014 – June 22, 2014 @GrindSpaces. This posting is about my experiences and a self interview/review of the projects and experimental hardware Microsoft allowed us to play with. If you’re interested please read on…
Question: So exactly what is a Kinect Hackathon?
The Microsoft Kinect for Windows team has been touring the world trying to push and expose the new Kinect for Windows (K4W) v2 device. They have gone to places like San Francisco, Germany, and China. More recently, they stopped off in New York City. At each location, the team introduced new features, and hardware to the masses showing off capabilities and potential. In NYC, the way in which the team did this was to have a Hackathon contest.
A Hackathon is simply a gathering of technical minded people ranging from inventors, to designers, to enthusiasts, to hobbyists, to developers, architects, and just plain ole smart people who have an idea. The goal is to take this idea and see if the people in the room can make it a reality by building a Proof of Concept (POC).
The contest part of it is to see which team can come up with the best working POC, for one or more ideas within 24 hours. Food and drinks are supplied all night, and team members and architecting, designing, developing, and testing all night until that cut off time.
Question: Wow, that sounds like fun, What was it like?
It was very fun!!! Let me explain why. The day started off by Ben Lower, community manager for Kinect for Windows team, introducing us to various members of the K4W Team: Carmine, Chen, Smarth, Ragul, Kevin, David and Lori. (Please excuse name spellings and if I missed anyone I apologize) and then explained about the new experimental firmware update to make the K4W v2 device support near mode – up to a potential 1cm, although at the Hackathon the current edition was up to 10cm. Ben also talked about Kinect Ripple a new framework which allows you use the floor to map or calibrate a grid for a pseudo menu/command control system for an application, while still keeping the K4W normal functionality – body tracking, audio, etc.
The next this that transpired was opening the floor for ideas, and forming a team. A little feeling slighted note… the winners of the contest were teams which were pre-planned and prepared prior to this event, but that was ok.
People took the microphone in droves… I wish I had recorded all the great ideas and thoughts people envisioned with this device, because I could just quit my day job and work on each idea, one project at a time. Each idea has the potential to make profit, and benefit humanity. The few ideas I did remember ranged from tracking animals, plants, and people, to robots avoiding obstacles, to field sobriety tests, to automated CAD designs, to virtual orchestras, playing instruments with your body, occulus rift + kinect skeleton tracking, simple funny gestures, to move the virtual egg but don’t wake the rooster farm game, to robotic hand tracking
, to Kinect ripple based Mine sweeper, a kinect ripple based match that shape game, and of course but not least my idea of the Windows 8 Store app: Kinect Virtual Doctor.
After the idea forming came the teams. I pitched my idea, others pitched their ideas, and we just went around forming teams if you didn’t have one. At first I was afraid my heart rate idea (based on my initial code here) would just be a copy and paste app for Windows 8 until a young college student named Mansib Rahman decided to pair up with me.
We changed the game…
We started binging (googling in reality – but binging sounds WAYYYY better) potential algorithms for various medical rates using the HD Camera, IR Camera, and the new Near Mode firmware features of the Kinect. We learned a lot. We worked all night when I re-imagined and realized the potential for a medical library for use with the K4W v2 device was huge. That’s when we decided to create the Kinect Virtual Doctor windows 8 store app. The application could potentially be placed inside your home, and you can stand in front of your mirror, while the application could tell you your breating rate- O2 Levels, pulse, blood pressure, stress mood, alertness, and many other things. But first we needed to make sure it was plausible and doable. We took the rest of the night trying to determine which algorithms we could implement in 24 hours. It turns out the heart rate, and breathing rates were the easiest, but we only ended up with re-writing my heart rate sample for Windows 8 utilizing the algorithm posted here.
One of the funniest stories of the night in my opinion was the “Pied Piper” green T’s group, at least that’s what I call them. Kudo’s by the way to sticking it out, and passing me a working audio sample – thanks to Andras (K4W MVP). Oh and before I forget – thank you Velvart Andras and James Ashley – (K4W MVP’s) for helping me out with ideas and coding.
These “Pied Piper” guys started out with the idea playing musical instruments with your body. For example if you hit your arm, it plays a drum, if you chest it changes the octave or plays another instrument. Sitting next to these guys was painful because of the terrible sounds coming from that area of the room. Envision akward musical notes with no melody constantly sounding off around 3am in the morning… Then on the other side of me was the Roosters crowing “Cocka-doodle-doo” right afterwards. I swear I felt like Noah or something. In any case the piped piper guys realized it was a little more difficult to do the playing music with your body routine. So they started to give up. A couple of them left and had some drinks – and in my opinion came back slightly wasted. That’s when the only logical thing for them to do appeared… “Let’s make a field sobriety test with the Kinect”. The app was simple – walk a straight line and repeat a tounge twister phrase. If the Kinect tracked you walking the straight line and the you said the phrase correctly, you didn’t go to jail.
This was hilarious and straight out of the HBO movie series “Sillicon Valley” and their fake Pied Piper web site mixed with the intoxication app from Google’s based “The Internship”… Now we went from 3 in the morning bad music to rooster crows to “Ricky Rubio went around the reading rainbow wrong” or something like that – PRICELESS!!!
Question: So what was your experience with the experimental firmware for the Kinect?
I will simply say this, for my application the 10cm worked better for obtaining the face and calculating the heart rate, however not everyone had the same success for their applications during the event.
Question: What was your experience with the Kinect Ripple?
I thought this was another great implementation for the Kinect. I can see Museums, Science exhibits, Schools, Convention centers and the like all utilizing this funtionality. In case you’re wondering what exactly it does… here’s a quick image:
and video: http://youtu.be/RfJggcO7zZ8
Question: So would you say the Kinect Hackathon was a success?
Yes, I most definitely would!