Hack Upstate Community Members Compete (and win!) at the World’s Largest XR Hackathon
The following is a guest blog written by Melissa Schmitz. Are you interesting in sharing your story with the Hack Upstate community? We’d love to hear from you! Send us a note at jesse@hackupstate.com if you’re interested in getting involved.
Two active members of the Hack Upstate community, Melissa Schmitz and Joshua Marris, recently attended the MIT Reality Hack held January 16–20 last month. The largest mixed reality hackathon in the world, it is a 5-day event held every January at the MIT Media Lab, one of the world’s premier hubs for augmented and virtual reality (often categorized together as “XR”). Interested coders, designers, and industry experts from all over the world apply and are selected to attend based on their various backgrounds and skill levels to come together for tech workshops, community engagement, and, of course, hacking.
Melissa Schmitz, Emerging Technologies Engineer and Virtual Reality Tech Lead at The Raymond Corporation, was particularly excited to attend for the community and creativity.
“Many hackathons focus almost solely on the technical aspect of things. I support that. I usually write code and work on technical projects at these hackathons. But one of the unique features of the MIT Reality Hack is the scale with which the design community gets involved. Almost 50% of the participants at the Reality Hack were designers! You don’t see that emphasis anywhere else in the hackathon world because XR specifically requires artists and creative thinkers to create the stunning visuals and compelling stories we witness in these experiences. I would like to see more artists and creatives engage in the general hackathon community as much as they do in XR because every application instantly transforms from good to grand with the right design and user experience.”
Melissa’s team made a VR application called MiXR, a spatial audio experience for the Valve Index. The game takes place in a futuristic vaporwave recording studio outfitted with acoustic foam panels and carefully placed neon lights. Some users commented that, despite the neon, it was a calming and inviting space. Focused on the visual design in Unreal Engine, Melissa stated, “We wanted the space to evoke curiosity and encourage people to explore.” Mythical synthesizer harmonies play as you carefully wander in room-scale VR, modulated by Vive Trackers strapped to your feet.
In the game, users pick up random objects to discover that they all make sounds. From the clickity-clack of a computer mouse to a mouthy table that exclaims in a female voice, “Table! Table! Table!” or even a honking golden goose statue, this whimsical acousmatic sandbox was intended to allow people to tinker and make music with the everyday objects around them.
“Our original concept was more abstract, like a Tilt Brush for sound where you could compose a music piece in VR,” she explained. “But that concept ended up being too vague and difficult to accomplish in a few days, so we pivoted to something that anyone could easily understand and interact with. An experience that brings imagination and quirkiness to the mundane.”
For Joshua Marris, currently a System Administrator at QPK Design and a small business owner, this was his first-ever hackathon. “It was exciting and really everything I hoped to experience at my first hackathon. I was lucky to find a group of people who were just as nervous and excited as I was. We’re still in touch each week, sharing what we’ve done since then and encouraging one another to continue.”
Going into the start of the hackathon, his goal was clear: iterate as much as possible in the short amount of time available, meaning many commits and pushes to test. Of his team members, Josh had the most experience using the collaborative tools required to submit their work and share the project, so he made sure to spend time helping his group become comfortable. “The nice thing about hackathons is that the people you meet have skills and interests that overlap your own. We had the opportunity to flex our skills and to try something we had wanted to but needed time to explore.”
Josh’s team worked on Brain Drain, a VR educational experience that allows students of all ability levels to answer academic questions with unique input methods, including eye-tracking with the HTC Vive Pro Eye VR headset. Inspired by teammate Shannon Putman’s 13 years of teaching experience, particularly special needs students with an autism spectrum disorder, she observed firsthand the desperate need for change in how we approach educational instruction. “We need to stop trying to change the way [students] learn and start changing the way we teach,” she remarked in their DevPost description.
The game tells the tale of two evil doctors, Dr. Apprentice and Dr. Novice, who created a “Phantom Virus” transmitted through junk food kids know to be unhealthy but still enjoy. When exposed to the virus, children transform from hard-working, intelligent students to dazed “phantoms” lurching about, asking academic questions on repeat for all of eternity… unless someone can answer their question correctly. The virus already spread worldwide, the world’s only hope was placed in the hands of a brother and sister duo Watson and Spiller, with help from their two beagle friends.
The design of the game was playful and cartoonish, with characters following a blocky style similar to the ultra-popular Minecraft. Teachers create student profiles with options for inputs (eyes, body, hands), subject (reading, math, technology), and grade (4–8), and students jump in to explore seasonal environments ranging from snowy hills to verdant forests. As the student answers questions correctly, the phantoms transform back into students to ultimately help defeat the evil doctors and save the world.
From their DevPost, “The ultimate challenge of Brain Drain is that there is no mouse or keyboard. You are the controller! If your character needs to jump, then you need to jump. If your character has to run, put your running shoes on because you will be moving! Students are moving, jumping, running, and learning all at the same time!”
Brain Drain won an award for Best Use of Eye Tracking sponsored by HTC, which provided an HTC Vive Pro Eye VR headset for the team. Josh commented, “We were surprised and elated to win! Our group had a lot of experienced teams with whom to compete, so it was nice to have a track that allowed us to be among one of the few to implement a feature like eye tracking.”
Two weeks following the end of the MIT Reality Hack, Josh and Melissa gave a joint presentation entitled, “Anyone Can Make a Game! Rapid Prototyping for All” on January 31st at the inaugural Syracuse Game Jam hosted by Hack Upstate’s Student Success Coordinator Jason Scharf and TCGPlayer’s Manager of Customer Success Ed Forth.
The presentation focused on how participants of any skill level can get started in creating a game ranging from how to capitalize on the tools and skills you already know to tips on how to think through the process of building your Most Viable Product (MVP) in as little as 24 hours. You can find more resources like these on the ‘Cuse Game Jam website.
Interested in joining the Hack Upstate community? Sign up for the monthly newsletter to stay current on upcoming community events, ranging from Women in Tech meetups, open code nights, to the bi-annual hackathons held every April and October at The Tech Garden in downtown Syracuse. Go to the Hack Upstate website to learn more!
