Blockbeats
VR game for composing music spatially
Date
2022
Duration
10 weeks
Project Type
VR game prototyping
Keywords
spatial audio, gestural interaction, Unity
Role
Ideation, Prototyping
Team
Aria Xiying Bao, Nix Liu Xin, Emmanuel Serrano, Yinghou Wang
Musicraft is a VR audio-visual gaming experience and platform where the user creates and remixes music and architecture at the same time. It explores the idea that architecture is frozen music and music is flowing architecture.
Concept sketch by Yinghou Wang
Game Mechanics
Stack and build freely with architectural building blocks.
Each block is a musical track. Layer and remix music while building architecture.
Step into the audio-visual space and experience it immersively.
Interaction Design
Prototyping
The project is built with Unity. On the software side, we are utilizing Meta’s Oculus Integration SDK to develop and deploy the VR experience. Specifically, we use Oculus Interaction SDK to enable vision-based hand-tracking and leverage its hand interaction infrastructure (hand physics, collision, gestures, custom hand poses, etc.). Unity Shader Graphs are used in combination with custom materials and textures to render the desired toon visuals resembling that of the game Monument Valley.
A major technical task for the project is the object snapping mechanism when modeling. A snapping mechanism is necessary because of the limited effects we add to the audio tracks based on the way objects are combined. Some of the technical considerations are a 3D grid system and preset snapping slhouettes custimized to each object.
Vision
An immersive spatial audio-visual experience where the audience is inside a morphing building synced to the remixing of the music in real time.
Vision images by Nix Liu Xin