Team Roles
Individual Project
Davide Zhang: Research, Product Audit, Ideation, Prototyping, Evaluation
Overview
Type: Final Project for GSD6338: Intro to Computational Design
Instructor: Jose Luis Garcia del Castillo Lopez
Time Frame: Nov 2020 - Dec 2020
Keywords: Oculus Quest 2, Unity, C#
Meshly is an intuitive low-poly mesh modeling application in VR. It is developed in response to the steep learning curve of 3D modeling software. Its goal is to be adopted by designers in their workflow as well as hobbyists without training in 3D modeling.
Problem
Imagine that you'd like to model a low-poly version of your cat to be 3D printed later, but you are not trained in 3D modeling software like Blender. You just received your Oculus Quest 2 headset and could not find intuitive VR 3D modeling applications either.
There is a lack of fast, intuitive ways to model low-poly meshes.
Solution
Meshly, an intuitive vertex-based low-poly modeling VR app that offers:
• Low-poly mesh creation (from scratch or by tracing real objects)
• Low-poly mesh manipulation
• Low-poly mesh coloring
• Low-poly mesh import/export
Process
Research
User Interviews
To learn about the current 3D modeling experience, I performed informal interviews with 5 friends and acquaintances who have/use 3D printers. The pool consists of both creatives with training in 3D modeling and those without. Questions include:
• What's something you like about 3D printing?
• Something you don't like?
• What do you usually 3D print?
• Where do the 3D models come from? Do you model yourself?
• Have you tried 3D modeling software?
• Have you tried VR 3D modeling software?
From the interviews, I learned that:
• Untrained users would love to 3D print their own models, but find learning 3D modeling too time-consuming/difficult
• Trained users would also love to have a quick way of modeling in their work flow
• While VR modeling apps appear more accessible, there is still a significant learning curve
• Users like low-poly models in addition to high-fidelity ones
"I'd love to 3D print my own creation, but having no training, the complicated interfaces of 3D modeling apps are just intimidating"
"It's more intuitive to model in VR, but those apps take a while to learn as well, which kind of defeats the purpose"
Product Audit - VR Creation Apps
I performed a product audit to get a better sense of how current VR creation apps approach modeling. I evaluated them based on the fidelity of models and the learning curve. Apps like Quill and Gravity Sketch offer a variety of features (such as animation and collaboration, respectively) that cater towards professionals, suggesting a steep learning curve. While Blocks is low-poly-friendly, its modeling method is based on aggregating primary shapes, thus still requiring prior knowledge of 3D geometry.
Conclusion:
There is space for a low-poly modeling app that is intuitive to use.
Ideation
Opportunity Statement
How might we make low-poly modeling intuitive for untrained users?
I was inspired by the small scale sculpting process where one holds and rotates the object in their non-dominant hand and uses their dominant hand to sculpt.
Small scale object sculpting
Image reference: TheCrafsMan SteadyCrafting Youtube Video
Concept Sketch
Interaction Concept Sketch
In Meshly's modeling environment, the user could use the left controller as their “hand” that holds and orients the object. The right controller is then a metaphor for the “tool” that manipulates the object, much like a chisel or any other sculpting tool.
Design
Degrees of Freedom (DoF)
3DoF (Left) and 6DoF (Right)
While Oculus Quest 2 is capable of 6DOF, I decided to maintain 3DOF after testing to stay true to the simplicity of the interaction paradigm that I am referencing. Instead of moving the entire body in virtual space and work on different parts of the mesh, the user could simply rotate the mesh towards them, rendering the extra degrees of freedom redundant. Fixing the user’s position in virtual space means fewer variables for the user to track cognitively, thus making the experience more akin to the small scale sculpting process and hopefully, more intuitive.
Interface
There are 3 modes of interacting with the mesh: create, sculpt, and color. Besides the controller buttons, the two major components of the user interface are the radial menu and the color picker. Their design processes are summarized below.
Menu Design Process
The tripartite radial menu was chosen for the final menu design based on user testing. Specifically, the radial menu, aside from aesthetic correspondence with the controller ring, uses the least number of steps to perform the task. Compared to the possibly multiple joystick movements of the list menu and the point-and-click operation of the raycast model, the radial menu only needs one joystick motion to finish selection.
Color Picker Design Process
For the color picker, a combination of hue ring and value triangle is selected because it offers the most options for fine-tuning while only adding one extra step compared to the other two options.
Prototyping
1. Create Mode
Radial menu diagram - Create
Create mode manual
Create mode selection using joystick
The user can create, highlight, select, and delete vertices, and make triangle faces through clockwise selection of 3 vertices.
Deleting vertices will in turn delete triangle faces related to those vertices.
This also enables the user to trace real-world objects in virtual space.
Create vertices, select vertices, and make faces
2. Sculpt Mode
Radial menu diagram - Sculpt
Sculpt mode manual
Sculpt mode selection using joystick
It is toggled from the radial menu via the left controller thumbstick.
It allows for selecting and moving vertices. In future iterations, more ways of sculpting the mesh could be added, such as attractor point deformation.
Select vertices, move with the right controller, and the mesh will update
3. Color Mode
Radial menu diagram - Color
Color mode manual
Color mode selection using joystick
The user could color the vertices. Drag the Hue Ring selector and simply touch the Value triangle to finish color selection.
The user is able to achieve a blended look or a discrete look depending on the number of vertices having the same color.
Face coloring could be implemented in the future in addition to the current vertex coloring method.
Pick color and color the vertices
4. Orienting the Mesh
Aside from the menu, the left controller rotates and moves the entire mesh by pressing its index button.
5. File I/O
Finally, there is also a simple file IO functionality with the left controller. The Y button imports an OBJ file from a specified path, while the X button exports the current mesh to the same path.
The application runs via Oculus Link so the path is on the computer, but could be changed to an on device location if deployed as a standalone app.
Evaluate
User Testing
Because of the time constraint and limitations due to the COVID-19 pandemic, I was not able to invite participants to test the prototype. Instead, I showed the same 5 users I interviewed a demo video that walks through the features and asked for their feedback, which are summarized below:
• All agree that Meshly looks intuitive to use
• Most would see themselves using this app
• Needs option for face coloring in addition to vertex coloring
• Needs ability to save colors
• Needs ability to scale model
• Most would see themselves using this app
• Needs option for face coloring in addition to vertex coloring
• Needs ability to save colors
• Needs ability to scale model
Reflection
I really enjoyed prototyping Meshly and learned a lot about designing and developing for virtual reality. I look forward to further exploring the design process mediated by emerging technologies such as VR. The abstract of this project has been submitted to eCAADe.
Things that I liked:
• Exploring different interaction paradigms in VR
• Trying to balance ease of use and functionality
What could be improved:
• More systematic user research and testing
• More iterations
• Better code design
