BinaryVR, Inc. does a different kind of tracking for VR. The California-based startup, founded by Jihun Yu and Kenneth Ryu in 2015, has developed technology that tracks facial expression. Its technology enables users to create a personalized 3D face from 2D facial photos to track and retarget their facial animations in real-time in any transformed CG character or avatar mode.
Facial expression is more challenging than body motion capture since it requires higher resolutions to capture subtle movements of the eyes, mouth, lips and cheeks that give expression to how a person is feeling. These miniscule movements, often times only sensed by the onlooker, but elemental to expression, require high fidelity and resolution and specialised filtering techniques to capture realistically.
What this means for VR users is that a virtual reality headset embedded with BinaryVR’s performance capture technology can transpose facial expressions on to virtual characters. This has great applications for social VR where avatars represent users in a virtual world. The technology has been demonstrated with the HTC Vive at CES 2016.
So, it’s not surprising to learn that BinaryVR is working with High Ficelity, an open source virtual reality platform that allows users to build their own worlds. BinaryVR has integrated its technology into High Fidelity, allowing a user’s mouth to be tracked while wearing a VR headset. High Fidelity was founded by Philip Rosedale, who founded Second Life in 2003.
Yu, who is Korean, studied computer graphics while doing a doctoral program in computer science at New York University. He worked in the Facial Performance Capture team at Industrial Light & Magic (ILM), the motion picture company founded by renowned film director George Lucas from 2011 to 2015. It’s here that his idea for the startup was born. BinaryVR also has an office in Yeoksam-dong, Seoul.
Besides the 3D avatars, BinaryVR is planning to produce a 3D avatar that resembles the user in addition to tracking facial expressions.
The startup has received funding from K Cube Ventures, a venture capital fund run by Kakao and was also a member of Boost VC Tribe 6. Boost VC is a start-up accelerator in Silicon Valley.
Yu’s ultimate goal is to incorporate Binary’s tracking technology a default option on different VR devices in order to create more realistic virtual reality.