NHK Enterprises, Inc. and NHK Global Media Services have long been committed to better serving the deaf community in Japan, becoming an accessibility pioneer in the media industry. This has led to heavy research into Japanese Sign Language (JSL) and investment into technology that can make important news available to JSL speakers. Our solution: Kiki, a virtual human interpreter that generates full JSL dynamic interpretations from text input.
Kiki uses a database od over 16,000 pre recorded animation capture files to generate the final motion. The user selects those bvh files with a custom web app and that get’s piped into an Unity instance running on AWS.
My work in the projected involved understading the BVH database making sure all those files can be read and piped into Unity animation clips.
Due to the complex rig needed for animation retargeting, we also created a system to blend between the BVH animations and keyframed animations. In the above video you can see the debug view with multiple skeletons in wireframe.
This project won 2 Cannes Lions and 2 Webby awards
more info at Media.Monks website: Kiki | Monks