Current Lip Sync Program Tests

CrazyTalk

CrazyTalk is a well known animation program that can record audio, take prerecorded audio or use a script to create a lip synced animation for other uses. CrazyTalk is fairly responsive in its lip sync and requires little processing time considering my short audio tests, however it can't be used in a live environment.

The animation timeline can be exported into other programs like Unity, but still it's not a live performance and lip sync animation.

Unity

These are current plugins for Unity that suppose real time lipsync. 

LipSync Pro

LipSync Pro doesn't automatically have a system to do live microphone input. Instead, you have to feed the program prerecorded clips to generate the animation. I'm sure there's a way to script LipSync pro to take live input though.

Still would have to code a live mic input, but I don't think that's quite as important as getting a neural network to recognize emotional differences. I think I should stop playing around with current lip sync programs and try to figure out how to train a neural network into doing what I want.