Want to have a sneak preview how our motion overlay technology looks on Apple TV?
In the video you can see two layers merged together to create a single video stream. The B&W layer shows the ghost image, how you should be doing the exercise, while the color layer shows your current body posture. On the sidebar you find the proceeding with the training program in terms of time left and calories burnt. The bottom bar shows how many reps you’ve got left for the current exercise.
Ok, but how does this stuff work?
First we record the exercise, in this case the “sawing with a bottle” (thanks to Mark 🙂 ) and create a video loop of it. As a second step we record the ideal trajectory of the upper arm during the exercise that we call the reference signal. Finally we store both the video and the reference signal within our app.
As the user performs the exercise, the algorithm running on the phone compares the live gyroscope and accelerometer measurements with the reference signal to estimate the current body posture that we transform into the appropriate video frame.
We are experimenting with further video effects for merging the live and ghost images to a video stream. Besides alpha-blending we are testing side-by-side and picture-in-picture effects as well to find the perfect user experience for each individual exercise. 🙂 Further eye candies are yet to come 🙂