FaceTrackerRC3

FaceTracker is a plugin node for Foundry Nuke created for facial tracking without mocap rigs and markers. The tracking information can be later used for retouching, adding scars, relighting, face replacement, aging and de-aging, etc. For best results we recommend using it with FaceBuilder. FaceTracker is similar to GeoTracker but in addition to model's position it also tracks facial gestures. Our tracking algorithms are stable contrary to neural networks solutions, precise, allow full manual control and don't require any kind of on-stage preparation like facial motion capture rigs.

Accurate & Fast Tracking

We want to bring latest achievements of the scientific world to the production pipelines, meaning you have the best algorithms and approaches at hand when you use our tracking plugins. Speaking of the speed, we can't guarantee you the realtime tracking, but we are really close to it, so you wouldn't need to wait long.

Integrated Workflow

You don't need to export and import anything from and into Nuke, the whole tracking process is happening inside: you create a FaceTracker node and start tracking using a 2D Viewer, and checking the results in 3D Viewer when needed.

What You See Is What You Get

You don't need to meddle with sliders and knobs, you just create 'pins' on the face mesh in the Viewer and drag them to the corresponding position. The face model is adapting while you drag the pins.

Manual Control & Precision

While we try to make everything full automatic requiring as little as possible human work hours, we still focus on precise tracking results, so you can control the process and adjust the results and keyframes to a full degree.

User Tracks (Helpers)

Usually you can just rely on our own tracking algorithms, but just in case, you can improve tracking quality using tracking data from other trackers. For example, you can import tracks of Nuke's built-in Tracker node and FaceTracker will take them into account with highest rating.

Surface Masks

You can mask out some polygons of the geometry that you want to exclude from tracking. For example some reflective parts of skin or eyeballs. Though usually you don't need to do this, since FaceTracker deals with them pretty well on its own.

Roto Masks

Another way to improve tracking (in those rare cases when it actually fails) is excluding occlusions from tracking. To do it you can connect a roto node to the Mask input of FaceTracker and (re-) launch tracking.

Feels Like Built-In

Our plugins are written with C++ using low-level Nuke API. So you can be sure you're working with tools with native-like experience.

FaceTracker FAQ

Where can I find a tutorial?

It's waiting for you here.

Do you have some examples to play with?

Indeed! They're on the examples page.

What model should I use as an input for FaceTracker?

We assume that you're going to use FaceTracker along with FaceBuilder, using FaceBuilder's output as an input model for FaceTracker. But you can easily use FaceTracker without FaceBuilder, check the question about a custom model below to find out how.

Can I use FaceTracker without FaceBuilder?

Yes, the workaround is described in the Custom Model question below. You don't need a license for FaceBuilder to export a default model.

Can I use a custom head model?

You can export the default model of FaceBuilder, and then modify the shape keeping vertices count and order. Then you can import it back, and connect to FaceTracker. It uses vertices to understand what is what in the model (e.g. nose, lips, eyes, etc). So, you can manually move vertices if FaceBuilder's result doesn't suit you, remove or add faces, change normals and UVs, wrap a scanned model with our default model, etc.

Can I use custom blendshapes?

Not at the moment, but we are considering adding support for custom blendshapes in future.

Do I need to start tracking from a neutral face expression?

Not really, but it'd be easier if you can find a neutral expression to start from.

If I abort the tracking, the results would be lost?

No, the tracking information is saved frame by frame, so you can abort tracking at any moment keeping the tracking information you have already got intact.

I have a license for FaceBuilder, do I need a license for FaceTracker?

Yes, the licenses for FaceBuilder and FaceTracker are independent. You need both to work with both plugins.

How to export an animated model to a file?

You can use Nuke's built-in WriteGeo node with ABC format. After pressing "Execute" button specify the frames you want to export.