Written Guide

Last updated: Feb 2025

Introducing FaceTracker for Blender.

Get easy access to markerless facial mocap and 3D facial animation with custom head meshes generated by the built-in FaceBuilder module right inside of your favourite 3D creation suite.

Installation

If you already have any other KeenTools add-on installed in Blender, just update it. The new FaceTracker tab will appear on the sidebar (Shortcut: N — hide/show).

Blender 4.2: Download KeenTools Blender Pack, open Blender Preferences > Add-ons, click on the little arrow in the top right corner and select Install from Disk. Navigate to the downloaded zip file and install it. If you’re on Mac, find ZIP in the downloaded DMG file.

For Blender versions 2.8–4.1 check out this installation video

Next, install Core Library. You can do it either online or offline.

Unfold KeenTools extension and agree to the Licence Agreement. If your machine is connected to the internet, simply press Install online. Core Library will be downloaded and installed automatically.

For offline installation, download Core Library manually from our website. After download, press Install from disk and navigate to the downloaded file. Please keep in mind that the add-on and the core library versions should match.

Inputs

Start by pressing N to view the sidebar and then click on the FaceTracker tab to open the main panel. Press Create new FaceTracker. This will open the main panel with Inputs on the top. The fields marked in red are empty fields that must be filled to proceed to tracking.

Camera

By default, FaceTracker picks up the Camera from your Blender scene. If you accidentally delete it, you can create a new camera by clicking Add > Camera in the top left of the viewport.

You can plug a new camera into the Camera Input from the dropdown list or directly from the viewport with the picker (Shortcut: E).

Clip

Click on the folder icon next to the Clip field and load your footage. This can be either an image sequence or a movie file.

When loading an image sequence, FaceTracker reads the name of the first file in that sequence and if there’s a suffix, it sets it as the start frame on the Blender timeline. E.g. Sequence_1001.png will automatically set the playback to start off with frame 1001.

You can reposition the playback on Blender timeline by entering a specific frame number in the Start frame selector just below the Clip input. Be mindful that setting the start frame for your playback has to be done before analysing and tracking takes place.

For faster tracking you will need to analyse the clip. Press Analyse, select the frame range and hit OK. FaceTracker will scrub through the timeline forward and backwards and create an advanced motion vector file (.precalc ) on your disk, which you can use later for tracking other objects in the same shot. By default, it will be saved in your temporary folder on MAC or disk C on Windows. But, of course, you can choose a different location.

Head

You will need a face geometry with FaceBuilder topology to track a face in the clip. You can either plug an existing FaceBuilder head in here or create a new one on-the-go by pressing New.

Create Face Geometry

Once you press New, you jump into the face building part of the pipeline. The built-in FaceBuilder module allows you to create custom face mesh using snapshots of video frames and image files.

Take a snapshot of a video frame

Scroll through the timeline and choose a frame in which the actor’s face is seen clearly and looks most three-dimensional. A ¾ view usually works best. Once you’ve selected the frame, press Take Snapshot.

Customise your face mesh

FaceBuilder automatically detects the face and matches it with the background image. Although, sometimes it may look like a perfect match, it is still a rough estimation of the face shape which needs to be adjusted manually by dragging pins (red dots) in the viewport to line up the facial features. We suggest first deleting (right-click) the pins whose positions seem irrelevant. Create and drag (left-click) new pins until the mesh is by and large lined up.

Try to keep the number of pins as low as possible. Sometimes deleting a pin works better than creating a new one.

Tracking accuracy will greatly depend on the accuracy of your 3D model. You can improve your face mesh by adding more views. Press + snapshot to choose another frame with a different enough angle. Or add images from files by clicking on + img file. The most valuable angles for face building are: ¾, profile and front. For each new view a new camera will be automatically added to your scene and a new view button will appear in the Views tab. You can switch between the views simply by clicking on them. If you need to delete a view, click on the hamburger button on the right and select Delete.

Line up face mesh in each of the newly added views. We suggest using the Auto Alight option first, and then dragging the pins in the viewport until everything is lined up correctly.

FaceBuilder module is a seamless integration of FaceBuilder inside FaceTracker. You can actually go over to the FaceBuilder tab and ‘sculpt’ your face geometry from there, as well as get access to appearance settings (wireframe colour and opacity) in the Appearance tab. All changes you make in the integrated module are automatically applied in the FaceBuilder add-on and vice versa.

Once you’re finished face building, press Back to FaceTracker. Before you move on to tracking, you can check how your model looks in 3D and whether you’re happy with it. If you discover anything weird or positively wrong, you can always press Edit and fix that by going through the views, removing wrongly placed pins, lining up things that are off, removing or adding new views.

Learn about creating high-precision 3D heads with FaceBuilder in our detailed video tutorial and written guide.

Tracking

The actual tracking takes the following steps: model placement, tracking and refining.

Model Placement

Press Start Pinmode in the Tracking tab. What you need to do now is overlay your face geometry on top of the face in the initial frame. This does not necessarily have to be the first frame of your clip. Find a frame in which the face is seen clearly and has an expression most close to neutral. Press Auto Align for automatic model placement. Click and drag pins to match the mesh with the face. Left-click anywhere on the mesh to create new pins. Right-click on the pins you want to remove.

Note that pins are only used for adjusting the model’s position. Higher number of pins will not improve your tracking. What matters is the accuracy of the model placement.

When you press Auto Align or adjust mesh position, FaceTracker creates a new manual keyframe (green dotted line on the timeline) which stores the model position, rotation and facial expression in the current frame.

Wireframe colour and opacity are adjustable, as are the size and sensitivity of the pins. You will find the basic settings in the Appearance tab. More settings in Preferences > Add-ons.

Camera setup

The key to successful model placement and perfect tracking results is using correct camera settings. If you know the focal length and film size of the camera on the set, just type them in in the dedicated Camera tab. In case you don’t or unsure about your camera data, you can tick Estimate focal length checkbox to get that value calculated for you. Note that automatic focal length estimation works only for the current frame while you’re pinning the model, and you need to have at least 4 pins set on the wireframe to activate it.

Track & Refine

Select tracking mode depending on your needs:

Head — facial mocap + head motion.

Camera — facial mocap + camera track.

Click on the Track Forward button. You’ll see FaceTracker estimating head position and facial expression in the consequent frames. Whenever you see the mesh is off, pause tracking and adjust its position and facial expression manually using the existing pins or creating new if necessary. Every time you make manual adjustments FaceTracker will create a new manual keyframe and store the new position, rotation and facial expression data in it.

But the range between the manual keyframes needs to be updated too. You can update it automatically by pressing Refine. Keep on tracking and refining until you’re satisfied with your face track.

You can also track backwards or frame by frame using the tracking buttons in the Tracking tab.

The resulting face track is a combination of head movement in 3D space relevant to camera (or camera relevant to head, when Camera tracking mode is selected), and blendshapes animation representing the facial performance. The exact location and rotation of the 3D head in each frame can be seen in the Item tab. The facial animation is stored as blendshapes and can be accessed through the Shape Keys tab in Object Data Properties.

Make it Stable!

Improve tracking accuracy with Lock View

Sometimes it’s hard to assess the tracking quality especially in the dynamic shots. Use the Lock View button or keyboard shortcut L to stabilize the clip in the viewport. With no pins selected, FaceTracker will snap the view to your Head geometry. But you can also lock the view to a specific point on your 3D model by selecting one or a group of pins. Learn more about this feature in our video tutorial.

Masking

FaceTracker offers three types of masks to deal with almost any occlusion you might encounter: Surface, Compositing, and Sequence masks.

Surface mask allows you to select polygons right on the surface of the face geometry to exclude them from tracking. This can be useful when dealing with accessories like glasses, hats or hair falling in the face by taking out the parts of the face mesh interacting with them.

To apply a Surface mask, switch to Edit Mode and select the polygons you want to mask out, then go to Object Data Properties, create a new vertex group by clicking on +, and press the Assign button below. Return to Object Mode and select the newly created group in the Surface mask dropdown list. You can then switch to Pinmode to view the mask.

Another way of dealing with occlusions is 2D masking, which works for most overlays like the actor’s hand or props getting in the line of sight. You can either draw a 2D mask right in Blender using the Compositing mask option or import a pre-made mask using the Sequence option. Note that you can’t select both Compositing and Sequence masks within one tracking session. You can only use one at a time.

In order to use a Compositing mask, go to Movie Clip Editor and select your clip. Switch the Tracking mode to Mask and click on Add > Circle (or other shapes that fit your situation). You can now press Move and place your mask over the element you want FaceTracker to ignore when tracking. Learn more about editing masks in this Blender manual.

You can adjust your mask position in other frames by creating new keyframes. Note that the keyframes you create in the Movie Clip Editor are independent of those generated by FaceTracker. Once you’re happy with the mask shape and position, go back to the 3D viewport and select your mask in the Compositing mask tab. Go to Pinmode to see your mask in action.

If you have a pre-made mask, you can import it using the Sequence mask option. You can then choose which of the RGBA channels you want to use for masking.

Just like with the loaded clip, FaceTracker reads the name of the first file in the mask sequence and if there’s a suffix, it sets it as the start frame on the Blender timeline. E.g. Mask_1001.png will set the mask to start off with frame 1001. If you need to adjust its position on the timeline, enter a specific frame number in the Start frame selector below.

Scene Tab

You can easily manipulate your scene from the Scene tab on the FaceTracker panel. The Transform section lets you Scale and Position your scene with the two corresponding buttons. You can scale either the whole scene relative to selected Pivot point or Geometry relevant to Camera, or Camera relevant to Geometry. And when doing the latter, you can also lock the camera visualisation scale.

You can select either Camera or Geometry as the pivot point to position your scene in 3D space. You can set the Location and Rotation values manually or use preset positions such as World Origin and 3D Cursor.

Go to the Animation section when you need to convert your Camera animation to Head or vice versa. Note that if both the Camera and Head are animated, pressing the Convert button will convert all animation to one, and then this operation will not be reversible!

Press Bake if you need to unparent Camera and/or face geometry. This will bake them, along with their animation, to World space.

Important! If your model rotates more than 360° between certain keyframes, we recommend using the Unbreak Rotation option. The algorithm behind it will go through all keyframes and make rotation continuous thus avoiding 360 degree jumps. It means that if rotation was 355° in the keyframe #24 and 10° in the keyframe #25, pressing this button will add 360° to keyframe #25, so the resulting rotation of the model will be 370°. It works backwards as well: going from 10° to 355° will make it -5°.

You need to be careful if your model rotates both forward and backwards between the nearest keyframes — then this operation will most likely make things worse.

Texturing

The Texture tab lets you create an animated texture map or texture blended from selected frames using background video as source.

When in Pinmode, unfold the Texture tab, scroll through the timeline and add source frames by pressing + next to the Add frames field. Use — to remove a frame from the list.

For best results, choose frames where the face is clearly seen from different enough angles. Avoid frames that look blurry and dark.

As you click Create Texture, FaceTracker projects the texture from the selected frames and bakes it onto your face geometry. Click the gear icon to access Texturing settings. Here, you can adjust the size of your texture and also set up things like Angle Strictness, allowing you to control how much the pixel colour will be affected depending on the viewing angle, and Expand Texture Edges which lets you cover dark spots on the texture by expanding the edges.

Click Export to get your texture as a PNG, JPEG, or EXR file.

Press Save to Sequence to export an animated texture map.

Animation Retargeting

You can transfer facial animation to a 3D character from the Export tab. Facial animation can be converted either to ARKit blendshapes or a Rigify rig with the dedicated toggle switch. Select the target object or rig in the dropdown or using the eyepicker and press Convert. You can also export animation data as a .csv file by pressing the Save as .csv button to use it on other 3D characters or digital doubles outside Blender.

Note that while head motion is represented by the xyz translation and rotation of the 3D model accessible through the Item tab, facial animation is essentially the change of the model’s shape from one frame to another.

Facial animation is stored as Shape Keys in Object Data Properties. Each of these shape keys account for the overall shape per frame. You can get more controls in for your face track if you go to the FaceBuilder tab > ARKit Blendshapes tab and press Create. This will create 51 additional shape keys based on FACS (facial action coding system) allowing you to adjust emotions and facial expressions dynamically blended with the face action of your 3D model by changing the Value parameter of each shape key.

Render Wireframe

You can render playback with overlaid wireframe or just animated wireframe by pressing the Render Wireframe button. In the render output panel you can adjust controls like wireframe line width and color, and use options like Lit wireframe, Backface culling, Highlight facial features. If you need to render only animated wireframe without the playback, deselect Use background. It will be rendered out as a series of images according to the selected file format: PNG, JPEG or EXR.

Other export options

In the Export tab you can export one or multiple Empties by specifying the source in the dropdown menu and pressing the Export button below it:

Head — head animation

Camera — camera animation

Selected pins — one or multiple Empties (drop a pin on the mesh and set its orientation before export)

Check Linked to keep exported animation synced with further changes in the tracking data.

Video tutorials

Check out these video guides to learn more about FaceTracker for Blender.

Find more tutorials on our YouTube channel!

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

FaceTracker for Blender Guide was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

Create a digital copy of the head in FaceBuilder

FaceBuilder for Blender lets you make fully-textured 3D heads based on photos. Inside Blender, go to the FaceBuilder tab, click on Create New Head and load images using the Add Images button. Check Allow facial expressions and Lock neck movement.

Creating heads with FaceBuilder is semi-automatic. Select the top image on the list and click Align Face. FaceBuilder will detect the face in the image and estimate the position of the mesh in 3D. The next step is to line up the mesh with the face manually. Drag the existing pins or add new ones to accurately match the facial features. Pay special attention to eyes, brows, nose and mouth. This will help you get a better texture in the end. Repeat the same operations for the remaining images.

The next step is projecting texture from the images onto Geometry. Select maxface UV type in the Texture tab, then go to Advanced and check Equalize brightness and Equalize colour to smooth the texture and make it more even. Then click on Create Texture and hit OK. The digital copy of the head is now ready for export to Character Creator. Go to the Export tab and hit the Character Creator button. FaceBuilder will run the software and transfer the 3D head model and the texture right into it.

Important! You need to have the latest versions of Character Creator and Headshot plug-in installed on your computer for this operation. Visit the official Reallusion website to download the software.

Generate body in Character Creator

Character Creator offers extensive settings to give your 3D character a unique appearance. We’ll stick to a very basic setup to demonstrate a quick path to creating a character.

Export from FaceBuilder to Character Creator gets you to the final stage of generating a head called REFINE. Here you can explore the mesh in the viewport and adjust everywhere it doesn’t quite match up using a number or brush tools. For example, click on the Project button and go over those areas on the mesh that didn’t fill out right versus the FaceBuilder profile. You can speed up this process using the Symmetry Edit option that will mirror your adjustments to the other side of the face. As soon as you like the way your head looks, click on ATTACH TO BODY.

In the popup first select a body type for your character and also Mask type, which is basically how much of the original texture you want to use. You can go with No Mask to use the full head at this point and adjust that later. Click on Generate to finally put the head and the body together.

Customise your 3D character

Here are some effective steps on customising your 3D character to make it look like the person in the original photos.

Go to the Content tab to set up the overall appearance of your character. For example, you can choose a hairstyle in the Hair tab or dress up your model in the Clothes tab. You can also add and remove various parts of your model in the Scene tab. The Motion button lets you quickly and easily select a pose for your character.

Control your character likeness with the Headshot plug-in

The Headshot 2 plug-in lets you tweak every single element of your head model inside Character Creator. Go to the Headshot tab, scroll down to the bottom, and activate Auto Update to have all your changes updated automatically. Inside this tab, you will find some very useful texture settings.

Skin

You can select Skin Type from the number of presets and also change skin colour using the colour picker. You can also localise the texture imported from FaceBuilder by specifying Face Mask in the Skin Settings section.

A little tip. Try different options to see which mask works best for your head, paying attention to ears, cheeks, and forehead. Likewise, you can improve texture around the nose and mouth by choosing a Mouth and Nostril Mask. Selecting an Eyelid Mask may help you fix issues like eyelashes baked in the texture.

Eyes

You can select Eye Color at the very top of the Texture Adjustment section. But there’s a lot more you can do with the eyes in the Morphs tab on the Modify panel. Go to Actor Parts and search for Eyeball Iris Scale for example to adjust that value. Back in the Headshot tab, you can go to Shape Adjustment, check Activate Sculpts Morph, click on the Eyes button and then adjust their position simply by dragging them right in the viewport. You can then rotate the head side-on, zoom in on the eye and adjust its depth.

Hair

Go to the Scene tab, select Hair Base, then go to the Texture tab on the Modify panel, select Hair in the Material List, then scroll down to Shader Settings. Here in Strand Color you can change the colour of all main hair parts. If the hairstyle contains more elements, you can adjust their colour in the same way by selecting them in the Scene tab.

A little tip. You can change the scale and size of all body parts in the Morphs tab on the Modify panel. Use search for specific elements, for example, Head Scale, Neck Scale, and Shoulder Top Scale. You can also go to the Attribute tab, make your model open the mouth with the Display Mouth button, and then back in the Morphs tab, search for elements like upper and lower teeth to change their height and scale.

Export to Blender

Go to File > Export > FBX and select the Clothed Character option. In the export menu select Blender as destination, then select Mesh in the FBX Options, uncheck Embed Textures, and then click on Export and save it to disk as model.fbx.

To get full control over your CC4 character we recommend installing the Character Creator add-on. Go to Add-ons in Blender Preferences and activate the Rigify option to be able to use the local rig. Download CC/iC Tools from GitHub. After you install it, you’ll get a new tab named CC/iC Pipeline on your side panel. Go to that tab, click on Import Character, navigate to the model.fbx file, and hit Import.

Explore Character Creator Add-on settings

The Character Creator add-on gives you access to many useful settings. You can go to the Scene tab and select a lighting scheme, like CC3 for example. This will make the materials inside your model visible in the viewport. Another thing you will surely need for your model is a rig. Go to the Rigging and Animation tab and click on the Rigify button. This way you get control over all elements of your model inside Blender.

A little tip. If you plan to export your model further on to other 3D environments, it makes sense to optimise the materials applied to your model. Go to the Character Build Settings tab, select Basic, and then click Rebuild Basic Materials. This will create a simplified version of the materials for your model.

Animate your model

The quickest way to animate your model is to use animation export from Character Creator. Inside Character Creator, click on the Motion button and select a specific animation. You can preview it by hitting the Play button right below the viewport. If this is the way you want your character to move in Blender, go to File > Export > FBX> Clothed Character. Select Blender as a destination once again, and then choose Motion in the FBX Options. You will also need to activate Current Animation and All checkboxes, and also deselect First Frame in Bind-Pose. Now hit Export and save it as motion.fbx.

Back in Blender, on the CC/iC Pipeline panel, click on Import Animations, select motion.fbx, and hit Import. Then go to the Rigging and Animation tab, select the animation you’ve imported in Source Armature and also Source Action, and then hit Preview Retarget. Preview the imported animation by hitting Play.

For more info, check out our dedicated webpage on FaceBuilder x Character Creator 4 integration.

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

FaceBuilder for Blender and Character Creator 4 Integration Guide was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

Written Guide

Last updated: Feb 2025

Geometry-based object and camera tracking add-on for your favourite 3D creation suite. Easily track objects in your videos and bring them to life in 3D space, all within Blender.

Installation

If you already have any other KeenTools add-on installed in Blender, just update it. The new GeoTracker tab will appear on the sidebar (Shortcut: N — hide/show).

Blender 4.2: Download KeenTools Blender Pack, open Blender Preferences > Add-ons, click on the little arrow in the top right corner and select Install from Disk. Navigate to the downloaded zip file and install it. If you’re on Mac, find ZIP in the downloaded DMG file.

For Blender versions 2.8–4.1 check out this installation video

Next, install Core Library. You can do it either online or offline.

Unfold KeenTools extension and agree to the Licence Agreement. If your machine is connected to the internet, simply press Install online. Core Library will be downloaded and installed automatically.

For offline installation, download Core Library manually from our website. After download, press Install from disk and navigate to the downloaded file. Please keep in mind that the add-on and the core library versions should match.

Inputs

Start by pressing N to view the sidebar and then click on the GeoTracker tab to open the main panel. Press Create new GeoTracker. This will open the main panel with Inputs on the top. The fields marked in red are empty fields that must be filled to proceed to tracking.

Camera

By default, GeoTracker picks up the Camera from your Blender scene. If you accidentally delete it, you can create a new camera by clicking Add > Camera in the top left of the viewport.

You can plug a new camera into the Camera Input from the dropdown list or directly from the viewport with the picker (Shortcut: E).

Clip

Click on the folder icon next to the Clip field and load your footage. This can be either an image sequence or a movie file.

When loading an image sequence, GeoTracker reads the name of the first file in that sequence and if there’s a suffix, it sets it as the start frame on the Blender timeline. E.g. Sequence_1001.png will automatically set the playback to start off with frame 1001.

You can reposition the playback on Blender timeline by entering a specific frame number in the Start frame selector just below the Clip input. Be mindful that setting the start frame for your playback has to be done before analysing and tracking takes place.

For faster tracking you will need to analyse the clip. Press Analyse, select the frame range and hit OK. GeoTracker will scrub through the timeline forward and backwards and create an advanced motion vector file (.precalc ) on your disk, which you can use later for tracking other objects in the same shot. By default, it will be saved in your temporary folder on MAC or disk C on Windows. But, of course, you can choose a different location.

Geometry

Select Geometry that you will use to track the target object in your clip. By default, GeoTracker picks the cube from the Blender startup scene. You can use any model from your scene, including imported 3D models (File > Import). Click on the field next to Geometry to select it from the dropdown list, or use the object picker (Shortcut: E) to select it right in the viewport.

Similar to Geometry, GeoTracker picks up the default Camera from your Blender scene. If you accidentally delete it, you can create a new camera by clicking Add > Camera in the upper left of the viewport. You can always pick the right camera from the dropdown list or use the picker (Shortcut: E).

Before moving on to tracking, it’s important to press Analyse and hit OK. GeoTracker will analyse the clip and save the analysis data as a .precalc file on your disk. By default, it will be saved in your temporary folder on MAC or disk C on Windows. But, of course, you can choose a different location. This will make tracking much faster, though you can skip it by deselecting Use analysis cache file.

Pinning

Next step is overlaying Geometry on top of the target object in the clip. Press Start pinmode in the Tracking tab. Click anywhere on the mesh to create the first pin. Drag it to align the model with the object you’re going to track. Create more pins and adjust their positions to ensure a precise fit. Right-click on the pins you want to remove. Note that pins are only used for adjusting the model’s position. Their number and exact positions will not affect your tracking.

Wireframe colour and opacity are adjustable, and so are the size and sensitivity of the pins. You will find basic settings in the Appearance tab. More settings in Preferences > Add-ons.

Camera setup

The key to good tracking results is correct camera settings. If you know the focal length and film size of the camera on the set, just type them in in the dedicated Camera tab. In case you don’t or unsure about your camera data, you can tick Estimate focal length checkbox to get that value calculated for you. Note that automatic focal length estimation works only for the current frame while you’re pinning the model, and you need to have at least 4 pins set on the wireframe to activate it.

Tracking

Select tracking mode depending on your needs using these buttons in the Tracking tab:

Geometry — to follow the motion of an object

Camera — to track the camera position in a physical space (e.g. when tracking static objects).

Click on the Track Forward button. You’ll see GeoTracker estimating the position of the target object in the consequent frames. Whenever you see the mesh is off, pause tracking and adjust its position manually using the existing pins or creating new ones if necessary. GeoTracker will store the adjusted position and rotation by creating new keyframes. Note that the manual keyframes created this way do not affect the tracking data stored in the other keyframes. Press Refine to update the tracking data after each adjustment. Keep on refining your tracking until the motion of your 3D model is smooth and accurate.

Improve tracking accuracy with the Lock View option

Sometimes it’s hard to assess the tracking quality especially when dealing with dynamic shots. Use the Lock View button or keyboard shortcut L to stabilize the clip in the viewport. With no pins selected, GeoTracker will snap the view to your Geometry. But you can also lock the view to a specific point on your 3D model by selecting one or a group of pins. Learn more about this feature in our video tutorial.

Masks

Oftentimes various elements in the video get in the way of tracking, and that results in Geometry slipping away from the target object. Masking is a technique that lets you exclude problematic areas from tracking. GeoTracker offers not just one but three types of masks to deal with almost any occlusion you might encounter: Surface, Compositing, and Sequencemasks.

Surface mask allows you to select polygons right on the surface of your Geometry to exclude them from tracking. This can be useful when dealing with reflection or when you need to take out certain elements, like adjacent parts, of your 3D model from tracking.

To apply a Surface mask, switch to Edit Mode and select the polygons you want to mask out, then go to the Object Data tab in the Properties, create a new vertex group by clicking on +, and press the Assign button below. Return to Object Mode and select the newly created group in the Surface mask dropdown list. You can then switch to Pinmode to view the mask.

Another way to fight off occlusions is 2D masking, which works for most overlays like the character’s hand or other elements in the foreground crossing the path of the track. You can either draw a 2D mask right in Blender using the Compositing mask option or import a pre-made mask using the Sequence option. Note that you can’t select both Compositing and Sequence masks within one tracking session. You can only use one at a time.

In order to use Compositing mask, go to Movie Clip Editor and select your clip. Switch the Tracking mode to Mask and click on Add> Circle (or other shapes that fit your situation). You can now press Move and place your mask over the element you want GeoTracker to ignore when tracking. Learn more about editing masks in Blender on this page.

You can adjust your mask position in other frames by creating new keyframes. Note that the keyframes you create in the Movie Clip Editor are independent of those generated by GeoTracker. Once you’re happy with the mask shape and position, go back to the 3D viewport and select your mask in the Compositingmask tab. Go to Pinmode to see your mask in action.

If you have a pre-made mask, you can import it using the Sequence mask option. You can then choose which of the RGBA channels you want to use for masking.

Just like with the loaded clip, GeoTracker reads the name of the first file in the mask sequence and if there’s a suffix, it sets it as the start frame on the Blender timeline. E.g. Mask_1001.png will set the mask to start off with frame 1001. If you need to adjust its position on the timeline, enter a specific frame number in the Start frame selector below.

Scene Tab

You can easily manipulate your scene from the Scene tab on the GeoTracker panel. The Transform section lets you Scale and Position your scene with the two corresponding buttons. You can scale either the whole scene relative to the selected Pivot point or Geometry relevant to Camera, or Camera relevant to Geometry. And when doing the latter, you can also lock the camera visualisation scale.

You can select either Camera or Geometry as the pivot point to Position your scene in 3D space. You can set the Location and Rotation values manually or use preset positions such as World Origin and 3D Cursor.

The Rig button lets you create an Empty with parented Camera and/or Geometry that you can use as a handle when building your 3D scene.

Go to the Animation section when you need to convert your Camera animation to Geometry or vice versa. Note that if both the Camera and Geometry are animated, pressing the Convert button will convert all animation to one, and then this operation will not be reversible!

Press Bake if you need to unparent Camera and/or Geometry. This will bake them, along with their animation, to World space.

Important! If your model rotates more than 360° between certain keyframes, we recommend using the Unbreak Rotation option. The algorithm behind it will go through all keyframes and make rotation continuous thus avoiding 360 degree jumps. It means that if rotation was 355° in the keyframe #24 and 10° in the keyframe #25, pressing this button will add 360° to keyframe #25, so the resulting rotation of the model will be 370°. It works backwards as well: going from 10° to 355° will make it -5°.

You have to be careful if your model rotates both forward and backwards between the nearest keyframes — then this operation will most likely make things worse.

Texturing

The Texture tab lets you grab texture from the tracked object in the video and then bake it onto the 3D model. You will first need to choose which frames you want GeoTracker to take the texture from. Go to the source frame on the timeline and press +. The snapshotted frame will then appear in the Add frames list. If you want to remove it, click -.

For best results, choose frames where the object is clearly seen from different angles to provide GeoTracker with ample data for creating texture. Avoid frames that are blurry or have something in the foreground that blocks the view of the object.

As you click Create Texture, GeoTracker projects the texture from the selected frames and bakes it onto your Geometry. Click the gear icon to access Texturing settings. Here, you can adjust the size of your texture and also set up things like Angle Strictness, allowing you to control how much the pixel color will be affected depending on the viewing angle, and Expand Texture Edges which lets you cover dark spots on the texture by expanding the edges.

Click Export to get your texture as a PNG, JPEG, or EXR file.

You can also save the texture from all frames as an animated sequence by clicking the Save to Sequence button.

Export

Empty

In the Export tab you can export one or multiple Empties by specifying the source in the dropdown menu and pressing the Export button below it:

Geometry — geometry animation

Camera — camera animation

Selected pins — one or multiple Empties (drop a pin on the mesh and set its orientation before export)

Check Linked to keep exported animation synced with further changes in the tracking data.

Render Wireframe

You can render playback with overlaid wireframe or just animated wireframe by pressing the Render Wireframe button. In the render output panel you can adjust controls like wireframe line width and color, and use options like Lit wireframe, Backface culling, Highlight facial features. If you need to render only animated wireframe without the playback, deselect Use background. It will be rendered out as a series of images according to the selected file format: PNG, JPEG or EXR.

Video tutorials

Check out these video guides to find out more about GeoTracker for Blender.

More tutorials are on their way!

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

GeoTracker for Blender Guide was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

Out of the box 3D head tracking in AE

The new 5th primitive — generic head model — has been added to the existing 4: Plane, Sphere, Cube and Cylinder. From now on, when you track the character’s head in your shot to either replace it, dress it up or attach anything to it, you don’t need to go out searching for a random 3D model. Just pick the head primitive, adjust its width and height, align it with the character’s head and then track it! That should be enough for most cases.

If you’re after precision, we suggest using FaceBuilder for Blender to create more accurate head models that can be passed directly to GeoTracker.

In addition, all primitives now can be exported from GeoTracker as .OBJ or .FBX with the same parameters as in your After Effects project.

Surface Mask

2D masks are good friends for many After Effects users and help greatly to mask out overlapping elements that go over the tracking area. But what are you going to do when the object in the video has elements that get in the way of tracking, like reflections on car windows or a fringe bouncing on the character’s forehead? Or when you want to drop off adjacent parts of the loaded 3D model. All those things can make tracking go off the rails!

For tasks like these, we’ve added a new type of mask — Surface mask — that lets you select certain polygons of the 3D model and exclude them from tracking.

Watch our new video tutorial to learn more about masks in GeoTracker.

Nuke package and FaceBuilder for Blender

Though this release touches primarily on GeoTracker for After Effects, Nuke package and FaceBuilder for Blender have also undergone updates and got some bug fixes and minor improvements, so don’t forget to update them too! We also did some preliminary work for further releases. Something exciting is coming!

Download links

– GeoTracker for After Effects
– FaceBuilder for Blender
– Nuke Package

We crave your feedback, so please, talk to us: team@keentools.io

KeenTools 2023.1: Track heads in After Effects! was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

GeoTracker for After Effects: Stable version

This September we released GeoTracker Beta for AE — the first 3D object tracker for After Effects. We’re extremely grateful to you for the invaluable feedback we’ve been receiving from you all this time.

This led to 5 free beta versions in total, each gradually improving the plug-in: we’ve added tracking smoothing settings, ExtendScript support, and fixed many minor issues. GeoTracker now works even in the latest After Effects 23.2 Beta which, by the way, now has a proper built-in 3D space and native support for 3D objects!

Today we release a stable version of GeoTracker, which means it becomes a paid tool with a 15-day trial period for everyone regardless whether they’ve already tried Beta or not. Our subscriptions are cross-platform, so if yours is an active one for GeoTracker for Foundry Nuke, it will work in After Effects too.
Please, find the pricing for annual and monthly plans on our Buying page.

As early adopters, you can subscribe to annual plans with discounts for the first year up until March 16 using the promo codes below:

• Freelancer Annual (node-locked subscription for individuals): FREELANCER-AE for $̵1̵7̵9̵ $149 (excl. VAT)
• Studio Annual (floating/node-locked subscription for companies): STUDIO-AE for $̵4̵9̵9̵ $399 (excl. VAT)

For a quick start, please check our tracking basics and export options tutorials. A detailed written guide is also there! Additionally, we’ve prepared two more videos on the two long-awaited integrations: Cinema 4D and Element 3D.

Nuke 14 support

Recently, Foundry upgraded their 3D system with the release of Nuke 14. In our 2022.3 update, we’ve made our tools compatible with the latest Nuke. So far, they work only as classic 3D nodes, but we plan to upgrade them in the near future. Stay tuned!

Licensing updates

Lately, at KeenTools we’ve upgraded our licensing by bringing out proper subscriptions that require no manual update, along with the new plans including monthly options for individuals. Please read the full announcement here.

Note: Current update includes better support for new subscription keys. It’s important to upgrade to 2022.3 to make all features work properly.

Download links

— GeoTracker for After Effects
 — FaceBuilder for Blender
 — Nuke Package

As always, we’re hungry for your feedback, so don’t hesitate to send your messages to team@keentools.io.

Merry Christmas and Happy New Year!

Yours,
KeenTools Team

KeenTools 2022.3: Stable Release of GeoTracker for After Effects, Nuke 14 Support was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

Brief summary:

• Proper subscriptions — renewed automatically
• Monthly plans for individuals
• ‘Personal’ licenses replaced by the Freelancer plan
• ‘Floating’ and ‘Commercial’ licenses replaced by the Studio plan
• Active licenses remain active
• Prices are increasing
• Steps to take

New licensing

From now on, two subscription types are available for each of our products:

Freelancer. For individuals only, allows commercial usage. Products under this subscription type can be used on a single machine (node-locked), there are annual and monthly options.

Studio. Standard choice for companies. This type gives you floating subscription keys that can be shared across an unlimited number of machines using an instance of our license server running in your network: one key for one computer at a time. It can be used as node-locked too.

All old licenses remain active until expiration dates.

Please note that we’re going away from ‘Commercial’ licenses (node-locked) because they weren’t popular and often led to confusion. However, we’ll keep supporting all current users of ‘Commercial’ licenses. In a short while, they will receive an email from us with a special offer to switch to Studio with a discount price for the first year of the subscription equal to the previous annual ‘Commercial’ price.

When purchasing the ‘FaceBuilder + FaceTracker’ bundle, you’ll receive a single subscription key for both products. After the key is activated, both plug-ins installed on one machine will be able to use the key. These products can’t be used on different computers at the same time using one FaceBundle key. That also applies to the floating license server.

New pricing

For 7 years since the foundation of KeenTools, we’ve kept our prices intact. It’s been our major concern to postpone the increase for as long as we could. We understand that this is always an unpopular measure, but we have to make this difficult step now to support and improve our current products and, more importantly, work on the new releases.

The new prices are listed on our Buying page.

For all users we offer a 3-month period until March 16, during which you can subscribe to any of our annual plans for the old prices for the first year. Use these coupons:

• Freelancer — PERSONAL-TO-FREELANCER
• Studio — FLOATING-TO-STUDIO

We will be reaching out to all our customers over the next few weeks with this update.

Important! If your current license ends in more than 3 months and you would like to renew it for the next year under the previous conditions, we can solve it if you contact us at team+prorate@keentools.io. Please provide the following details:
- License type
- Order ID
- Email address you used for this purchase

Example: your current ‘Personal’ FaceBuilder license ends on July 25, and you want to update to the new subscription on December 25. That means you still have 7 months of active subscription.

We’ll take the price of the old ‘Personal’ annual subscription, subtract the part reflecting your remaining 7 months from it, and this will be the price of your new FaceBuilder Freelancer annual plan for the first year. You’ll be provided with a personal discount coupon, your old licenses will be terminated afterwards.

The offer is valid until March 16, 2023.

Proper subscriptions

Apart from changing names and types, we finally get to proper subscriptions. Now you don’t need to re-purchase and re-install new license keys every year. Once you start your subscription, it’ll be valid until you decide to cancel it. No reactivation or any other actions needed on your side. All users can now manage all their subscriptions on our User Portal, which also allows downloading offline keys for machines not connected to the Internet.

Monthly subscriptions for individuals

Following multiple requests from our individual users, we’re introducing a new monthly subscription option for all of our products. Find the prices on our Buying page.

Steps to take

Here’s a simple algorithm to tell you if you need to take any actions:

1. You haven’t purchased our licenses before, or they are expired → simply subscribe now
2. You have an active license:

• Your licenses expire before March 16 → wait until they expire and subscribe to a new plan using an old price coupon from above
• You have licenses expiring after March 16 → let us know to update them at old pricing via team+prorate@keentools.io. This offer is valid until March 16.

In any case, we’re always there to answer any questions, don’t hesitate to drop us a message at team@keentools.io!

Thank you for your support,
KeenTools Team

Updated licensing: new subscription types, increase in prices, proper annual and monthly plans was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

True 3D results

Wait, there’s no real 3D in After Effects, right? Yes and no. Tracking with GeoTracker happens as usual — in the 2D viewport, but when tracking is done, you get both 2D and 3D transformations as results.

You can apply effects using 3D or 2D coordinates, or pass the track through to Element 3D, Cinema 4D, particle generators and other true-3D plugins for After Effects.

3D models

GeoTracker can load FBX, OBJ and DAE files as 3D models to track. As for another option, you can always use four customisable built-in primitives: cylinder, sphere, plane and cube.

Focal length estimation

Focal length estimation is included as well, so if you don’t know camera settings or the focal length is changing in the footage, GeoTracker can handle that.

Occlusions and reflections

Those who tried GeoTracker in Nuke know that it deals with occlusions and reflections quite well on its own. For difficult cases, we made it possible to use native After Effects’ 2D masks, so you can exclude confusing areas from tracking.

Open beta

GeoTracker is in open beta now, so it’s completely free until we release a stable version. No registration required — just download, install and have fun!

We crave your feedback as it’d help us make GeoTracker a top-notch plug-in for After Effects. Please don’t be shy and bombard us with your ideas and bug reports!

Links

Download GeoTracker for After Effects from our website.

To get started, please watch these brief tutorials: Basics, Exporting results.

For those who like reading more, here’s the written guide. More tutorials are on the way!

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

Meet an Open Beta of GeoTracker for After Effects was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

Installation and setup

Download GeoTracker from our website, make sure After Effects is closed. Launch the installer and follow the instructions. Once the installation is complete, GeoTracker will appear in the Effects & Presets list.

Removing GeoTracker from your system is also pretty simple. If it was installed via the downloaded package or installer, on macOS: run the uninstaller from Applications/Utilities. On Windows: look for the entry in Programs and Features. Don’t forget to close After Effects before uninstalling GeoTracker. In case it was installed manually on Windows or macOS, just delete the files.

For a quick start, add footage to the timeline and apply the GeoTracker effect. The plug-in will automatically start analyzing the video, the progress bar can be seen in the Effect Controls panel.

Built-in primitives

Use one of the built-in primitives to track something of a basic three-dimensional shape:

Plane — for screens, walls or any flat surfaces
Sphere — for balloons, sport balls, heads and other spherical objects
Cube — for boxes, vehicles, buildings, etc
Cylinder — for bottles, cups, and so on

GeoTracker 2022.3.1 and higher features a built-in head primitive. Try FaceBuilder for Blender, which lets you create precise head models from images or even a single picture.

Custom models

Load your 3D model by clicking the dedicated button in the Geometry collapsible tab. Supported formats are FBX, OBJ, and DAE. Once the model is loaded, you’ll see its wireframe on top of the image in the viewport — this overlay is visible only while the effect is selected.

Pinning

The next step is placing the model in the initial position. Click anywhere on the mesh to create the first pin. Drag it to align the model with the object you’re going to track. Create and drag more pins to stretch the model and make it fit the object the best you can. Right-click on the pins you want to remove.

The colour and opacity of the wireframe are adjustable. You can customise its look in the Appearance tab.

By dragging the pins you change the model’s position and rotation in 3D space. You can do the same by changing values for position and rotation in the Results tab. Using pins, however, lets you align the model with the object right on the image, which makes it a lot easier and quicker. Note that pins are only used for adjusting the model’s position. Their number and exact positions will not affect the tracking.

Camera

Successful tracking depends a lot on the correct camera settings. By default GeoTracker picks the camera from After Effects, which is called Active Camera and is the upper layer among all layers in the composition. It’s good if you know which focal length and film size were used to film the video. Then you can create a new camera layer, go to camera settings, and just type them in. The newly created camera becomes active and can be found in the topmost camera layer in the composition, but you can select other camera layers from the collapsible Camera menu.

If you don’t know the exact camera settings in the footage, you can switch Camera Source to Custom mode and try to calculate them by ticking the Estimate Focal Length checkbox. By default, Custom camera uses Fixed Lens mode, assuming that the focal length value remains unchanged throughout the video. If you switch Fixed Lens to Zoom, you will notice the new Track focal length checkbox, which allows you to track both the object and the changing focal length at the same time.

Tracking

Once your model is in place, the camera is set, and the footage analysis has finished, you can move on to tracking.

Press the Open Toolbar button to see the GeoTracker panel that contains the main tracking controls. Click on the track button (Play), you’ll see the model following the object in the video, which means the tracking is in progress. Whenever you see the mesh is off, pause tracking and adjust the wireframe position with the existing pins or create new ones if necessary. GeoTracker will store the new position data automatically by creating a new keyframe. Note that the keyframes you’re creating this way don’t affect the tracking data in other frames, so the range between the keyframes will still contain an old tracking information. Press the Refine button in the Toolbar to update tracking data. This will make the model’s movements between keyframes more smooth.

You can quickly jump between keyframes using keyframe navigation buttons in the Toolbar. Continue tracking, refining, and adjusting keyframes until the model’s movement recreates that of the object as accurately as possible in every frame. We don’t recommend creating too many keyframes, though, to avoid jerky movements.

Masks

Most often tracking goes smoothly. But sometimes various elements in the video get in the way like the character’s hand going over the tracking area or hair falling in the face and so on. At that moment, the primitive or model you’re using may bounce off the object, and your tracking goes rough. In this case, you can use two types of masks to exclude problematic areas from tracking.

For 2D masks, create an adjustment layer and draw a mask around the element that needs to be masked out. Then unfold the 2D Mask collapsible menu and select that layer in the Layer drop-down list. Choose the source for that mask in the Source drop-down list. For drawn masks, select Mask and leave Alpha in the Channel menu.

Surface mask lets you select the polygons you want to mask out right on the 3D model. This can be helpful when you deal with reflections on car windows or if you want to use only a certain part of your 3D model for tracking without the adjacent elements, for example. Go to Surface Mask, pick the brush, and mark those polygons you want GeoTracker to skip while tracking. If you click on something you didn’t mean to, use the Eraser to deselect the marked polygons. If you want to clear all, hit the Clear Mask button. Additionally, there is an Ignore checkbox, which allows you to switch the mask off for the chosen frames or even the entire video and ignore it in tracking.

Exporting results

Tracking results can be exported as a Null layer which can be further used in your pipeline. Go to the Export menu, unfold the Object Track tab and press the Export Null button. This will create a Null containing all your tracking data.

You can export null objects as 2D layers for 2D animation purposes by selecting the 2D mode in the Object Track tab.

If your task is narrowed down to attaching a CG element to a point on the surface of the object, you can export your results through the Surface Point tab.

By default, the null object is normal-oriented, which means its Z-axis is perpendicular to the surface of the tracked object. However, if you want your null object to be oriented in the same way as your model, you need to switch Orientation from Normal to Object.

When exporting tracking results, make sure the camera in GeoTracker matches the camera in the Composition. Otherwise, the transform data of the exported Null will be incorrect. You don’t need to export the default Active camera. However, if you use a Custom camera, you need to export it along with the tracking results by clicking the Export Camera button in the Camera tab.

Links:

Download GeoTracker for After Effects from our website (15-day registration-free trial included). Monthly and annual plans can be found here. Watch the tracking basics video and check out the detailed export video guide. Additional video tutorials on two integrations: Cinema 4D and Element 3D.

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

GeoTracker for After Effects Guide was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

This release brings a groundbreaking improvement to FaceBuilder — now, when you drag pins, they affect nearby regions in a smart way, leaving the unrelated facial parts unchanged. With this improvement, pining becomes more detailed, for example, lip corners, curved noses and skin folds are not a problem now. As an outcome, 3D models resemble people from photos far better than before, increasing the variety of potential use cases for FaceBuilder.

One of them can be creating a MetaHuman out of a real person. As you must have heard, Epic Games recently released the Mesh to MetaHuman plugin which can wrap any 3D model of a human head and create a MetaHuman character based on the wrapped mesh. This of course means that you can also load a FaceBuilder into Unreal Engine and create a MetaHuman out of it. But to make the union of FaceBuilder and MetaHuman even better, in this release we added the new ‘MH’ texture map which is compatible with the texture layout of MetaHuman. It gives you a way to improve the MetaHuman texture by transferring facial features such as birthmarks or scars from photos with the help of the texture generated in FaceBuilder. The whole procedure is covered in this guide.

In conclusion, the mix of FaceBuilder and MetaHuman is perhaps the easiest way to transfer a real person to a game engine with the highest level of likeness!

Speaking about our package for Nuke, apart from the refreshed FaceBuilder, users might be pleased to find updated transform handles (aka Axis controls) in all of our 3D nodes — their reliability was improved in 3D mode, and now they can be used in 2D mode too. By the way, it’s worth mentioning that since this update, our plugins for Nuke 11 and earlier will receive only maintenance updates. All new features will be available in Nuke 12+.

The full list of features in FaceBuilder for Blender:
– FaceBuilder can reconstruct faces with finer details
– Added MetaHuman-compatible UV layout
– FaceBuilder now supports many heads in one scene
– Many minor fixes and UI tweaks

The Nuke package:
– FaceBuilder can reconstruct faces with finer details
– Added MetaHuman-compatible UV layout to FaceBuilder
– GeoTracker, FaceTracker: added transform handles in 2D
– FaceBuilder, FaceTracker: fixed automatic face alignment for anamorphic footage and translated cameras
– Nuke 11 is deprecated
– Many minor fixes and UI tweaks

Download the new FaceBuilder for Blender and the updated Nuke package from our website!

KeenTools 2022.2: The next level of likeness in FaceBuilder, MetaHuman compatibility was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.

FaceBuilder: Creating a head model

First, let’s download and install FaceBuilder for Blender. Once it’s done, create a FaceBuilder head and load some photos. You can see them in the stack on the right tab.

Now you can start the process we in KeenTools call “Pinning”: click one of the buttons with an image file name on the Views panel — you will switch the Pin mode on. In the viewport, you’ll see the photo you loaded and the mesh of the FaceBuilder 3D model. Now you can start pinning the mesh to the photo.

It can be done manually or using automatic face alignment. For the second option, press the Align Face button on the Views panel. Then a couple of neural networks will find a face on the photo and set up some pins to match the position and the shape based on facial landmarks. It doesn’t work 100% accurately but it definitely allows you to start building your model faster.

The pinning process is iterative and we recommend starting with approximate positions across all of the views first. This looseness of positioning allows FaceBuilder to estimate camera positions without diving too deep into the details of the shape.

Once you’ve done with the approximate pinning, it’s time to start adjusting the shape adding more details on different views one by one going back and forth until you’re satisfied with the result.

It usually takes up to 30 minutes to get a really detailed 3D model.

Exporting a texture and model

Once the head and face are pinned across the desired number of views, FaceBuilder can create the texture — open the Texture tab, select the mh UV map and set the size to 1024 x 1024 pixels, then it will match the default MetaHuman texture.

Click the Create texturebutton and once the texture is ready, you can export it in a file or work with it inside Blender. Let’s export it because you’ll need it later for our MetaHuman.

Apart from that, you also need to also to export the model as an FBX file which you will load to Unreal Engine.

Mesh to MetaHuman: Aligning the model

Here we come to Unreal. First, let’s import the model using the exported FBX file with its texture saved as a PNG file. Then, please check you have the Mesh to MetaHuman plugin installed and activated.

Create a MetaHuman asset, call it as you wish and select the imported FBX file as a base model. Here it’s better to narrow the view angle, so it won’t be so wide and the face will be less distorted. Then we frame the face to see the frontal view with all its details and launch auto-tracking, which takes a couple of moments and gives us the automatically detected facial landmarks.

Lock the camera and check if the landmarks are good enough. Once you’re sure that everything looks good, launch the Identity Solve button. This function uses the landmarks to produce a MetaHuman model based on the imported FaceBuilder mesh. Now you’ve got your model wrapped with a MetaHuman topology!

If after comparing the imported FaceBuilder mesh with the solved MetaHuman model you notice that some parts of the face aren’t perfect. It’s easy to fix. In this case, frame the head to see the inaccurate part better, launch auto-tracking once again, switch on the landmarks of this part and fix them, so they accurately reflect the geometry. Launch the Identity Solve button one more time to get a new MetaHuman model.

When you’re satisfied with the results, choose a body from the list on the left, and click the Mesh to MetaHumanbutton to upload it to the MetaHuman online backend for further modification.

MetaHuman Creator: set up the appearance of the character

In the online MetaHuman editor, choose the character and tune it to match the person from the photos as closely as possible by tweaking the skin colour, eyes, facial hair, clothes and so on.

Using the online editor, you can reach a very good level of likeness, but there’s a way to make it even better. To achieve this we need to replace the generated MetaHuman texture with something more realistic and feature-rich.

Photoshop: Preparing the texture for MetaHuman

MetaHuman uses four textures for different facial expressions including the neutral one. We take them all to Photoshop and merge with the mh texture previously extracted from the photos by FaceBuilder.

The mh texture map of FaceBuilder was created specifically to make this process as simple as possible. It matches the layout of the built-in MetaHuman texture perfectly, so all facial parts will align naturally, you just need to set up the size of the texture properly as we did while creating and exporting the texture.

At this stage, you need to accurately join high and low-frequency details from both textures and match their colours. Don’t forget to also clean up the standard MetaHuman texture from the features that it has, but your real person hasn’t.

Unreal Engine: adding the new textures

In Unreal, let’s create a sample MetaHuman project and import back the character we tweaked in the online MetaHuman editor. Replace the default character in the scene with your MetaHuman.

The final step — you need to replace the textures of the model with the ones you created earlier in Photoshop out of the MetaHuman ones and the ones extracted from the photos by FaceBuilder. Do it one by one for every expression.

Once you’ve completed it, you will see the distinct skin features of the person on the MetaHuman character!

That’s it!

In the short future, we will release a detailed video guide covering this process. Subscribe to our YouTube channel to not miss it!

Download FaceBuilder for Blender 2022.2 from our website.

Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit

KeenTools FaceBuilder x MetaHuman Guide was originally published in KeenTools on Medium, where people are continuing the conversation by highlighting and responding to this story.