Camera Tracker

‣
Table of Contents

Notes

Inputs

Source

Plate to be tracked.

Mask

Mask used to define areas of the Source that are not to be tracked.

This is particularly important to omit areas of the plate that have low-contrast features, such as clouds in skies, or moving elements, like people running through a plate.

Properties

Verify the following properties to identify, analyze, solve, clean and resolve tracks. Note that the properties are outlined in the specific order that they should be addressed for setting up and generating tracks.

Camera Tracker Tab

The more information that can be specified here based on the camera/lens used to capture the plate, the better the camera tracking will result.

Camera Motion

Used to define how the plate was captured with the camera positioning, such as hand-held camera, camera on a fixed tripod, camera mounted on a moving dolly, or other configuration. This constrains the motion to the specified option.

  • Free Camera (default): Hand-held camera operation
  • Rotation Only
  • Linear Motion

Lens Distortion

This confirms that the plate has been undistorted. Although the option to let this Camera Tracker to undistort, it is better to undistort using the Lens Distortion node.

Focal Length

  • Unknown Constant (default)
  • Known: Provides the option to specify the focal length of the lens used to capture plate.

Film Back Preset

Presets for setting size of camera sensor

Sony > DSLR > Mirrorless A7

User Tracks Tab

This allows for user-defined tracks to serve as accurate references for the auto-track operations. Auto track would work even without these user-defined tracks, but the user-defined tracks add a level of accuracy that the process can benefit from.

Import Tracker

Import tracks computed with Tracker node/s.

Pop-up dialog with available Tracker nodes appears.

Import process may take some time.

Settings Tab

Number of Features

Default value: 150

The default value may not be sufficient as this number does not fully get deployed in the plate, so a higher number may be warranted. Use the Preview Features option to investigate.

Detection Threshold

This is the threshold to detect feature contrast in the plate and determines the placement of the features for tracking. Higher values concentrate the features in areas of high contract details. Lower values disperse the features throughout the plate, but may inevitably place features in low contrast areas without detail and therefore not be able to track adequately.

Feature Separation

Minimum distance between features. An alternative to adjusting this is to enable Refine Feature Locations.

Refine Feature Locations

Distributes tightly packed features to nearby corners and detailed areas, particularly if features are less than the Feature Separation minimum distance.

Preview Features

This displays the specified Number of Features that Nuke intends to deploy in the shot in the Viewer.

Tracking

The properties in this section are thresholds for retaining features being tracked. For instance, the Minimum Length default is 3, so any feature that does not retain tracking within 3 frames is discarded. The same applies for the other properties in this section.

Default settings are good for starting point.

Camera Tracker Tab

Analysis: Track

Verify that current frame is set to start frame on timeline.

Once the above properties have been addressed, then it is time to enable this Track analysis.

Allow Tracker to process entire frame range forward and backward.

Results display each feature and their tracks.

Auto Tracks Tab

One the analysis has output tracks, then it is time to clean extraneous tracks that are faulty or do not contribute stability to overall tracking.

Graph Controls

  • Of the options available in the left panel, select one and view in graph.
  • To select multiple options in the left panel and view simultaneously, press CTRL and select additional options or SHIFT to select sequence of options.
  • Press F to frame content within graph visibility.
  • MMB to pan.
  • Mouse scrolling wheel to zoom in/out.

Track Length - Minimum

Select Track Len - Min and Min Length options from left list.

The leftmost tracks on the graph correlate with those tracks that meet the bare minimum track length set under Settings: Tracking: Minimum Length, which may not truly be fully stable since they only exist for a few frames beyond the minimum and then disappear.

Min Length

  1. Use knob at bottom of graph to increase minimum length to a value to be set as the new minimum.
  2. This will display all tracks below new threshold as red in the Viewer.

Delete Rejected:

  1. Press this button to delete those tracks below the new minimum set by the Min Length knob.
  2. A dialog pop-up will confirm that the rejected tracks are to be deleted.
  3. The rejected tracks are then removed from Viewer and graph.

Manually Delete Tracks

With the minimum length tracks refined, manually delete tracks that are not deemed suitable.

Verify that the Camera Tracker node is still active and visible.

Select tracks in Viewer individually or with selection marquee. These can include those tracks in low-contrast areas like skies, plain surfaces, or movable assets, like moving people.

Press Delete key to delete selected tracks.

Note that this process needs to be repeated for each frame in frame range as tracks appear and disappear over course of frame range.

Tip: A refined Roto mask applied to this Camera Tracker before the Track Analysis may mitigate cleanup steps, but either way, it takes time to setup and clean tracks.

Camera Tracker Tab

Analysis: Solve

Verify that current frame is set to start frame on timeline.

Once tracks have been cleaned, it is time for the Camera Tracker to solve the track using this Solve button.

The results are noted in the Viewer by applying colors to each track:

  • Green: Good track solutions.
  • Red: Tracks with too many errors; unstable.
  • Yellow: Tracks that the Solve process was unable to resolve.

Error

A good track results in an Error value of less than 1.0.

Auto Tracks Tab

Following the Solve analysis, use this tab to clean up resulting tracks.

Solve Error

This notes the error output by the Solve analysis in terms of RMS (root mean square). The intent is to lower this to be below 1.0.

Delete Rejected

Delete the red tracks by clicking this button.

Pop-up dialog box will confirm deletion.

Note that deleting the rejected tracks will decrease the error noted in the Solve Error property above.

Delete Unsolved

Similar to the red tracks, the tracks colored yellow by the Solve analysis provide no positive value since they were not able to be resolved. This button deletes only the yellow unsolved tracks.

Pop-up dialog box will confirm deletion.

Note that the Solve Error value at the top should not change since the yellow tracks provided no contribution to analyzed solution.

Resolve Max Error

Select Error - Max and Max Error options from left list.

Max Track Error

  1. This will mark the max threshold for RMS pixel error.
  2. Use knob at bottom of graph to decrease threshold.

Max Error

  1. This marks the max threshold for pixel errors in tracks.
  2. Use knob at bottom of graph to decrease threshold. Note the bar in the graph denoting the threshold relative to the solved tracks.
  3. Mitigate peaks in the graph.

Delete Rejected

  1. Solved tracks that are denoted above the max error threshold are now noted with red in the Viewer.
  2. Use Delete Rejected, as applicable, and note the revised Solve Error at top of panel.
  3. Repeat Analysis: Solve analysis to refine error.
  4. Repeat process as necessary to achieve adequate Solve Error.

Scene Tab

Define Ground Plane & Transforms

With the point cloud generated via the tracks, it can be viewed in the Viewer's 3D space. Note that the ground plane needs to defined.

  1. In the plate view, scrub to a frame where tracks on the ground plane are noticeable.
  2. Select these track points and right-click > ground plane > set to selected.
  3. Tab back to 3D space and note that the point cloud and camera are now oriented relative to the specified ground plane.

Camera Tracker Tab

Export

There are various options available to export select items or all items from Camera Tracker node. This will output the relevant content (Camera, Scene, Lens Distortion, Point Cloud, and Scanline Renderer) to the Node Graph.

  • Scene+

    • (Preferred) This option outputs Camera, Scene, Lens Distortion, Point Cloud, and Scanline Renderer

Link Output

This option is enabled by default. Disabling it allows for the output Camera to be duplicated and retaining their own independent transform values, rather than always being linked with relative values to the Camera Tracker node.

Resources

Refining Solve