All language subtitles for 1. Painting and Patching

In the previous video we set up a complex non-linear retime.

We were concentrating on getting the timing correct,

but we weren't paying attention to the quality of the retime results,

but in this module I'm going to show you some of the various approaches we can

use to remove the artifacts caused by retiming algorithms.

Guessing the perfect frames for my retime can take a lot of work,

so it's always important to make sure we have the correct timing

before we start worrying about cleaning up the frames.

Show your retime to the director, the VFX supervisor,

editorial department or anyone else who might have any feedback on

the timing before you start the cleanup process.

In this class we are going back to the clip of the

skateboarders from module two.

We're going to slow down the clip to half speed.

This version of the clip runs from frame 210 to 245.

When we double the length of the clip it's going to run

to frame 280 of the Nuke timeline,

so I'm going to adjust our project settings to match that range.

Based on what we learned in module three,

I'm going to set up the retime with a TimeWarp node.

By default,

the TimeWarp node will add a key frame at the first and last frame of the clip.

The first frame doesn't need changing,

but we need to move the second key frame along the

timeline from frame 245 to frame 280.

This gives us the correct timing for the shot,

but we know we can get smoother results by using optical flow,

so I'm going to use an expression to connect a Kronos node to this timewarp.

I'm going to connect the source connection of the Kronos node to the Read node,

and click the reset button next to the Input range of the clip.

Then I'm going to change the timing dropdown from Output Speed to Frame.

To add the expression that links this to the TimeWarp node I need to

right-click on the property and choose Add expression.

The expression I need is simply TimeWarp1.

lookup, making sure that I get the upper case T and W in the word TimeWarp.

Now I have a Kronos node referencing the retime from the TimeWarp node.

In module two we found that setting the vector detail to one in the

Kronos node gave us a pretty good result all around,

so let's do that.

In Nuke X the foundry has updated the optical flow

algorithms that are used in Kronos,

and the quality of results are massively improved from previous versions,

but because Nuke is trying to create frames that never existed

the results are never going to be perfect,

and sometimes the errors can be very ugly and distracting.

You usually see issues at the edges between two layers moving in different

directions or when objects appear or disappear completely.

In this clip the most obvious issues are the road that

appears at the bottom edge of frame, the skater's arm that is moving too quickly,

the detail in the background behind the main hero skater,

and the car that drives into the shot from behind the building.

Cleaning up retimes can be a slow and endless task.

Sometimes it's impossible to make everything perfect in the time you have,

so you have to pick the most obvious issues to fix first,

and then keep working until you run out of time.

In this example the first thing we need to fix is the road.

One of the simplest ways of improving retime results is to use

different settings for different areas of your shot.

In module two I showed you how the Vector Detail slider and the

Strength slider can make a big difference in the results.

If I duplicate this Kronos node and in the new node

reduce the Strength slider to 0.

1 you can see that we get a much less distracting result for the road,

but overall the results are much less smooth.

If I draw some loose shapes around the corners of the frame

we can use a Keymix node to combine the different results

from the different Kronos nodes.

I'm going to connect the Kronos with the good

results for the road to the A input, and the other Kronos to the B input.

Then I'm going to connect my roto shape to the mask input.

This isn't perfect,

but it's definitely a big improvement in a short amount of time.

In module two I had roto for the foreground skater.

That made a big difference to the retime results.

The roto was done on the original plate before the retime had been applied.

Let's connect the roto to the main Kronos node,

and set the Matte channel to Matte Alpha.

Again, this seems to be a big improvement,

but if you look closely it actually introduces

artifacts that weren't there before, like these sharp edges,

and this roto took a significant amount of time to create.

We're going to use this roto to fix some of the other issues,

but if you didn't already have full body roto available it

might not be worth the time or effort to create it just to

plug into a Kronos node, but as I have it I'm going to use it.

Sometimes combining different Retime nodes together

can get you a good enough result,

but when you start using multiple optical flow nodes your

script can really start to slow down.

If you're using this approach you might want to

render out each different retime,

and then read them back in and combine the rendered results.

I still have some cleanup to do,

so I'm going to render out these two Kronos nodes and use

the renders for the rest of my work.

Now that I have rendered out the Kronos nodes,

and I've done some broad stroke fixes, it's time to get into the little details.

There are two main techniques that I'm going to show you

here; both are basic rig removal techniques,

painting, and patching.

Both use different parts of different frames or different areas of

a frame to recreate the areas that need fixing.

Using the Paint node to fix each frame at a time is relatively simple,

but it is tricky to create consistent work over many frames.

It's good for fixing individual frames or areas that are moving

fast and are quite different on every frame,

but if the movement is slow and steady you can quickly see popping or

boiling in your paint work when the results are played back.

Creating patches takes a lot more work to set up,

but once you have them working they're smooth and consistent.

We could spend hours or days cleaning up everything in this shot,

so I'm just going to show you a few examples of how

I would use these approaches.

I'm going to start by using the Paint node to clean up the

section of the shot where the skater's arm lifts up and

crosses the skater behind him.

We can see that the version of Kronos that uses the roto gives us a nice,

consistent, solid shape for the arm for most of the action,

but it does have some ugly, sharp edges that need cleaning up.

Because the arm is moving fast in this section of the shot we

can use the frames before and after to source clean areas of

the background to fix these edges.

Let's fix up frame 243.

Connect a Paint node to the Read node,

and choose the Clone tool from the menu on the left of the viewer.

By default, the Clone tool clones from different areas of the same frame,

but if we set the source to bg using this dropdown at the top of the

Viewer we reveal the time offset option labeled delta t.

This lets us choose to clone from a different frame.

The arm is moving upwards at this point,

so if we want to clone a clean area of the face we

need to source the previous frame.

So let's set the time offset to -1.

Everything in this shot is moving between each frame,

so we need to line up the source frame to the destination frame.

We can do this using the onion skin feature.

Press T to overlay the source frame over the current frame.

Now hold down the Command or Ctrl key and drag on the viewer to line the two up.

Use the hard edge of the white sunglasses to get

the lineup as close as possible.

Hit T again to turn off the overlay.

Start painting over the edge of the arm.

This will clone the clean image of the face from the

previous frame over the areas that need fixing.

If we tried to paint on the lower edge of the arm we will

clone in more arm from the previous frame,

so we need to change the clone source.

Change the time offset from -1 to positive 1 to clone from the

next frame rather than the previous frame.

Hit T again to show the overlay, and line up the new source frame.

Then paint in the lower edge of the arm to fix the hard edge.

We can see that the arm on frame 242 looks okay,

but if you move backwards through the sequence from here the

arm breaks or doubles up between 232 and 242.

The truck behind it also breaks up a lot.

We're going to use a couple of patches to fix these areas.

We're going to work from the back layer first,

then restore the foreground layers on top of the clean background,

so let's start with the truck.

We're going to create a patch for the truck between frames 232 and 242.

Frame 232 shows more of the front of the truck than

any other frame in the sequence, so we can use this frame to create the patch.

Let's add a FrameHold node, and set it to 232.

Then I'm going to add a Roto node below the FrameHold node,

and draw around the top half of the truck,

and then use a premult to apply this roto shape to the image.

We need to track this patch over the moving image.

I would usually try to use a Tracker node to track the movement,

and then apply that to the patch,

but because of how the skater covers the area there

isn't much that we can track.

We only need to use this patch for 10 frames, so we can track it in by hand.

I'm going to add a Transform node below the Premult node,

and on frame 232 add a key frame to the translate,

rotate, and scale properties.

We can also hold down the Command key or Ctrl if you aren't on a Mac,

and drag the control handles to the middle of the patch.

Now we can merge this over the main image.

On this frame we can set the mix value and the merge

node to 0 and add a key frame.

Then go to the next frame and set the mix to one.

If we look before and after this Merge node we can now see that

the patch doesn't line up with the truck behind.

We're going to need to manually line that up on each frame,

but because the camera was pulling backwards at a reasonably

steady pace the scale is going to change,

as well as the position.

Because this is a steady change we should set a key frame on the last frame,

and then hopefully we won't need to adjust the scale on every frame.

Let's go to frame 242 and line up the patch on this frame (clicking).

Once the patch is lined up we can set the mix property in

the Merge node to 0 for this frame, and go back 1 frame,

and set it back to 1.

Now we have the patch onscreen from 233 to 241,

and we have lined up the patch with the frames before and after that.

The next step is to go through the eight frames in that range,

and manually line up the patch on each frame.

If we constantly compared the before and after we'll make sure that

we get the lineup as close as we can (clicking).

Once it's done it should look something like this.

Now we need to restore the skater back over the top,

and fix some of the issues with the arm as we do it.

Because we have the full body roto of the skater we can mask

the skater and run a premultiplied version of the skater

through another copy of the Kronos setup.

Because we have removed all the surrounding details this actually

produces a much cleaner result of the skater.

There are still a few bad edges,

and on frame 239 there is a weird artifact in the alpha channel,

but we can clean these up with a few paint nodes,

and place this layer over everything else.

If we compare the chart before and after we can see

that we have a much better result.

There are still lots of areas that need fixing,

but we can use similar techniques to these to fix them all up.

In the next video I'm going to show you how we can use camera

tracking and 3D projections to help with the process.