Composite a 3D object into video footage with Boujou, Maya and After Effects

Matchmoving is a visual effects technique that allows the insertion of computer graphics into video footage. In this tutorial we’ll start by taking a look at how we can track our footage using Boujou. We’ll then jump into Maya and use that tracking information, along with a custom HDR image, to create and render a 3D object which we can insert into the scene. We’ll then use After Effects to composite the 3D object into the original video. The techniques used in this tutorial can be applied to almost almost all of your 3D integration shots.

Step 1

First we need to convert our video into sequence of images so that it can be read by Maya and Boujou. Open After Effects and go to “File > Import > File…”.

Step 2

Navigate to your footage, select it and click “Open” to load the video into After Effects.

Step 3

To create a composition with the same characteristics as our source footage, just drag the video to the new composition button.

Step 4

You should now be able to see the video along with it’s timeline.

Step 5

We now need to change our composition’s frame rate to match the frame rate in Maya. Go to “Composition > Composition Settings…”

Step 6

As the frame rate for this footage is 30fps, I’m going to set the frame rate to 30 frames per second and click “Ok”.

Step 7

To render the video, go to “Composition > Add to Render Queue”.

Step 8

Once the render queue has opened, click on the word “Lossless” in the output module section.

Step 9

In the output module settings, select “JPEG Sequence” from the dropdown and then press “OK”.

Step 10

Click here to change the name of your rendered image sequence.

Step 11

Select the folder where you want your sequence to be rendered to (remember that we’re going to be creating a lot of images!) Now change the name of your video to match this format : “Name of your video.[####].jpg”. The “.[####]” is extremely important as it ensures that our images are named sequentially. Also, make sure that you use a dot and not a under score in the filename, as Maya won’t recognize the sequence if it is not in the above format.

Step 12

We’re now ready to render the sequence, so click “Render”.

Step 13

Open Boujou and click on “Import Sequence”.

Step 14

Find the sequence that you rendered with After Effects, select the first file and click “Open”.

Step 15

Click on “OK”.

Step 16

Click on the “Edit Camera” button.

Step 17

We need to make sure that Boujou is working with the same frame rate as our sequence, so change the frame rate to 30 and click “OK”.

Step 18

Click on the “Track Features” button.

Step 19

I usually leave the settings as default, so just make sure “All frames” is active and click on “Start”.

Step 20

Boujou will now start the initial tracking process.

Step 21

Now that Boujou has finished, we have 2-dimensional tracking data for our shot. However, in order to integrate a 3D object into the footage we need to calculate the position and rotation of the camera. To begin this process, click on the “Camera Solve” button.

Step 22

Make sure that “All frames” is active and click on “Start”.

Step 23

Boujou will now start calculating the scene camera’s position and rotation.

Step 24

Once Boujou has finished tracking, we now need to align our scene to match the footage. Scroll through the video and select 2 or more points that lie in a straight line, in this case on the tabletop. These points will form our X-axis.

Step 25

Click on the “Scene Geometry” button.

Step 26

Click on “Add Coord Frm Hint”.

Step 27

Select the X-axis from the “Type” dropdown and click on “Connect to Selected”.

Step 28

With our X-axis in place, we now need to define at least one other axis so that Boujou can correctly orient our scene. Close the previous window and now select 2 or more points that lie in a straight line, but at 90 degrees to our X-axis. This will be our Z-axis and in this case travels away from camera across the tabletop.

Step 29

Click on the “Scene Geometry” button.

Step 30

Click on “Add Coord Frm Hint”, select the Z-axis from the “Type” dropdown and click on “Connect to Selected”.

Step 31

If you can confidently select points in order to calculate the Y-axis, do so and repeat the process as before, making sure to select the Y-axis from the “Type” dropdown before clicking “Connect to Selected”. In this scene, we’re unable to select Y-axis points so we’ll move straight on to the next step.

Step 32

Click on the “Scene Geometry” button, then click “Update Coord. Frame” and close the window.

Step 33

Activate the “Ground Plane” checkbox in the overlay tab on the bottom-right of the Boujou window.

Step 34

We can now see the floor plane that has been calculated for our scene. It should stay in place throughout the video (try scrolling through the footage to make sure that this is the case.) If it does not however, repeat steps 24-31 above, choosing different points to orient your scene.

Step 35

To make sure that our track is correct, we’re going to create some test objects. Click on the “Add Test Objects” button.

Step 36

Select the object of your preference and close the window, I usually choose “Ladybird”.

Step 37

Play the video to see if the object stays in place throughout. As with the ground plane overlay, if your object does not sit correctly within your footage, your scene might not be oriented correctly. If this is the case you should repeat steps 24-31 above choosing different points for each axis.

Step 38

We now need to export our tracking information to Maya. To do this, go to “Export > Export Camera Solve”.

Step 39

Browse to your project folder and choose a filename, change the export type to “Maya 4+” and then make sure the move type corresponds to your footage. As this footage was shot using a handheld camera, I’ve chosen “Moving Camera, Static Scene”. With this done, click “Save”.

Step 40

After opening Maya, go to “File > Open” and open the file that we just exported from Boujou. Your viewport should look similar to this image.

Step 41

In the menu at the top of the active viewport, go to “Panels > Perspective > Camera_1_1”.

Step 42

We will now be looking through the matchmoved camera created for us by Boujou.

Step 43

Go to “Create > Text” and click the option box.

Step 44

Enter your text and use the available options to add bevels to the text object. Click “Create” when done.

Step 45

As we aligned our scene within Boujou, our new text object should automatically sit on top of the table in our original footage.

Step 46

Rotate, scale and position the text as desired, making sure that it remains sitting right on top of the grid.

Step 47

Go to “Window > Rendering Editors > Hypershade”.

Step 48

Select “Create Mental Ray Nodes” from the dropdown on the left, and click “mia_material_x” in the “Materials” group to create a new material.

Step 49

With the new material still selected, press “Ctrl-A” to open the attribute editor and change the diffuse color of your material to your desired color, in this case a red/orange. We also want to change the following options : Reflection Color to grey, Reflectivity to 1.000, Glossiness to 0.350 and Glossy samples to 0.

Step 50

Close the Hypershade. Right click our text object in the viewport and go to “Assign Existing Material > mia_material_x1″ to apply our new material to the text.

Step 51

Open the “Render Settings” window.

Step 52

Select “mental ray” from the “Render Using” dropdown. Now change the following options in the Common tab : Choose a file name prefix (in this case I have added a folder structure before the name itself to ensure that the files are saved to the correct location,) change the image format to TIFF, change the “Frame/Animation ext” to name.#ext, set the “Start frame” and “End frame” values to match that of your animation, select “Camera_1_1” from the “Renderable Camera” dropdown, and make sure that the rendered image size matches that of your footage (in this case I have selected the HD 1080 preset.) With that done we can now close the render settings window.

Step 53

We now need to make sure that Maya’s frame rate matches that of our footage. Go to “Window > Settings/Preferences > Preferences”.

Step 54

In the “Settings” category change the time to match that of your footage. Here I’ve selected “NTSC (30 fps)”. Click “Save” When done.

Step 55

Re-open the Render settings window.

Step 56

In the Quality tab we need to change the following options: set the “Max Sample Level” to 2, “Multi-pixel Filtering” to “Gauss”, activate “Jitter”, “Raytracing Reflections” and “Refractions” to 3, “Raytacing Max Trace Depth” to 6, set “Motion Blur” to “Full” and set the “Motion Blur By” value to 2.000.

Step 57

Go to the “Indirect Lighting” tab and create an Image Based Lighting Environment. Then activate “Final Gathering” and change “Accuracy” to 200, “Point Density” to 0.100 and “Secondary Diffuse Bounces” to 2.

Step 58

If the attributes editor isn’t visible, click “Ctrl-A” to activate it. Click the folder icon next to “Image Name” and browse to your HDRI file.

Step 59

Go to “Create > Lights > Area Light”.

Step 60

Making sure that the light is selected, go to “Panels > Look Through Selected” in the menu at the top of the active viewport.

Step 61

Position the light so that the shadows cast will match direction of the shadows in the original footage.

Step 62

Go to the Attributes Editor with the light selected and activate “Use Ray Trace Shadows”. With this done, change the “Shadows Rays” to 15.

Step 63

With the light still selected go to “Edit > Duplicate”.

Step 64

Select the duplicated light and go to “Panels > Look Through Selected” in the menu at the top of the active viewport.

Step 65

Position the new light at the same height but on the opposite side of our text. This helps to produce softer shadows in the final render.

Step 66

Go to the attributes editor of the new light and turn off “Use Ray Trace Shadows”.

Step 67

In the active view menu go to “Panels > Perspective > Camera_1_1”.

Step 68

Now we need to create a surface to act as a shadow-catcher. Go to “Create > Polygon Primitives > Plane”.

Step 69

Move and scale the plane so that the shadows of the text will fall within it’s boundaries.

Step 70

Go to “Window > Rendering Editors > Hypershade”.

Step 71

Select “Create Maya Nodes” from the dropdown on the left, and click the “Use Background” material in the “Surface” group. This material allows our plane to receive shadows and reflections, but ensures that we can still successfully composite the final render on top of our original footage.

Step 72

With our new “Use Background” material selected, go to the attributes Editor and reduce “Reflectivity” to 0.

Step 73

Apply this material to the plane, by right clicking the plane object and selecting “Assign Existing Material > UseBackground1″.

Step 74

We’re now going to double-check that our objects sit exactly on top of the groundplane. In the menu at the top of the active viewport, go to “Panels > Orthographic > Side”.

Step 75

Move the objects into position if required.

Step 76

Go back to the “Camera_1_1” camera.

Step 77

Create a test-render of the scene by clicking the “Render” icon in the main toolbar.

Step 78

You can see in our render how the object integrates perfectly within the scene. If the shadows are cut off at this point, close the render view and scale up your shadow-catcher object. Make sure to do another test-render to ensure the problem has been fixed.

Step 79

Select the “Rendering” menu from the main toolbar dropdown and go to “Render > Batch Render” to begin rendering the sequence.

Step 80

When Maya finishes the rendering process jump over to After Effects and go to “File > Import > File…” .

Step 81

Navigate to find your newly rendered sequence, select the first frame and, making sure that the “TIFF Sequence” option is active, click on “Open”.

Step 82

A window asking for the alpha channel will pop up. Select “Straight – Unmatted” and click “OK” to import the footage along with it’s alpha channel.

Step 83

We now need to import the original video sequence. Go to “File > Import > File…”.

Step 84

Browse for the sequence of images we initially rendered out from After Effects, select the first frame and, making sure that the “JPEG Sequence” option is active, click “Open”.

Step 85

You should now see both sequences in the project bin.

Step 86

Drag the .tif sequence we rendered from Maya to the new composition button.

Step 87

Now drag our original .jpg sequence into the composition we just created, making sure to position it below the .tif sequence.

Step 88

Our two layers are now composited together in the viewport.

Step 89

If you zoom in, however, you’ll notice that there’s currently a glitch. This is because whilst the foreground sequence begins on frame 1, the background sequence begins on frame 0.

Step 90

To correct this error, zoom into the time line using the small slider at the bottom of the screen, and drag the green bar representing our background sequence 1 frame to the left.