Integrating Light & Shadow

The purpose of this project was to successfully integrate a rounded CG object which would move through a live action background complex shadows. We were encouraged to shoot our own background footages and collect all the camera and lighting information from the scene to have a better understanding what needs to be done in post production. To do so, I used a Canon 5D Mark IV camera, a 3 inch chrome ball, a 2 inch wooden cube, and a 2 inch white soft ball.

Each image had a different purpose. The clean plate was used for composting, the cube was used for matching the camera angle and position, the white soft ball helped figuring out the key light position and the key to fill ratio, and the chrome ball helped creating the hdri for this scene.

Knowing the exact measurements of the cube, what camera was used and with what focal length, I could align a CG cube with the original one to match the perspective.

We had to pick a rounded CG object to see how the shadows would wrap around it, so I decided to use watermelons. I created the textures in Substance Painter and used and AiColorJitter node to add some variety to the 4 watermelons, which I also modified the shapes for, to make them look more different.

In order to make the watermelons roll realisticly, I created a simulation where I added a ramp of the frame and put gravity on the watermelons. I also had to set the ground plane and the ramp as a passive collider and then I just experimented with the rigid body attributes to see what looks realsitic. This is one of the settings for the watermelons:

After that, I created multiple render layers, so I could easily combine different elements in Nuke.

The render layers include: Beauty, Shadow, Occlusion, Shadow Projection Prep, Shadow Projection, Ground Reflection, and Ground Shadow Reflection ( I had to create seperate layers for each watermelon for ground shadow reflection, because they were visible in each others reflection, which I tried turning off by a layer override, but still didn't work.)

What was different from previously made CG integration projects, is creating the gobo alpha for the watermelon. In this method, we added a camera that matches the position and angle of the keylight and render out just the ground plane to see how it would look from that angle. Then I made a black and white image out of that render and used it as a projection onto the watermelons, which created this shadow wrapping over the objects.

Unfortunately, this alpha would affect the whole object, while it should only be visible on the direct channel. So, I could have only use it to color grade the diffuse direct and specular direct, but most of the color information was coming from the indirect channels so I had to improvise.

I created a mask out of the green channle of the direct passes and multiplied it with the inverted alpha of the gobo, which gave me the exact result I wanted. I also used the inverted alpha from the green channle to color grade the sahdow on the watermelon.

After that I made a "shadow plate" from the original clean plate in Photoshop, and used the shadow render layer to reveal that area as the watermelons are rolling through the scene. The occlusion pass was responsible to create the core shadow and its falloff.

For more realism, I included the motionvetor AOV in the Beauty render layer, and used that to add motionblur with a VectorBlur node in Nuke.

Combining verything gave me a photorealistic CG integration with realistic motion. Here's a screenshot of the Nuke nodegraph and a turntable of the watermelon:

Overall, this project had its challenges but I enjoyed every step of it from shooting to compositing and learned a lot of the process of CG integration in general.