Process Blog - Collaborative Project under The Mill
More than Just a Car... A UFO??
This week, we finally reached our end result after 10 weeks of hard work. Focusing on the smaller details, I adjusted the densities of the volumes, refined the lighting comp by adjusting the light groups and noise levels, and tweaked the color correction of the short grass. For such an ambitious project, I am happy with the outcome. Through this project I was able to experiment a lot with lighting and integrating volumetric effects, which is something that I have wanted to gain experience with for a while. I think it was also a valuable experience troubleshooting effects that are moved from Houdini to Maya and learning to work with aiStandins.
If we were to do it again or have more time, I feel that we should spend a lot more time planning the effects and working on the comp. When we were first planning this project, we didn't anticipate just how many layers of effects we would need to pull off the force of the tractor beam. The ground fracture and debris that we ended up with are working well, but could still use more layers and debris pieces. For the comp as well, I think a lot more time could be spent on the day-to-night comp in shot 2. To me, the sky feels very flat and is lacking detail, and the short grass seen between the cg road and tall grass has a lot of issues including visible roto edges. For myself, I would have loved to have the time to texture the car rather than leave it with only shaders. Especially for shot 1, having the extra details would have helped tremendously. Overall, I'm grateful I had this opportunity to develop a great project under the guidance of The Mill and grow closer with people that I otherwise would not have had the chance to talk much with, and I can't wait to see where everyone on my team ends up next.
March 5-6, 2020
February 29 - March 1, 2020
This week I was able to implement many changes from the notes we were given. For shot 1, I adjusted the exposure of my lights--increasing the intensity of the beam and decreasing the intensity of the streetlight coming from the right. I also re-timed the animation of the beam so that it hits much earlier and decreased the density of the dust particles. For shot 2, I added in the new continuity dust effect and increased the interaction between the headlights and the volumes by placing spotlights within the headlights. The purple of the car paint was also reduced using a color lookup node on the direct diffuse aov in Nuke.
Addressing the buzzing of the headlights and the repeat of the road texture was the most difficult. For the headlights, I tried increasing various samples but did not achieve any significant difference with a reasonable render time. Since the noise was coming from the transmission, I removed as much of the internal glass as possible and rendered the outer glass separately. This significantly reduced the noise, and as a bonus revealed much more detail within the headlights. For the lookdev of the road, the new color map Elyssa gave me was much too flat so I brought out much more range using color correction in Nuke. I also found that using the color map as a bump map worked better than the normal map, and the height map made no significant difference as. To help reduce repeat in the pattern even further, I mirrored the repeat in the place2dTexture node. Finally, cracks were added by subdividing the road geometry 1 iteration.
shot 2: lighting comp progress
shot 1: lighting comp progress
February 25, 2020
shot 1: beam and streetlight exposure wedge
re-time of tractor beam in shot 1
shot 2: beam exposure and car color wedge
Wedge Tests: I rendered wedge tests for both shots to primarily test what tractor beam exposure would work best for each shot. Since I am using light groups in comp, this was fairly quick to test--rather than having to render a lot of frames. For shot 1, I adjusted the beam and streetlight exposures, since one of the notes from the last session was to reduce the exposure of the streetlight. For shot 2, I adjusted beam exposure and car color to address the note that the car should appear less purple. I also re-timed the tractor beam in shot 1 so that it hits earlier with the effects.
February 22-23, 2020
shot 1: lighting comp progress
shot 2: lighting comp progress
For this edit, I focused on adjusting the falloff and color of the tractor beam and on optimizing the render for the second shot. Following Todd's feedback, I looked at reference of underwater photography to get a better idea of color and falloff. Since the beam consists of two disk area lights, I adjusted the color by making the central, more narrow light a warmer green. I then adjusted the falloff through the aiAtmosphereVolume's density by multiplying the cloud noise that causes the god rays by a planar projected ramp.
To optimize shot 2, I first began by testing what was most contributing to the long render out of the new elements added--the higher quantity and size of files in the project folder or the three layers of particles added to the scene. After getting similar times rendering only 20 frames on the farm (and thus having significantly less .ass files to transfer), it was clear that the particles were causing the issue. To resolve this, Felipe converted the three layers of particles to a volume using vdb, and I rendered this using aiStandardVolume in Maya. This significantly helped render time. I also reduced the size of the hdr, decreased unnecessary light samples and aovs, and conserved samples where necessary. For example, while the beauty layer rendered on 6, 3, 2, 2, 0, the cryptomatte layer rendered on 6, 0, 0, 0, 0. For each layer, I also turned on matte or added black surface shaders where possible. Together through both of these optimizations, the shot rendered significantly faster; before we were getting 2 hour kills on 80 cores and 94 ram, and this time all layers rendered in 1 hour on 32 cores and 64 ram. Overall, I am excited to see how much progress we have made with this render, and moving forward I plan to continue refining the night lighting and work with Felipe on troubleshooting alignment issues with the effects.
February 19, 2020 - Questions for the Mentors
- Has the beam lighting improved? Any comments or tips for improving the lighting overall?
- How can we better optimize shot 2 to reduce render time; would it help to split up elements among layers further?
- Any tips for look development and lighting of the VDB and particle effects?
February 16-18, 2020
shot 1: lighting comp progress
shot 2: lighting comp progress
Beginning to address the notes from the last session, I brightened the tractor beam significantly. While it does have a much bigger impact on the scene, I think this could still be pushed further. I also still need to address the falloff of the beam to emphasize scale, and would like to experiment more with noise in the aiAtmosphereVolume. For the headlights, I changed from using aiStandardSurface emission to mesh light in the shape node's arnold tab. This created much more detail in the headlights, which was then pushed further with three point lights constrained to the mesh. I then isolated the headlights in comp using cryptomatte so that Elyssa can add glow and flares. Finally, I added the license plate texture that Felipe created to the car, and am excited to see it added to the next render. For next week, I plan on focusing on lighting and scene optimization so that hopefully all of the rendering issues I faced this week can be avoided.
license plate texture
For this week's review, we primarily focused on getting all of the effects integrated in the shots--with the only effect missing being the dirt simulation for the second shot. It was a new experience working with vdb's and particles in Maya Arnold, which added difficulty to the process and left less time for tweaking the lighting. The largest issue I had was render time for shot 2. The frames themselves didn't take long (at most around 18 minutes a layer), but the large amount of files that need to transfer before render caused many frames to 2 hour kill. As a result, part way through the render I had to split up the beauty layer and lower samples.
February 9-10, 2020
progress edit with night lighting
For the newest version of the edit, I focused primarily on the night lighting. The tractor beam was created with two circular area lights and aiAtmosphereVolume. The small area light in the center adds variation to the intensity of light, and the god-ray like beams were created using a stretched 3D cloud texture. I also began working on the road texture, although it is still very rough and needs displacement to add more variation.
There are a few issues in these renders that I plan on addressing. In the first shot, the shader on the pebbles flickers and the last 20 frames are frozen. I did have issues with the shader flickering before, and had to re-assign the material. For the frozen frames, I know it is most likely a render farm issue since the first job (frames 1-100) were animated while the second job (frames 101-120) were frozen, and will continue to keep an eye out for this issue.
In the second shot, there is an alignment issue between the road fracturing effect and the small debris effect. This is because there were alignment issues between the road I exported from Maya to Houdini, and I had to manually adjust the road effect back into place. However, I forgot to adjust the placement of the debris effect as well to match this. I also noticed with the day-to-night composite that the sky slips at the end, and the grass next to the road in front of the tall grass needs to be lifted.
Another issue I faced in comp is that the emissive light coming from the ufo was included in all of my light group aovs. As a result, the emissive light visible in the specular and coat indirect aovs were practically quadrupled in the final result. For the next render, I plan on isolating this light in order to gain more control in comp. Overall, I am excited to continue refining the night lighting and work on troubleshooting the issues we faced. We also tested exporting the .ass files properly with motion blur, so the next render will include motion blur on the effects as well.
February 7, 2020
playblast: new camera, layout, and animation
After testing if a new track would work, I went forward and completed a new track. In this track, the 40mm lens worked much better and was less shaky than my previous test. I only tracked the footage until the road disappeared, and then I continued to match the camera movement by manually keyframing the rotation in Maya. This of course wasn't ideal, but was the fastest solution and turned out well--especially since the car is floating. After locking the camera, I adjusted the layout so that the UFO is more overhead; however, I am unsure if it is too close to the car.
new camera track with 40mm lens
Beginning to address the intense angle and distortion of the car and ufo, I rendered a wedge test with different focal lengths (31, 35, 40, 45, and 50mm) and ufo scales (1, 1.3, 1.6, and 1.9). While the plate was filmed with a 35mm lens, I double checked the tracked camera in Maya and found that it was currently at 31mm--resulting in more distortion. Looking at the different renders, we found that we liked the 40mm lens with ufo scale 1.9 the most; however, changing the focal length of the camera to this results in a ruined track. Because of this, we are unsure if we will be able to render with a 40mm.
While I am not responsible for camera tracking on this project, I wanted to run a quick test to see if it would be possible to quickly redo the track and get a better result. The first track I did was sticking almost instantly, but has a focal length of 35mm. This does match the plate better, but is still not the ideal 40mm. I tried again and forced the track to be 40mm, which resulted in a more shaky track. It still stuck fairly well, but would need more tweaking to get it right. Overall, redoing the track feels like a promising solution to reducing the distortion--even if we have to stick with the 35mm--and continuing to refine the ufo placement and car animation will help as well.
February 5, 2020
February 1, 2020
progress edit with effects
This weekend, our main goal was to begin integrating the effects into each shot. Since I already had the scene set up for shot 1, I began by setting up the layout of shot 2. Rather than have the UFO directly overhead the car, I moved it back in the scene and plan to set the tractor beam at an angle. This will ensure that the car will have dramatic rim lighting instead of being flatly lit from the top. I have included a test of this in the images above as well; currently I am using an area light for the tractor beam to get a rough idea, but next I plan on experimenting with layered spotlights.
For pipeline and rendering, this week we tested using sequences of .ass files for the effects from Houdini to Arnold standins in Maya. This had many benefits, making integration easier and bypassing any issues with camera alignment and scale between Houdini and Maya. The main issue I had with it was look development, in that textures were slipping when using triplanar projection. This can be seen in my first render for shot 1, with the rocks swimming through the texture. To fix this temporarily, I used aiColorJitter on the faces to create variation in color; however, moving forward I plan on completing the look development in Houdini so that it will be brought in with the effect into Maya. I am also interested to see how well the standin will work for volumes and dust.
rock texture slipping in first attempt
Overall, I am excited to see the elements coming together in the edit. It is a lot easier to see what we need to focus on next, especially continuity between the two shots. Next I will be focusing on the road texture to create a match between the two shots. I also want to start working with the day-to-night plates to progress the lighting into nighttime. I also have not had the chance to address adding more texture detail to the car for the close up, and would like to begin working on that as soon as possible.
Another benefit of the standins in Maya is that it works with our school's renderfarm, while Arnold Houdini is currently having issues on the farm. I currently have two render layers for each shot, but I plan on isolating elements a lot more than I currently am in order to have more control in comp. The first shot consists of a beauty and shadow/occlusion layer, while the second shot consists of a beauty and tractor beam layer. Isolating the light beam enabled me to plus it over the comp; otherwise it would have been clipped by the alpha.
January 28, 2020
During the last feedback session, the main notes I received were to adjust the shadow in the shot one comp and add detail like dust and scratches to the front of the car. I first addressed the shadow issue by creating a shadow plate through color correcting and grading in nuke. I then merged this plate in the shadow pass, and merged that on top of the clean plate. This solution is beginning to work well, but I think the shadow plate could be adjusted further to create a closer match. It especially feels weird on the edges where the shadow is very soft.
To prepare to add dust and scratch detail to the car, I also began gathering references. I plan on adding detail subtly to maintain a newer, commercial car look. Next I will need to plan out and uv map the exact areas that will need the additional detail, paying attention to detail that could possibly be added procedurally rather than painted.
January 26, 2020
Before the next progress update, I took the render layers that I had prepared in the previous blog post and put together a rough comp. I think the perspective and key light direction is working well; however, I am still unsure of the placement of the car and the lightness in the shadow directly underneath the car. I also need to adjust my hdr fill light, as it feels too dark currently. Of course, the lighting will change dramatically though once the day to night composite is finished. For not having worked on the lighting for long, I think the integration is coming together nicely and the car materials feel accurate.
January 23-24, 2020
turntable with updated materials
Moving forward, I continued to adjust the car materials by focusing on each shader individually. Once finished, I then viewed all materials at once on the car and made further refinements as necessary. At this point I am happy with the overall look development, and will focus next on adjusting small details like the brushed metal and glossy paint. Texturing of elements like the license plates and hood logo--as well as imperfections--need to begin as well.
ufo rough look development
I also began a very rough look development of the UFO. Since the object does not exist in real life, I am having a hard time clearly deciding what each material should be; however, I plan on finding more precise shader reference to overcome this. For this lookdev, it was fun to play with the emissive surfaces and tractor beam lighting.
With lookdev going well, I also began to work on the lighting for the first shot. I began with aligning the camera to the cube reference, and moved forward with basic lighting. I currently have one key (spotlight) and one fill (sphere with surface shader) in the scene. Next, I set up two basic render layers--beauty and shadow/ground occlusion. For the projection of the ground in the beauty layer, I set up a shading network that multiplies in the shadow of the car. This creates a fake shadow that prevents light spill underneath the car, allowing for easier compositing. I also added the shadow matte and ambient occlusion together so that they can be rendered as one layer instead of two and shuffled out in comp.
January 17-20, 2020
My main focus this weekend was on filming with my teammates; however, individually I spent most of my time working on the car look development. I first refined my lookdev rig for this project, including two new lighting setups (day and night), three different reference sphere positions, and an adjusted camera height. These refinements were added to my already existing lookdev scene generator (scripted in python), which also enables a user to easily scale the scene to the model and set the timing of model and light turntables.
turntable with rough shaders
car reference for basic lookdev
To test the lookdev rig, I rendered a full turntable of the car and lights with the rough shaders that I worked on last week. I then separated the rough materials into a quilt of spheres and began working on refinement. The first material I began refining was the car paint, using reference from this video: https://www.youtube.com/watch?v=oXtBvZhgIjo. Looking at the reference, I recognized a gradient of color from bright saturated red to a deep maroon. To create this effect in the shader, I used the facing ratio attribute of the sampler info node to drive a fresnel effect with ramps. I used this for both diffuse color and specular color, and brought it all together with a coat on top.
Since our main concern with the footage is successfully pulling off the day-to-night comp, we shot two days this weekend. I was in charge of photography, and had the help of my teammates for adjusting the reference objects and shooting an hdr on a samsung 360 camera.
On the first day, the sky was perfectly clear and it was very windy. The hard shadows are not ideal for the day-to-night; however, the plate most likely would not need a sky replacement and the wind is great for giving the effect of the UFO flying nearby. The second day, on the other hand, was overcast and still. This allowed me to get plates without hard shadows, but the sky would need to be replaced and the footage does not have the bonus of the wind. For this reason, I am leaning towards using the footage from day one.
During the two shoots, I also captured reference of the railroad gravel in shot one and took texture photos of the road for the ground replacement. I already turned one of these into a seamless texture, and merged the chrome ball photos as well for use in lighting.
January 14, 2020
Next, I finished organizing the car mesh by material and began rough look development to check that each object was in the right group. One mistake I noticed, for example, is the white car paint rim on the edge of the side windows. Starting lookdev was also helpful for pointing out potential challenges, such as headlights and tail lights.
Preparing for production, I set up my team's project directories for use with my pipeline tools. This involved generating many folders through python, editing a custom Maya shelf and Nuke menu, updating a bash custom (for automatically transferring the newest shelf and menu among teammates), and integrating the files we already had within the pipeline structure. It was really interesting to go through the process of setting up the pipeline for a second project, and it gave me new ideas for improving this process in the future.
January 12, 2020
Overview: This week, I was paired with a team of three other artists and tasked with coming up with a car-centered project that is 'more than just a car'. My team quickly decided on a UFO themed spot consisting of two cg-integrated shots, in which a sitting car is suddenly lifted by a spaceship. Focusing on quality over quantity, I feel that this project gives us a great chance to demonstrate our strengths while working though an interesting set of challenges. For instance, we will gain experience with day-to-night compositing for the first time, and finding reference on such a fictional subject has already proven itself difficult. Overall I am very excited to start production, especially since my team has agreed to use the custom pipeline tools I developed for my senior project (tool breakdown here). This will help us work much more efficiently and keep files organized with significantly less effort.
Location: In order to push for cg-integration on this project, I started looking for a filming location early on. After a few hours of exploring Savannah through Google Maps, I had roughly ten locations that could work with the UFO theme. From this, I narrowed down the list to two ideal locations and scouted them myself. In the end we decided to film at Woodcock St.--an isolated back road running alongside a train and surrounded by wide fields. The open sky gives us plenty of room to add the UFO, and the gravel of the railroad will tie in well with the rocks that will shake in the first shot.
Reference: One challenge this project posed early on was the difficulty of finding real-world UFO reference.
As a lighting artist, the first element I searched for was the tractor-beam. I found that I liked the look of foggy streetlights, searchlights, and flashlight beams. I'm really excited for the challenge of bringing the tractor beam to life, especially since I do not have a lot of experience with volumes.
The ground fracturing and lift effect was especially difficult to find reference for, so I ended up helping the effects artists on my team. I think what we have now is a good start, but more research is needed.
Ground Fracture & Lift Effects Reference
Model: Rather than decide on a specific car first, we looked online for the best car model we could find for free. It was especially important to us that the model have an interior and detailed lights, as well. After a lot of searching, we decided on this Mercedes AMG by Behr Bros.
One issue I faced was figuring out how to export the mesh properly from 3ds Max to Maya. Once in Maya, I spent a lot of time cleaning out unnecessary detail (since the model is very high poly), organizing the mesh by material, and setting subdivision and opaque attributes.