My Micro-Credential in Virtual Production Journal

I was extremely honoured and grateful to have been selected to participate in the Sheridan College Micro-Credential in Virtual Production, with a generous full scholarship by Warner Bros. Discovery Access Canada. The course took place online and in studio at SIRT Centre - which is one of the original research centres focusing on VP before it exploded. I was very impressed by their information session and their careful attention to a thoughtful curriculum that showed me that there was true insight into the complexity of VP. Specifically, I liked that they had streams - Art Dept, UE On-Set Technician, and MOCAP technician. While I had a Ph.D. that focused on "Developing a Learning Model to Teach Film Production Online" that was completed in 2017 also before universal adoption of VP, I felt I would benefit greatly in a deeper dive into Unreal Engine and the ICVFX pipeline of VP. I had recently collaboratively run an award-winning 'In-Engine' Pipeline at my own institution, Humber College, whereby students made films entirely virtually. My next step is to teach students how to harness the aspects of the pipeline using an 'indie' set up for ICFVX and thereby learn the fundamentals with a limited budget.

Day 1 - AT SIRT

On the first day we were introduced to all our Professors, met some of the admin involved and got an overview of the program. Then we immediately got to see a scene set up with the LED wall working, someone in MOCAP live-linked to a background character (in this case a bartender) and some of my fellow classmates as stand-in actors. We practiced shooting with a camera that was being tracked. It was very interesting, but also most of us did not have any idea of what level of detail we were about to dive into in the coming weeks to understand how such a shot is prepped and truly captured. This was a great in-person day, to start out a module that would be mostly virtual and extremely dense.

Module 1 - Unreal Engine

Our instructor, Spencer Kenji Idenouye, is possibly one of the most patient people alive. I say this because he had a virtual room full of students who were at very different levels of software understanding. Some students had been professional animators and were versed in software that definitely makes using UE 'not that much of a stretch' to some, and then those like myself (although I had already worked minimally in UE myself) who really found UE to be a very dense and challenging software to understand. What was shocking and pleasantly surprising is that with Spencer's astute teaching, we were able to slowly, but surely, grasp the basics and start to understand the true magic of UE. For example, before this class, I used to look at Blueprint and it would give me serious anxiety and now I actually love the idea of Blueprint and having coded pretty simple HTML back in the day, I realize how beautiful Blueprint is for non-coders. Don't mind my poor shot below, but sometimes I'd just take actual pictures instead of screenshots.

There were several assignments in this class, including creating a basic environment from scratch, setting up basic scenes, lighting and 'editing' them and exporting via sequencer. This entire module was online, except the first day and the last day, where we actually put our nDisplays to LED TV's to simulate aspect of the on-set UE tech responsibilities. This involved measuring tapes, looking at X,Y,Z axis and working with "Listener"...oh I am already realizing that you really have to use this on the daily to retain it all.

I thought this module really addressed all the aspects of the pipeline, as a good intro to the basics and then we were ready to split off into our streams.

Module 2 - MOCAP STREAM

Our instructor, Jason Hunter, is a mocap guru. I must admit, he was one of the reasons I wanted to sign up for this micro-credential specifically. At the information session, he was very knowledgable and could answer just about any question you throw at him. MOCAP by nature is more physical, so our stream was the one that was in-studio the most (by about a day). I really appreciated how 'simple' things started, which essentially was, take the Optitrack cameras out of the boxes, and set up a volume on simple stands. Essentially, all you needed was a large space, stands, Optitrack cameras, lots of POA network cable, sand bags for stands, and then of course some decent computers, one with Motive by Optitrack and the other with UE.

The next few weeks involved us getting into suits ourselves - as it's always good to put yourself in the performer position to know what they deal with - and also putting trackers on others in suits. Cue the many sequences of nonsense messing around with swords and lots of fun 'action' that we captured.

I am forgetting to mention wanding...you have to wand in order to set up the volume....which is actually very beautiful to look at on Motive.

Once you have all your takes (and they are called .tak files) you have to review them, clean them up and there are other steps that involve editing together files for reference with time code and reference video, so that you can hand of your MOCAP to the production team and they will have what they need. One thing that I found interesting/ frustrating is that you have to stream via LiveLink into UE if you want to showcase your MOCAP in-engine, because if you just take existing .tak files, export as .FBX, then you will have to re-target. I hope that one day soon, re-targeting might be avoided in some magical way. Sorry, if I have lost you. MOCAP, in summary, is a highly detailed job. You really have to be exact, in set up and especially in clean-up, and once everything is set up and working, you can 'almost' relax. I have to give a shout out to my classmates, especially Deepak Nambiar, who covered for me many times, when I was dealing with family responsibilities. This is something I have not even touched on yet, but what really made this MC memorable, were the amazing classmates who were stellar people from all different backgrounds and fields. One of our 'major' assignments in MOCAP was to create all the previs for the final shoots. So we broke down scenes and each 'directed' the mocap, performed by some awesome performers that Sheridan had secured.

Module 3 - Leadership and Collaboration

This will be covered in a future blog post...stay tuned!