PEV04 – Norman Hollyn on Performance Enhancing Visual Effects

pev04_fi

This interview is part of the larger series about Performance Enhancing Visual Effects. Norman Hollyn is an editor best known for his work on Heathers. He’s a professor at USC, where he teaches editing. He wrote two books on this topic as well, “The Film Editing Handbook” and “Lean Forward Moment” which I heartily recommend. For those more visually inclined, a great course of his, “Foundations of Video: The Art of Editing“, is available on Lynda.com, and you can find a few of his webinars on Moviola.com

Before I published my previous interviews with Alan E. Bell and Zbigniew Stencel, I sent Norman drafts of those and asked for his opinions. Here is his reply. see more

Adobe Premiere Pro and After Effects Plug-in Development Course on fxphd.com

sys204

I’m happy to announce, that this term I’m teaching plug-in development for Adobe Premiere Pro and Adobe After Effects on fxphd. I’ll be covering the basics of C/C++ and Adobe After Effects SDK in a bit more detail, as well as GPU acceleration for Premiere. All these will be illustrated with a simple vignette effect that we are going to create during the course with the students. Believe it or not, any similarities to the very first effect that I wrote are coincidental. see more

PEV03 – Zbigniew Stencel (Editor) About Performance Enhancing Visual Effects

pev03_fi

This interview is part of the larger series about Performance Enhancing Visual EffectsZbigniew Stencel is a Polish film editor. He worked on recently released  No Panic, With A Hint of Histeria, and Czułość, which is going to arrive to the cinemas somewhere in the first quarter of 2016. Read on what he has to say about PEVs.


Please briefly introduce yourself.

My name is Zbigniew Stencel. I’m an editor based in Poland. I graduated from the Polish National Film, Television and Theatre School in Łódź (specialization in film editing), I have cut documentaries, music videos and ads. During recent years I focused on various 3D stereoscopic projects. Just now I edited two feature films.

How did you first learn about Performance Enhancing Visual Effects (PEVs)?

I guess Kirk Baxter, Angus Wall and Alan Edward Bell got me inspired. I read and watched various interviews and videos with them, where they described the process. But I used these kind of effects intuitively before. I thought it was cheating – and I was ashamed that I couldn’t find a “proper” editing solution to a problem in the movie. I was taught to always find a way using a simple cut, not to resort to “tricks”. When I learned about PEVs I was relieved, that maybe it’s not cheating – that it is the evolution of editing techniques. I think using PEVs gets editing even closer to directing.

Could you describe PEVs in your own words?

PEVs are techniques of merging audiovisual elements of film that are at editor’s disposal. It’s mainly visual elements, but I extend that to audio, with all the ways that sound can help “sell” a shot or a scene. It’s the synthesis of what was shot on set but not at the same time. It’s a collage with consistency. PEVs create what was intended on set but never happened there.

Can you recall the first time you used these effects, and can you tell us what your thought process was at that time?

I guess it was one of the films in film school. Two characters were sitting on a bench. I wanted to hold a shot of them sitting for a little longer, but one of the characters got up and walked away. I did a split screen with slowdown on this moving character, so the two of them sat together a little longer. It looked acceptable, so I left it in the movie.

I too have heard of PEVs referred to as “tricks” not related to storytelling even by experienced editors. Where do you think this perception might come from? Is it the unfamiliarity with the technique, a misunderstanding, or is there something more to it?

Experienced editors might think that PEVs will replace basic skills and techniques of storytelling in film editing. PEVs are expansion of our skill set. Maybe some not tech-savvy editors are afraid that these techniques are difficult, or that the “classic-NLE” means of editing will become obsolete. I think any problem can be resolved with simple cuts and make particular thing work in the movie. With PEVs we can not only make something work in the edit, but make it look better.

You said that PEVs make editing more like directing. This is a very astute observation. Can you elaborate on that?

When you make some actors do things that they didn’t do on set, you’re stepping into the shoes of the director. Using PEVs is like doing a re-shoot. In most cases PEVs help with problematic situations with actors in the footage. Directors and actors are very close on set, but in the editing bay the intimacy starts to grow between the footage and editors. This relationship is not as direct as its counterpart on set, but editors are still “saying” to actors “do this, do that” with cuts, and even more with PEVs.

Zbig in the editing/colour suite. He decided to avoid showing his favourite NLE - because he loves them all.

Zbigniew in his editing/grading suite. He decided to avoid showing his favourite NLE – because he loves and knows them all.

Do PEVs influence the way you approach editing?

I try to use PEVs as the last resort. I dive into the footage to find what I may have omitted, to look at it from a different perspective, to find gems. Only when I’m sure I will not find what I’m looking for, I use PEVs. It would be lazy to use PEVs before that. All those editors in the history of cinema resolved difficult situations with their creativity – we cannot lower our standards and standards of our profession.

Does the knowledge and ability to perform PEVs allow you to be a better editor and tell better stories?

It’s always about telling a better story. Be it PEVs, transitions, effects – they all should serve the story. Cliché, but that’s what I think. But I’m a fan of a simple cut. I’m more proud of myself, when I resolve a problem with straight cuts than with PEVs.

Do PEVs change the way you look at the footage and think about it?

Yes, but I try to use “PEVs perspective” on footage only when all other ways of problem solving in editing failed. When I see the footage for the first time I’m not automatically analyzing “Here I will use split-screen, here I will do a slowdown and add a mask”. All these ideas come later if they’re really required.

What differentiates PEVs from the “true” editing?

I think PEVs don’t make the editing any less “true”. But maybe cutting on Steenbeck, KEM or Moviola was “true” editing. It was all about cuts. Any dissolves involved sending a cut to laboratory, not mentioning more difficult processes. I was editing on Steenbeck for a year in film school and I thought about simple cuts, not masks, slowdowns, cloning or repositions. So maybe the “true edit” is something that was available almost since the dawn of cinema: a cut.

Do you think PEVs should be taught to new editors in film schools? To what extent?

When students understand the basic principles of editing and they can solve issues with techniques known to the editors in the era of film – then yes, they should be taught to use PEVs. But in the first years of education, maybe, just maybe, they should be forbidden to use them.

Do PEVs influence your relationship with the director?

Yes. At first, the directors were hesitant to use this technique. They didn’t know that the tools were in an NLE. After few shots done this way which looked good, they were like: let’s merge this part of this take with the other shot into one. If I wasn’t sure that I searched through all the footage for particular scene or sequence I didn’t agree to use PEVs.

How often are you using PEVs?

I try to minimize the use of this technique, as I explained before. It can range from 5 to 15 shots in a movie.

How much are you doing yourself, and how much is passed to the VFX team?

I’m doing an offline version with offline resolution footage in an NLE. VFX guys get online resolution clips and they’re doing the final shot. I have to be sure to pass all the necessary metadata and references so that they can recreate what was approved in the offline. So the idea is created in editing room but it’s realized by the VFX department.

What was your simplest shot that you created?

I think that it was the one created in film school, still intuitively. I like these kind of simple treatments, but my ambition is to create – if a film I am editing requires it – some PEVs with face replacement or roto to smoothly add body parts. These kind of manipulations are still not my league.

Can you tell us about your most complex shot?

It was a reaction shot of six people. In each take they weren’t synchronized in their movements. I had to merge their reactions so they would turn their heads and bodies at the same time. The shot was static, but you had to compensate for slight differences in camera position and positions of the actors. It’s not that hard technically, but the choice of particular takes and timing is crucial. Of course, the VFX team made it even better.

And the most satisfying one?

The most complex shots are the most satisfying ones.

How much time do you spend on a shot?

From a few minutes to half an hour at the most. The time is precious in the editing bay so If I can’t achieve it in the short time I just make it less polished and leave it for the VFX team. Sometimes shots are more complicated, so NLE tools are not enough.

What kind of tools do you use to make PEVs in your daily work?

In today’s post production world there is no comfort in knowing only one NLE. So I use the one that’s desired for particular project. Inside Media Composer there’s the Animatte effect. Inside Premiere Pro – masks, that are now very nicely built into clips effects controls and other effects. In Final Cut Pro X – the Draw Mask effect. I also use retiming on selected composited shots – my favorite is FluidMotion retiming in Media Composer. So masks and retiming are basic tools for PEVs.

Thanks a lot for your thoughts.

PEV02 – Alan E. Bell (ACE) About Performance Enhancing Visual Effects

pev02_fi

This interview is part of the larger series about Performance Enhancing Visual Effects. Alan E. Bell (ACE) is the editor who invented the term and made it his trademark. He is most known for his work on The Green Mile, (500) Days of Summer, The Amazing Spiderman, The Hunger Games: Catching Fire, and recently The Hunger Games: Mockingjay part 1 and part 2. Read on to learn his thoughts about PEVs.

see more

PEV01 – An Introduction to Performance Enhancing Visual Effects

PEVs 01

The term “Performance Enhancing Visual Effects” (PEVs) or “Performance Based Visual Effects” was coined by Alan E. Bell, the editor of (500) Days of Summer, The Amazing Spider-Man”, and most recently The Hunger Games: Catching Fire and two parts of Mockingjay. It encompasses all manipulations of the source material aimed at making an actor’s performance better. Usually the alteration takes part only in part of the frame, as opposed to typical cutting and juxtaposition of whole frames. In my opinion the availability and recent popularisation of these techniques constitute a significant shift in the history and process of editing.

see more

The Inevitable Convergence III – The NLE/Grading Edition

feature-convergence-iii

With the introduction of Resolve 12, suddenly the race towards a unified NLE/Grading tool become very interesting. It’s hard to argue, that colour correction and grading became an integral part of post-production workflow. It’s also seemingly one of the low-hanging fruits, as opposed to visual effects and audio. Let’s see what is going on for all major players in this regard.

see more

The Bane of UI Minimalism

ui-feature

I am appalled at some trends in the User Interface design these days. Touch screen devices became an entrenched market a couple years ago, and suddenly everyone and their mother decided to optimise the User Experience (UX) to take into account this new animal. Nothing bad about the general idea, but unfortunately in some cases the results seem to have been the apparent devolution of the UI design.

see more

Rumours That Leave Some Terrified…

Adobe-nuke

When the news first came out that The Foundry is going to be available for sale in 2015, and my friend told me that there were rumours that Adobe might be interested in acquiring it, at first I dismissed it as rather unlikely. However, this week, first an Adobe employee tweeted The Foundry’s reel, and soon after an article in Telegraph confirmed this possibility, which makes things almost official.

Some of the early tweeter world reactions were not very enthusiastic, to say the least:

twitter

If you are familiar with the profiles of these two companies, the initial wariness is understandable. Adobe delivers tools for everyone, while The Foundry has been traditionally associated with high-end workflows for larger visual effects studios. Perhaps the dismay of some The Foundry users comes from the fact, that The Foundry does not really need anything from Adobe to supplement their great products in the niche that they positioned themselves at. Were it not for the venture capital (The Carlyle Group in this case) that simply wants to profit from their investment, there would hardly be any interest for The Foundry in mingling with Adobe. From Adobe’s perspective though, acquiring The Foundry is a perfect opportunity to fill in the areas, which have always been their weak points – true 3D (Modo, Mari, Katana) and high-end compositing (Nuke).

Personally I would not mind having an additional set of icons added to my Creative Cloud license. Depending on how this (potential) marriage is handled, it can be a beginning of something great, or a potential disaster to some. I am cautiously optimistic.

Both companies have their own mature line-up of products that are mostly self-sufficient. The real challenge is immediately obvious: integrating these is not going to be a piece of cake. For example, Adobe’s internal scripting platform revolves around JavaScript, while The Foundry’s is centered around Python. These are not compatible in any way, shape or form. Adobe has their own UI framework called Drover, while The Foundry is using a Linux-gone-multiplatform standard of QT. This is also very unlikely to change, and perhaps shouldn’t. To cater for the needs of large studios, The Foundry delivers not only for Windows and OS X, but also – and perhaps most importantly – for Linux. This is an area where Adobe has arguably limited experience – they released one version of Photoshop for Unix once, which was subsequently discontinued because of a total lack of interest. Will Adobe then have to develop at least the Creative Cloud Desktop application for Linux to handle licensing? This might be interesting.

The questions appear almost instantaneously: what will happen to the alliance between Adobe and Maxon, when they acquire their own 3D software package (Modo)? If Nuke becomes the main compositing tool for Adobe, how it will impact the development of After Effects as a platform, and what will happen with quite a few compositing plug-ins? This is the most obvious place where these technologies can clash, and some third-party developers might be left out in the cold. How much of the development power will be focused on integration, and creation of Dynamic-Link engines in all applications that talk to each other, as opposed to implementing new, cutting-edge features or fixing bugs? Without a doubt, it would be great to see a link to Nuke composition in Premiere Pro – and this might in fact be not so difficult to achieve, since Nuke can already run in the render mode in the background. However, how will it impact the development of “Flame Killer” Nuke Studio itself? Hard-core Nuke users will most definitely see the necessity to use Premiere as a hub as a step back, especially when it comes to conforming – an area, which is known to be an Achilles heel for Premiere (see my previous notes about it) – and the vfx workflow. And if we are to take hint from what happened with the acquisition of SpeedGrade, when most development resources were moved towards creating Dynamic Link with Premiere, and the actual development on SpeedGrade itself almost stalled, this might be worrying.

Certainly there are some valid concern about responsiveness of Adobe towards the usual clients of The Foundry, as the market audience for the products will inevitably shift. At the same time Adobe does crave to work on the higher end, and it’s much easier for high-profile people like David Fincher to ask for the features, and receive them, as opposed to common folks like you and me. So the studios will still have the leverage on Adobe. However, a challenge will come in the fact, that The Foundry tools (with the exception of Modo) are not as accessible and intuitive, as Adobe’s, and very often require extensive personal training to use properly. Again, Iridas acquisition being an example, Adobe will try to make small changes in the UI, where necessary, but in general the efforts will be spent elsewhere. Personally I don’t ever envision myself using a Katana which is most definitely a specialised relighting tool for high-end 3D workflows, mostly working with assets coming from the software owned by Autodesk. If I were to name a single product that is most likely to be dropped after the acquisition, it would be Katana. It would take quite a pressure from the studios using it to keep it in development. Adobe would have no skin in this game – in fact, possibly quite the opposite. One way or another, I highly doubt Katana will make it to the hands of Adobe’s typical end-user. It might become a separate purchase, like Adobe Anywhere is now.

On a good side, this acquisition will indeed make Adobe’s video pipeline next to complete. We used to snicker at the slides and demos suggesting or even insisting, that it’s possible to do everything within the Creative Cloud. We knew that making even a simple 3D projection in After Effects was an effort often destined to fail. A lot of great work has been made in After Effects despite its shortcomings, but the workarounds are often time-consuming – with Nuke at our disposal this would no longer be the case. It indeed has the potential to make Adobe a one-stop shop in post-production. And even more good news? The drop in price is inevitable, especially with the recent acquisition of Eyeon by Blackmagic Design.

If I am to make predictions, I’d say that initially some The Foundry products (After Effects plug-ins, Modo, Nuke, Mari and Mischief, if it doesn’t get integrated into Photoshop/Illustrator) will immediately become part of the Creative Cloud offer. Adobe will be showcasing Modo and Nuke to sell more CC licenses. A lot of users who just shelled out thousands of dollars for their Nuke licenses will be unhappy, but Adobe will most likely give some grace period for them – maybe in the form of free Creative Cloud licenses to current The Foundry users without active CC subscription or something similar. However, to avoid legal issues with Linux users, where Adobe is not able, and will most likely never be able to deliver their full line of Creative Cloud products, a separate offering will be made for this platform – perhaps on custom order, similarly to CC for Enterprise customers. Linux versions will keep up feature-wise at the beginning with their counterparts, but depending on the number of licenses sold this way, they might stall or be discontinued. Katana is most likely the first to go. The whole Nuke line will be integrated into a single product – hopefully Nuke Studio, but possibly to what is now known as NukeX. The latter would be unfortunate, as there is quite a lot of potential in Nuke Studio, but I’m not sure Adobe folks will understand it at the moment, as they seem to be only now learning about high-end vfx workflow. Hopefully outcry from the clients will be enough. Hiero, however will also most likely be dropped, as it essentially is redundant to conform part of Nuke Studio.

I hope some of the original The Foundry branding will be retained, but I am a bit afraid that we will quite fast see either square icons with Nuke symbol, or even letters Nu, Mo, Ma, Mc. Hopefully someone can point Adobe Media Encoder icon as a precedent, and at least the Nuke symbol remains intact. Adobe letter salad becomes a bit tedious to keep up with.

Again, if we are to take hint from Iridas acquisition, The Foundry development team will remain mostly the way it is – unless people decide that they don’t want to have anything to do with Adobe as a company, which does happen from time to time – but it will be integrated into Adobe culture. Adobe seems to be pretty good in this kind of thing, so the falloff should be minimal. Development-wise, most certainly the attempts at making exchange between various application easier will get priority right after making sure Creative Cloud licensing works. An importer of Modo files into After Effects, perhaps a bridge between After Effects and Nuke, sharing cameras, tracking data, scene geometry, and some layers; or attempts at Dynamic Link between Nuke and Premiere – these are my initial guesses. Perhaps even the XML exchange between Premiere and Hiero/Nuke Studio will finally be fixed, and at some point The Foundry applications will be able to read and/or write Premiere Pro project files. Adobe’s XMP model of metadata will most likely be employed throughout the Collective.

On a good side, it will allow The Foundry to focus – I had the impression that for some time this company began to behave like Adobe in times of CS5-CS6 – trying to expand the market, pumping out new flashy features instead of focusing on stability and bug fixing, and diluting Nuke line, or in general trying out to lure people to buy their products or updates. Creative Cloud subscription model, regardless of how it was perceived when introduced about two years ago, helps in this regard quite dramatically, as there is less pressure on the developers to cater to the needs of marketing department (vide introduction of Text node in Nuke) and maintaining various versions of the software. This should translate into more time and manpower being directed towards the “core” development – the good stuff.

I think this is promising – if it ever happens. There already has been a precedent of a lower-end company acquiring high-end tools and making them available for public without necessarily watering down their value. We’ve all seen it. Most of us loved it. The company’s name is Blackmagic Design, and the tools were daVinci Resolve and Eyeon Fusion. Here’s to hoping that Adobe handles this acquisition in a similarly efficient and successful manner, bringing the high-end 3D and compositing tools to the hands of many. That is, if this buyout ever happens. Because you know what? Why wouldn’t Blackmagic simply outbid them just for the sheer thrill of disrupting the market?