Applying the Power Window effect

01-Applying-Power-Window-

I’ve been both terribly busy and sick lately, thus the lack of the tutorial last week. Hopefully, something interesting will come out of my business next week, and I would advise you to follow my blog during NAB, if you are interested on my take on a few things that might come up there.

But until then, here is this week’s tutorial:

Major improvements in Vignette and Power Window

matrices

Some of you have complained that my plugins sometimes take a lot of memory to run, and that they are not CUDA accelerated. Well, GPU acceleration will have to wait for a moment longer, but I have coded a few improvements to the Power Window and Vignette which will result in less usage of memory, and much quicker render times.

It used to be, that on the faster machines the CPU usage during rendering these plugins fell from 90% to 15-20%. I did not really know why. It was not multithreading – this is something that Premiere handles on its own. As it turned out, it was an issue of poor coding, and lack of knowledge.

I fixed that, and I also added clamp input black control to Power Window which should fix the problem with low blacks causing problems in gamma calculations, and added various alpha falloffs to the Vignette. The impatient ones may follow to the downloads page, and grab the newest versions. Those interested in details please read on.

Both Vignette and Power Window use the same part of code to calculate the position of the mask. Up until this release, these calculations were performed separately for the move, scale and rotation, for each pixel, for each frame. I was clever enough to calculate the sinuses and cosinuses at the effect setup, but it still turned out to be a significant number of calculations.

Those of you who know about matrices, affine transformations and homogeneous coordinates, have already most likely buried their faces in their hands, crying “why, oh why did you do this? Didn’t you know any better?” As it turns out, I didn’t. And I should have.

I’m a physics major. The knowledge and skills I gained during the studies are really helpful in many aspects of my life, including coding these plugins, and – of all things – understanding how camera sensors work. I used to learn algebra and matrices as far back as high school. And I was pretty decent in understanding them.

And yet, I missed the obvious: that I should have created a single transformation matrix for all the transformations, and apply them all in one go. And had I attended a course on computer graphics during my studies, I would most likely know, that the mathematicians have already came up with some nifty way to resolve my problems. As it was, I learned about the proper way to do this only recently, when I found strange 4D matrices for 3D transformations in Nuke. After doing some research on homogeneous coordinates, with a subtle push from my friend Wojtek Bagiński, I finally decided that I’m going to apply this knowledge to my plugins as well.

The result was far more dramatic, than I would have ever anticipated. Finally the plugins started to use 90-100% of CPU power, and the memory usage dropped down considerably.

Originally there were 11 multiplications, 3 divisions, 4 subtractions and 2 additions to calculate an ellipse. I changed it to 4 multiplications and 4 additions, so it’s a reduction by the factor of 3-4 with a few type conversions lost as well. And the end result is that, even though these plugins are not yet GPU accelerated, you can preview them in the real-time on most of the machines.

It’s good to learn new things. And even better – it’s better to know things so that you don’t have to reinvent the wheel, since somebody has already done it for you. When you don’t know that something exist, you don’t even know what to look for, and your solutions to common problems will most likely be suboptimal.

Go out and learn then :)

The brighter side of Anywhere

AdobeAnywhere

Today another interesting thought about Adobe Anywhere struck me. Essentially, part of the idea of Anywhere is to totally divide the UI from the renderer. The concept in itself is nothing new, and all 3D applications use it. However, as far as I know, it is the first time that it has been applied to an NLE.

The obvious implication is the possibility of network rendering, and using more than a single machine for hefty tasks. Of course, there are a lot of caveats to multi-machine rendering, and we already know that not all problems scale well, and sometimes the overhead of distributing processes is higher, than the gain in speed. But in general, the more means the better.

The less obvious implication is the possibility of running various types of background processing, like rendering or caching parts of a sequence that are not currently being worked upon, if the resources allow. This means a quicker final render time. It also allows to run multiple processes at the same time, or even multiple parts of the rendering engine at the same time, allowing for better utilization of server’s computational power.

However, the most overlooked implication is the fact, that the renderer is UI agnostic. It doesn’t care what kind of client gets connected, it only cares that the communication is correct. And this has potentially huge implications for the future of applications in Adobe’s Video Production line.

Pipeline

Anywhere Renderer most likely already consists of several separate modules – After Effects, and Premiere, possibly even Lumetri (SpeedGrade), which can be chained in any sequence. For example, the client sends a request to render a frame at half-resolution consisting of the following stack: V1 P2 clip, V2 Red One clip with Ultra Keyer on it, V3 Dynamic Linked AE clip with lower thirds, and a color correction via Lumetri on top of it all.

Adobe Anywhere makes Premiere renderer open a Premiere sequence, finds a dynamic linked clip overlaid on a source material, and asks After Effects to render it, at the same time composits the V1 and V2, and once AE is done, composits the result together. Then it sends the whole thing to Lumetri for color correction. Each stage is most likely cached, so that when the CC or the keyer is changed, the AE does not have to re-render the thing. Then the frame is rendered and sent out to the client.

Running locally

However, it’s all in the client-server architecture – you might say. How about those of us who use the programs on a single machine? Anywhere will not have any impact here… or will it?

There is nothing stopping Adobe from installing both the server, and the client software on the same machine. After all, the communication does not care about the physical location of client and server that much. It cares about the channels, and whether the messages are being heard. Perhaps the hardware might be a little bit problematic, considering the fact that both UI, and render most likely use GPU acceleration. But other than that, all Adobe needs to do is to distribute the Adobe Anywhere Render Engine as part of any local application. It would perhaps be custom-configured for local usage, to streamline some tasks, but it’s going to be the same Anywhere.

And that, dear readers, could be huge.

Implications

Separating the UI and a renderer is a brilliant move. In the long run it allows Adobe to alter the client without rewriting or even incorporating the renderer code in the application. For all Anywhere cares, the UI could be as simple as an HTML5 application which would send and receive the proper messages. Need I say anything more? Let your imagination run wild already.

The regular readers perhaps already see where I am going with it. The newcomers are encouraged to read about my vision of Adobe conforming tool, and – why not – Stu Maschwitz’s proposition of merging Premiere Pro and After Effects, or my hopes of seeing a Smoke-like ubertool from Adobe. Any such application could access Anywhere’s backend, and could be optimized to suit specific needs, giving birth to a number of tools specific to certain needs. Tools which are easily written, quick to update, perhaps even accessible via mobile devices. And in time they might also work locally, on a single powerful machine. Or on a number of them. Wherever you prefer. Anywhere.