AI relighting after the shoot: where it works and where it breaks

«We'll fix the light in post» now has a measured answer. On 14 October 2026, a team at the Hochschule der Medien in Stuttgart published a peer-reviewed evaluation of three AI relighting tools in Frontiers in Communication, using real test footage and eight lighting professionals scoring the results. They hold up on wide shots. They break on skin, on fine texture, and the moment anything moves.

Until now, judging that promise meant trusting demos picked by the vendor. This study changes the starting material: shots that were actually filmed, a panel that lights for a living, and the scoring criteria stated up front. On the same day, Maxon shipped an AI lighting tool for After Effects, although it does not do what the three evaluated tools do.

What comes out of it is usable in a production meeting: a rule of thumb by shot size, by source material, and by who signs off on the light.

An exam with eight professionals in the room

David Brenner, Katja Schmid and Jan Adamczyk put three tools through five test shots and ran the results past four cinematographers, two gaffers and two colourists.

The three tools take different approaches. NVIDIA DiffusionRenderer is open source and driven by HDRI maps: you pick the hour and the sky, not the position of your key or how it is shaped. There is no flag and no diffusion inside an HDRI. Beeble Studio runs locally, up to 4K, placing virtual lights over a physical estimate of the scene. SwitchX, also from Beeble, is generative: you guide it with a reference image, and it runs in the browser at 720p.

The shots were not picked at random either. Two wide exteriors: one in hard backlight, the other in low late-afternoon sun, converted to night. Two over-the-shoulder interiors. And a bicycle shot, with plenty of movement and 35 mm grain.

Where it works

Relighting delivers on wide and establishing shots, and on shoots constrained by weather, schedule or budget. The day-to-night conversion with SwitchX was judged convincing, though the authors present it as an indicative case rather than a general conclusion.

One point of vocabulary, because it gets muddled daily: what these tools do is not day for night. Day for night is a capture technique: you shoot in daylight, underexposing and protecting the sky. This is a conversion in post, which is precisely what it would replace.

The fit is economic before it is aesthetic: it earns its place on shots whose reshoot means going back to the location with a crew and a day, not on shots you fix by moving a lamp.

Where it breaks

The failures cluster where the eye is least forgiving. Smoothed skin, altered features, simplified vegetation, elements that appear or merge for no reason. On the bicycle shot, the one with the most movement, flicker showed up.

The study notes that none of the three preserved the photochemical grain of the 35 mm take. That one deserves reading carefully, because on its own it says little: any VFX pipeline degrains, works clean and regrains at the end using grain from the negative itself. It is routine, not a wall.

The real problem sits underneath. The model does not only remove grain: it takes the microtexture that was under it: skin pores, the weave of a fabric, the relief of a wall. No regrain brings that back, and it is what makes a relit shot look a little too clean.

Each tool also fails in its own way, which shapes the workflow around it:

• DiffusionRenderer offers no preview: every attempt is a blind render.
• Beeble Studio is the fastest and most interactive, but scored worst, with a look the interviewees described as flat and digital.
• SwitchX earned the best marks in exchange for less predictable control: backlights and flares turn up that nobody asked for.

The trade-off is the familiar one in generative work. Predictable control with a poor result, or a good result with little control.

Maxon's Scene Light does something else

On that same 14 October, Maxon released Red Giant 2027.0, and the headline feature was Scene Light, an After Effects plug-in. It has been reported as bringing AI relighting to the edit suite. That is not quite it, and the difference matters when you are quoting a job.

The three tools in the study replace the lighting: they estimate how the scene is built and light it again. Scene Light is additive. It adds lights on top of the shot you already have, but it does not take the original lighting away. A badly lit shot is still badly lit afterwards.

What it does is estimate depth and normal maps from the footage with AI, so that the lights you add respond to the relief in the image. According to Maxon, they fall behind foreground objects and wrap around surfaces instead of lighting a flat image.

That description needs bounding. Depth and normals estimated from a single camera give you 2.5D: relief, not geometry. You get occlusion order in Z and shading response on whatever faces the lens, but no shadow cast onto another surface, no bounce, no speculars from anything off-frame. On set, light comes back; here it only modulates the pixels you can see. It is the same reason Beeble gets called flat.

It accepts depth and normal passes from Cinema 4D or Redshift, which is where the real mileage is. If you export them, watch three things or it will not line up first time: the normal space and the orientation of the green channel, depth remapped linearly to near and far instead of scene units, and data passes rendered with no filtering or antialiasing, because edges interpolate values that do not exist.

Two practical notes from the release. According to Maxon, all processing runs locally, which settles the conversation about confidential material. And there are two known bugs: edge softening does not currently work, and the buttons for point lights and spots can freeze After Effects when the interface is set to a language other than English.

That second one gets misdiagnosed: it looks like a driver or GPU problem and costs you an afternoon. If you work in another language, switch the interface to English before opening the effect. Be aware it requires a restart and renames the effects, so it breaks expressions and scripts that call them by their localised name.

What this changes if you shoot in the Canary Islands

Here the light runs the schedule, not the other way round. The calima, the Saharan dust haze, rolls in mid-morning and flattens contrast for the rest of the day; a coastal exterior gives you hard backlight at almost any hour; the golden-hour window comes down to twenty usable minutes. It is the scenario the study marks as favourable, because these are shoots constrained by weather, schedule or budget.

In our work it fits landscape and wide establishing shots, where the piece rests on the open frame and the time of day, and archive material, which starts from the same problem of keeping what was filmed and changing only what is needed, with the added requirement of recording what was touched.

We do not use it on close-ups, interviews or testimonials. And relighting does not go into a shot without sign-off from whoever shot it, even when the shot gets touched months after the wrap.

Six questions before you commit a shot

1. What colour space and bit depth does it take in and put out? These models work in Rec.709. Coming from S-Log3, LogC or ACEScg means transforming both ways, or the falloff lands on the wrong curve. And if the only source is a compressed 4:2:0 8-bit master, the answer is no.
2. What resolution does it deliver? A 720p tool is for deciding a look, not for delivering.
3. Does it match the neighbouring shots? The shot does not exist alone: it sits between two lit on the day. Relighting one breaks continuity with the ones either side, and that is what sinks most of these jobs.
4. Does the face own the frame? If so, physical light.
5. Is there heavy movement? That is exactly where flicker shows up. Ask for a test, and budget it: it is a day of somebody's time.
6. Who approves the light, and in which room? That is the condition the professionals in the study set themselves.

From generating to modifying what was filmed

The Stuttgart study, Scene Light and much of what the labs published in October point the same way: start from the footage and change one layer. The demo is no longer generating the whole shot, it is swapping its lighting.

For a production company that pushes work backwards, into the shooting plan. You decide before you leave what you really capture, what you leave for post, and how you cover the shot if the tool does not get there. When a shot's lighting is going to be decided later, put it on the call sheet and in the budget, not in a conversation.

AI does not replace lighting on set, but it serves as a selective tool for correction and extension, and it moves some lighting decisions into post-production.

That is the authors' conclusion. In production terms: it is there to save a shot you cannot reshoot, not to take an electrician off the crew.

Sources

Checked as of 17 October 2026:

• Brenner, D., Schmid, K. and Adamczyk, J., «AI-based relighting in moving images», Frontiers in Communication (14 October 2026). Peer-reviewed study.
• Maxon, Red Giant 2027.0.0 release notes (14 October 2026), including the known Scene Light issues.
• Digital Production, «Maxon 2027.0: More Trees, Fewer Detours».

If you have a piece where the light is the risk (an exterior with a short window, or archive that needs recovering) our AI VFX service starts exactly there: from what is already on the card.