Visual effects in film and TV 3

Zapsat Zobrazit rozvrh

Garant předmětu

Antonín WEISER

Jméno vyučujícího (jména vyučujících)

Antonín WEISER

Katedra

Předmět zajišťuje Katedra kamery

Obsah

Předmět navazuje na přednášky „VISUAL EFFECTS IN FILM AND TV I.+II. Na filmových ukázkách ze světové tvorby i z kameramanské praxe přednášejícího provedu posluchače technikami 3D ve filmovém a televizním průmyslu. Výrobou používaných efektů ve 3D. Jejich aplikací do různých technologiií a současným využíváním v kinematografii. Současně s přednáškou probíhá seminář praktických úkolů „VFX cvičení“ v atelieru katedry v LP v pracovně LDK. Cvičení provádí posluchači samostatně pod dohledem odb.as,George Pinkavy a doc.A.Weisera.

LED PROJECTION / VIRTUAL PRODUCTION

doc. Mgr. Antonín Weiser

1. Principle of LED Projection / Virtual Production

Basic Principle

• The LED wall displays the virtual environment directly during filming.

• Compared with greenscreen, it provides a finished background, interactive lighting, and real

reflections.

• Virtual production shifts a significant part of the work from post-production to pre-production.

Real-time rendering – types of content displayed on the LED wall

2D – pre-recorded image / plate

• Conventional video or a still image displayed on the LED wall.

• The image perspective is fixed – without true parallax.

• The camera should remain predominantly static.

• Suitable for simple backgrounds, car-driving scenes, or landscapes.

• Lowest computational requirements.

2.5D – spatially separated layers

• The 2D image is divided into several planes at different virtual distances.

• Camera movement creates a limited simulation of parallax.

• Allows greater camera movement than pure 2D.

• Lower computational demands than full 3D.

• A compromise between a pre-recorded plate and real-time 3D.

3D – real-time virtual environment

• The environment is created as a true 3D scene, typically in Unreal Engine.

• Camera tracking sends the camera’s current position and orientation to the engine.

• Perspective is recalculated in real time.

• The frustum displays the correct perspective for the camera’s current position.

• True spatial parallax is created.

• Highest demands on rendering, tracking, synchronization, and system performance.

• Main risk: latency.

Remember: 2D = no parallax → 2.5D = simulated parallax → 3D = real-time parallax.

1. LED Wall – Technical Parameters Important for the Cinematographer

Pixel pitch

• The distance between the centers of adjacent LEDs.

• Determines the minimum usable distance between the camera and the wall.

• Smaller pixel pitch = the camera can be positioned closer.

• As a guideline: 2 mm ≈ 4 m; 10 mm ≈ 20 m.

• P2.6 – common VP standard; P1.9 – higher quality and greater demands.

• Main risk: moiré and visible pixel structure.

Refresh rate

• Determines the refresh speed of the LED panel.

• Event walls around 3,840 Hz may be borderline for camera use.

• For virtual production, approximately 7,680 Hz or higher is preferable.

• A low value may cause banding and problems when changing fps or shutter settings.

Scan rate

• Determines how many LEDs are refreshed simultaneously.

• Higher multiplexing can create bands or unsynchronized areas of the image.

• For VP, approximately 1/16 at minimum; 1/8 is preferable.

Viewing angle

• For camera use, an angle of approximately 170° or more is desirable.

• At extreme side viewing angles, green or magenta color shifts may occur.

Brightness

• For VP, a wall of approximately 1,000 nits or more is suitable.

• Wall brightness is a working parameter, not merely a technical specification.

• Insufficient brightness can limit interactive lighting and exposure options.

Contrast and black level

• An unlit LED is not perfectly black.

• Panels and reflectors bounce studio light and raise the black level.

• Protect the wall from stray light – use flags, directional lighting, and distance.

• During preparation, switch off the image and check reflections, seams, and hot spots.

Color gamut and spectrum

• An LED wall is primarily a display device, not a full-featured film lighting fixture.

• A narrow RGB spectrum can reproduce cyan, orange, and yellow poorly.

• The LED wall alone is not ideal for skin tones.

• RGBW / RGBCA systems may provide a more suitable spectrum.

• Standard colorimeters may not measure narrow-band LED light reliably.

1. Rendering, Tracking, and Synchronization

Camera tracking

• The system must know the camera’s position and orientation in real time.

• Tracking sends data to the rendering engine, which changes the background perspective.

• Fixed optical systems or sensors mounted on the camera are used.

• Tracking accuracy must be checked and calibrated during shooting.

Lens mapping

• The system must include focal length, focus, distortion, and other lens parameters.

• Lenses should be mapped during the preparation day.

• Every lens change requires an update of frustum and parallax parameters.

Genlock

• The camera, tracking system, engine, media server, and LED processors must be synchronized.

• Genlock aligns the timing of image capture and frame generation.

• Without synchronization, line-scan aliasing and other image artifacts may occur.

• A film camera can also be synchronized using a film/video sync box.

Shutter and frame rate

• Genlock alone may not solve all scan-rate issues.

• The shutter angle must match the capabilities of the LED processor and panels.

• ShutterSync makes it possible to adapt the wall refresh to the camera shutter.

• Always test the specific combination of camera, fps, shutter, panel, and processor.

Multiple cameras / high fps

• Maximum fps is limited by panel quality and the data capacity of the LED processors.

• Higher fps may require significantly more processors.

• Frame Remapping / GhostFrame can display different content for two cameras.

• Disadvantages: flicker visible to actors and reduced wall brightness.

1. Preparing Content for LED

Choosing the content type

• 2D – simplest and least demanding; without true parallax.

• 2.5D – a compromise between flexibility and computational demands.

• 3D – greatest spatial realism, but the highest demands on rendering and tracking.

• The content type must correspond to the planned camera movement.

Preparation

• The virtual environment must be prepared and approved before filming.

• Final framing, lenses, lighting, and continuity with the physical set must be taken into account.

• The more complex the real-time environment, the more important the test days become.

1. Choosing the Camera and Lenses

Camera

• A global shutter or a very fast rolling shutter is advantageous.

• The ability to synchronize precisely is important.

• The quality of the low-pass filter affects susceptibility to moiré.

• A larger sensor makes it possible to use longer focal lengths and shallower depth of field for the

same framing.

Lenses

• Longer focal lengths and wider apertures help blur the LED pixel structure.

• Softer rendering can help blend the physical foreground with the LED background.

• Very sharp lenses can emphasize the wall structure.

• Anamorphic lenses can help through their rendering character, but they require more thorough

preparation.

• In wide shots, make sure the camera does not move outside the LED volume coverage.

Filters

• Diffusion filters reduce edge sharpness and may decrease the visibility of the pixel structure.

• Their effect must be tested with the specific lens and LED wall.

1. Exposure and Camera Work on Set

Exposure

• Do not use LED wall brightness only as a tool for matching exposure.

• Treat the wall similarly to a real location.

• In bright scenes, exposure can be adjusted with ND filters while preserving the working

brightness of the wall.

• In practice, it may be advantageous to keep the wall at high or maximum brightness.

Matching foreground and background

• The most important factor is matching the brightness and color of the physical foreground to

the virtual background.

• Backlight, sunlight, and other strong sources are critical.

• False Color and EL Zone help control brightness relationships.

• An incorrect lighting ratio between foreground and background quickly reveals the illusion.

1. Color Management

Calibration

• The LED wall is calibrated for a specific camera system.

• A different camera model may show color deviations on the same wall.

• A dedicated camera test is necessary even in a previously calibrated studio.

• The reference white of the LED wall is typically D65 ≈ 6,504 K.

Color evaluation

• Do not evaluate LED light by eye alone.

• Use the camera and a calibrated monitor.

• Standard light meters and colorimeters may be inaccurate with narrow-band LED spectra.

Corrections

• The image on the wall is also light falling onto the scene.

• Therefore, changing the background color also changes the color of the interactive light.

• Keep the base content as neutral as possible.

• Check final decisions through the camera / DIT workflow.

• In the case of spectral problems, the content can be corrected selectively, but always according

to a camera test.

1. LED Wall as a Light Source

Interactive lighting

• The LED environment creates natural ambient light and colored reflections.

• Reflections in glass, metal, windows, and glossy surfaces are a major advantage.

• Light cards outside the frustum can serve as additional virtual light sources.

Limitations

• The LED wall is not ideal as the only source for lighting skin tones.

• Strong sources – sun, fire, hard highlights – usually need to be supplemented with film lighting.

• Light from the wall may have incorrect softness or falloff relative to the virtual distance of the

source.

• If the physics of the light do not match the image, replace the lighting effect with a real fixture.

Distance from the wall

• Scattered studio light reduces the black level of LED panels.

• The ideal distance between the set and the wall may be approximately 2–3 m; in practice it is

often smaller.

• The transition between the physical and virtual floor is particularly problematic.

1. Typical Image Problems

Moiré

structure.

• Caused by interference between the LED wall pixel structure and the camera sensor pixel

• The risk increases at short distances and with high background sharpness.

• Greater distance, a longer focal length, a wider aperture, and focusing in front of the wall can

help.

• In critical situations, the LED wall should not be in sharp focus.

Banding / tonal transitions

• Dark and low-contrast scenes can reveal an insufficient number of tonal steps.

• The result may be visible bands and posterization.

Black level

• An LED panel cannot produce absolute black.

• Stray light can turn black into a milky gray.

• In very high-contrast scenes, the foreground black level can be kept slightly higher and lowered

in post-production.

Color shifts

• At an unsuitable camera angle to the panel, green or magenta shifts may appear.

• Check the viewing angle of the panels.

Texture and atmosphere

• Film grain or film emulation can help unify the physical and virtual parts of the image.

• Haze can unify the image, but it also reduces wall contrast – use with caution.

1. Latency and Camera-Movement Limitations

Latency

• The signal passes through the chain: tracking → engine → rendering → LED processor → panel.

• Each stage adds delay.

• Latency is most visible during fast movement and with strong spatial reference points.

Problematic movements

• Fast pans and whip pans.

• Dynamic crane movements.

• Fast circular tracking shots.

• Very unstable handheld camera work.

Possible solutions

• With 2D / 2.5D, some dynamic movements can be executed without real-time parallax.

• For dynamic tracking shots, allow for unusable acceleration and deceleration at the beginning

and end of the move.

• A selected part of the frustum can be pre-rendered to reduce computational load.

• Every dynamic shot must be tested in advance.

1. Physical Set and Integration with the Virtual Environment

Foreground

• The physical set is crucial to the credibility of the final result.

• Build several depth layers between the camera and the LED wall.

• The virtual environment should continue naturally beyond the physical set.

Floor

• The transition between the physical and virtual floor is a critical area.

• Check continuity of perspective, texture, lighting, and shadows.

• The virtual sun must not light the background differently from the adjoining physical area.

Reflective surfaces

• Reflections from the LED environment are a major advantage of the technology.

• Large curved surfaces can also reveal the edge of the wall or the studio.

• For cars, glass, and chrome, the largest possible LED coverage is advantageous – ideally

including the ceiling.

• Framing must respect the actual extent of the volume.

1. Production Workflow, Preparation, and Limits of LED Use

Workflow

• LED is not a cheaper greenscreen – it is a distinct method of VFX shooting.

• The biggest change is the shift of part of post-production into pre-production.

• Tests of the camera, lenses, background, tracking, synchronization, and color are essential.

• Time must be allocated on set for calibration and technical adjustments.

• Tracking accuracy may change during the day – check it continuously.

When not to use LED as the final background

• LED can be used only as an interactive light source.

• The wall can display a clean keying background for subsequent post-production.

• The background can be intentionally defocused and replaced in post-production.

• A hybrid workflow may be more advantageous than trying to finish everything in-camera.

Final principle

shot.

• The technology works best when the method of using LED is chosen according to the specific

• The decisive factor is not the wall itself, but preparation, testing, and the integration of camera,

lighting, set, and virtual environment.

Výsledky učení

Seznámit posluchače se všemi okruhy 3D postprodukčních technologií. Absolvent, magisterského oboru CINKK, musí mít orientaci, znalost použití a schopnost zasvěcené diskuze s tvůrci 3D, aby mohl zůstat výhradním autorem obrazové složky kinematografického díla.

Předpoklady a další požadavky

Absolvování přednášek ing.Hřebačky - Digitální Workflow 1+2..

ing.Bernas – Kolorimetrie, Visual effects in film and tv I+II..- odb.as.J.Ježek, doc.Mgr.A.Weiser

Kurzy: odb.as.J.Ježek – SkatchUP, a Frame Forge ,

Literatura

Richard Rickitt „The History and Technique Special effects“,

Steve Wright „ Digital Compositing for Film and Video“, First edition

Steve Wright „Digital Compositing for Film and Video“, Second Edition

Dough Kelly :“Digital Compositing in Depth“ - Coriolis Group

Jon Gress „Digital visual effects & compositing“, 2015

Hodnoticí metody a kritéria

Credit is awarded based on Lecture and class participation. The final exam is individually as a test and interview; topic questions, process explanations for particular examples.

Grading: level of knowledge and bearing in the study.

Poznámka

in progress

Rozvrh na zimní semestr 2026/2027:

• Týdenní přehled
• Seznam

Rozvrh na letní semestr 2026/2027:

Rozvrh zatím není připraven

Předmět je součástí následujících studijních plánů

• Cinematography_3_2021 (Povinné předměty)