Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare
Ryan Schindler
4 Weeks
Lorem ipsum
• Create 3 different overlays on a 50 second clip of the girl walking through a hallway
• Themed to a silent hill style subway, a sand cave and a library
• Have the girl and camera move the same but distort the background
The biggest challenge face here was definitely keeping the camera movements and the motion of the main character moving the same with no random faces appearing while completely distorting the background. Keeping the consistency in a long clip especially with the zooming in of the hair where it seemed AI always wanted to get a little too creative.
After a lot of trial and error we eventually found a flow allowing us to do the full 50 second clip and keep good consistency and movement when creating these clips in the different themes. This was later layered into the original clip and short film and with some editing was used to father elevate the story.
To start of the project we had several discussion to understand the goal of the movie and the expectations for us. How do we come into play to really help tell the story and add to the horror.
We sat down to discuss the different scenes and themes we had to create. Our goal was to transform the scene of the main girl walking through the hallway into 3 different worlds. Themed as a sand cave, a library and a silent hill styled subway. This was all to help elevate the story of the short film and really put into place that the main characters mind was being manipulated.
We started creating some initial tests and examples to try get a feeling for the general vibe and what was possible and the important elements to keep. Which was the camera movement, the way the girl moves and if possible to keep the bright lights of the hall shining through.
With a short deadline in place and some idea of where to start we took to experimenting in order to find a flow that would allow us to essentially alter the entire background but keep the camera motion and girl the same and in motion. We knew comfyUI was the method. It was also important to make sure that it didn’t look too signature of a specific AI tool. We didn’t want the first thing for people to do is look at it and go “oh that was made in Dali.”
On ComfyUI we started playing around with control net types, different strengths and combinations. Switching up the prompts and the negative prompts and continuing to learn the little tricks of comfyUI before arrive to a near perfect flow. But due to the length of the video and the demands of the control net our GPU couldn’t quite handle the stress of a lengthy video.
There’s no doubt that every project has it’s challenges and it’s safe to say we had a few here. With AI being somewhat unpredictable and having an unrequited love for adding random faces everywhere, to a short deadline and a GPU that suffered burnout on a video longer than 20secs we had quite a few hurdles to jump over.
Flexibility and patients was something needed in great abundance. We had found our flow allowing for consistency and changing the background but took some altering the prompts and denoising in order to prevent a random face from appearing. After struggle with this we were eventually able to find a flow allowing for even more consistency and allowing us to do the full 50 second clip instead of just 20 secs.
After some back and forth and trial and error we were able to get to a point allowing us to create the AI special effects to theme and keep the movement of the girl and the lights in the original hallway prominent in the scene. It was later upscaled and sent over for color correction and editing.
Tap into our Creative Insight Hub for expert advice on crafting immersive and interactive experiences. Whether you're at the idea stage or finalizing details, our focused guidance ensures your vision comes to life with impact and innovation. Let's make your project shine.
