Experimenting with a new AI-driven restyling workflow.
I used ComfyUI with Mickmumpitz’s SDXL workflow to stylize the first frames of each shot. Then I took the original 3D previz clip and the stylized frame to Runway to let it stylize the rest of the video with the same look.
I suspect the FLUX workflow with LoRAs might give even better results, but I haven’t had the time to test it yet. The benefit of this method is that it gives you control over the camera and characters within the AI-generated image. In Unreal Engine 5, you can select lenses and cameras like the ARRI Alexa, giving it a cinematic touch right from the start.
There are some limitations, though. One major challenge is that characters can’t walk into the frame since they need to be visible from the very beginning of the shot. Still, it’s an interesting way to blend AI stylization with previz.
