Embedded Files
Introduction:
As the generative AI lead for the Pink Floyd music video project, sponsored by RunPod, I collaborated closely with our director, Matin Nakhl Ahmadi, to innovate in this artistic endeavor. Our goal was to explore the limits of AI-driven creativity while ensuring the artistry remained at the forefront of the production.
Experimentations and Strengths
Technology Exploration:
Tested various AI models including Dall-E, Midjourney, and Stable Diffusion to determine the best fit for our artistic vision.
Chose Stable Diffusion (SD) for its flexibility and control, utilizing the controlnet method to guide AI generation with simple animations.
Team Collaboration:
Onboarded artists on using SD on RunPod’s powerful GPU computing platform, ensuring all team members were proficient.
Developed a comprehensive Notion board for seamless team collaboration and interaction.
Creation Challenge
Question: How can we create an AI-generated video that stands out in a sea of AI art and retains our artistic integrity?
Problem Statement
We needed to balance production costs and output while maintaining visual consistency across different parts of our storyboard. This required a cost-effective yet efficient AI model and pipeline suitable for our indie team's resources.
Solution Architecture
After extensive R&D, we settled on using Stable Diffusion as our primary tool, employing the controlnet method to ensure AI served as a tool, not the director. This method allowed us precise control over the generated image sequences, aligning them closely with our predefined animations.
Problem Solving Moments
Innovative Feature Experimentation:
Implemented the M2M controlnet feature to use random seeds for each frame, avoiding the common fixed seeds and enhancing the uniqueness of our photo-motion animation.
Wrote a Python script to adapt the UI for SD, allowing video path reading from directories instead of direct uploads, overcoming cloud system limitations.
Dynamic Prompting:
Leveraged dynamic prompting and wildcards with the Fast SD model to tailor specific shots, enhancing the visual narrative's flexibility and impact.
Conclusion
This project successfully merged AI technology with artistic creativity, resulting in a unique music video that made it to the official Pink Floyd YouTube video contest. This experience has been a pivotal milestone in my career, demonstrating my ability to lead in technology-driven artistic environments.
Page updated
Google Sites
Report abuse