Rising up in a metropolis neighboring Lyon, France, I’ve been lucky to witness the celebrated “Fête des Lumières” firsthand. This fascinating annual event bathes Lyon in an beautiful showcase of sunshine, metamorphosing its thoroughfares, iconic websites, and edifices into awe-inspiring masterpieces. Annually, thousands and thousands of tourists from across the globe flock to witness this extraordinary spectacle, immersing themselves within the enchanting realm of sunshine, shade, and innovation that defines the Fête des Lumières.
In the present day, I made a decision to embark on a private problem: utilizing Houdini, a 3D software program, to create and simulate a lightweight artwork set up paying homage to these seen on the “Fête des Lumières”. This concept has been brewing in my thoughts for fairly a while; I’ve envisioned a novel strategy. I aimed to put the customer on the heart of the expertise by incorporating an onsite immediate system. This technique would dynamically generate the projected picture over the constructing primarily based on prompts offered by guests, using generative AI know-how. With that match plan in thoughts, let’s see how we are able to create a proof of idea that does this.
In a earlier article, I demonstrated the usage of SideFx Houdini software program (sidefx.com) to generate AI-generated photos by the MLOPs bundle (github.com/mlops). You may take a look into the detailed rationalization within the earlier article (here) to understand the basics of my Steady Diffusion picture technology workflow. On this article, I’ll delve deeper into extra points of the method, constructing upon the groundwork laid out beforehand.
First issues first, let’s discovered a 3D mannequin price of our inspiration. On my aspect, I’ve settled on utilizing a 3D mannequin of the door from the French medieval church of Sainte-Radegonde (Poitiers). The mannequin is freely obtainable for obtain (here).
Now that we’ve downloaded and imported our mannequin into Houdini, we’ll apply a sequence of transformations to align it with the viewport grid. We then cut up the mesh into Houdini teams, which we’ll later use to populate a customized attribute named “building_parts”. This attribute will show helpful shortly after we delve into the generative AI step, as it is going to be used to create masks I’ll use to merge all our generated AI layers utilizing compositing.
For this proof of idea, we’ve restricted ourselves to a few zones of curiosity: the principle door group, encompassing all areas throughout the arch; the sidewalls, consisting of the 2 aspect columns and the principle wall; primarily all the things else. At that time, our setup that handles the mesh seems just like the one under.