Day 22 – May 22 2023

Welcome to day 22 of my 365 day challenge!

Happy Monday!

This week I’m exploring how a simple abstract image can be transformed via diffusion model to create these exotic images.

Generative AI works in mysterious ways

I used Stable Diffusion, an AI system that takes text prompts and magically creates images. Stable diffusion is a text-to-image,  latent diffusion model which means it generates images by gradually removing noise from an image of random static.

The model identifies patterns in the randomness then matches those patterns to it’s interpretation of the text prompt. There are many different models, trained on specific kinds of data, means each model will interpret the text prompt differently.

For example, in this week’s desktop wallpaper images, you will see male and female figures emerging from the noise. If I had used a different model, we may have seen cars or cartoon characters emerging from the noise.

One of the newish ways to work with diffusion models is to provide them with a starter image instead of starting with random noise.  This is a very interesting new way to interact with the model.  What happens if you feed it a picture of a square and provide it the text prompt of “circle”, for example?  The results are often suprising and can provide clues as to how the model processes words and images.