Angelic Pretty
A Bunny in a Bomber-Jacket
Victorian Dress
A boat
Giraffe / Penguin
Bubble Fairy
Dog / Elephant
Dog / Sloth
Seagull / Bunny
We're excited to share our demo, "Diffusion Illusions: Hiding Images in Plain Sight," which showcases how optical illusions and steganography can be created using text-to-image diffusion models. A video demonstration is coming soon! Our demo shows how Score Distillation Loss, introduced in DreamFusion, can be used for more than just creating 3D models. For example, as shown in our paper "Peekaboo: Text to Image Diffusion Models are Zero Shot Segmentors", that loss can even be adapted for segmentation. In this demo, we use a variant of score distillation loss to optimize different optical illusions - hiding images in plain sight.
Source code: github.com/RyannDaGreat/Diffusion-Illusions. If you like it, please give it a star!
We'll keep this brief so we can dive right into the pictures - check the bottom of the page for more details about this project!
There are three kinds of illusions presented: Flippy Illusions, Rotating Overlays, and Hidden Characters
If you wait a few seconds, the below images will flip upside-down - to reveal a new image!
You can
Try it on Colab!In this section, we have examples with two images. As the one on the top is rotated, the image changes! The one on the top is multiplied by the one on the bottom (pixelwise), simulating how light would be filtered through two transparent sheets with images printed on them. Effectively, we've made 4 images out of 2!
This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.
You can
Try it on Colab!In this section, we have sets of 4 images. You can
This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.
I won't spoil it for you in this paragraph; try it out!
Try it on Colab!Click the this link to access a short 2-page whitepaper about this project, including mathetmatical formulae.
The project is comprised of three demonstrations:
The project will include physical items such as transparencies for people to interact with, as well as a live demonstration that will allow people to create their own illusions on the spot. If space permits, we might bring a printer to let people play with the illusions they create. And if we have even more space, perhaps a robot arm to automatically demo the flippy images.
In this website you'll find multiple interactive exhibits where you can drag images around as if they were pieces of paper / transparencies. Until we demo at CVPR, this is the next best thing to the physical demo!
Another project with a similar goal was released in February: Illusion-Diffusion, authored by Matthew Tanick. That project only achieves the first part of this demo, aka image flipping.
Our paper, "Peekaboo: Text to Image Diffusion Models are Zero Shot Segmentors", has an extremely similar formulation to the illusions on this website - it optimizes images using a variant of score distillation loss to perform zero shot segmentation.
To help explain how these were calculated, I've included some easy-to-read pseudocode that covers how each of these illusions were made. The full source code can be found at github.com/RyannDaGreat/Diffusion-Illusions, or at any of the Google Colab links throughout this webpage.
Diffusion illisions are generated through an iterative process: we optimize the output images with gradient descent, starting from pure noise. Here are some timelapses showing how they evolve over time.
For reference, here are the end results of the above timelapse video:
