Diffusion Illusions:

Diffusion Illusions:

Diffusion Illusions:

Hiding Images in Plain Sight

By Ryan Burgert, Xiang Li, Abe Leite, Kanchana Ranasinghe, Michael Ryoo

rburgert@cs.stonybrook.edu

Introduction

This website is fully interactive playground! In this project, we generate optical illusions and perform steganography using off-the-shelf diffusion models - all of which work in the physical world too!

Illusion Types

We present four variants of illusions:
Flip Illusions
Rotation Overlays
Twisting Squares
Parker Puzzles
Hidden Overlay Illusions

News

Navigation

More Details: Check the bottom of the page to learn more about this project!

Our Paper: diffusionillusions.pdf - please give it a read!

Source code: github.com/RyannDaGreat/Diffusion-Illusions. If you like it, please give it a star !

Make Your Own

Google ColabFlip Illusions

Google ColabRotation Overlays

Google ColabHidden Overlays

Google ColabParker Puzzle

Google ColabTwisting Squares


Google ColabRotation Overlays

Google ColabHidden Overlays

Google ColabTwisting Squares


Matt Parker: Stand Up Maths!

Steve Mould!

Teaser Video

Full SIGGRAPH 2024 Talk



Parker Puzzles


Flip Illusions

If you wait a few seconds, the below images will flip upside-down - to reveal a new image! You can click and drag an image to rotate it by yourself, and double click to reset it.


Click here to reset all rotations Pause

Angelic Pretty
A Bunny in a Bomber-Jacket
Victorian Dress
A boat
Giraffe / Penguin
Bubble Fairy
Dog / Elephant
Dog / Sloth
Seagull / Bunny

A real-life demo:


Google ColabTry it on Colab!


Rotation Overlays

In this section, we have examples with two images. As the one on the top is rotated, the image changes! The one on the top is multiplied by the one on the bottom (pixelwise), simulating how light would be filtered through two transparent sheets with images printed on them. Effectively, we've made 4 images out of 2!

This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.

You can click and drag them to inspect the top and bottom images separately, or right click and open image in new tab to print them out for yourself. To reset the position of an image, double click it.



Click here to reset all positions Pause














A real-life demo:


Google ColabClick here for a printable PDF full of these images, so you can try it yourself!

Google ColabTry it on Colab!


Hidden Overlays

In this section, we have sets of 4 images. You can click and drag them on top of each other! They will all be multiplied together, and when they're all on top of each other they will reveal a secret 5th image. To reset the position of an image, double click it.

This works in real life too: you can overlay printouts of these images on top of each other for the same results, as seen in the youtube video.

I won't spoil it for you in this paragraph; try it out!



Click here to reset all positions


Click and drag these 4 pictures together! Can you find the hidden 5th picture?
(hint: it's easiest to see on the white background!)




Try it with these too!






Click here to reset all positions


(Hatsune Miku) * 4 = ?






Click here to reset all positions








Click here to reset all positions








Click here to reset all positions






A real-life demo [WARNING: SPOILERS!!!]


Google ColabTry it on Colab!


Folding Tile Illusions

Auxetic mechanical linkages of connected squares. Click and drag them from left and right!


Rotation: Hinged Pause

Folding Tile Illusions: Videos

Here are some videos of the above illusion, in case your browser is too slow to render them nicely.


Bunny + Astronaut
Cat + Dog
Donut + Ice Cream
Woman Face + Floofy Dress
Woman Face + Hatsune Miku
Anime Girl + Hatsune Miku
Emma Watson + Walter White
Wonder Woman + Skeleton
Yoda + Weeping Angel

Project Details

Our Paper

Click the this link to learn much more about Diffusion Illusions!

Overview

The project is comprised of four demonstrations:

  1. Flip Illusions: Generating images that appear as different objects when flipped upside down - For example, a giraffe could look like a penguin. The image satisfies two different textual prompts dependint on its orientation.
  2. Folding Tile Illusions: A mechanical linkage of small squares - We splice an image up into a checkerboard-grid of small tiles, and rotate each even tile 90 degrees and each odd tile -90 degrees. This can be built as a bunch of small squares with hinges on them!
  3. Rotation Overlay Illusions: Revealing multiple images through image rotation - Overlaying two images to reveal four different images when rotated. The top image, when overlayed on the bottom image and light shines through, will reveal an image such as Darth Vader. When the top image is rotated 90 degrees, it becomes a different picture such as Pikachu. Another 90-degree rotation reveals another image and a final 90-degree rotation reveals a fourth image.
  4. Hidden Overlay Illusions: Revealing a hidden image through image overlay - Overlaying four images, such as Hatsune Miku, a bear in a field, Mario, and a pumpkin, to reveal a fifth image, Darth Vader, when light is shone behind it. The four images are optimized to look like the original objects when viewed individually, but when combined, they reveal the hidden image of Darth Vader.

Similar Projects

Another project with a similar goal was released in February 2023: Illusion-Diffusion, authored by Matthew Tancik. That project only achieves the first part of this demo, aka image flipping.

Our other paper, "Peekaboo: Text to Image Diffusion Models are Zero Shot Segmentors", has an extremely similar formulation to the illusions on this website - it optimizes images using a variant of score distillation loss to perform zero shot segmentation.

Inspired by our Diffuion Illusions project, Daniel Geng et al wrote Visual Anagrams. It has beautiful results and is definitely worth checking out!

How it Works: Pseudocode

To help explain how these were calculated, I've included some easy-to-read pseudocode that covers how each of these illusions were made. This code has been oversimplified for the sake of simplicity - for all the details please read our paper. The full source code can be found at github.com/RyannDaGreat/Diffusion-Illusions, or at any of the Google Colab links throughout this webpage.

Flip Illusions


Rotation Overlays


Hidden Overlays

Timelapses

Diffusion illusions are generated through an iterative process: we optimize the output images with gradient descent, starting from pure noise. Here are some timelapses showing how they evolve over time.

Flip Illusion Timelapses

For reference, here are the end results of the above timelapse video:

Click here to reset all rotations Pause

Giraffe / Penguin
Angelic Pretty / Victorian Dress


Hidden Overlay Timelapses




Bibtex Citation