Running Stable Diffusion Locally with Easy Diffusion

When using various online tools to conjure up AI images, the underlying tech is usually handled by DALL·E 3 (which generated the cover image), Midjourney, or Stable Diffusion. Free tools typically have usage limits, whereas others are paid based on quantity and/or quality.

A third option is to run your AI image generator locally, that is, on your own computer, which will have no such limitations. Until recently, it has been quite a complex process to get running, but this is not the case with Easy Diffusion. As the name suggests, it is an easy-to-use GUI for Stable Diffusion that still offers plenty of options to experiment with.

Stable Diffusion System Requirements

It is perfectly possible to generate AI images using only the computer’s processor (CPU), but it is far too slow – even when using the streamlined standard settings.

Consequently, you will want a  reasonably fast GPU to handle the heavy calculations. Nvidia graphics cards are currently the best option but there is also support for the integrated graphics in Apple’s M1/M2 chips. Via Linux, I’ve also managed to run Easy Diffusion with a dedicated AMD graphics card.

At least 8 GB of RAM is also required. However, if you use an M1 Mac with an integrated GPU (shared memory), a lot more RAM is preferable.

Installing Easy Diffusion

Windows users are in luck, as the installation process is no more difficult than running the exe file from the GitHub page.

MacOS (and Linux) Installation

Installing on MacOS or Linux is a nearly identical process. Unfortunately, it’s a little trickier because these Unix-based operating systems are far less lenient than Windows regarding what you install and how.

1. Unzip the file you downloaded from GitHub. Then preferably move the folder to a more practical location than the Downloads folder. The reason is that everything that is subsequently downloaded ends up in the same ‘easy-diffusion’ folder.

2. Open the folder in a Terminal window. The fastest way to do this in MacOS is to right-click the path in a Finder window and choose ‘Open in Terminal’.

3. In Terminal, run the script with ./start.sh, which initiates the download of all files required to run Stable Diffusion.

4. When your downloads are ready, Easy Diffusion opens in a web browser window.

That’s it. To restart the app later, you simply run the start.sh script again (you don’t have to download everything again).

Settings and Options

The default settings in Easy Diffusion will not necessarily result in mind-blowing works of art. But there are endless possibilities to modify the settings, making it more versatile than most commercial web-based tools.

By default, images are generated by telling Stable Diffusion what to make (text-to-image generation). You can however also use image-to-image generation, and/or use a wide range of image modifier presets that are added to the prompt.

Among the more advanced settings is the option to replace the tensor (mathematical model) used for image generation. This may greatly affect both the time taken and the results. Safetensor files are dropped into the easy-diffusion/models/stable-diffusion folder and can then be selected in the GUI.

Your choice of sampler also affects the results, but in terms of processing time, the number of inference steps is crucial. Stable Diffusion starts the image-generating process with what looks like random noise and builds it in a step-by-step process. The time taken from start to finish increases linearly with the number of steps.

What doesn’t increase linearly is the quality of the result. The example above shows the difference between 25, 50, and 75 steps. In other words, adding more steps is a good way to refine images you’re happy with, but shouldn’t be used to create samples. The quality difference in the upper range, e.g. between 50 and 100 steps, is rarely substantial.

In any case, Easy Diffusion is a fun and inspiring way to experiment with image generation, using methods you rarely have access to in free or paid web-based alternatives. The interface is easy to use and educational, as it goes to lengths to explain what to expect from the various options.