Midjourney vs Stable Diffusion: Which AI Image Generator Should You Use?

We have witnessed a sudden explosion of generative AI tools in recent months. And nowhere is that more true than in the image generator arena. Between free and paid, open-source and proprietary, there are many options to choose from. But if you’ve paid any attention to discussions of AI-generated images, you’ve no doubt heard of the rivalry between Midjourney and Stable Diffusion.

While both AI image generators do a great job, they differ in terms of customization features, art styles, ease of use, pricing, and more. So, in this article, let’s take a look at what separates the two and which one you should use.

If you’ve only recently heard about Midjourney or Stable Diffusion, it’s because they are some of the latest AI image generators available today. Both launched to the general public in mid-2022, but with completely different goals. Midjourney was launched as a for-profit service, while the developers of Stable Diffusion released it under an open-source license.

Stable Diffusion’s open nature has made it one of the most customizable and widely available image generators. You can use a basic version of it online, for free or through a paid subscription. But if you want to play around and get the best possible results, you’ll probably want to generate images on your own computer. Frontend software like the Stable Diffusion Web UI project often includes many advanced features and allows you to use custom art models. The downside is that it takes some time and technical skills to set up.

Halfway through the trip, on the other hand, doesn’t take nearly as much effort. We have a dedicated guide to using Midjourney, but you can just access it through the Discord chat app. You can add a few parameters to your prompts, but that’s about all you can do with customization.

What about the actual quality of the images? You should get fairly accurate results from both AI image generators. That said, Midjourney’s latest V5 model generates high-quality images that match the text prompt very well. So in that ease of use it has a small advantage. But if you’re willing to put in the effort of tweaking inputs and adjusting settings, Stable Diffusion can also match or even surpass Midjourney’s results.

At a glance, Midjourney and Stable Diffusion can look like they offer the same feature set. However, both image generators have some unique strengths and weaknesses.

Take for example upscaling, which you could previously achieve with the buttons U1, U2, U3 and U4 in Midjourney. That is no longer possible because Midjourney’s latest model does not yet support scale-up models. The open nature of Stable Diffusion, on the other hand, means you can download different upscaling models to experiment with. And if your computer has enough video memory, you can also generate higher-resolution images than Midjourney’s current one-megapixel limit.

Here is a short list of some other feature differences between Midjourney and Stable Diffusion:

• Painting in and out: Stable Diffusion allows you to use inpainting to modify specific parts of an existing image. Similarly, outpainting allows you to generate new details beyond the boundaries of an existing image. Midjourney does not yet support any of these editing features.
• Image prompts: You can upload an image (or two) as part of your prompt in Midjourney. The bot combines the image with your text to generate an output that is visually similar to the input.
• Custom art styles: Midjourney offers an anime-optimized model that you can call up with the --niji parameter. However, Stable Diffusion lets you download custom models trained in a variety of art styles, from realistic to origami.
• Censorship: While Stable Diffusion’s official base models don’t allow you to generate explicit images, you can work around these limitations with custom models. This is not possible with Midjourney and your account may even be banned if your prompts contain explicit or suggestive language.

Keep in mind that you need to run Stable Diffusion on your own hardware to get the most benefits. Online tools don’t offer the same level of flexibility, making Midjourney as capable as Stable Diffusion.

As a for-profit company, it’s not surprising that Midjourney has some restrictions on how often you can use it. Even with a paid Midjourney plan, you only get a limited number of free image generation hours each month. The $30 and above plans offer unlimited hours of relax mode, but that means waiting several minutes for each task. Moreover, there is no single free tier or trial.

Stable diffusion works exactly the opposite way. The source code is officially available for download, which means you can use it for free. However, you need a powerful computer with a dedicated graphics card. Most models consume at least 4 GB of VRAM, so a modern gaming PC fits that well. But if you don’t already have one, that kind of hardware can easily cost you well over a thousand dollars. Fortunately, you can use Stable Diffusion online or even in a cloud-based virtual machine like Google Colab if you’re feeling adventurous.

It’s worth noting that the makers of Stable Diffusion also offer a web-based image generation app called DreamStudio. It’s easy to get started with, just like Midjourney, but there is a cost involved. Still, it’s a lot cheaper since $10 gets you 1,000 credits. You can also try it for free, as all new accounts get 25 credits upon signing up.

If you’re looking for a free and easy-to-use option, I suggest you consider a third-party alternative like Bing Image Creator instead.

Let’s look at some real-world examples to see which AI image generator comes out on top. For this comparison, I’m using DreamStudio to generate images through Stable Diffusion because it’s as easy to use as Midjourney. I might be able to get better results with custom models, but that would take a lot more effort. To keep things fair, let’s compare the one-click experience of using Midjourney and Stable Diffusion.

The first image was generated with a rather simplistic prompt: “Star Wars Darth Vader, but make it cyberpunk”. I think both image generators did a pretty good job with the limited information provided, even if they took drastically different approaches.

The next prompt clearly showcases one of Midjourney’s greatest strengths: realism. If you want to generate images that look like they’re grounded in reality, you’ll often get better results than most other AI image generators. While Stable Diffusion matches everything in the prompt, it falls short of the “photorealistic” part.

For our final example, see how the two image generators handle fantasy prompts. The text prompt called for a large white serpent-like dragon sitting atop a rock looking toward the horizon. Again, it’s clear that Midjourney can generate more intricate details. However, if you look closely at the bottom of the image, you’ll see a series of mangled letters where Midjourney tried to replicate a watermark it learned from its training data.

It’s worth reiterating that advanced users can get much better images out of Stable Diffusion than the examples shown here. However, when comparing the two easiest-to-use options, I’ve found that Midjourney often produces better results for the same input prompt. Which AI image generator do you prefer?