I wanted to test a variety of upscalers which include both open-source and commercial options, to see how they stack up against each other in handling upscaling tasks. Specifically, I’m looking at SUPIR, Clarity Upscaler, Topaz Image Upscale, Recraft Crisp Upscale, UltraSharp v2, and Remacri.

I found UltraSharp v2 to be the best local model and Recraft Crisp Upscaler ($0.006 per image) to be the best paid model considering the cost savings to effectiveness ratio. SUPIR still performs better, but the cost is substantially higher ($0.10 per image) for not exactly substantially better results.

Source Images & Testing

The images used in this comparison are freely available and have both low and high resolution variations available. You can download them here: portrait, illustration, landscape.

Since each of these images is available at high resolutions, for example, the landscape image is available at 5979 x 3986, I am able to see clearly how the upscaler manipulates an image by comparing not only the low resolution to upscaled image, but also the upscaled image to the true high-resolution original.

This provides a benchmark for evaluating the accuracy and fidelity of the upscaling process.

Upscalers

There are hundreds of upscalers freely available on OpenModelDB. You should check and test multiple models for your own use case as you may find something better than what I am testing here.

What my goal here is to test a few free and paid options to see what I can use for everyday use across many different image types (i.e. Anime, Pixel Art, Portrait, etc.) without having to think if I’m using the right model for the task at hand.

Portrait Comparison

Okay, so we’ll be looking closely at all the original details, especially the fine textures of the skin, the clarity of the eyes, and the definition in the facial hair.

Here’s all the comparisons between the models testing:

• UltraSharp V2 (Free)
• UltraSharp V2 + Ultimate SD Upscale (ComfyUI Download Link) (Free)
• Remacri (Free)
• Recraft Crisp Upscale
• SUPIR
• Clarity Upscaler
• Topaz

Many of the results from the models above are quite remarkable. For free I prefer the UltraSharp V2 model, which does a great job of preserving the details in the skin and eyes while also being performant. SUPIR has nice skin texture preservation, but Topaz generally does a bit better in keeping details from the original image along with text preservation (look at the word Minolta on the camera).

However, to truly see big comparisons, look at the original image vs. the upscaled image:

• Original vs. UltraSharp V2
• Original vs. Topaz

Here you can see how upscale models tend to smooth out skin textures or flatten details particularly in the facial hair. This is also prevalent in the sweater the subject is wearing.

Illustration Comparison

When dealing with illustrations, the upscaling challenge shifts from preserving natural textures to maintaining sharp lines and vibrant colors that are characteristic of drawn or computer-generated artwork.

Here’s a look at how each upscaler handles the illustration:

• UltraSharp V2 (Free)
• UltraSharp V2 + Ultimate SD Upscale (ComfyUI Download Link) (Free)
• Remacri (Free)
• Recraft Crisp Upscale
• SUPIR
• Clarity Upscaler
• Topaz

SUPIR does a very nice job here of preserving the general look and feel while adding a bit more depth to the image and colors. However, SUPIR does take quite a bit of creative liberties when comparing it to the original high resolution illustration.

Clarity Upscaler kind of acts like a “hires fix” alternative and adds a great deal of texture, detail, and color vibrancy — enhancing the overall look of the illustration. UltraSharp V2 again performs well in keeping true to the original while keeping lines sharp and colors vibrant, this is especially prevalent when comparing it to the original high resolution illustration: UltraSharp V2 vs. Original.

Landscape Comparison

Upscalers often fail on still life or landscape images as they can look overly processed/sharpened. I like to say that they looked “fried” or “cooked”. Here’s how each upscaler handles the landscape image:

• UltraSharp V2 (Free)
• UltraSharp V2 + Ultimate SD Upscale (ComfyUI Download Link) (Free)
• Remacri (Free)
• Recraft Crisp Upscale
• SUPIR
• Clarity Upscaler
• Topaz

SUPIR I think generally shines best in this comparison as it does a good job of keeping the background trees looking natural and not overly sharpened. However, when compared to the original high resolution landscape image, it isn’t exactly the best, nor true to the original — showing that upscale models still have a long way to go. SUPIR vs. Original.

Topaz and Clarity Upscaler also do a fine job, whereas Remacri completely flattens and over-sharpens the image, making it look overly processed. Remacri vs. Original.

The 3 P’s: Price, Performance, and Privacy

UltraSharp v2 and Remacri are free and open-source models allowing you to not only use them without any cost but also are fast (with generations typically less than 10 seconds per image on an RTX 3060), but are also locally executable, which means your images don’t need to be uploaded to external servers.

SUPIR is by far the most expensive at about $0.10 per image to run on Replicate and therefore get quite expensive if you need to upscale large batches of images, videos, or animated content. Additionally, its performance can vary depending on the complexity of the image. Topaz is half the cost at $0.05 per image, Clarity Upscaler at $0.013, and Recraft Crisp Upscale at the cheapest of $0.006 per image.

Worth noting is SUPIR, Topaz, and Clarity Upscaler all have several parameters where you can tweak the output to your preference, which can be beneficial for specific use cases depending on the project you are working on.

Testing Different Upscalers (Paid vs. Free Options) was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

Personally, I prefer BEN2.

I’ll spare the technical details, as quite honestly, it’s above my head. But if you want the arXiv papers, you can get them here and here (respectively).

Anyhow, these models are a complete game-changer. Here’s a side-by-side comparison:

Crazy, eh?

Here’s another test image of REMBG-2.0 (the old background removal tool that was commonly used) versus BEN2 and BiRefNet-HR on a much more complex scenario — hair:

BiRefNet-HR definitely appears to do much better with hair compared to other models.

I noticed that BiRefNet-HR does sometimes over-correct and causes poor results:

So be sure to test the two models to make sure you have the right one for the task at hand.

Of course, if you do this for video, then it really becomes a game-changer:

Here was a screencap of the original video:

Pretty remarkable!

How to Remove the Background in ComfyUI

This is actually really easy and runs well on low-powered machines like my Apple laptop.

You can download the workflow from here.

You’ll just want to first install the ComfyUI-RMBG custom node. It has support for a host of models, which include the following: RMBG-2.0, INSPYRENET, BEN, BEN2, BiRefNet models, SAM, and GroundingDINO.

To get BEN2 and BiRefNet-HR, you’ll just need to pass an image through the Remove Background node once, and it’ll download automatically to the comfyui > models > rmbg directory:

The workflow just has an input and output image node and the remove background node sitting in the center:

One nice feature within this specific node is the background option.

You can switch this out to various solid colors and as an alpha channel, allowing for transparent PNG images:

The author of the plugin also has a bunch of terrific tutorials over on their repo for face segmentation, clothes, and fashion segmentation if you are looking for character tweaks.

And that’s how you can remove the background of any image in a couple of steps!

Background Removal in ComfyUI Just Got Really Really Good was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

If not, here’s the short of it. ComfyUI is a versatile open source application that allows you to interact with diffusion models and produce images, short videos, etc.

But what you may not have known is that ComfyUI is likely to surpass Midjourney in search volume in the not-too-distant future:

Was anyone aware of this?!?

What does this mean for the future of image generation?

One word: Control

By their design, diffusion models are messy.

Put in a prompt and click “generate” 5 times, and you get 5 different images (assuming random seed values, of course).

Generalist models, including Midjourney, Flux 1.1 Pro Ultra, and Ideogram, are black boxes where users have little control over the input and output of their creations.

While the addition of features like — sref, — cref, mood boards, inpainting/outputting canvases, and other tools help to address the problem, they still don’t solve the fundamental issue: users want control.

Control can be understood in many different ways, from the ability to fine-tune the weight of input images and prompts, set desired poses, own their outputs, maintain the privacy of their generations, and control the cost of membership, among other things.

Closed-source platforms just aren’t there yet and are incredibly slow to adapt to change — just look at how long it took Midjourney to move out of Discord if you need proof.

Signs ComfyUI is Here to Stay

Unlike Automatic1111 Stable Diffusion WebUI, WebUI Forge, SD.Next, and other open-source apps that allow you to generate images with diffusion models, newly released models, including Wan 2.1, Hunyuan, LTX, Flux.1 Dev, Stable Diffusion 3.5 Large, and countless others, have day-one support in ComfyUI to allow you to test models’ capabilities immediately.

Given the flexibility, modularity, and design of ComfyUI, I don’t see this changing anytime soon, which supports the idea that this is quickly becoming the industry standard for computer vision tasks.

The Achilles’ Heel of Open Source

The adoption and progress of ComfyUI have been embraced by the open-source community, but it is still highly dependent on companies making new models available.

Of course, if closed-source models make tremendous breakthroughs, companies may hold off on releasing watered-down open-source models to help solidify their market dominance.

When you couple this with the cost of training an open-source model without much in return, you won’t see progress as quickly.

If history is any indicator, the community sees a new model release about once a year on average:

Between these noteworthy releases, various enhancements are delivered in the form of fine-tunes, LoRAs, ControlNets, and more to keep the quality of outputs on par with commercial alternatives.

But let’s not forget two huge important factors:

Learning curve and GPU Compute

Setting up ComfyUI is unnecessarily complicated (although it’s getting better) and thus eliminates a large swath of people.

Not to mention workflows that look like you need to map many nodes just to generate a nice-looking image:

Additionally, the number of people that have a modern GPU that costs a significant amount of money is relatively slim — making the total addressable market even smaller.

So paying $8/mo for say Ideogram is way easier than dealing with the headaches above for many.

Does This Mean Closed-Source Image Models are Dying?

No, definitely not — there’s too much money at stake.

The first Immutable Law of Marketing, which is Leadership, could be summarized as follows:

The foundational principle of marketing is not to create a superior product, but to be the first in a new category. Being first establishes leadership in the customer’s mind, which is far more powerful than being “better.” Once a brand becomes synonymous with a category (e.g., Coca-Cola in cola, Xerox in copiers), competitors struggle to displace it, even with better products. First-mover advantage locks in perception, often making the pioneer’s name generic to the category (e.g., “FedEx” a package). Timing and relevance matter — some firsts fail due to bad ideas (e.g., dog ice cream) or poor timing (e.g., a national newspaper in the TV era).
Source: Al Ries

Midjourney was one of the first notable AI image generators that came onto the market.

I suspect that they will continue to assert their market dominance for the foreseeable future, pending any significant external factors.

Not to just pick on Midjourney, we are seeing more and more companies bake AI image generation into their products, including Adobe, Canva, and even Microsoft Paint.

So, no, the closed-source companies will continue to thrive for some time.

Where it Goes From Here

Not only is the gap in mindshare between both open source and proprietary models closing quickly — so too is the quality in the outputs.

So if you’ve ever been curious to try open source image generation and haven’t yet taken the plunge — now is better than ever to get into the scene.

ComfyUI is About to Become More Popular than Midjourney was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the problem that ThinkDiffusion solves.

But are they good?

Here’s what you should know:

What You Probably Want to Know…

For straightforward image generation, ThinkDiffusion works exceptionally well. With a pre-loaded balance of at least $10 in your account, you can spin up an instance of Automatic1111 with several models pre-loaded in a few minutes.

The top tier plan, which has as much VRAM as an RTX 4090, which retails for roughly $1,500, will only cost you about $1.75/hr.

There, you can generate upwards of 2,000 images per hour with the best open-source diffusion models available.

Advanced users will surely have some issues when it comes to setting up a workspace, particularly with file management. However, once set up, the reliably fast render times will help to increase overall productivity.

What is ThinkDiffusion?

ThinkDiffusion is a cloud-based platform that provides access to powerful GPU resources, allowing users to run state-of-the-art diffusion models without the need for expensive hardware. It offers a range of pre-loaded models and interfaces, making it easy to generate high-quality images and explore the possibilities of AI-generated art.

The Process

Starting up an instance is super easy; all you have to do is select one of the following options:

Once selected, within about 3–5 minutes, you’ll have a fully functional Automatic1111 instance with several models pre-loaded and ready to use.

Here’s a look at the interface:

And here’s a look at all the models that come pre-loaded with the instance:

As you can see, ThinkDiffusion offers a wide range of models, including many of the most popular available on Civitai.

Additionally, ControlNet models are also pre-loaded, so you can use those as well.

As for out-of-the-box extensions, quite a few are available, including Infinite Zoom, ADetailer, Infinite Image Browsing, Deforum, Inpaint Anything, and others. Here’s a look at everything that’s available:

These extensions can greatly enhance your image generation experience, and it’s nice to see ThinkDiffusion offering them out of the box, so you don’t have to spend the time and effort installing and configuring them yourself.

Lastly, all the images generated through ThinkDiffusion can be easily downloaded, and the platform also provides an option to save your favorite models and configurations for future use. The image browser, located on the right panel in the appropriate directory (i.e., a1111 > outputs > date):

Note: Instead of downloading each image individually, you can zip the folder and download everything at once.

File Management

File management is a bit cumbersome, especially for more advanced users. The lack of renaming, drag-and-drop, and folder organization capabilities can make it difficult to upload and correctly organize files, especially when working with multiple projects or complex model configurations.

For example, while I love that you can input a URL and have ThinkDiffusion download the model rather quickly, some models, like those related to Clip Vision, which are a critical part of IP Adapter, are labeled as model.safetensors within HuggingFace.

So when you import it, you cannot rename the file to CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors. Instead, you must download the 1.7GB file, rename it locally, and then re-upload it, which is a tedious and time-consuming process.

Additionally, you have no way to access your files unless you are within an active instance. So, you must be charged for the instance time even when you are waiting for models to download and/or upload.

Pro Tip: Use the lowest powered machine when managing files.

Lastly, if you step away from Think Diffusion after 48 hours, all your instances are automatically terminated:

So you’ll need to set up everything again from scratch, which can be frustrating, especially if you had a complex workspace established and simply enjoyed the weekend.

However, you could upgrade to the TD-Pro plan, which is monthly and offers persistent storage, eliminating the need to re-upload files and allowing access to your files at any time.

This upgrade would mitigate some of the file management issues mentioned above, but it comes at an additional cost of $30 per month.

Try ThinkDiffusion Today — Sign Up Here

Other Applications

Aside from Automatic1111 WebUI, which is by far one of the most popular interfaces when working with diffusion models, ThinkDiffusion also offers ComfyUI, Fooocus, InvokeAI, Kohya SS, StableSwarm UI, and FaceFusion — providing you with a range of interfaces to choose from, each catering to different user preferences and workflows.

Alternatives to ThinkDiffusion

At this time, the closest competitor to ThinkDiffusion is likely to be RunDiffusion — another very-much done-for-you platform that allows users to run diffusion models without extensive technical knowledge or hardware.

While both platforms share similarities, there are some key differences. For instance, RunDiffusion provides access to many more applications, including Audio-WebUI, Enfuge, Oobabooga, and other AI-related tools.

Allowing you to use your credits across many different applications, whereas ThinkDiffusion is more focused on providing access to a range of diffusion model interfaces.

However, from a pricing perspective, ThinkDiffusion is far more competitive, with an hourly rate of $1.75 for a high-end GPU, compared to RunDiffusion’s $2.50 hourly rate for a similar high-end GPU.

Aside from RunDiffusion, other options include Vast.ai, RunPod, PaperSpace, Vultur, Amazon SageMaker, and more — but these require some scripting know-how to get them up and running.

Pricing

The three tiers of pricing are straightforward here and range generally from $0.59/hr to $1.99/hr, depending on the application you are planning to use.

If you plan on using the service extensively, TD-Pro could be an option — but the price is a bit steep:

At $30/mo, you get access to persistent storage, which means your files won’t be wiped after 48 hours of idle time. Additionally, you’ll get better support if you have any questions. $10 of the $30 is converted into credits that can be used to offset hourly costs, making it a more cost-effective option for heavy users.

Enterprise customers may want to check out TD-Teams for shared workspaces, models, settings, etc., which can be customized to meet the specific needs of a company.

Competitively priced at $60/mo, it can be a nice upgrade to the creative department to storyboard or plan out projects together.

Customer Support

Visiting their Discord server, you can see that their support team is highly responsive and knowledgeable.

They have a dedicated channel for support requests, and community members are also active in helping each other out.

The documentation and resources provided by ThinkDiffusion are comprehensive, making it easy to get started and troubleshoot any issues that may arise.

Try ThinkDiffusion Today — Sign Up Here

This post contains affiliate links.

ThinkDiffusion Review was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

After a bit of experimenting, when you combine the style extraction of IPAdapter along with the precise segmentation of the Grounding Dino and Segment Anything models, you can get remarkably accurate and hassle-free outfit changes with minimal post-processing.

Here’s how:

Workflow & Full Guide: If you want to watch a video of this in action or download the workflows, they are available for free over on the Prompting Pixels website.

Final Results

Before getting into the details of the workflow, here’s a visual representation of a before and after picture:

Setting Up the Workspace

I found that separating this into three different groups to be a good approach — from an organizational standpoint to what is happening in the overall process. These three groups are as follows: Basic Workflow, IPAdapter, and Segmentation. Here's what's going on in each:

Basic Workflow

This is basically the default workflow you start with in ComfyUI. The most notable changes here are that you’ll want to get an inpainting checkpoint loaded rather than a generational one, as we are working with a partial image.

I also recommend a good SDXL checkpoint, such as RealVision, ICBINP XL, or the base SDXL Inpainting checkpoint.

As for your prompt, since we will be making changes to a specific segment, then you’ll want to define what you want to appear — in our example, we’ll convert a basic shirt to a jazzy Hawaiian shirt.

IPAdapter

IPAdapter stands for Image Prompt Adapter. Essentially, these nodes can transfer a style or the general features of a person to a model.

Think of this like a mini LoRA or textual embedding.

Therefore, we need to provide an image as an input through the Load Image node along with extracting the relevant bits of information from the image.

To get the image prompt adapter (IPAdapter) set up in ComfyUI, you’ll need to get the CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors and ip-adapter-plus_sdxl_vit-h.safetensors models loaded. You can find these and other relevant models available on the IPAdapter GitHub repo.

Once placed into the relevant directories, you’ll pass your image output from the Load Image node into the IPAdapter Advanced node.

In addition, you'll bring in the CLIP model with the Load CLIP Vision and the IPAdapter Unified Loader both into the IPAdapter Advanced node. Be sure to set the preset to PLUS (high strength) if you are using the same IP adapter model mentioned previously.

Lastly, in the IPAdapter Advanced, be sure to change the weight_type to style transfer.

The model output from the IPAdapter Advanced goes directly into the KSampler node, where the modified model file will now accurately draw an image/style based on your desired input.

Segmentation

One of the coolest parts of this process is the implementation of the GroundingDino model. This segmentation model can be given a textual prompt and then find that within an image and segment it out accordingly.

This is extremely powerful.

While in our workflow, we are using this for segmenting a shirt, you could segment just about anything, from a hat to a pair of shoes, and even entire backgrounds. The possibilities are endless.

To set this up, you’ll need to bring in the Segment Anything custom node (available in ComfyUI manager or via the GitHub repo).

Like IPAdapter, when segmenting, an image will be the first input.

Therefore, set up your Load Image node and then pass it through to the GroundingDinoSAMSegment node.

In addition, you'll need to bring in a Segment Anything model with the SAMModelLoader node along with the GroundingDinoModelLoader. On first run, these model loader nodes will go out and download the relevant models (~3GB) and then pass them into the GroundingDinoSAMSegment node.

As you will notice, the GroundingDinoSAMSegment node has a text field where you can input a word of the object you want to segment, such as shirt, glasses, etc.

Now you can define multiple objects like shirt glasses, however, I noticed that it won't be nearly as reliable:

As for the threshold, generally speaking, a lower value allows the model to be more liberal with its selection, whereas a high value will be more confident. Too high a value may result in an error where nothing is selected.

Once segmented, you’ll just need to pass the mask to both the VAE Encode (for Inpainting) (mask input) and IPAdapter Advanced (attn_mask input) nodes. The original image should also be passed to the VAE Encode (for Inpainting) node.

Optional: You can add a FeatherMask node to the mask, which softens the edges and may help produce better results.

Queue Prompt & Review Results

Once all connected, you can queue the prompt and review the results. You should see a fairly accurate representation of your desired outcome, with the shirt transformed into a jazzy Hawaiian shirt.

Take note of the output image and adjust the settings as needed to achieve the desired outcome.

Remember to experiment with different settings and test various models to find the perfect combination that works for you.

Advantages and Limitations to this Process

Here’s a general pros/cons overview of this approach that may be helpful in deciding how to use this workflow to your advantage:

Pros:

• Zero-shot object detection by GroundingDino allows you to automatically segment and retouch images in a batch process.
• Accurate segmentations allow for the model to only retouch the correct area.
• Potential consumer-facing applications such as a ‘virtual try-on’

Cons:

• Segmentation eats up a few more GB of VRAM, which may cause problems if using with other nodes like ControlNets, LoRAs, AnimateDiff, etc.
• Only style application, not physical application. For example, if a long-sleeve shirt is the input for the IPAdapter, the final image will still have short sleeves as that is what is segmented.

👉 Want to power-up your AI art skills? Then consider checking out the FREE Prompting Pixels courses.

How to Change Outfits in ComfyUI was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

After playing with it for a couple of days — I am still blown away at the results.

This new state-of-the-art method allows for changes on a per pixel basis rather than an entire region.

Another way to put this in perhaps more layman’s terms, or at least this is the way I understand it — is that instead of masking out an area to make changes when inpainting, differential diffusion allows you to apply a mask as a gradient instead, with the light areas of the mask more likely to change and the darker areas less likely to change. This results in cleaner and better end results without any jagged edges.

To quickly demonstrate the difference.

Here’s a side-by-side demonstration of what an inpainting mask looked like before (left) and what it’ll look like after using differential diffusion (right).

While this doesn’t necessarily look groundbreaking. Trust me on this — the results are super good.

Demonstrating the Process

As always, if you want to check out how this all works, check out the Prompting Pixels website where I the workflow available to download.

Don’t worry; I’ll still be going over much of the same information in this article, but I think that page may give you a little better understanding.

There’s also a video on YouTube as well:

The Nodes

The workflow to set this up in ComfyUI is surprisingly simple. You’ll just need to incorporate three nodes minimum:

• Gaussian Blur Mask
• Differential Diffusion
• Inpaint Model Conditioning
• Convert Mask to Image (optional — helpful to review the mask)

The Workflow

In this short demonstration, we’ll be using a standard inpainting workflow. This means an image will be provided as an input, and then we’ll modify it through mask editing and prompting.

So for the first half of the workflow:

We’ll set up our Load Image node where I'll be putting in the original image I want to adapt. In this case, I am using this cropped image from Unsplash.

Then, by right-clicking on the image, I’ll open up the Mask Editor to select the area I want to modify:

In this example, my goal is to have her wearing glasses. So, with that in mind, I’ll mask out the appropriate area.

Once masked, you’ll put the Mask output from the Load Image node into the Gaussian Blur Mask node. This node applies a gradient to the selected mask.

Think of the kernel_size as effectively the size of the brush (larger value = bigger brush), whereas the sigma is the strength of the softness (larger value = softer gradient).

The workflow should look like this:

Pro Tip: The softer the gradient, the more of the surrounding area may change. So, don’t soften it too much if you want to retain the style of surrounding objects (i.e. in this example it would be the nose, mouth, etc.).

The checkpoint you’ll just use a standard generational checkpoint — not an inpainting checkpoint (more on this in a minute).

As for your prompt, it’s just like regular inpainting. The prompt should be both what your image currently is and what you want it to be. So, instead of just a woman, it becomes a woman wearing glasses.

Second half of the workflow:

The second half is where things get interesting. Unlike traditional inpainting that you must download a separate inpainting checkpoint, differential diffusion works well with standard generational checkpoints (can still provide good results with an inpainting checkpoint as well).

Here’s what the second half of the workflow looks like:

However, you’ll need to condition the model by passing your model through the Differential Diffusion node.

Additionally, your prompts, mask, etc. must also be passed through the Inpaint Model Conditioning node.

These are native to ComfyUI.

Pro Tip: If you don't see them, make sure you update your application accordingly in order to load these nodes.

Lastly, for the KSampler, you will just set the values that work best for your checkpoint.

The one notable change you'll want to look at here is the denoise value in the KSampler node. Essentially, denoise is how much noise is added to the masked region. So, if denoise = 1, then the model will draw completely new results and have no knowledge of what is under the pure white areas of the mask. Whereas a denoise value of 0, and the image won't change.

Typically, somewhere in the range of .6 - .8 is good for inpainting, as the final result will change depending on your prompt while still having some knowledge of the information underneath the mask.

Once all set, just hit that Queue Prompt button and review the results.

Here’s a look at our before and after:

Pretty cool and incredibly consistent between the cheeks and glasses — you’d never know this was effectively two different images. Notice that subtle details like her eyebrows and nosering changed slightly — this could be reduced by being more selective on the mask and reducing the sigma value in the Guassian Blur Mask node.

However, admittedly, the top line of the glasses where the eyelashes meet is a little strange-looking, but nothing that couldn’t be cleaned up after an iteration or two.

So, there you have it, a cool new way to do better inpainting.

Want to learn more about diffusion models? Check out the Prompting Pixels website where we have in-depth tutorials and videos.

How to Use Differential Diffusion for Better Inpainting in ComfyUI was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

Flux.1, released on 8/1/24 by Black Forest Labs, is a new text-to-image generation model that delivers significant improvements in quality and prompt adherence compared to other diffusion models, including Stable Diffusion, Midjourney, and OpenAI’s DALL-E.

If you’re a hobbyist familiar with SD 1.5 or SDXL, this article is designed to get you up to speed on why you might want to consider using Flux.1 for your image generation tasks going forward.

Who Made Flux.1?

Many of the key individuals from the original Stable Diffusion team are behind Black Forest Labs and their flagship model, Flux.1.

Think of it as the unofficial successor to SDXL (we are purposely ignoring SD3 since it is believed that many of these folks had already left Stability at the time of the release).

What Makes Flux.1 Special?

As Black Forest Labs puts it:

All public FLUX.1 models are based on a hybrid architecture of multimodal and parallel diffusion transformer blocks and scaled to 12B parameters.

When reading the linked paper they cited, one interesting approach is the reliance on two CLIP models (G/14 and L/14) and T5 XXL.

Note: SDXL uses two versions of CLIP as well.

As mentioned in the paper about the impact of T5:

We observe T5 to be important for complex prompts, e.g., those involving a high degree of detail or longer spelled text (rows 2 and 3). For most prompts, however, we find that removing T5 at inference time still achieves competitive performance.

Furthermore, the ELO score, which is a method to compare relative skills of models/players/etc., demonstrates that Flux.1 is the best-performing model:

How Prompting Is Completely Different

When prompting in Flux.1 you can be much more descriptive and speak naturally instead of using archaic SD 1.5 or SDXL prompting. For example, in the image below, I used the following prompt:

Picture of a woman wearing glasses white blouse black denim shorts and casual shoes standing in a new york city alley and holding a cardboard sign that has writing on it in black paint that says “WHAT THE FLUX?!”, natural lighting and people walking in the background

For the most part Flux.1 got everything spot-on without issue — only exception is that there is no reference to her shoes in the final image!

Here’s the other key part:

Flux.1 doesn’t require negative prompting — this is understandably hard to get used to. However, with descriptive prompting you can typically get the results you’re looking for in fewer attempts.

As for nuanced prompting, Flux.1 also performs quite well. Here’s a sample prompt I wrote:

On the floor is a green pyramid. Next to the green pyramid is a red square block. Behind the blocks is a wooden chair. The view is overhead. Use natural lighting.

Here’s the output:

Impressive that Flux.1 got all the right order of elements in the image. The view could perhaps use some adjustments, but overall not bad.

What’s With All The Different Versions of Flux.1?

There are three distinct versions of Flux.1: Pro, Dev, and Schnell.

You can read more about the differences at the official Flux.1 announcement.

• Pro is the best model, optimized for high-quality, high-resolution images. Access is available via an API to Black Forest Labs or through third-party API providers like Replicate, Segmind, and Fal.ai.
• Dev is nearly identical to Pro but is optimized for research and development. You can download the model via HuggingFace and run it on your own hardware. It comes with a non-commercial license. However, it’s quite large, making it somewhat limited for casual local use.
• Schnell (German for “fast”) is the fastest model, optimized for speed and local development. It was released under the Apache 2.0 license, which allows for commercial use. You can download the model via HuggingFace and run it on your own hardware.

GPU Requirements for Flux.1

If you decide to use Flux.1 Dev, it’s ideal to have a GPU with at least 12GB of VRAM.

Quantized versions of the model are available, such as those provided by City96 on HuggingFace, which work well with ComfyUI. For those who prefer WebUI Forge, check out this post to get things up and running with lower VRAM usage.

Folks on Reddit are reporting varying inference times depending on their GPU. For some hard numbers, check out this thread, which summarizes the performance of the various models:

+-------------+-------------+
|     GPU     | Render Time |
+-------------+-------------+
| RTX 3060    |     116.81  |
| RTX 3090    |      29.77  |
| RTX 4090    |      20.06  |
| RTX 4060 Ti |      45.00  |
| GTX 1060    |     600.00  |
| GTX 1070    |     120.00  |
| GTX 1080Ti  |     445.57  |
+-------------+-------------+

*Render time in seconds.

Fine-Tunes & LoRAs

While Flux.1 was just released, the community is already hard at work creating LoRAs and fine-tunes. You can find them at the usual places, including Tensor.art, Civitai, and HuggingFace.

When testing, I found that LoRAs tend to work much better with the base model than always downloading a fine-tuned model. While it’s still early, be sure to give them a chance before writing them off.

ControlNets & IP-Adapters

While the rapid release of LoRAs is exciting, the community is eagerly awaiting adaptations specifically for both ControlNets and IP-Adapters.

Currently, some initial releases worth checking out are available from ShakkerAI Labs, Misto.AI, and XLabs-AI.

However, compatibility with NF4 or GGUF formats may be limited at this time.

Want to learn more about diffusion models? Check out the Prompting Pixels website where we have in-depth tutorials and videos.

A Stable Diffusion User’s Guide to Understanding Flux.1 was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

Here’s how you can set up ComfyUI with Python with an image-to-image workflow.

👉 All files referenced in this article are available to download for free on the Prompting Pixels website.

Setting Up ComfyUI

1. Install the ComfyUI tooling node custom node. This allows you to send and receive images, especially useful if ComfyUI is installed on a separate web server.

2. Enable dev mode in ComfyUI settings:

3. Also with settings, go to Light Graph and scroll down to the node section and make sure the Node ID badge mode is set to “show all”:

Creating an Image-to-Image Workflow

The most, and relatively only, important aspect in this workflow that you should learn is the input and output nodes. In this case its the Load Image (Base64) and Send Image (WebSocket) nodes. We’ll be communicating to both of these via Python.

When saving this workflow, make sure you select the Export (API) option.

Connecting Python to ComfyUI

In a directory, store your input image, main.py file, and workfow file:

For the main.py file, I used Claude Sonnet to do most of the heavy lifting. Basically, this file loads the workflow, encodes the PNG input image to Base64, sends it to the server, and awaits the response to download the output image.

You can adapt upon it for your own needs however you want.

Before you execute the code, be sure to update the local IP address to the machine that is running ComfyUI. Additionally, also update the ID value to match that of the Load Image (Base64) node:

A successful call to the server should look like this:

And consequently place a generated_image.png file in the same directory that the script was executed in.

What’s Next?

Now that you have Python talking with ComfyUI — you have a lot of options you could take this.

From basic batch processing, prompting with LLMs, to scraping and beyond, with great power comes great responsibility.

Did you like this tutorial? Check out Prompting Pixels for other great tutorials on ComfyUI and more.

Unleashing the Power of ComfyUI with Python: A Step-by-Step Guide was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

AI image creation tools have gotten much better over time, which made me think about whether I was using the best tool for what I needed.

While I don’t think Ideogram is the best tool for everyone, I believe it’s great for beginners, especially when you want to create text-based images or realistic pictures.

Here’s why:

A Quick Note

In the interest of full disclosure — I receive a commission if you click on a link like this one to Ideogram and make a purchase — at no additional cost to you. I pay for the service out of my own pocket and have personally used Ideogram for several months.

This is my opinion of the service, and I try my best to approach this review with an open mind. However, it is only one data point and should be treated as such.

If you are on the fence reading this, then consider just paying the $8/mo and giving it a shot — that way you can instantly find out if it’s right for you. You can cancel your subscription at any time.

Where I Find Ideogram Most Useful

While recent advancements in diffusion models have made text generation far better than it was only a year ago, Ideogram is by far the best model for the job if you want stylized textual outputs.

Text Generation

It seems to get things right more often than not and can handle longer strings of text to short sentences.

I’ve personally found this to be most helpful with logo generation and t-shirt design.

Here are a few short sample outputs to showcase the performance:

It got three out of four correct (capabilities was spelled wrong on the second design).

Very impressive!

For print-on-demand, it is a game changer.

Logo Design

Logo design can also be quite strong, but requires a little more patience. Consider checking out the ‘Explore’ tab and filtering for Logo designs if you need help with prompting tips and inspiration:

Batch Generation

A feature sorely missing from other model providers such as Midjourney, batch generation can be super helpful if you are looking to scale artwork for various activities.

It could be for mood boards, unique character poses, or simply selling images on stock services like Adobe Stock.

Ideogram provides you with a simple CSV template where you just add the rows and then upload:

Even when submitting > 200 rows of prompts, the total generation time is no more than a few minutes.

Canvas

The canvas is suitable for image editing and allows you to do many of the basic manipulations including remixing (image-to-image), inpainting, outpainting, applying text, and basic layer management.

I personally find canvas useful if you are making iterative edits to a single image rather than pure generation.

The UI within canvas does take a little getting used to. I found the layer management to be a little underwhelming as it is not always apparent if newly generated images were under the current layer.

After an hour or two of playing in canvas, you can expect to master the tool.

Prompt Adherence

Recent advancements with AI image generation have been around prompt adherence — or the ability to generate complex images based on a prompt.

My prompt is as follows: “an illustration of a wooden chair. on the right side of the chair is a red sphere, on the left side of the chair is a green pyramid. On top of the chair is a beach ball.”

Here are the outputs:

As you can see, 3 out of 4 images were correct.

Magic Prompt

Magic prompt is Ideogram’s way of helping out beginners and pros alike to help expand a prompt to generate more desirable outputs. For example, if I put in the prompt:

“a group of people at an indoor birthday party.” it generates the following outputs (Magic Prompt is turned off):

Ideogram expands this to:

“In a warmly lit indoor space filled with vibrant balloons and colorful streamers hanging from the ceiling, a single joyful child stands center-stage, blowing out candles on a beautifully decorated cake. The scene is captured in a soft, glowing photograph style, with a cozy, inviting atmosphere that radiates happiness and celebration.”

As you can see, the latter is more aesthetically pleasing when it comes to both the composition and the lighting.

API

Ideogram offers an API — which is really terrific! It is available through their site or through other inference providers like Replicate.

The API is easy to use with endpoints consisting of generate (text-to-image), edit (inpaint), remix (image-to-image), upscale, and describe.

Costs have come down significantly over the past week(!) and moved this from a previous downside of Ideogram to now a strength. It used to cost $.08 per image now costs $0.04 (2a) or as low as $0.025 (2a turbo).

That being said, this is still relatively expensive compared to their web interface as you get 4 images per prompt rather than one. For a cost per image comparison:

Cost per image comparison:

• API (2a): $0.04 per image
• API (2a turbo): $0.025 per image

For the paid plans (using 2a turbo for best value):

• Basic: $8/3,200 = $0.0025 per image
• Plus: $20/8,000 = $0.0025 per image
• Pro: $60/28,000 = $0.0021 per image

The paid plans are significantly more cost-effective than the API:

• Basic/Plus plans are 10x cheaper than API (2a turbo)
• Pro plan is about 12x cheaper than API (2a turbo)

Even compared to regular 2a API, the paid plans are 6–7x cheaper.

The takeaway: If you are a developer, you don’t really have an option — and need to go with the API. However, if you are an individual creating a high number of images — you would be better off programmatically generating CSVs and using the batch generator to reduce costs and get 4x the outputs.

Where Ideogram Comes Up Short

While Ideogram works well for many AI image generation tasks, there are some noticeable areas where it could be improved.

People

A common fault with diffusion models is when generating many people in a scene, and Ideogram hasn’t fully overcome this shortcoming yet either. As you can see in the sample output below, faces are generally deformed, fingers are deformed, and generally the image is bad:

To counteract this, you may need to limit the number of people in a frame, go for closer shots, and clean up in inpainting/Canvas.

Upscaler

When enlarging images, the upscaler is good, but takes a lot of creative liberties that may not work for all use cases. Ideogram provides two parameters within the upscaler — resemblance and detail. Resemblance, as the name suggests, determines how closely the model should follow the original input (0 = little resemblance/more creative; 100 = more resemblance/less creative). Detail is how much the model should fill in for details (0 = little added details and 100 = more details).

Here’s where I found this to not be the case. When setting the parameters to 100 resemblance and 10 for detail, significant changes were still seen in the final output of the image:

👉 Click here to see the comparison slider on Imgsli.

You can see that the eye colors change (from blue to brown), a lot of coloring added particularly under the eyes and nose, and the lips go from a light pink to nearly flesh tone.

That being said, the quality is phenomenal, but the details are just a bit too different given the parameter settings.

Here are a few other tests:

• 1 detail / 100 resemblance
• 100 detail / 1 resemblance
• 50 detail / 1 resemblance
• 50 detail / 100 resemblance
• 100 detail / 100 resemblance

Alternatives

Aside from Ideogram, commercial alternatives (i.e., paid platforms) consider Midjourney, Leonardo, Recraft, Canva, Freepik, Adobe, and many others. These all have a polished interface that works well.

Open source models including Stable Diffusion 1.5, XL, and 3.5 are viable models from Stability AI. Black Forest Labs has Flux Schnell for commercial purposes (see Flux.dev for non-commercial/research).

Overall Value & Final Thoughts

With plans starting at $8/mo with nearly the full suite of tools, it’s affordable for budget-minded to commercial users alike.

So if you want to try a new image generation service and in particular want one that can generate text, give Ideogram a shot, I think you’ll be pleased with the results.

Ideogram AI Review: Worth Trying In 2025? was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.

It also comes with an asynchronous queue system, dynamic node composition, model merging, prompt conditioning and editing, and a command line option for low-VRM usage.

ComfyUI also offers a node-based interface, allowing users to create complex workflows without any coding required.

Despite its numerous features and benefits, running ComfyUI locally can be resource-intensive, especially for users with limited hardware resources.

Here’s how you can run ComfyUI in the cloud to alleviate these resource constraints and enhance your AI image generation experience.

Done For You Services

Several cloud-based options are available for running ComfyUI, providing an easy way to access the software without the need for a powerful GPU or local installations. These services offer different features, pricing models, and user experiences to cater to various needs and budgets.

When choosing a cloud-based ComfyUI service, consider the following factors:

1. Pricing model: Some services charge based on usage (e.g., GPU time), while others offer monthly subscription plans with a set amount of credits or workflow runs.
2. GPU performance: The type and power of the GPU can impact the speed of your workflow execution. Some services offer a choice of GPUs with different VRAM capacities.
3. Parallelism: The ability to run multiple workflows simultaneously can significantly reduce overall execution time, especially for complex projects or batch processing.
4. Customization: Depending on your requirements, you may need a service that allows you to install custom nodes, extensions, or models.
5. User experience: Some services provide a user-friendly interface, pre-built workflows, or template galleries, which can be particularly helpful for beginners or those looking for a streamlined experience.
6. Community and support: Having access to an active community of creators and comprehensive support can be valuable for learning, troubleshooting, and inspiration.

Comparison Table

When selecting a cloud-based ComfyUI service, carefully evaluate your specific requirements and budget.

Consider the trade-offs between pricing, performance, customization, and user experience to find the best fit for your needs.

Build Your Own

If you prefer more control over your ComfyUI setup, you can build your own instance using cloud computing platforms. \

This approach allows for greater customization and flexibility compared to the “done for you” services.

Here are a few popular options:

Runpod

Runpod offers a simple way to get started with ComfyUI in the cloud. They provide a pre-configured template that makes it easy to spin up a ComfyUI instance with just a few clicks.

Runpod recommends selecting a pod with at least 16GB of RAM for experimentation, and you can easily scale up for generating larger images.

Their blog post “How to get Stable Diffusion Set Up With ComfyUI on Runpod” provides a detailed walkthrough of the process.

Vast.ai

Vast.ai is another cloud platform that makes it simple to run ComfyUI. They offer a one-click template that gets you up and running quickly.

Just head to their Templates page, select ComfyUI, choose your desired machine (GPU) and storage, and click the “Rent” button to start your instance.

Personally, I’ve found Vast.ai to be a great option as its really cheap and intuitive. For example, to rent a RTX 3090 for an hour will only cost about $0.25! Not bad considering a brand new RTX 3090 will cost about $1,000.

Vultr

Vultr is a cloud computing platform that offers a ready-to-use ComfyUI image in their Marketplace.

This pre-configured image allows you to launch and start using ComfyUI instantly, without any setup hassles. Vultr’s cloud infrastructure provides scalability, reliability, and global accessibility for your ComfyUI projects.

Regardless which option you choose, running ComfyUI in the cloud can greatly enhance your AI image generation experience by providing access to powerful hardware resources without the need for local installations.

But be sure to consider factors such as GPU performance, memory, storage, and pricing.

Each platform has its own pricing structure and available resources, so compare the options to find the best fit for your needs and budget. Building your own instance provides the most flexibility and control, but may require more technical knowledge compared to the “done for you” services.

ChatGPT, Anthropic’s Claude, and other LLMs can be super helpful in creating the essential scripts and configurations for spinning up my ComfyUI instances in the cloud. They have saved me countless hours of research and trial and error.

ComfyUI Workflows

ComfyUI Workflows .json or PNG files that allow you to share pre-configured workflows with others or easily reproduce your own creations.

👉 We’ve got a bunch of workflows for free over on the Prompting Pixels website.

These workflows encapsulate all the nodes, settings, and connections required for a specific image generation task, making it simple to replicate or adapt the process for different inputs or outputs.

Users can drag and drop existing workflows into ComfyUI and start generating art right away, or they can use these workflows as a base and modify them to fit their own needs.

Workflows can be created to support a wide variety of models and extensions, such as Hires Fix, Img2Img, Inpainting, Lora, Hypernetworks, Embeddings/Textual Inversion, Upscale Models, and many more.

Your Options for Running ComfyUI in the Cloud was originally published in The Latent Space on Medium, where people are continuing the conversation by highlighting and responding to this story.