AI Art Generation Handbook/Node Prompting in Stable Diffusion

This type of "programming language" used in ComfyUI may overwhelm especially to the user whom is not familiar to before compared to other local UI (such as Auto1111, Foooocus, Oomost and such...) but if we bite the bullet and get through it, this may become easy.

As you can see, there are square blocks that are connected to one another using multiple coloured lines. In this context, the square blocks are known as "node" or "block diagram" and the multiple coloured lines are known as "wires" or "pipelines"

Control
Once you successfully installing ComfyUI, we learn how to control inside ComfyUI

Anatomy of the nodes
Every nodes have this 3 characteristic : Input Data Types (Red): The data flows into the nodes through this pipelines. Take note that the wire colours are matching to the input data types Output Data Types (Blue): The data flows out of nodes through this pipelines. Take note that the wire colours are matching to the output data types Control Parameter (Green): The results of the calculations performed within this node is largely influenced by the setting values inside control parameters

Basically, how this works is the whole nodes (as shown on the left) is basically a big "black box" where it takes data flow into the nodes and perform abstractions as well as the complex calculations (based on the control parameters within its nodes). After the calculations is done, the data is outputted to the next nodes for other processing.

Node Programming


To learn node programming, we can clear the whole workflow by clicking "Clear" button on the right. As seen on the diagram of the Text-2-Image of how it works, we can recreate the workflow.

Node Programming - Add in new nodes
To add new nodes, we can double click on any empty spaces in canvas and using search functions to search required block diagrams easier:

We add this few important block diagram

Node Programming - Connecting the nodes
We can start connecting the block diagrams by the pipelines.

''Note: Either way is OK but if you wanted to connect many to one nodes, the recommended method is to connect from the block diagrams input (left side) to the block diagrams output (right side) to minimises potential bugs and issues later on. See Connecting CLIP to K-Sampler parts ''

Let's start with the Load Checkpoint block diagrams and K-Sampler block diagrams. Click on the MODEL output (with lavender circle), you can notices that only model input is highlighted meanwhile the rest is grayed out (Meaning it is not connectable).

Another point to take note that, pay attention to the colour of input and output data types (Only both of same colours are able to be connected together)

Now, Load Check Point should be connected to K-Sampler through the MODEL pipelines.

There are CLIP to be connected. As mentioned above, the CLIP is related to Text Encoder, therefore, we can connect both CLIP Block Diagrams. But However, before we started to connect those, rename the Block Diagrams first into more appropriate name to prevent confusions. Right click on the block diagrams and click on "Properties Panel". There should be Title, Mode, Color and Node name for S&R, change the both Title and Node Name for S&R to more self explanatory names likme  as examples.

Note : We want to distinguish which block diagrams is for positive and for negative prompts. Connect the Load Checkpoints CLIP Output to both Positive and Negative Block Diagrams clip input (See yellow pipelines). Lastly connect the CLIP CONDITIONING output to positive and negative K-Sampler input (See orange pipelines)respective to the block diagrams names

You should have the connections as seen in the left screenshot by now

Note: There are potential bugs of connecting output data types CONDITIONING to input data types positive or negative, therefore reverse the sequence to connect from input positive / negative to output CONDITIONING.

As we can see now in Load Checkpoints block diagram, VAE block diagram is still unconnected. We can click on VAE output datatypes (red pipelines) to see which it can connect to, only VAE Decode Block Diagram vae input is highlighted.

Repeat the same in VAE Decode diagram block for sampler input datatypes (pink pipelines) to be connected to K-SAMPLER 's latent output data type

Finally, repeat the same for VAE Decode IMAGE output datatypes (blue pipelines) to be connected to images input datatypes in Save Images Block Diagrams.

You should get the similar diagrams as below and left only one connections between the K-Sampler and Empty Latent Image block diagrams to make it functional.

Connect the LATENT in Empty Latent Image to KSampler 's latent_image to finish the workflow.

Congrats, you finished the basic text to image workflow and have slightly better understanding on how this all works.

Node Programming - Generate Image
Try to click the "Queue Prompt" on the right button on the sidebar. You may not notice at first but the KSampler node is suddenly highlighted in green colour and then, there is green colour bar that starting to go across. It meant that the AI Art Models is working.

To know what does each of the setting meant, go to AI Art Generation Handbook/Stable Diffusion settings to learn more

Save Prompt Names as Image Ouput File Name
As we generate the images, we get the generic names of ComfyUI_0001.png as default, but do you know that with some additional steps , we can save prompt as filenames in ComfyUI ?

Here is how we do it very easily. On the Positive CLIP block diagram, right click on the block diagrams to search for the name of  

Use the values that you key in earlier (in this case "Positive CLIP") and type the following into filename_prefix inside Save Image block diagram:

%Positive_CLIP.text% , refer to picture on the left for clarity.

From now onwards, you can see the filename changes to the saved images if the Save Image block diagram is not altered in anyway.

For more customisation on the name, here are the page  where we can get to change the custom names (especially prompt names)  for the ComfyUI output files.

Improve Image Generations Quality
SDXL and above are trained on 1024 x 1024 px images, therefore anything less than that will results in low quality of images as shown per this experiments here.

Therefore, on Empty Latent Image block diagram you can adjust both width and height up to 1024px to get more aesthetic image generations.