Download MANUAL.md from data-archetype/dflow_2b: direct link, hf CLI and curl.
- Browser
- Download file 50 kB
-
https://huggingface.co/data-archetype/dflow_2b/resolve/main/MANUAL.md
- Command line
-
hf download hf://data-archetype/dflow_2b/MANUAL.md
-
curl -L -o MANUAL.md https://huggingface.co/data-archetype/dflow_2b/resolve/main/MANUAL.md
dflow_2b Manual
Model card Β· Example gallery Β· Technical report Β· Release notes
Contents
- Installation
- A first image
- Prompts
- JSON layouts
- Image sizes
- Sampling
- Guidance
- Self-attention gain
- Python API reference
- ComfyUI node reference
- Troubleshooting
Installation
dflow_2b needs Python 3.12 or later, PyTorch 2.13 with CUDA, and an NVIDIA GPU with bfloat16 support. Install a CUDA build of PyTorch first; the PyTorch installation selector gives the command for your system.
The repository holds the denoiser weights, the text encoder, the dflow Python
package and the ComfyUI nodes. The autoencoder,
dinac3_96, downloads into the
Hugging Face cache the first time the model is loaded.
ComfyUI
The repository is a ComfyUI custom-node folder. Download it into
custom_nodes and install its requirements into ComfyUI's Python environment:
cd ComfyUI/custom_nodes
hf download data-archetype/dflow_2b --local-dir dflow_2b
pip install -r dflow_2b/requirements.txt
Restart ComfyUI. The nodes appear under the DFlow category. Start ComfyUI
with --preview-method auto for live previews.
Python
Download the repository and install the package from it. The install is editable, so Python runs the code in that folder:
hf download data-archetype/dflow_2b --local-dir dflow_2b
pip install -e dflow_2b
diffusers is needed only for the optional
diffusers wrapper.
Compilation
The denoiser, the text encoder and the autoencoder are compiled with
torch.compile by default. The first generation takes roughly 30 to 45 seconds
longer while the graphs are built.
To turn compilation off, pass compile=False in Python or switch
compile_model off in the DFlow Loader node. The model then starts at once and
each image takes longer. On an RTX 4090 a 50-step 832Γ1216 image takes about 10
seconds compiled and about 13 seconds eagerly, with a peak of about 7 GB of
VRAM.
A first image
In ComfyUI
Load dflow_2b/comfyui/workflows/dflow_default.json and queue it. It uses the
default preset. You can also drop any PNG from the gallery onto
the canvas: each one carries the workflow that made it.
The default graph has ten nodes, connected as follows:
- DFlow Loader outputs the model, the text encoder and the autoencoder.
- DFlow Text Encode takes the text encoder and the prompt.
- DFlow Guider takes the model and the encoded prompt.
- DFlow Empty Latent sets the image size.
- DFlow Sampler feeds DFlow Time Schedule, which also takes the latent and outputs the sigmas and the sampler.
- RandomNoise sets the seed.
- SamplerCustomAdvanced takes the noise, the guider, the sampler, the sigmas and the latent.
- VAEDecode takes the result and the loader's autoencoder, and SaveImage stores the image.
RandomNoise, SamplerCustomAdvanced, VAEDecode and SaveImage are ComfyUI's own nodes. The node reference lists every input.
In Python
from dflow.inference import GenerationSettings
from dflow.pipeline import DFlowPipeline
pipe = DFlowPipeline.from_pretrained("dflow_2b")
settings = GenerationSettings(width=832, height=1216, seed=7)
image = pipe("A lighthouse on a cliff at dusk.", settings).images[0]
image.save("lighthouse.png", pnginfo=pipe.png_info(image))
"dflow_2b" is the folder downloaded during installation.
GenerationSettings() with no arguments is the default preset, and the seed
fixes every random draw of the call.
Prompts
Plain captions
Natural descriptive captions work best. Write what a careful caption of the finished image would say: the subject, what it is doing, the setting, the light and the medium (a photograph, an oil painting, a movie still). Full sentences work better than lists of keywords, and a long, detailed prompt is fine. The limit is 6,144 tokens.
Quoted text for lettering
To put words in the image, write them in double quotes inside the caption:
A neon sign in a rainy street that reads "OPEN LATE".
Everything inside double quotes is encoded one character at a time, so the model sees the exact spelling. Use double quotes only for text that should be visible in the image. To place text precisely, use a JSON layout.
Prompt emphasis
A phrase can be given more or less weight with the markup (phrase:weight):
A portrait of a woman in a (red:1.4) coat, (blurred background:0.7).
The weight changes how much attention the rest of the model pays to the tokens of that phrase. A weight of 1 is neutral, a weight above 1 strengthens the phrase and a weight between 0 and 1 weakens it.
The markup follows a few rules:
- The weight is the number after the last colon of a parenthesised group. The phrase itself may contain colons.
- Groups can be nested. An inner weight replaces the outer one for its own words.
- Parentheses without a numeric weight are ordinary text, and
\(and\)always are.
Two settings control how a weight is applied. Let be the weight and the strength.
The mapping turns a positive weight into a bias added to the attention
logits of the weighted tokens. linear adds . log adds , which at strength 1 multiplies the attention those tokens
receive by exactly . The linear mapping is the stronger of the two: at
strength 1, a weight of 2 multiplies the attention by about 7.
The strength scales every bias. A strength of 0 turns emphasis off without removing the markup.
A weight of 0 or below works differently. It scales the content the tokens contribute by . At strength 1 a weight of 0 mutes the phrase and a weight of β1 subtracts it.
The weights act in every attention layer of the denoiser. The encoder switch applies them inside the text encoder as well.
| Setting | Default | Python (EmphasisSettings) |
ComfyUI (DFlow Text Encode) |
|---|---|---|---|
| Emphasis on | yes | emphasis=EmphasisSettings(...), or None to read prompts verbatim |
emphasis |
| Mapping | linear |
mapping |
emphasis_mapping |
| Strength | 1.0 | strength |
emphasis_strength |
| Encoder | on | encoder |
emphasis_encoder |
Emphasis applies to plain-text prompts. Negative prompts and JSON layouts are encoded exactly as written: in a layout, parentheses and colons in captions or text stay literal and carry no weight.
Negative prompts
A negative prompt is read by classifier-free guidance (CFG) and by the contrastive modes of path-drop guidance, both described under Guidance. With those modes, an empty negative prompt selects the model's learned null text. The default guidance conditions its reference prediction on the prompt itself and ignores the negative prompt.
JSON layouts
A layout prompt describes an image as a canvas with a list of boxes. It gives control over where things are and over the exact text that appears. Layouts suit posters, covers, title cards and other designed images.
The stored format
A layout is a JSON object with exactly three keys: summary, frame and
items.
summary describes the image as a whole.
| Field | Type | Meaning |
|---|---|---|
meta |
list of strings | Tags for the kind of image, such as concert_poster |
canvas |
string | The canvas type; single in the examples here |
caption |
string | A caption of the whole image |
frame describes the canvas: the overall arrangement, style and colours.
| Field | Type | Meaning |
|---|---|---|
caption |
string | How the image is composed |
tags |
list of strings | Style and composition tags |
palette |
list of strings | Main colours as hex codes |
items is a list of boxes, one per element.
| Field | Type | Meaning |
|---|---|---|
label |
string | character, object or text |
box_2d |
four integers | The box as [y0, x0, y1, x1] on a 0 to 1000 scale |
text |
string | The exact string to render; required for text items |
caption |
string | A description of this element |
tags |
list of strings | Tags for this element |
palette |
list of strings | Its colours as hex codes |
Boxes are measured from the top-left corner of the image. y0 and y1 are the
top and bottom edges as thousandths of the image height, and x0 and x1 are
the left and right edges as thousandths of the width. The vertical coordinate
comes first.
Optional fields
The required fields are the three top-level keys and, for each item, label
and box_2d, plus text on a text item. Every other field can be left out:
training dropped the optional fields at random. Captions, tags and palettes
give the model more to work with, and a layout with labels and boxes alone
still works.
Unknown keys raise an error.
Compiling a layout to a size
Before encoding, a layout is compiled for one image size: the frame becomes a
first item, labelled panel, that covers the whole canvas, and every box is
converted from the 0 to 1000 scale to integer pixels measured from the centre
of the image. For an image pixels high, the top edge and bottom
edge of a box become
and the left and right edges are computed the same way from the width. At
832Γ1216 the panel box is [-608, -416, 608, 416]. The result is one line of
JSON with two fields naming the convention (abs_centralized, yxyx), in the
format of the MOSS-Image layout prompts.
The compiled boxes are in pixels, so a compiled layout belongs to one image size: the same stored layout compiles to different numbers at 832Γ1216 and at 1216Γ832. A compiled layout whose boxes do not fit the image size raises an error.
Keep layouts in the stored format, which is independent of size, and let the code compile them:
- In ComfyUI, the DFlow Layout Encode node takes the stored JSON together
with
widthandheight. Set them to the size of the latent you sample. The guider stops with a message when they differ. - In Python, pass the stored JSON as the prompt. The pipeline compiles it at
the size in
GenerationSettings. The function that does this isdflow.inference.resolve_prompt, anddflow.layout.compile_layoutreturns the compiled text if you want to inspect it.
MOSS-Image layouts
The model reads the compiled form, which is the layout prompt format of MOSS-Image. MOSS-Image layouts work wherever a stored layout does, and are converted as follows:
- MOSS-Image layout prompts (
bboxin pixels from the centre,abs_centralized,yxyx), such as the output of ComfyUI's MOSS-Image Layout Prompt node, keep their items. Every layout the model was trained on began with a full-canvaspanelitem, so a prompt without one gets a bare panel (label and box only) put first. Every box must fit the image size. - MOSS-Image editable drafts (
width,height,bboxin pixels from the top-left,abs,xyxy) are compiled the way MOSS-Image'scompile_prompt.pycompiles them, then given the panel. The draft'swidthandheightmust equal the image size.
In ComfyUI, give either to DFlow Layout Encode; DFlow Text Encode encodes a MOSS-Image layout prompt exactly as written, without the panel.
Worked example: a jazz festival poster
This layout describes an 832Γ1216 poster with a sun, a saxophonist and two lines of text:
{"summary": {
"meta": ["concert_poster", "flat_illustration", "mid_century_modern"],
"canvas": "single",
"caption": "A mid-century style jazz festival poster shows a saxophonist in silhouette against a large orange sun on a deep teal background. The title \"BLUE HARBOR JAZZ\" runs across the top in cream capitals and the line \"JULY 12 β 14, 1996\" sits at the bottom."},
"frame": {
"caption": "The saxophonist stands in the centre, cut off at the knees, with the sun behind his shoulders. The title fills the top fifth and the date line sits in a band along the bottom edge. Flat colour shapes with a light paper grain.",
"tags": ["flat_color", "centered_composition", "paper_grain"],
"palette": ["#0F4C5C", "#F28C28", "#F5E9D0", "#1B1B1B"]},
"items": [
{"label": "object",
"caption": "A large flat orange circle, the sun, behind the musician.",
"tags": ["sun", "circle"],
"palette": ["#F28C28"],
"box_2d": [230, 170, 720, 830]},
{"label": "character",
"caption": "A black silhouette of a man in a hat playing a tenor saxophone, facing right, leaning back slightly.",
"tags": ["saxophonist", "silhouette", "hat"],
"palette": ["#1B1B1B"],
"box_2d": [260, 250, 880, 760]},
{"label": "text",
"caption": "The title \"BLUE HARBOR JAZZ\" in bold condensed cream capitals on one line.",
"text": "BLUE HARBOR JAZZ",
"tags": ["title", "condensed_sans", "uppercase"],
"palette": ["#F5E9D0"],
"box_2d": [60, 70, 180, 930]},
{"label": "text",
"caption": "The date line \"JULY 12 β 14, 1996\" in small spaced cream capitals.",
"text": "JULY 12 β 14, 1996",
"tags": ["date", "small_caps"],
"palette": ["#F5E9D0"],
"box_2d": [905, 300, 950, 700]}]}
Reading it from the top:
- The summary caption describes the finished poster in one paragraph, as a plain prompt would, and quotes both lines of text.
- The frame caption says how the canvas is arranged, and the palette lists the four colours of the design.
- The sun comes first in
itemsand the saxophonist second. Their boxes overlap, which is how the layout says that the figure stands in front of the sun. - The title box
[60, 70, 180, 930]spans from 6% to 18% of the height and from 7% to 93% of the width: a wide band near the top. Thetextfield holds the string to render and the caption describes the lettering.
The same layout ships as the workflow
comfyui/workflows/dflow_layout_poster.json. In Python:
from pathlib import Path
from dflow.inference import GenerationSettings
from dflow.pipeline import DFlowPipeline
pipe = DFlowPipeline.from_pretrained("dflow_2b")
layout = Path("jazz_poster.json").read_text(encoding="utf-8")
settings = GenerationSettings(width=832, height=1216, seed=42)
image = pipe(layout, settings).images[0]
At 832Γ1216 the sun's box [230, 170, 720, 830] compiles to
[-329, -275, 268, 275]: pixels from the image centre, top edge first.
Tips
- Write the exact characters in
text, including capitals, punctuation and spaces. The model renders that string; the item's caption describes its style. - Keep the number of items small. A handful of well-described boxes works better than many small ones.
- Boxes may overlap or contain one another.
- Give each box the shape of its content. A single line of text wants a wide, short box.
- Small text and long passages often come out wrong in this release, so prefer short strings in generous boxes.
Image sizes
Width and height must be multiples of 16, because the autoencoder maps each 16Γ16 block of pixels to one latent cell.
The model works best at about one megapixel, the area it was trained on, in portrait, square or landscape. The default is 832Γ1216.
| Shape | Sizes of about 1 MP |
|---|---|
| Portrait | 832Γ1216 |
| Square | 1024Γ1024 |
| Landscape | 1216Γ832 |
Larger images work, for example 1536Γ832, 1216Γ1088 or 1024Γ1536. Composition and anatomy become less reliable as the area grows, and the self-attention gain should move towards 0. Shapes far from the trained aspect ratios are also less reliable.
Sampling
Sampling starts from noise and moves to the image in a number of solver steps. Time runs from , pure noise, to , the finished image. Three choices define a sampling run: the solver, the schedule of times it visits, and a shift applied to that schedule.
Solvers
A solver is the numerical method that takes each step. Some are deterministic, and some are stochastic: they add fresh noise at each step. For a stochastic solver the noise scale multiplies that noise; deterministic solvers ignore it.
| Name | Method | Stochastic | Noise scale |
|---|---|---|---|
er_sde (default) |
Extended reverse-time SDE, three-stage | yes | 1.0; 0 makes it deterministic |
er_sde_bt |
The same with Brownian-tree noise, so runs with different step counts follow one noise path | yes | 1.0 |
euler |
First-order ODE | no | ignored |
euler_maruyama |
First-order reverse SDE | yes | 1.0 is the SDE's own noise level |
dpmpp_2m |
DPM++ 2M, second-order multistep ODE | no | ignored |
dpmpp_2m_sde |
DPM++ 2M SDE | yes | Acts as eta; use 0.5 |
abm2 |
AdamsβBashforthβMoulton predictor-corrector | no | ignored |
abm2 evaluates the model twice per step, so it takes about twice as long per
step as the others.
Schedules
The schedule sets where the solver's time points lie between noise and image, before the shift.
| Name | Spacing |
|---|---|
beta_1_25 (default) |
Quantiles of a Beta(1.25, 1.25) distribution: slightly denser in the middle |
beta_0_6 |
Quantiles of Beta(0.6, 0.6): denser at both ends |
linear |
Evenly spaced |
custom_beta |
Quantiles of a Beta distribution you specify by beta_mean in (0, 1) and beta_concentration above 0 |
For custom_beta the two Beta parameters are beta_mean Γ beta_concentration
and (1 β beta_mean) Γ beta_concentration. A mean above 0.5 puts more steps
near the noisy end. A concentration of 2 at mean 0.5 is the linear schedule;
higher values gather steps around the mean and lower values push them to the
ends.
Steps
The default is 50 steps, which the gallery uses. With the default guidance each step calls the denoiser twice, once in full and once with its middle layers skipped.
Resolution-dependent shift
A larger image needs more noise to hide the same amount of structure. The model was trained with a timestep shift that grows with the image area, and the sampler applies the same shift by default.
The shift is expressed in log signal-to-noise ratio. For an image of width and height in pixels,
Each schedule time strictly between 0 and 1 is then moved to
where is the logistic function. A negative shift moves the time points towards the noisy end.
| Size | Area | Shift |
|---|---|---|
| 832Γ1216 | 1.01 MP | β2.99 |
| 1024Γ1024 | 1.05 MP | β3.02 |
| 1536Γ832 | 1.28 MP | β3.22 |
| 1024Γ1536 | 1.57 MP | β3.43 |
Two settings control the shift. base_log_snr_shift is , default β0.25.
resolution_shift adds the size term and is on by default; with it off, the
shift is alone at every size. The defaults match how the model was
trained.
Guidance
Guidance sharpens an image and strengthens its adherence to the prompt by comparing two predictions at each step and pushing the result away from the weaker one.
Path-drop guidance
dflow_2b was trained so that it can also run with its 18 middle layers skipped, which gives a cheaper and weaker prediction from the same weights. Path-drop guidance (PDG) uses that weak prediction as the reference and moves the result away from it, towards the full prediction, by a scale. The technical report explains why this works.
With the full prediction , the weak one and the scale , the guided prediction is
A scale of 1 is no guidance, and the weak pass is then skipped. The default is 2.5.
Suffix caption bias
By default the weak pass reads the prompt less in its last six blocks: a bias
of β4 is added to their attention on the prompt's tokens, on every step. This
strengthens prompt following. Values closer to 0 weaken the effect and 0 turns
it off; move towards β2 if colours or contrast look too strong. Below about β8
there is almost no further change. It is PdgGuidance(suffix_caption_bias=...)
in Python and pdg_suffix_caption_bias on the DFlow Guider node. The
technical report gives the details.
Classifier-free guidance
Classifier-free guidance (CFG) is the familiar alternative. Its reference is the full model conditioned on the negative prompt, or on the learned null text when the negative prompt is empty. The formula is the same, with that reference as . CFG is off by default. It can be used alone or combined with PDG through the modes below. With the plain PDG modes, CFG and PDG cannot share steps: give them disjoint step windows, or use a mode that includes CFG.
Modes
The PDG mode decides what the reference is at each step. "Negative text" means the negative prompt, or the learned null text when it is empty.
| Mode | Reference |
|---|---|
full (default) |
Middle skipped, conditioned on the prompt (read with the suffix caption bias) |
full_contrastive |
Middle skipped, conditioned on the negative text |
three_quarter |
One image token of each 2Γ2 group goes through the middle; conditioned on the prompt. Width and height must be multiples of 32 |
alternate_contrastive_positive |
full_contrastive on even steps, full on odd steps |
alternate_pdg_first, alternate_cfg_first |
Contrastive PDG and CFG on alternate steps, starting with the one named |
alternate_positive_pdg_first, alternate_positive_cfg_first |
The same with prompt-conditioned PDG |
combined_cfg_pdg |
CFG and contrastive PDG on every step, their two corrections averaged |
pdg_with_alternating_cfg |
Contrastive PDG on every step, with CFG averaged in on odd steps |
cfg_to_pdg |
CFG before the PDG step window, prompt-conditioned PDG inside it |
Steps are counted from 0. The modes that involve CFG need a CFG scale even when plain CFG is switched off.
In the two averaging modes the scales are extra strengths added on top of the full prediction, so a PDG or CFG value of 0 there means no contribution from that side.
Scale curves
The PDG scale can be constant or vary along the trajectory.
| Curve | Behaviour | Parameters and defaults |
|---|---|---|
| Constant (default) | One scale at every step | scale 2.5 |
| Ramp | Starts at one scale at the noisy end, moves linearly to another over the first part of the schedule, then holds | start 4.5, end 2.5, length 0.3 |
| Endpoint curve | Goes from a scale at the first step to a scale at the last, along a chosen shape | noisy scale 2.0, clean scale 2.5 |
| Bezier | 1 plus a multiplier times a fixed fitted curve over time | multiplier 3.0 |
The ramp's length is a fraction of the schedule: 0.3 with 50 steps means the scale reaches its end value at step 15.
An endpoint curve has one of four shapes:
| Shape | Interpolation between the two scales |
|---|---|
linear |
Linear in the step index |
power |
(step / steps) raised to power, default 3: stays near the noisy scale for longer |
s_curve |
A logistic curve with its midpoint at s_curve_knee (default 0.5) and slope s_curve_sharpness (default 8) |
linear_logsnr |
Linear in the log signal-to-noise ratio of each step |
The ramp cannot be combined with the cfg_to_pdg mode, which switches on the
step window.
Step windows
PDG and CFG each have a step window: a first and a last step, both inclusive, outside which that guidance is off. The default window covers every step. In ComfyUI a stop step of β1 means the last step.
Projected guidance (APG)
High guidance scales raise contrast and saturation. Projected guidance reduces
this by splitting the guidance correction into a part parallel to the full
prediction of the clean image and a part orthogonal to it, and keeping only a
fraction of the parallel part. That fraction is parallel_weight: 1 is plain
guidance and 0 removes the parallel part.
The projection has two forms. global projects each image as a whole.
per_band splits the latent into five spatial frequency bands, from the mean
colour (band 0) and the coarsest structure (band 1) up to the finest detail
(band 4), and projects each band from first_band upwards, leaving the lower
bands unprojected.
Projected guidance needs PDG or CFG to be on. It cannot be combined with guidance rescale or with the two averaging modes.
Guidance rescale
Guidance rescale is a simpler remedy for the same problem. It pulls the magnitude of the guided prediction back towards that of the full prediction. A value of 0 is off and 1 matches the magnitude fully.
Sparse middle steps
This is a speed option. On the chosen steps the full pass sends only one image
token of each 2Γ2 group through the middle layers, a configuration the model
was trained on. percent is the share of steps that run this way, and the
schedule is prefix for the first steps of the run or cyclic to spread them
evenly. The default is 0, which is off. Width and height must be multiples of
32.
Defaults
| Setting | Default |
|---|---|
| PDG | on, mode full, constant scale 2.5, all steps, suffix caption bias β4 |
| CFG | off |
| Self-attention gain | β0.03 on the full pass, 0 on the guidance pass |
| Projected guidance | off |
| Guidance rescale | 0 |
| Sparse middle steps | 0% |
| Negative prompt | empty |
Recipes
Each recipe changes one thing from the defaults. The Python snippets build a
GuidanceSettings object, which goes into GenerationSettings(guidance=...).
They use these imports:
from dflow.guidance import (
BezierPdgCurve,
CfgGuidance,
ConstantPdgScale,
EndpointCurveKind,
EndpointPdgCurve,
GuidanceSettings,
PdgGuidance,
PdgMode,
ProjectedGuidance,
ProjectionMode,
RampPdgScale,
SparseMiddleSteps,
SparseStepSchedule,
StepWindow,
)
In ComfyUI, the mode, the constant scale, the step windows, CFG and the self-attention gain are on the DFlow Guider node. Curves, projected guidance, rescale and sparse steps are on DFlow Guider Advanced, which takes the guider as input.
A different PDG scale. Lower values are softer, higher values add contrast.
GuidanceSettings(pdg=PdgGuidance(scale=ConstantPdgScale(2.0)))
Contrastive PDG with a negative prompt. The negative prompt goes in
GenerationSettings(negative_prompt=...), or into a second DFlow Text Encode
node connected to the guider's negative input. A lower scale suits this mode.
GuidanceSettings(
pdg=PdgGuidance(mode=PdgMode.FULL_CONTRASTIVE, scale=ConstantPdgScale(1.75))
)
A ramp from 4.5 to 2.5 over the first 30% of the schedule.
GuidanceSettings(pdg=PdgGuidance(scale=RampPdgScale(4.5, 2.5, 0.3)))
An endpoint curve, here an S-curve from 2.0 to 3.0.
GuidanceSettings(
pdg=PdgGuidance(
scale=EndpointPdgCurve(
kind=EndpointCurveKind.S_CURVE, noisy_scale=2.0, clean_scale=3.0
)
)
)
The Bezier curve.
GuidanceSettings(pdg=PdgGuidance(scale=BezierPdgCurve(multiplier=3.0)))
PDG without the suffix caption bias.
GuidanceSettings(pdg=PdgGuidance(suffix_caption_bias=0.0))
PDG on part of the run only, here steps 0 to 39 of 50.
GuidanceSettings(pdg=PdgGuidance(window=StepWindow(0, 39)))
CFG alone.
GuidanceSettings(pdg=None, cfg=CfgGuidance(enabled=True, scale=4.0))
CFG and PDG on alternate steps. The CFG scale is set but plain CFG stays off; the mode runs the CFG steps itself.
GuidanceSettings(
pdg=PdgGuidance(mode=PdgMode.ALTERNATE_POSITIVE_PDG_FIRST),
cfg=CfgGuidance(enabled=False, scale=4.0),
)
Projected guidance, per band from the third band up with the parallel part removed, at a slightly higher scale.
GuidanceSettings(
pdg=PdgGuidance(scale=ConstantPdgScale(3.0)),
projection=ProjectedGuidance(
mode=ProjectionMode.PER_BAND, parallel_weight=0.0, first_band=2
),
)
Guidance rescale.
GuidanceSettings(rescale=0.5)
Sparse middle on half of the steps, spread evenly.
GuidanceSettings(
sparse_middle=SparseMiddleSteps(percent=50.0, schedule=SparseStepSchedule.CYCLIC)
)
Self-attention gain
The self-attention gain is a small correction to the full pass that reduces the excess contrast guidance produces. It is part of the default preset, and its best value depends on the image size.
What it does
In every image layer of the denoiser, the gain multiplies the attention queries by . A negative gain makes attention slightly more diffuse and a positive one makes it sharper. The main pass uses the gain you set and the guidance pass uses 0.
The guidance formula amplifies any difference between the two passes by the guidance scale, so a 3% softening of attention in the main pass becomes a visible reduction in contrast and in structural errors. The technical report gives the derivation.
Choosing a value
The default, β0.03, was set at the trained sizes of about one megapixel. For larger images, move the gain closer to 0 as the area grows.
| Image area | Example sizes | Gain |
|---|---|---|
| up to about 1.05 MP | 832Γ1216, 1216Γ832, 1024Γ1024 | β0.03 (default) |
| about 1.3 MP | 1536Γ832, 1216Γ1088 | β0.02 |
| about 1.6 MP | 1024Γ1536 | β0.015 |
| about 2.5 MP and above | 1280Γ1920 | 0 to β0.01 |
These values are roughly linear in the logarithm of the area, from β0.03 at 1 MP to 0 at about 2.5 MP, so sizes in between can be interpolated.
At about one megapixel, a gain closer to 0 gives oversaturation and more structural defects. At larger sizes, too negative a gain gives washed-out images.
In Python the gain is GuidanceSettings(self_attention_gain=...). In ComfyUI
it is self_attention_gain on the DFlow Guider node. The gain of the guidance
pass can also be set, as guidance_self_attention_gain, and should stay at 0.
Python API reference
All classes below are frozen dataclasses unless noted. Their defaults are the
default preset, so GenerationSettings() reproduces it.
DFlowPipeline
dflow.pipeline.DFlowPipeline loads the model and turns prompts into images.
DFlowPipeline.from_pretrained(
path_or_repository="data-archetype/dflow_2b",
*,
device="cuda",
compile=True,
revision=None,
cache_dir=None,
vae_dir=None,
)
| Argument | Meaning |
|---|---|
path_or_repository |
A local release folder, or a Hub repository id to download |
device |
A CUDA device |
compile |
Compile with torch.compile; False runs eagerly |
revision |
Hub revision, for a repository id |
cache_dir |
Hugging Face cache directory; None for the default |
vae_dir |
A local copy of data-archetype/dinac3_96, for offline use; None downloads the pinned commit |
Calling the pipeline generates images:
pipe(
prompt,
settings=None,
*,
num_images=1,
output_type=OutputType.PIL,
image_to_image=None,
callback=None,
)
| Argument | Meaning |
|---|---|
prompt |
One prompt, or a list of prompts that all have the same number of tokens |
settings |
A GenerationSettings; None uses pipe.default_settings |
num_images |
Images to make from a single prompt string |
output_type |
What to return; see OutputType |
image_to_image |
An ImageToImage, to start from existing latents |
callback |
A latent callback, called after each solver step |
A prompt may be a plain caption or a layout in any form that JSON layouts describes; a compiled layout's boxes must fit the requested size. The images of one call share the seed and differ because each gets its own noise.
The result is a DFlowOutput with three fields: images, a list of PIL images
for PIL output; tensors, the RGB or latent tensor for the other output types;
and settings, the settings used.
DFlowComponents
dflow.pipeline.DFlowComponents holds the loaded parts and is available as
pipe.components. DFlowComponents.from_pretrained takes the same arguments
as the pipeline's.
| Field | Meaning |
|---|---|
engine |
The denoiser and text encoder. engine.sample(prompts, settings, image_to_image=None, callback=None) returns latents |
vae |
The autoencoder. vae.decode(latents) returns RGB in about [β1, 1]; vae.encode(images) takes RGB in [β1, 1] with sides that are multiples of 16 |
compiled |
Whether the model was compiled |
Latents are float32 tensors of shape [B, 96, height / 16, width / 16].
GenerationSettings
dflow.inference.GenerationSettings holds everything except the prompt that
defines one generation.
| Field | Default | Meaning |
|---|---|---|
width |
832 | Image width, a multiple of 16 |
height |
1216 | Image height, a multiple of 16 |
seed |
0 | Seed of every random draw, from 0 to 2βΆΒ³ β 1 |
sampling |
SamplingSettings() |
Solver, schedule and shift |
guidance |
GuidanceSettings() |
Guidance and gains |
emphasis |
EmphasisSettings() |
Prompt emphasis; None reads prompts verbatim |
negative_prompt |
"" |
Read only by CFG and the contrastive PDG modes; empty selects the learned null text |
The starting noise is drawn on the CPU from the seed, so a seed gives the same noise on every GPU.
SamplingSettings
dflow.inference.SamplingSettings:
| Field | Default | Meaning |
|---|---|---|
solver |
Solver.ER_SDE |
A member of dflow.solvers.Solver |
noise_scale |
1.0 | Noise multiplier of stochastic solvers |
schedule |
TimeSchedule() |
Spacing and step count |
base_log_snr_shift |
β0.25 | The size-independent part of the shift |
resolution_shift |
True |
Add the image-size term |
dflow.schedules.TimeSchedule has kind (a ScheduleKind, default
BETA_1_25), steps (default 50), and beta_mean and beta_concentration
for CUSTOM_BETA (defaults 0.5 and 0.6).
from dflow.inference import GenerationSettings, SamplingSettings
from dflow.schedules import ScheduleKind, TimeSchedule
from dflow.solvers import Solver
settings = GenerationSettings(
width=1024,
height=1024,
seed=3,
sampling=SamplingSettings(
solver=Solver.DPMPP_2M,
schedule=TimeSchedule(kind=ScheduleKind.LINEAR, steps=40),
),
)
GuidanceSettings and its parts
dflow.guidance.GuidanceSettings:
| Field | Default | Meaning |
|---|---|---|
pdg |
PdgGuidance() |
Path-drop guidance; None turns it off |
cfg |
CfgGuidance() |
Classifier-free guidance |
self_attention_gain |
β0.03 | Gain of the main pass, from β0.5 to 0.1 |
guidance_self_attention_gain |
0.0 | Gain of the guidance passes |
projection |
ProjectedGuidance() |
Projected guidance (APG) |
rescale |
0.0 | Guidance rescale, from 0 to 1 |
sparse_middle |
SparseMiddleSteps() |
Sparse middle steps |
The parts, all in dflow.guidance:
| Class | Fields and defaults |
|---|---|
PdgGuidance |
mode=PdgMode.FULL, scale=ConstantPdgScale(), window=StepWindow(), suffix_caption_bias=-4.0 (from β10 to 0; 0 is off) |
CfgGuidance |
enabled=False, scale=None, window=StepWindow() |
StepWindow |
start_step=0, stop_step=None (the last step); both inclusive |
ConstantPdgScale |
scale=2.5 |
RampPdgScale |
start_scale=4.5, end_scale=2.5, length=0.3 |
EndpointPdgCurve |
kind=EndpointCurveKind.POWER, noisy_scale=2.0, clean_scale=2.5, power=3.0, s_curve_knee=0.5, s_curve_sharpness=8.0 |
BezierPdgCurve |
multiplier=3.0, plus the control points of the curve |
ProjectedGuidance |
mode=ProjectionMode.OFF, parallel_weight=1.0, first_band=0 |
SparseMiddleSteps |
percent=0.0, schedule=SparseStepSchedule.PREFIX |
CfgGuidance.scale must be set when CFG is enabled or when the PDG mode uses
CFG, and must be left as None otherwise. Inconsistent settings raise a
ValueError when the object is built, before any sampling.
EmphasisSettings
dflow.prompt_emphasis.EmphasisSettings has mapping
(EmphasisMapping.LINEAR or LOG, default LINEAR), strength (default 1.0)
and encoder (default True). See Prompt emphasis.
ImageToImage
dflow.inference.ImageToImage starts sampling from existing latents, noised to
the time of a later schedule step.
| Field | Meaning |
|---|---|
latents |
[B or 1, 96, h, w] latents of the generation size, for example from pipe.components.vae.encode |
start_step |
The step of the schedule at which sampling starts, from 0 to steps β 1 |
The schedule is built at its full step count. The latents are noised to the
time of start_step with the seed's noise, and the remaining steps run. A
later start step keeps more of the input image. Guidance step windows count
from the start step.
OutputType
dflow.pipeline.OutputType:
| Member | Result |
|---|---|
PIL |
images: PIL images that carry their generation metadata |
TENSOR |
tensors: [B, 3, H, W] float32 RGB in [β1, 1] |
LATENT |
tensors: undecoded [B, 96, H/16, W/16] latents |
Previews during sampling
A latent callback has the signature callback(latents, completed, total). It
is called after each solver step with the model's current estimate of the
finished latents and the step counts.
dflow.preview.LatentRgbPreview is a fitted projection from those latents to a
small RGB image, at one eighth of the image resolution.
from dflow.preview import LatentRgbPreview, preview_images
preview = LatentRgbPreview.bundled().to(pipe.components.engine.device)
def show(latents, completed, total):
image = preview_images(preview.project(latents))[0]
image.save(f"preview_{completed:03d}.png")
image = pipe("A lighthouse on a cliff at dusk.", callback=show).images[0]
PNG metadata
Each PIL image returned by the pipeline carries a JSON description of how it
was made in image.info["dflow"]: the model and package versions, the prompt,
the index of the image in its batch and the complete settings.
To keep it in the saved file, pass it to save:
image.save("out.png", pnginfo=pipe.png_info(image))
The text is stored in a PNG text chunk named dflow.
Diffusers wrapper
dflow.diffusers_pipeline.DFlowDiffusersPipeline offers the call signature of
a diffusers pipeline. It requires diffusers to be installed and wraps a
DFlowPipeline; the model, sampler and guidance are the same.
from dflow.diffusers_pipeline import DFlowDiffusersPipeline
pipe = DFlowDiffusersPipeline.from_pretrained("dflow_2b")
image = pipe(
"A lighthouse on a cliff at dusk.",
height=1216,
width=832,
num_inference_steps=50,
guidance_scale=2.5,
generator_seed=7,
).images[0]
| Argument | Default | Meaning |
|---|---|---|
prompt |
The prompt | |
negative_prompt |
"" |
Accepted only with settings, whose guidance must read it (CFG or contrastive PDG); set it in those settings |
height, width |
1216, 832 | Image size |
num_inference_steps |
50 | Solver steps |
guidance_scale |
2.5 | The PDG scale |
num_images_per_prompt |
1 | Images to generate |
generator_seed |
0 | The seed |
settings |
None |
A full GenerationSettings; when given it replaces every argument except the prompt and the image count |
guidance_scale here is the PDG scale, so values suited to CFG in other models
are too high. The wrapper is loaded with its own from_pretrained, as above.
It does not support save_pretrained or loading through
DiffusionPipeline.from_pretrained.
ComfyUI node reference
The nodes are in the DFlow category. Their defaults are the default preset.
DFlow Loader
Loads the denoiser, the text encoder and the autoencoder. Outputs: model,
text_encoder, vae. The vae output works with ComfyUI's VAEDecode and
VAEEncode nodes.
| Input | Default | Meaning |
|---|---|---|
source |
bundled |
bundled reads the weights in the node folder; local_dir another folder; hub a Hugging Face repository |
local_dir |
empty | Release folder, for the local_dir source |
repository |
data-archetype/dflow_2b |
Repository id, for the hub source |
revision |
empty | Hub revision; empty for main |
compile_model |
on | Compile the model with torch.compile |
cache_dir |
empty | Hugging Face cache directory; empty for the default |
DFlow Text Encode
Encodes a plain prompt. Output: conditioning.
| Input | Default | Meaning |
|---|---|---|
text_encoder |
From the loader | |
text |
empty | The prompt |
batch_size |
1 | Copies of the prompt; must equal the latent's batch size |
emphasis |
on | Apply (phrase:weight) markup |
emphasis_mapping |
linear |
linear or log |
emphasis_strength |
1.0 | Multiplier on every weight's effect |
emphasis_encoder |
on | Also apply the weights inside the text encoder |
For a negative prompt, add a second DFlow Text Encode node and connect it to
the guider's negative input. Stored layouts and MOSS-Image drafts go to DFlow
Layout Encode, which takes the image size they are compiled for.
DFlow Layout Encode
Compiles a layout at the image size and encodes it. Output:
conditioning.
| Input | Default | Meaning |
|---|---|---|
text_encoder |
From the loader | |
layout_json |
empty | A stored or compiled layout, or a MOSS-Image layout prompt or draft |
width |
832 | Image width the layout is for |
height |
1216 | Image height the layout is for |
batch_size |
1 | Copies of the prompt |
width and height must equal the size of the latent that is sampled.
DFlow Guider
Sets path-drop guidance, its suffix caption bias, CFG and the self-attention
gain. Output: guider.
| Input | Default | Meaning |
|---|---|---|
model |
From the loader | |
positive |
The prompt's conditioning | |
negative |
not connected | Optional conditioning for CFG and the contrastive PDG modes |
pdg_enabled |
on | Path-drop guidance |
pdg_mode |
full |
The mode |
pdg_scale |
2.5 | Constant PDG scale; 1 is none |
pdg_start_step |
0 | First PDG step |
pdg_stop_step |
β1 | Last PDG step; β1 for the last step |
pdg_suffix_caption_bias |
β4 | Suffix caption bias of the reference; 0 is off |
cfg_enabled |
off | Classifier-free guidance |
cfg_scale |
4.0 | CFG scale, also used by the PDG modes that involve CFG |
cfg_start_step |
0 | First CFG step |
cfg_stop_step |
β1 | Last CFG step; β1 for the last step |
self_attention_gain |
β0.03 | Gain of the main pass |
DFlow Guider Advanced
Takes a guider from DFlow Guider and returns it with further settings. It is
optional. Output: guider.
| Input | Default | Meaning |
|---|---|---|
guider |
From DFlow Guider | |
guidance_self_attention_gain |
0.0 | Gain of the guidance passes |
pdg_curve |
constant |
constant, ramp, linear, linear_logsnr, power, s_curve or bezier |
curve_noisy_scale |
2.0 | Endpoint curves: scale at the first step |
curve_clean_scale |
2.5 | Endpoint curves: scale at the last step |
curve_power |
3.0 | Exponent of the power curve |
s_curve_knee |
0.5 | Midpoint of the s_curve |
s_curve_sharpness |
8.0 | Slope of the s_curve |
ramp_start_scale |
4.5 | Ramp: scale at the noisy end |
ramp_end_scale |
2.5 | Ramp: scale after the ramp |
ramp_length |
0.3 | Ramp: fraction of the schedule |
bezier_multiplier |
3.0 | Height of the Bezier curve |
projection |
off |
Projected guidance: off, global or per_band |
projection_parallel_weight |
1.0 | Weight kept on the parallel part |
projection_first_band |
0 | per_band: first projected band, 0 to 4 |
rescale |
0.0 | Guidance rescale; 0 is off |
sparse_middle_percent |
0.0 | Percent of steps with a sparse middle |
sparse_middle_schedule |
prefix |
prefix or cyclic |
With pdg_curve at constant the scale is the pdg_scale of the DFlow Guider
node. The other choices replace it and read only their own inputs.
DFlow Sampler
Chooses the solver. Output: sampler, which goes to DFlow Time Schedule.
| Input | Default | Meaning |
|---|---|---|
solver |
er_sde |
One of the solvers |
noise_scale |
1.0 | Noise multiplier of stochastic solvers; eta for dpmpp_2m_sde |
Changing the solver in the ComfyUI interface sets noise_scale to that
solver's recommended value.
DFlow Time Schedule
Builds the schedule for the size of the latent. Outputs: sigmas and
sampler.
| Input | Default | Meaning |
|---|---|---|
sampler |
From DFlow Sampler | |
latent |
The latent that will be sampled; gives the image size | |
schedule |
beta_1_25 |
One of the schedules |
steps |
50 | Solver steps |
base_log_snr_shift |
β0.25 | The size-independent part of the shift |
resolution_shift |
on | Add the image-size term |
beta_mean |
0.5 | custom_beta only |
beta_concentration |
0.6 | custom_beta only |
Connect both outputs to SamplerCustomAdvanced. The sampler output carries the
schedule the sigmas were built from: its kind, its shift and the image size.
Guidance computes the ramp and the linear_logsnr curve from them and checks
the size against the latent. With a sampler taken straight from DFlow Sampler,
sampling stops with an error message.
DFlow Empty Latent
Creates an empty 96-channel latent. Output: latent. ComfyUI's stock empty
latent nodes produce a different latent format and cannot be used.
| Input | Default | Meaning |
|---|---|---|
width |
832 | Image width, a multiple of 16 |
height |
1216 | Image height, a multiple of 16 |
batch_size |
1 | Images per run |
Stock nodes
Seeds come from ComfyUI's RandomNoise node. A seed gives the same starting noise in ComfyUI as in the Python package.
SamplerCustomAdvanced takes the noise, the guider, the sampler and sigmas from
DFlow Time Schedule, and the latent. VAEDecode with the loader's vae output
turns the result into an image, and SaveImage stores it with the workflow
embedded.
Troubleshooting
"The dflow_2b weights are not in the node folder", or a file "is a Git LFS
pointer". This happens after a git clone without the large files. Download
them into the folder with the command the message prints,
hf download data-archetype/dflow_2b --local-dir <folder>, or set the loader's
source to hub.
The first image takes much longer than the following ones. The model is being compiled. This happens once per session.
Errors during compilation. torch.compile needs a working compiler
toolchain and a matching PyTorch build. To check whether compilation is the
cause, run eagerly: compile=False in Python, or compile_model off in DFlow
Loader. Eager mode runs the same model at lower speed.
"Please enter a prompt." Empty prompts are off in the release's
config.json, so a forgotten prompt fails instead of generating freely. To
allow them, set "allow_empty_prompt": true under "inference" there.
Out of memory. An 832Γ1216 image needs about 7 GB of VRAM at peak, and memory grows with the image area and the batch size. Generate one image at a time, reduce the size, and close other programs that hold GPU memory. In ComfyUI, other loaded models count against the same memory.
"The prompt has N tokens; the model takes at most 6144." Shorten the
prompt. Text in double quotes and text fields of a layout use one token per
character, so long quoted passages add up quickly.
"Prompts of one batch must have the same token count." A list of different
prompts in one Python call must have equal token counts. Run prompts of
different lengths as separate calls, or use num_images to repeat one prompt.
"This layout prompt was compiled for β¦ the image is β¦" A compiled layout is
being used at a different size. Compile the stored layout again at the image
size: in ComfyUI, set width and height on DFlow Layout Encode to the
latent's size.
"The schedule was built for β¦ but the latent is β¦" The DFlow Time Schedule node is connected to a different latent from the one being sampled. Connect the same latent to both.
"Connect the sampler output of DFlow Time Schedule β¦" SamplerCustomAdvanced is receiving the sampler from DFlow Sampler directly. See DFlow Time Schedule.
"The latent batch is β¦ but the conditioning holds β¦ prompts." Set the same
batch_size on the text encode node and on DFlow Empty Latent.
Large images look dark or burnt, or lose coherence. Adjust the self-attention gain for the image area.