Skip to content

Adding support for Bria's FIBO model #732

Open
kfirgoldberg wants to merge 9 commits intoostris:mainfrom
kfirgoldberg:kfir/add_fibo
Open

Adding support for Bria's FIBO model #732
kfirgoldberg wants to merge 9 commits intoostris:mainfrom
kfirgoldberg:kfir/add_fibo

Conversation

@kfirgoldberg
Copy link
Copy Markdown

@kfirgoldberg kfirgoldberg commented Mar 4, 2026 •
edited
Loading

Adding support for Bria's FIBO model, including training (with low VRAM flags) and inference.

kfirgoldberg and others added 7 commits February 3, 2026 10:41
@kfirgoldberg
Add FIBO model support
7810af7 
Add support for Bria AI's FIBO model including:
- FiboModel implementation with DimFusion text encoding
- Shifted logit-normal timestep sampling for training
- Model-specified default timestep type and content_or_style
- Fix dtype and broadcasting in flowmatch scheduler add_noise
@kfirbria
working with low_vram
08bded7
@kfirbria
removing custom scheduler logic
65b97a1
@kfirbria
undoing unnecessary changes to custom_flowmatch_sampler.py
bb60dd3
@kfirbria
Fix FIBO prompt layer cache ordering and validation
e608d0c
@kfirbria
Remove codex code review doc from branch
ad10717
@kfirbria
Merge upstream/main into kfir/add_fibo
6e75a99
@jaretburkett
Copy link
Copy Markdown
Contributor

jaretburkett commented Mar 28, 2026

Sorry it took so long, I am talking a look at this today.

@jaretburkett
Copy link
Copy Markdown
Contributor

jaretburkett commented Mar 28, 2026

I am running into a few potential challenges with this model since it is unique in the face that it is JSON-native and trained on long structured captions. Which is awesome, but AI Toolkit is not really setup to handle that currently. I am curious how you guys recommend handling this.

  1. The sample captions in AI toolkit are designed to be single line and short. The default sample prompts alone do not produce good results because of this. I am curious if there is a low information json template we could inject the string prompts into that are not detected to be json, to make the model function the best it can with minimum information for sampling?
  2. There is a similar issue with captions on the dataset. Many users when training in a new concept or character, do a simple trigger word, or very short descriptions of the images. Is there a bare minimum json template we could apply to simple captions to keep them from breaking the model when training. I know long json captions would be ideal, but as a bare minimum workaround, this could be helpful?
  3. What is the best method to caption a dataset to get this extremely descriptive json caption?
  4. For the unconditional, is that json formatted as well, or just a blank prompt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kfirgoldberg @jaretburkett @kfirbria
⚡