Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-Context LoRA #127

Merged
merged 4 commits into from
Mar 5, 2025
Merged

In-Context LoRA #127

merged 4 commits into from
Mar 5, 2025

Conversation

filipstrand
Copy link
Owner

@filipstrand filipstrand commented Feb 14, 2025

For this feature I did not follow any reference implementation, so I hope it is correct (but does look promising on some initial tests). Would be cool to include in v.0.6.0 which should be released soon (otherwise leave it for next release along with some other editing techniques).

Example:

In this set of two images, a bold modern typeface with the brand name 'DEMA' is introduced and is shown on a company merchandise product photo; [IMAGE1] a simplistic black logo featuring a modern typeface with the brand name 'DEMA' on a bright light green/yellowish background; [IMAGE2] the design is printed on a green/yellowish hoodie as a company merchandise product photo with a plain white background.

seed_42_step20of20

Uses the LoRA adapters from https://huggingface.co/ali-vilab/In-Context-LoRA

Left to do:

  • (DONE) Some cleanup
  • (DONE - Just accept a regular prompt for widest possible options) Think about the best kind of interface for this feature (e.g maybe have 3 separate input prompts - global, IMAGE1 and IMAGE2...the general setup seem to map a set of input images to a set of output ones, so it could be too restrictive to limit this to 1-in 1-out...)
  • (DONE) Automatically download the relevant LoRAs files from HF Hub, like we do for controlnet files etc.
  • (DONE) Write docs, include examples etc.

Also updates some of the existing interface for inputting images
init_image -> image
init_image_strength -> image_strength

Note to self: remember to document this in release notes

@filipstrand filipstrand self-assigned this Feb 14, 2025
@filipstrand filipstrand changed the title In context LoRA In-Context LoRA Feb 14, 2025
@filipstrand filipstrand marked this pull request as draft February 14, 2025 21:36
@filipstrand filipstrand mentioned this pull request Feb 22, 2025
@filipstrand filipstrand force-pushed the in-context-lora branch 15 times, most recently from df78926 to 2b095af Compare March 2, 2025 16:10
@filipstrand filipstrand marked this pull request as ready for review March 2, 2025 16:13
@filipstrand filipstrand force-pushed the in-context-lora branch 7 times, most recently from 741a6d2 to f3cb16e Compare March 5, 2025 06:17
@filipstrand filipstrand merged commit fb15edc into main Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants