阿里In-Context LoRA：基于F1身份一致性连贯视频分镜，10组风格无限创意
🌹大家好！欢迎来到破狼公众号。感谢大家的支持与鼓励。在AIGC探索道路上，我将与你一路同行。喜欢就星标关注破狼公众号或文末扫码加入交流群 ！本人仅运营公众号平台，未经授权严禁CSDN等其他平台抄袭和转载！
In-Context LoRA简介
近日有阿里团队发布了一款名为In-Context LoRA 的微调文生图框架（基于ai-toolkit），该框架能够生成具有可定制内在关系的图像集，可以选择性地基于一组图像进行条件生成，从而实现对广泛任务的适应。
在使用扩散变换器（DiTs）框是进行任务无关图像生成的方法，是通过简单地在图像间连接注意力令牌。然而，尽管投入了大量的计算资源，生成图像的保真度仍然不尽如人意。阿里研发团队采用了通过假设文本到图像的DiTs天生就具有上下文生成能力，只需要最小的调整就能激活它们，重新评估并简化了框架。并通过多样化的任务实验，定性地证明现有的文生图的DiTs架构是可以在没有任何调整的情况下有效地执行上下文生成。基于这一洞见，研发团队提出了一个非常简单的流程来利用DiTs的上下文能力：
1. 连接图像而不是令牌，
2. 对多幅图像进行联合描述
3. 使用小数据集（例如，20到100个样本）而不是全参数调整和大数据集进行任务特定的LoRA调整。
In-Context LoRA（IC-LoRA）方法不需要对原始DiT模型进行任何修改，只需要改变训练数据。IC-LoRA流程生成了高保真度的图像集，具有更好地遵循提示。虽然在调整数据方面是任务特定的，但是IC-LoRA框架在架构和流程上仍然是任务无关的。
• Github：https://github.com/ali-vilab/In-Context-LoRA
• huggface：https://huggingface.co/ali-vilab/In-Context-LoRA/tree/main
• 项目主页：https://ali-vilab.github.io/In-Context-LoRA-Page/
演示案例
01
This set of four images illustrates a young artist's creative process in a bright and inspiring studio; [IMAGE1] she stands before a large canvas, brush in hand, adding vibrant colors to a partially completed painting, [IMAGE2] she sits at a cluttered wooden table, sketching ideas in a notebook with various art supplies scattered around, [IMAGE3] she takes a moment to step back and observe her work, and [IMAGE4] she experiments with different textures by mixing paints directly on the palette, her focused expression showcasing her dedication to her craft.02
In a tender exploration of first love, [IMAGE1] we see Jamie nervously arranging flowers in a park, glancing around as if waiting for someone special, [IMAGE2] transitioning to the moment arrives, their eyes locking in a shy smile that speaks volumes, [IMAGE3] finally showing them seated on a bench, sharing stories and laughter, surrounded by blooming blossoms, embodying the magic of young romance.03
This set of four images showcases a teenage girl with curly black hair wearing a stylish denim jacket, each image highlighting her dynamic personality in urban settings; [IMAGE1] she is skateboarding down a graffiti-covered alley, a confident smile on her face as she maneuvers around obstacles; [IMAGE2] she is seated at a trendy café, typing on her laptop with focused determination, the bustling city life visible through the large windows behind her; [IMAGE3] she stands on a rooftop at sunset, her hair blowing in the breeze as she gazes thoughtfully over the city skyline; and [IMAGE4] she is laughing with friends at a vibrant street market, colorful lights and stalls creating a lively atmosphere around her.04
This set of four images captures a lively day spent at a beach between a mother and her son, highlighting their playful connection and shared joy; [IMAGE1] the boy runs towards the water, his arms wide open, with the mother following behind, smiling as she watches him; [IMAGE2] they are knee-deep in the ocean, laughing as they splash each other, the sunlight reflecting off the water; [IMAGE3] they sit on the sand, the boy intently building a sandcastle while the mother assists, both focused and relaxed; [IMAGE4] the final image shows the two walking along the shore at sunset, the mother’s arm draped protectively around her son’s shoulders, their footprints trailing behind them in the sand.05
The set of four images features a minimalist handwriting font for casual use. [IMAGE1] shows "Everyday" on a coffee cup; [IMAGE2] displays "Notes" on a small journal; [IMAGE3] has "Live Simply" on a white pillow; [IMAGE4] shows "Good Vibes" on a cozy blanket, perfect for lifestyle and home decor branding.06
This set of four images captures a colorful, nature-inspired living space with touches of green and earthy textures; [IMAGE1] features a cozy nook with a woven chair draped in green blankets, surrounded by potted plants and botanical prints on the wall; [IMAGE2] highlights a rustic wooden shelf adorned with small planters, candles, and woven baskets; [IMAGE3] displays a serene bedroom with a bed made up in white linens, a natural wood nightstand, and a forest-themed mural; [IMAGE4] shows a close-up of a large plant pot with unique textures beside a patterned area rug.07
This set of four images showcases a rustic-themed PowerPoint template for a culinary workshop; [IMAGE1] introduces "Farm to Table Cooking" in warm, earthy tones; [IMAGE2] organizes workshop sections like "Ingredients," "Preparation," and "Serving"; [IMAGE3] displays ingredient lists for seasonal produce; [IMAGE4] includes chef profiles with short bios.08
This image pair showcases the transformation of a cyclist through a sandstorm visual effect; [IMAGE1] features a cyclist in vibrant gear pedaling steadily on a clear, open road with a serene sky in the background, highlighting focus and determination, [IMAGE2] transforms the scene as the cyclist becomes enveloped in a fierce sandstorm, with sand particles swirling intensely around the bike and rider against a stormy, darkened backdrop, emphasizing chaos and power.更多演示案例：https://ali-vilab.github.io/In-Context-LoRA-Page/
In-Context LoRA ComfyUI体验
同时研发团队还提供Flux体验工作流和10 款多样化风格的Flux LORA模型。如果已经安装过Flux模型体验，仅需导入工作流，并下载对应风格LORA模型即可。风格包括：情侣头像设计（Couple Profile Design）、电影故事板（Film Storyboard）、字体设计（Font Design）、家居装饰（Home Decoration）、肖像画（Portrait Illustration）、肖像摄影（Portrait Photography）、PPT模板（PPT Template）、沙尘暴视觉效果（Sandstorm Visual Effect）、烟花视觉效果（Sparklers Visual Effect）、视觉身份设计（Visual Identity Design。另外，最新BizyAir也已经支持，低显存用户可考虑。
Flux文生图工作流
Flux文生图感兴趣的同学可参考LIBLIB在线运行工作流：FLUX[续篇]：12B参数23G最大开源文生图模型，Dev版直出惊艳美图欣赏
另外LIBLIBAI已支持本地客户端使用可首页下载体验。
本地体验演示:
In-Context LoRA工作流
In-Context LoRA工作流已上传LIBLIB平台：
https://www.liblib.art/modelinfo/fd5f7b35250d47a68beb27701096b505?versionUuid=d6526844494c427eac11d68546742a8d
01.电影故事版
[MOVIE-SHOTS] In a tender exploration of first love, [IMAGE1] we see Jamie nervously arranging flowers in a park, glancing around as if waiting for someone special, [IMAGE2] transitioning to the moment arrives, their eyes locking in a shy smile that speaks volumes, [IMAGE3] finally showing them seated on a bench, sharing stories and laughter, surrounded by blooming blossoms, embodying the magic of young romance.02.肖像画
This set of four images captures the serene moments of an elderly woman tending to her garden. [IMAGE1] She kneels beside a bed of blooming flowers, her hands gently pruning a rose bush, the soft morning light illuminating her silver hair; [IMAGE2] she stands with a watering can, her face calm and peaceful as she nurtures her plants; [IMAGE3] a close-up reveals her content smile as she examines a budding flower in her hand, a sense of pride and joy evident; [IMAGE4] she sits on a small bench, sipping tea with her garden behind her, surrounded by the vibrant colors of her hard work.03.字体设计
The set of four images features a minimalist handwriting font for casual use. [IMAGE1] shows "Everyday" on a coffee cup; [IMAGE2] displays "Notes" on a small journal; [IMAGE3] has "Live Simply" on a white pillow; [IMAGE4] shows "Good Vibes" on a cozy blanket, perfect for lifestyle and home decor branding.04.家具装饰
This vibrant set of four image captures a lively home decor scene filled with color and eclectic charm; [IMAGE1] the first image showcases a cozy living area with pastel-colored walls, a soft blue sofa, wooden storage units displaying colorful accents, and a unique layered pendant light, [IMAGE2] the second image features a kitchen setup with open shelves holding assorted kitchenware, a wire grid for organizing mugs above a white sink, and warm sunlight streaming onto the countertop, [IMAGE3] the third image highlights a bold art wall with an array of colorful, abstract paintings above a sage green sofa adorned with bright cushions, and [IMAGE4] the fourth image shows a cheerful dining nook with a blue table, vividly striped cushions, framed artwork on the sunny yellow wall, and a distinctive green pendant lamp casting a soft glow over the space.05.PPT模板设计
The set of four images presents a PowerPoint template designed for a charity fundraiser; [IMAGE1] introduces "Help Make a Difference" in large, bold text over a background of hands reaching out; [IMAGE2] lists causes like “Education,” “Healthcare,” and “Water Access” with heart icons; [IMAGE3] displays donation statistics; [IMAGE4] includes a call-to-action slide with links to donate and volunteer.06.情侣头像设计
The pair of images depicts a couple in a cartoon-style grocery shopping scene; [IMAGE1] one character reaches for a snack on a high shelf with a playful grin, while [IMAGE2] the other character with wide eyes and a towering cart of food holds a grocery list, all set in a colorful grocery aisle.07.视觉身份设计
In this set of two images, a modern mystical brand identity comes to life. [IMAGE1] Against a deep navy background, intricate star and moon motifs in metallic silver and soft blush pink shimmer in various sizes, creating a cosmic, dreamlike atmosphere. The brand name “Celestial Glow” is displayed in a sleek, geometric font that radiates a mystical yet minimalist vibe. [IMAGE2] The design is adapted onto a glowing glass misting bottle and a crystal-infused body lotion bottle, arranged on a soft, cloud-like velvet fabric with crystals and candles, showing the brand’s ethereal charm in self-care products.08.肖像插画
The image pair contrasts a photographic portrait with its illustrated counterpart, showcasing an artistic reinterpretation; [IMAGE1] the initial photo shows a woman with a high bun, dressed in a classic black trench coat, holding a bright yellow umbrella, standing on a rainy street, while [IMAGE2] the illustration accentuates her pose with exaggerated features, making the umbrella the focal point with vivid yellows and reds, transforming the rain into playful, curving lines.09.沙尘暴视觉效果
This image pair showcases the transformation of a cyclist through a sandstorm visual effect; [IMAGE1] features a cyclist in vibrant gear pedaling steadily on a clear, open road with a serene sky in the background, highlighting focus and determination, [IMAGE2] transforms the scene as the cyclist becomes enveloped in a fierce sandstorm, with sand particles swirling intensely around the bike and rider against a stormy, darkened backdrop, emphasizing chaos and power.更多推荐文章：
• Flux-NewReality:栩栩如生摄影级解禁模型,追求真实细节&风景&神话高品质艺术
• [ComfyUI]InstantIR：来自小红书团队模糊图像修复技术，效果是否惊艳？
• 更像了！5个百分点提升，字节写真换脸PuLID-F1再升级，小红书流量密码
• OmniGen：统一图像生成和多任务集成模型，任意人物自由合影，8位量化体验
• [ComfyUI]Flux：F.1多区域精确控图，无需LORA技术多区域自由构图工具
• [ComfyUI]MochiEdit：最新视频编辑工具，Mochi视频生成加速方案
• [ComfyUI]FaceAging：太好玩啦！仅需几秒看完你或她的一生，从出生到百岁面容
• [ComfyUI]Flux:低显存救星,无限创意!无需部署就能体验最新Joy2|PuLID|LLM等,CF无缝集成
• [ComfyUI]Flux：超治愈！民间青草编织手工艺术，顽强生命微观世界
