[ComfyUI]英伟达Cosmos:图生视频世界模型,生成具有物理意识视频和物理智能世界状态而设计

科技   2025-01-17 08:23   浙江  

英伟达Cosmos:世界模型图生视频ComfyUI官方支持

🌹大家好!欢迎来到破狼公众号。感谢大家的支持与鼓励。在AIGC探索道路上,我将与你一路同行。喜欢就星标关注破狼公众号或文末扫码加入交流群 !

Cosmos世界模型简介

在之前的文章(英伟达Cosmos:世界基础模型ComfyUI官方支持,旨在生成具有物理意识的视频和物理AI开发的世界状态而设计)已经推荐过由英伟达发布的世界模型CosmosCosmos扩散模型是一系列基于扩散的世界基础模型,能够从文本、图像或视频输入生成动态、高质量的视频。它可以作为与世界生成相关的各种应用或研究的构建块。在ComfyUI的最新官方版本更新已支持Cosmos模型的图生视频模型,在视频生成方便具有更高的可控性。今天文章主题将重点体验Cosmos世界模型的图生视频ComfyUI部署和体验。

Cosmos图生视频ComfyUI体验

当前ComfyUI官已支持Cosmos图生视频模型运行,因此仅需更新ComfyUI本体到最新版本即可,无需安装其他插件。(注:本文涉及模型可文末网盘获取

  • • Cosmos视频模型:下载模型并放置 /ComfyUI/models/unet 目录下。这里包含7B14B两个版本,建议使用7B模型。下载地址:https://huggingface.co/mcmonkey/cosmos-1.0/tree/main

  • • cosmos_cv8x8x8_1.0.safetensors - VAE模型:下载模型并放置 /ComfyUI/models/vae 目录下。下载地址:https://huggingface.co/comfyanonymous/cosmos_1.0_text_encoder_and_VAE_ComfyUI/tree/main/vae

  • • oldt5_xxl_fp8_e4m3fn_scaled.safetensors Clip模型:下载模型并放置 /ComfyUI/models/clip 目录下。该模型并非Flux等模型使用的基础T5模型,需要独立下载模型。这里包含fp16fp8两个版本,建议使用fp8版本。下载地址:https://huggingface.co/comfyanonymous/cosmos_1.0_text_encoder_and_VAE_ComfyUI/tree/main/text_encoders

Flux文生图&混元视频工作流

最新LIBLIBAI平台已支持Flux文生图混元视频ComfyUI工作流在线体验:

• F.1-绮梦流光-水湄凝香

https://www.liblib.art/modelinfo/134c6dd95aef48e98a22b24e003e026b

• 文生图-Flux文生图(PuLID|LORA|Joy|SUPIR)工作流

https://www.liblib.art/modelinfo/782aacd70f604da39e83368c696a02a8?versionUuid=9c5eceb01fb94d4d93d60fe2c0bd7468

• 文生视频-腾迅混元最强开源视频(LORA)工作流

https://www.liblib.art/modelinfo/35ee21d5f6a94204abb767ad194ab9cd?versionUuid=be674032ffa14e5597a08922556f4da0


Cosmos世界模型工作流体验

Cosmos世界模型工作流已上传LIBLIBAI平台可体验:
https://www.liblib.art/modelinfo/51ff567e892d4ba4a39f73a97ab7f421?versionUuid=fa46c87e1678406bbc35bfe04e00d017

• Cosmos世界模型:运行7B版本模型需要本地大约24G显存1280*704分辨率121帧5秒视频)采样耗时大约需要7分17秒

• Cosmos世界模型对于物理运动在物理世界的模拟取得相对不错的进展。但在人物处理方面手部容易崩溃和存在幻影。还需要社区进一步迭代。

01.新春祝福

The little girl held the Spring Festival blessing, wish everyone a happy New Year, A photo-realistic shoot from a close-up camera angle about a cute asian girl holding a red paper lantern with chinese characters. the image also shows a shallow depth of field, focusing on the girl's face and upper body. on the middle of the image, a 1-year-old chinese girl, who appears to be around 2-3 years old, with black hair styled in two pigtails, is standing. she is wearing a red dress with long sleeves and a red tassel at the bottom. the girl has a big smile on her face, looking directly at the viewer with her black eyes. her mouth is slightly open, showing her teeth, and she is holding a small red lantern with gold Chinese characters in her hands. the background is blurred, with red lanterns and a wooden wall, creating a festive atmosphere. 1girl, solo, looking at viewer, smile, black hair, long sleeves, hair ornament, dress, holding, teeth, indoors, blurry, teeth out, red dress, blurry background, holding object, holding lantern, chinese new year

02.自拍

Chinese girls dance indoors by twisting their dresses and swinging their hands, A young woman standing in a room, holding a phone and taking a selfie. she is wearing a strapless, pink, ruffled dress with a white, lace-trimmed bodice and a tiered skirt. her long brown hair cascades down her back, and she has a playful, mischievous smile on her face. the room has a white wall with a black and white floral pattern, and a black chair is visible in the background. the woman is standing in the middle of the image, with her upper body facing the camera, and her eyes are looking directly at the viewer. she appears to be in her early twenties, with a slim body and fair skin. her hair is styled in a long, straight look, with brown hair framing her face and brown eyes. her ears are pink and fluffy, adding a playful touch to her overall appearance. 1girl, solo, long hair, breasts, looking at viewer, brown hair, dress, holding, animal ears, cleavage, jewelry, bare shoulders, medium breasts, standing, full body, earrings, indoors, hairband, holding phone, off shoulder, chair, pink dress, strapless dress, animal ear headband

03.剑者

A woman waves her sword, A digital illustration shoot from a frontal camera angle about a fierce female warrior standing confidently in a cityscape at night, holding two swords in her hands. the image also shows a dramatic lighting effect with blue lightning and neon signs in the background. on the middle of the image, a 1woman, who appears to be in her mid-twenties, with long black hair and a serious expression, is standing with her full body facing the viewer. she has a slim body and is wearing a purple and black outfit with intricate designs, including a plunging neckline, long sleeves, and a cape-like capelet. her hair is styled in a long hair style, and she is wearing earrings and a necklace. she is holding a sword in each hand and has a confident and powerful stance. 1girl, solo, long hair, breasts, looking at viewer, large breasts, black hair, hair ornament, holding, closed mouth, cleavage, jewelry, standing, purple eyes, weapon, outdoors, earrings, boots, sword, holding weapon, armor, leotard, lips, night, armor on body, holding sword, night sky, dual wielding, glowing, glowing sword, lightning

Cosmos世界模型:关注公众号口令【Cosmos世界模型】下获取
更多推荐文章:
• 英伟达Cosmos:世界基础模型ComfyUI官方支持,旨在生成具有物理意识的视频和物理AI开发的世界状态而设计
• [ComfyUI]首块缓存:全方位模型推理加速神器。适用于黑森林Flux&腾讯混元视频&LTXV
• [ComfyUI]最强腾讯开源混元视频炼丹炉已就绪,国漫经典李慕婉,一致性写真视频轻松批量直出
• [ComfyUI]腾讯混元视频:官方极限优化8GB可运行!32G到8G极限优化,开源生态加速
• [ComfyUI]Flux:2025元旦快乐,新年心想事成!生肖蛇年之白蛇贺新年
感兴趣加入[AGI技术交流群]+V

    如果觉得文章不错,就请在看转发三连

破狼
关注AIGC、LLM、绘图作品、软件工程、技术学习。交流+V:shunshizhiwu。
 最新文章