ComfyUI 最强图像反推 JoyCaption2 安装步骤及问题解决

最新推荐文章于 2026-05-21 12:44:30 发布

原创最新推荐文章于 2026-05-21 12:44:30 发布 · 9.3k 阅读

15 ·

本内容遵循CC 4.0 BY-SA版权协议

GEO检测

收录于

ComfyUI

Qwen-Image 定制镜像 | RTX4090D CUDA12.4 大模型推理专用

基于官方 Qwen-Image 基础镜像定制优化，适配 RTX 4090D 24GB 显存环境，预装 CUDA 12.4、对应驱动 550.90.07，配置 10 核 CPU / 120GB 内存运行环境，内置通义千问视觉语言模型依赖库、推理脚本与常用工具，开箱即用，支持大模型快速加载、图像理解、多模态推理等任务，无需重复配置环境。

JoyCaption2（joy-caption-alpha-two）是当前ComfyUI生态中主流的图像反推工具之一，通过结合多模态模型和语言模型实现高精度图像描述生成，适用于AI绘画提示词生成、批量文件打标等场景。以下是其核心特性与技术要点：

一、功能特性

‌精细化图像反推‌
相比前代模型，反推结果包含更丰富的细节描述，支持对人物特征（如发色、服装）、场景元素（如光照、构图）、风格（如艺术流派）等17类细节的分类增强‌
‌多样化输出控制
支持9种提示词风格（如MidJourney风格、艺术评论模式）
可调节输出长度（从极简到超长描述）
支持NSFW/SFW内容识别与处理‌
‌高效批量处理‌
提供本地部署方案，支持低显存设备（最低10G显存）完成批量图像打标，兼容ComfyUI工作流节点化操作‌

二、技术架构

‌多模型协作‌
采用google/siglip-so400m-patch14-384提取图像特征，结合Llama-3.1-8B-Lexi语言模型生成文本，实现图文对齐优化‌35。
‌动态配置扩展‌
允许用户通过组合细节分类参数（如人物细节+构图风格+景深）定制反推结果，提升生成内容的灵活性与准确性‌。

三、应用场景

AI绘画工作流‌

与ComfyUI Flux插件结合，实现“图像→提示词→新图像”的闭环创作，支持Lora模型调用与多图融合‌13。

自动化数据标注‌

适用于训练集图像标签生成，支持CSV/TXT格式批量导出‌

以下是mac mini M4 下ComfyUI安装步骤及遇到问题解决

ComfyUI Manager 里直接搜索 ComfyUI_SLK_joy_caption_two 安装

四、模型下载

1.CLIP模型

模型地址：https://hf-mirror.com/google/siglip-so400m-patch14-384/tree/main
所有模型下载到

ComfyUI/models/models/clip/siglip-so400m-patch14-384/

2.LLM模型

unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
模型地址：https://hf-mirror.com/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit/tree/main
所有模型下载到

ComfyUI/models/models/ComfyUI/models/LLM/Meta-Llama-3.1-8B-Instruct-bnb-4bit/

3. Joy-Caption-alpha-two 模型

模型地址: https://huggingface.co/spaces/fancyfeast/joy-caption-alpha-two/tree/main/cgrkzexw-599808
所有模型下载到

ComfyUI/models/models/ComfyUI/models/models/Joy_caption_two/

下载完成后重启ComfyUI

五、问题与处理

运行后遇到以下问题猜测是transformers版本问题，
查询当前transformers 版本

pip show transformers
#显示当前版本号为 4.49.0
Name: transformers
Version: 4.49.0

推荐版本号 4.44.0
删了当前版本重新安装

 pip uninstall transformers
 
 pip install transformers==4.44.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

重新运行

Exception during processing !!! Failed to import transformers.models.timm_wrapper.configuration_timm_wrapper because of the following error (look up to see its traceback):
cannot import name 'ImageNetInfo' from 'timm.data' (/opt/miniconda3/lib/python3.12/site-packages/timm/data/__init__.py)
Traceback (most recent call last):
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/models/timm_wrapper/configuration_timm_wrapper.py", line 25, in <module>
    from timm.data import ImageNetInfo, infer_imagenet_subset
ImportError: cannot import name 'ImageNetInfo' from 'timm.data' (/opt/miniconda3/lib/python3.12/site-packages/timm/data/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/jerry/Documents/confyui/ComfyUI/execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jerry/Documents/confyui/ComfyUI/execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jerry/Documents/confyui/ComfyUI/execution.py", line 174, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/Users/jerry/Documents/confyui/ComfyUI/execution.py", line 163, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jerry/Documents/confyui/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 268, in generate
    self.loadModels()
  File "/Users/jerry/Documents/confyui/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 263, in loadModels
    self.pipeline.loadModels()
  File "/Users/jerry/Documents/confyui/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 229, in loadModels
    self.clip_model = JoyClipVisionModel(self.load_device, self.offload_device)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jerry/Documents/confyui/ComfyUI/custom_nodes/comfyui_slk_joy_caption_two/joy_caption_two_node.py", line 43, in __init__
    clip_model = AutoModel.from_pretrained(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 543, in from_pretrained
    has_local_code = type(config) in cls._model_mapping.keys()
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 781, in keys
    self._load_attr_from_module(key, name)
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 777, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 693, in getattribute_from_module
    if hasattr(module, attr):
       ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.timm_wrapper.configuration_timm_wrapper because of the following error (look up to see its traceback):
cannot import name 'ImageNetInfo' from 'timm.data' (/opt/miniconda3/lib/python3.12/site-packages/timm/data/__init__.py)