Google Gemini 2.5 零样本检测与分割实战：从 JSON 结果到可视化

原创于 2026-07-07 21:36:23 发布 · 18 阅读

1 ·

本内容遵循CC 4.0 BY-SA版权协议

GEO检测

收录于

人工智能入门

教程

Google Gemini 2.5 零样本检测与分割实战：从 JSON 结果到可视化

这篇教程是我根据近期学习 Gemini 2.5 多模态能力、零样本目标检测和实例分割的复现过程整理出来的。重点演示如何用 Gemini 2.5 Flash 直接根据提示词返回检测框或分割 mask，并通过 supervision 解析和可视化结果。

Gemini 2.5 的优势是不用训练就能通过自然语言描述视觉任务。本教程保留模型 prompt 和 API 参数，适合用来理解 VLM 输出 JSON、再把 JSON 转成可视化检测结果的完整链路。

本文会重点跑通以下流程：

配置 GOOGLE_API_KEY 并初始化 Gemini API 客户端
用 prompt 控制模型输出检测框或分割 mask
用 supervision 解析 VLM JSON 并可视化
保留英文类别名和 JSON 字段，避免影响模型输出格式

如果你正在系统学习计算机视觉、多模态模型或 Roboflow 项目复现，建议收藏本文；配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住，可以在评论区说明具体报错。

📚 文章目录

Google Gemini 2.5 零样本检测与分割实战：从 JSON 结果到可视化

⚙️ 环境准备

先配置 Google API Key、安装依赖，并下载两张示例图片。

import os
from google.colab import userdata

os.environ["GOOGLE_API_KEY"] = userdata.get("GOOGLE_API_KEY")

!pip install google-genai supervision

!wget -q https://media.roboflow.com/notebooks/examples/dog-2.jpeg
!wget -q https://media.roboflow.com/notebooks/examples/dog-3.jpeg

🔌 初始化 Gemini API 客户端

这里创建 Gemini 客户端，并设置安全策略。

from google import genai
from google.genai import types

client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])

safety_settings = [
    types.SafetySetting(
        category="HARM_CATEGORY_DANGEROUS_CONTENT",
        threshold="BLOCK_ONLY_HIGH",
    ),
]

🎯 目标检测 Prompt

先让 Gemini 根据提示词检测狗尾巴，并要求模型返回包含 box_2d 和 label 的 JSON。

MODEL_NAME = "gemini-2.5-flash-preview-05-20"
TEMPERATURE = 0.5

IMAGE_PATH = "/content/dog-3.jpeg"
PROMPT = "Detect dogs tail. " + \
"Output a JSON list of bounding boxes where each entry contains the 2D bounding box in the key \"box_2d\", " + \
"and the text label in the key \"label\". Use descriptive labels."

from PIL import Image

image = Image.open(IMAGE_PATH)
width, height = image.size
target_height = int(1024 * height / width)
resized_image = image.resize((1024, target_height), Image.Resampling.LANCZOS)

response = client.models.generate_content(
    model=MODEL_NAME,
    contents=[resized_image, PROMPT],
    config = types.GenerateContentConfig(
        temperature=TEMPERATURE,
        safety_settings=safety_settings,
        thinking_config=types.ThinkingConfig(
          thinking_budget=0
        )
    )
)

response.text

🖼️ 解析检测结果并可视化

使用 supervision.Detections.from_vlm 将 Gemini 返回的 JSON 转换为标准检测结果。

import supervision as sv

resolution_wh = image.size

detections = sv.Detections.from_vlm(
    vlm=sv.VLM.GOOGLE_GEMINI_2_5,
    result=response.text,
    resolution_wh=resolution_wh
)

thickness = sv.calculate_optimal_line_thickness(resolution_wh=resolution_wh)
text_scale = sv.calculate_optimal_text_scale(resolution_wh=resolution_wh)

box_annotator = sv.BoxAnnotator(thickness=thickness)
label_annotator = sv.LabelAnnotator(
    smart_position=True,
    text_color=sv.Color.BLACK,
    text_scale=text_scale,
    text_position=sv.Position.CENTER
)

annotated = image
for annotator in (box_annotator, label_annotator):
    annotated = annotator.annotate(scene=annotated, detections=detections)

sv.plot_image(annotated)

在这里插入图片描述

🧩 实例分割 Prompt

下面让 Gemini 返回 pepper 和 salt 的分割 mask。prompt 中的类别和 JSON 字段会影响输出，保持英文原样。

MODEL_NAME = "gemini-2.5-flash-preview-05-20"
TEMPERATURE = 0.5

IMAGE_PATH = "/content/dog-2.jpeg"
PROMPT = "Give the segmentation masks peper, salt. " + \
"Output a JSON list of segmentation masks where each entry contains the 2D bounding box in the key \"box_2d\", " + \
"the segmentation mask in key \"mask\", and the text label in the key \"label\". Use descriptive labels."

from PIL import Image

image = Image.open(IMAGE_PATH)
width, height = image.size
target_height = int(1024 * height / width)
resized_image = image.resize((1024, target_height), Image.Resampling.LANCZOS)

response = client.models.generate_content(
    model=MODEL_NAME,
    contents=[resized_image, PROMPT],
    config = types.GenerateContentConfig(
        temperature=TEMPERATURE,
        safety_settings=safety_settings,
        thinking_config=types.ThinkingConfig(
          thinking_budget=0
        )
    )
)

response.text

🎨 解析分割结果并可视化

实例分割结果会额外包含 mask，这里使用 MaskAnnotator 叠加到原图。

import supervision as sv

resolution_wh = image.size

detections = sv.Detections.from_vlm(
    vlm=sv.VLM.GOOGLE_GEMINI_2_5,
    result=response.text,
    resolution_wh=resolution_wh
)

thickness = sv.calculate_optimal_line_thickness(resolution_wh=resolution_wh)
text_scale = sv.calculate_optimal_text_scale(resolution_wh=resolution_wh)

box_annotator = sv.BoxAnnotator(thickness=thickness)
label_annotator = sv.LabelAnnotator(
    smart_position=True,
    text_color=sv.Color.BLACK,
    text_scale=text_scale,
    text_position=sv.Position.CENTER
)
masks_annotator = sv.MaskAnnotator()

annotated = image
for annotator in (box_annotator, label_annotator, masks_annotator):
    annotated = annotator.annotate(scene=annotated, detections=detections)

sv.plot_image(annotated)