Merge branch 'main' into remote_ctrl

Author: SmartArduino

.gitignore

@@ -7,6 +7,9 @@ sdkconfig.old
 sdkconfig
 dependencies.lock
 .env
-.DS_Store
 releases/
-.cache/
+main/assets/lang_config.h
+main/mmap_generate_emoji.h
+.DS_Store
+.cache
+main/mmap_generate_emoji.h

CMakeLists.txt

@@ -4,7 +4,7 @@
 # CMakeLists in this exact order for cmake to work correctly
 cmake_minimum_required(VERSION 3.16)
 
-set(PROJECT_VER "1.6.2")
+set(PROJECT_VER "1.7.0")
 
 # Add this line to disable the specific warning
 add_compile_options(-Wno-missing-field-initializers)

README.md

@@ -7,9 +7,7 @@
 
 👉 [视频介绍【bilibili】](https://www.bilibili.com/video/BV1icXPYVEMN/)
 
-👉 [ESP32+SenseVoice+Qwen72B打造你的AI聊天伴侣！【bilibili】](https://www.bilibili.com/video/BV11msTenEH3/)
-
-👉 [给小智装上 DeepSeek 的聪明大脑【bilibili】](https://www.bilibili.com/video/BV1GQP6eNEFG/)
+👉 [人类：给 AI 装摄像头 vs AI：当场发现主人三天没洗头【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
 
 👉 [AI-01模组使用手册](docs/AI-01_使用手册.pdf)
 
@@ -32,21 +30,20 @@
 已实现功能
 
 - Wi-Fi / ML307 Cat.1 4G
-- BOOT 键唤醒和打断，支持点击和长按两种触发方式
 - 离线语音唤醒 [ESP-SR](https://github.com/espressif/esp-sr)
-- 流式语音对话（WebSocket 或 UDP 协议）
-- 支持国语、粤语、英语、日语、韩语 5 种语言识别 [SenseVoice](https://github.com/FunAudioLLM/SenseVoice)
-- 声纹识别，识别是谁在喊 AI 的名字 [3D Speaker](https://github.com/modelscope/3D-Speaker)
-- 大模型 TTS（火山引擎 或 CosyVoice）
-- 大模型 LLM（Qwen, DeepSeek, Doubao）
-- 可配置的提示词和音色（自定义角色）
-- 短期记忆，每轮对话后自我总结
-- OLED / LCD 显示屏，显示信号强弱或对话内容
-- 支持 LCD 显示图片表情
-- 支持多语言（中文、英文）
+- 支持两种通信协议（[Websocket](docs/websocket.md) 或 MQTT+UDP）
+- 采用 OPUS 音频编解码
+- 基于流式 ASR + LLM + TTS 架构的语音交互
+- 声纹识别，识别当前说话人的身份 [3D Speaker](https://github.com/modelscope/3D-Speaker)
+- OLED / LCD 显示屏，支持表情显示
+- 电量显示与电源管理
+- 支持多语言（中文、英文、日文）
+- 支持 ESP32-C3、ESP32-S3、ESP32-P4 芯片平台
+- 通过设备端 MCP 实现设备控制（音量、灯光、电机、GPIO 等）
+- 通过云端 MCP 扩展大模型能力（智能家居控制、PC桌面操作、知识搜索、邮件收发等）
 
 ## 软件部分
-* ESP-IDF需要在5.3以上，推荐版本为5.3，参考[官方指南](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32c2/get-started/index.html)
+* ESP-IDF需要在5.4以上，推荐版本为5.4，参考[官方指南](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32c2/get-started/index.html)
 * 编译
     ```
     idf.py @main/boards/doit-ai-01-kit/boards.cfg build    
@@ -110,6 +107,7 @@
 | 你好小智            |                  |
 | 小艾小艾(需升级支持) | Hey Alice(需升级支持) |
 
+👉 [新手烧录固件教程](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
 
 ### 机器人
 

README_en.md

@@ -1,68 +1,68 @@
-# XiaoZhi AI Chatbot
+# An MCP-based Chatbot
 
-([中文](README.md) | English | [日本語](README_ja.md))
+(English | [中文](README.md) | [日本語](README_ja.md))
 
-## Introduction
+## Video
+
+👉 [Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
 
-👉 [Build your AI chat companion with ESP32+SenseVoice+Qwen72B!【bilibili】](https://www.bilibili.com/video/BV11msTenEH3/)
+👉 [Handcraft your AI girlfriend, beginner's guide【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
 
-👉 [Equipping XiaoZhi with DeepSeek's smart brain【bilibili】](https://www.bilibili.com/video/BV1GQP6eNEFG/)
+## Introduction
 
-👉 [Build your own AI companion, a beginner's guide【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
+This is an open-source ESP32 project, released under the MIT license, allowing anyone to use it for free, including for commercial purposes.
 
-## Project Purpose
+We hope this project helps everyone understand AI hardware development and apply rapidly evolving large language models to real hardware devices.
 
-This is an open-source project released under the MIT license, allowing anyone to use it freely, including for commercial purposes.
+If you have any ideas or suggestions, please feel free to raise Issues or join the QQ group: 575180511
 
-Through this project, we aim to help more people get started with AI hardware development and understand how to implement rapidly evolving large language models in actual hardware devices. Whether you're a student interested in AI or a developer exploring new technologies, this project offers valuable learning experiences.
+### Control Everything with MCP
 
-Everyone is welcome to participate in the project's development and improvement. If you have any ideas or suggestions, please feel free to raise an Issue or join the chat group.
+As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.
 
-Learning & Discussion QQ Group: 376893254
+![Control everything via MCP](docs/mcp-based-graph.jpg)
 
-## Implemented Features
+### Features Implemented
 
 - Wi-Fi / ML307 Cat.1 4G
-- BOOT button wake-up and interruption, supporting both click and long-press triggers
 - Offline voice wake-up [ESP-SR](https://github.com/espressif/esp-sr)
-- Streaming voice dialogue (WebSocket or UDP protocol)
-- Support for 5 languages: Mandarin, Cantonese, English, Japanese, Korean [SenseVoice](https://github.com/FunAudioLLM/SenseVoice)
-- Voice print recognition to identify who's calling AI's name [3D Speaker](https://github.com/modelscope/3D-Speaker)
-- Large model TTS (Volcano Engine or CosyVoice)
-- Large Language Models (Qwen, DeepSeek, Doubao)
-- Configurable prompts and voice tones (custom characters)
-- Short-term memory, self-summarizing after each conversation round
-- OLED / LCD display showing signal strength or conversation content
-- Support for LCD image expressions
-- Multi-language support (Chinese, English)
-
-## Hardware Section
+- Supports two communication protocols ([Websocket](docs/websocket.md) or MQTT+UDP)
+- Uses OPUS audio codec
+- Voice interaction based on streaming ASR + LLM + TTS architecture
+- Speaker recognition, identifies the current speaker [3D Speaker](https://github.com/modelscope/3D-Speaker)
+- OLED / LCD display, supports emoji display
+- Battery display and power management
+- Multi-language support (Chinese, English, Japanese)
+- Supports ESP32-C3, ESP32-S3, ESP32-P4 chip platforms
+- Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
+- Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
+
+## Hardware
 
 ### Breadboard DIY Practice
 
 See the Feishu document tutorial:
 
-👉 [XiaoZhi AI Chatbot Encyclopedia](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
+👉 ["XiaoZhi AI Chatbot Encyclopedia"](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
 
-Breadboard demonstration:
+Breadboard demo:
 
-![Breadboard Demo](docs/wiring2.jpg)
+![Breadboard Demo](docs/v1/wiring2.jpg)
 
-### Supported Open Source Hardware
+### Supports 70+ Open Source Hardware (Partial List)
 
 - <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="LiChuang ESP32-S3 Development Board">LiChuang ESP32-S3 Development Board</a>
 - <a href="https://github.com/espressif/esp-box" target="_blank" title="Espressif ESP32-S3-BOX3">Espressif ESP32-S3-BOX3</a>
 - <a href="https://docs.m5stack.com/zh_CN/core/CoreS3" target="_blank" title="M5Stack CoreS3">M5Stack CoreS3</a>
-- <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">AtomS3R + Echo Base</a>
-- <a href="https://docs.m5stack.com/en/core/ATOM%20Matrix" target="_blank" title="AtomMatrix + Echo Base">AtomMatrix + Echo Base</a>
+- <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">M5Stack AtomS3R + Echo Base</a>
 - <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="Magic Button 2.4">Magic Button 2.4</a>
 - <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">Waveshare ESP32-S3-Touch-AMOLED-1.8</a>
 - <a href="https://github.com/Xinyuan-LilyGO/T-Circle-S3" target="_blank" title="LILYGO T-Circle-S3">LILYGO T-Circle-S3</a>
 - <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="XiaGe Mini C3">XiaGe Mini C3</a>
-- <a href="https://oshwhub.com/movecall/moji-xiaozhi-ai-derivative-editi" target="_blank" title="Movecall Moji ESP32S3">Moji XiaoZhi AI Derivative Version</a>
-- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">CuiCan AI pendant</a>
+- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">CuiCan AI Pendant</a>
 - <a href="https://github.com/WMnologo/xingzhi-ai" target="_blank" title="WMnologo-Xingzhi-1.54">WMnologo-Xingzhi-1.54TFT</a>
 - <a href="https://www.seeedstudio.com/SenseCAP-Watcher-W1-A-p-5979.html" target="_blank" title="SenseCAP Watcher">SenseCAP Watcher</a>
+- <a href="https://www.bilibili.com/video/BV1BHJtz6E2S/" target="_blank" title="ESP-HI Low Cost Robot Dog">ESP-HI Low Cost Robot Dog</a>
 
 <div style="display: flex; justify-content: space-between;">
   <a href="docs/v1/lichuang-s3.jpg" target="_blank" title="LiChuang ESP32-S3 Development Board">
@@ -77,23 +77,17 @@ Breadboard demonstration:
   <a href="docs/v1/atoms3r.jpg" target="_blank" title="AtomS3R + Echo Base">
     <img src="docs/v1/atoms3r.jpg" width="240" />
   </a>
-  <a href="docs/AtomMatrix-echo-base.jpg" target="_blank" title="AtomMatrix-echo-base + Echo Base">
-    <img src="docs/AtomMatrix-echo-base.jpg" width="240" />
-  </a>  
-  <a href="docs/v1/magiclick.jpg" target="_blank" title="MagiClick 2.4">
+  <a href="docs/v1/magiclick.jpg" target="_blank" title="Magic Button 2.4">
     <img src="docs/v1/magiclick.jpg" width="240" />
   </a>
   <a href="docs/v1/waveshare.jpg" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">
     <img src="docs/v1/waveshare.jpg" width="240" />
   </a>
-  <a href="docs/lilygo-t-circle-s3.jpg" target="_blank" title="LILYGO T-Circle-S3">
-    <img src="docs/lilygo-t-circle-s3.jpg" width="240" />
-  </a>
-  <a href="docs/xmini-c3.jpg" target="_blank" title="Xmini C3">
-    <img src="docs/xmini-c3.jpg" width="240" />
+  <a href="docs/v1/lilygo-t-circle-s3.jpg" target="_blank" title="LILYGO T-Circle-S3">
+    <img src="docs/v1/lilygo-t-circle-s3.jpg" width="240" />
   </a>
-  <a href="docs/v1/movecall-moji-esp32s3.jpg" target="_blank" title="Moji">
-    <img src="http://github.com/docs/v1/%3Cspan%20class="x x-first x-last">movecall-moji-esp32s3.jpg" width="240" />
+  <a href="docs/v1/xmini-c3.jpg" target="_blank" title="XiaGe Mini C3">
+    <img src="http://github.com/docs/v1/%3Cspan%20class="x x-first x-last">xmini-c3.jpg" width="240" />
   </a>
   <a href="docs/v1/movecall-cuican-esp32s3.jpg" target="_blank" title="CuiCan">
     <img src="docs/v1/movecall-cuican-esp32s3.jpg" width="240" />
@@ -104,41 +98,53 @@ Breadboard demonstration:
   <a href="docs/v1/sensecap_watcher.jpg" target="_blank" title="SenseCAP Watcher">
     <img src="docs/v1/sensecap_watcher.jpg" width="240" />
   </a>
+  <a href="docs/v1/esp-hi.jpg" target="_blank" title="ESP-HI Low Cost Robot Dog">
+    <img src="docs/v1/esp-hi.jpg" width="240" />
+  </a>
 </div>
 
-## Firmware Section
+## Software
 
-### Flashing Without Development Environment
+### Firmware Flashing
 
-For beginners, it's recommended to first use the firmware that can be flashed without setting up a development environment.
+For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.
 
-The firmware connects to the official [xiaozhi.me](https://xiaozhi.me) server by default. Currently, personal users can register an account to use the Qwen real-time model for free.
+The firmware connects to the official [xiaozhi.me](https://xiaozhi.me) server by default. Personal users can register an account to use the Qwen real-time model for free.
 
-👉 [Flash Firmware Guide (No IDF Environment)](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
+👉 [Beginner's Firmware Flashing Guide](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
 
 ### Development Environment
 
 - Cursor or VSCode
-- Install ESP-IDF plugin, select SDK version 5.3 or above
-- Linux is preferred over Windows for faster compilation and fewer driver issues
-- Use Google C++ code style, ensure compliance when submitting code
+- Install ESP-IDF plugin, select SDK version 5.4 or above
+- Linux is better than Windows for faster compilation and fewer driver issues
+- This project uses Google C++ code style, please ensure compliance when submitting code
 
 ### Developer Documentation
 
-- [Board Customization Guide](main/boards/README.md) - Learn how to create custom board adaptations for XiaoZhi
-- [IoT Control Module](main/iot/README.md) - Understand how to control IoT devices through AI voice commands
+- [Custom Board Guide](main/boards/README.md) - Learn how to create custom boards for XiaoZhi AI
+- [MCP Protocol IoT Control Usage](docs/mcp-usage.md) - Learn how to control IoT devices via MCP protocol
+- [MCP Protocol Interaction Flow](docs/mcp-protocol.md) - Device-side MCP protocol implementation
+- [A detailed WebSocket communication protocol document](docs/websocket.md)
+
+## Large Model Configuration
+
+If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the [xiaozhi.me](https://xiaozhi.me) console for configuration.
 
-## AI Agent Configuration
+👉 [Backend Operation Video Tutorial (Old Interface)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
 
-If you already have a XiaoZhi AI chatbot device, you can configure it through the [xiaozhi.me](https://xiaozhi.me) console.
+## Related Open Source Projects
 
-👉 [Backend Operation Tutorial (Old Interface)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
+For server deployment on personal computers, refer to the following open-source projects:
 
-## Technical Principles and Private Deployment
+- [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Python server
+- [joey-zhou/xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) Java server
+- [AnimeAIChat/xiaozhi-server-go](https://github.com/AnimeAIChat/xiaozhi-server-go) Golang server
 
-👉 [Detailed WebSocket Communication Protocol Documentation](docs/websocket.md)
+Other client projects using the XiaoZhi communication protocol:
 
-For server deployment on personal computers, refer to another MIT-licensed project [xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server)
+- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Python client
+- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Android client
 
 ## Star History