AgentScope employs a multi-tiered testing strategy to ensure the reliability of its SDK, model adapters, and execution environments. The infrastructure spans from local unit tests with mocked models to integration tests running in sandboxed containers and automated CI/CD pipelines.
The testing suite is designed to validate both the logic of the SDK and the connectivity of the model layer. It utilizes pytest for unit testing and specialized scripts for verifying LLM provider integrations.
The following diagram illustrates how tests interact with different layers of the AgentScope ecosystem, mapping high-level test types to specific code entities.
AgentScope Testing Data Flow
Sources: .github/workflows/unittest.yml30-34 src/agentscope/workspace/_applecontainer/_applecontainer_backend.py23-25 src/agentscope/workspace/_applecontainer/_applecontainer_workspace.py75-85
The core of the test suite resides in the tests/ directory. It covers every major component, including agents, middleware, formatters, and storage backends. To avoid high API costs and latency during development, AgentScope provides a comprehensive mocking utility.
MockModel class allows developers to simulate complex LLM behaviors, including streaming responses, structured JSON outputs, and API exceptions, without making actual network calls.tests/backend_applecontainer_test.py, verify that shell commands and file I/O operate correctly within isolated containers via AppleContainerBackend tests/backend_applecontainer_test.py39-46For a detailed breakdown of test categories and mocking patterns, see Unit & Integration Tests.
Sources: tests/backend_applecontainer_test.py1-118 tests/workspace_applecontainer_test.py1-73
Located in scripts/model_examples/, these scripts serve a dual purpose: they act as executable documentation for users and as a functional test suite for LLM provider adapters.
run_tests.py script provides a single entry point to detect available API keys and run provider-specific tests.ChatModelBase implementations across different providers.For details on setting up environment variables and running these scripts, see Model Example Scripts.
AgentScope includes robust testing for its sandboxed execution layers, ensuring that tools (like Bash or File I/O) behave consistently across different backends.
AppleContainerWorkspaceManager manages container lifecycles with TTL-based caching and background sweeping src/agentscope/app/workspace_manager/_applecontainer_workspace_manager.py37-48 Tests verify that AppleContainerWorkspace correctly pulls images and bootstraps the gateway src/agentscope/workspace/_applecontainer/_applecontainer_workspace.py157-191AppleContainerBackend implements exec_shell, read_file, and write_file using the container CLI src/agentscope/workspace/_applecontainer/_applecontainer_backend.py23-52 Unit tests mock asyncio.create_subprocess_exec to verify command construction and error handling tests/backend_applecontainer_test.py57-75Sources: src/agentscope/workspace/_applecontainer/_applecontainer_backend.py1-140 src/agentscope/app/workspace_manager/_applecontainer_workspace_manager.py37-114 src/agentscope/workspace/_applecontainer/_applecontainer_workspace.py75-154
The project uses GitHub Actions to automate testing across multiple operating systems and Python versions.
| Component | Description |
|---|---|
| Workflow | unittest.yml triggers on every push and pull request .github/workflows/unittest.yml1-3 |
| Matrix | Runs on ubuntu-latest, windows-latest, and macos-26 .github/workflows/unittest.yml8-11 |
| Dependencies | Uses uv for fast dependency installation and bubblewrap for Linux sandboxing tests .github/workflows/unittest.yml17-25 |
| Coverage | Generates a coverage report using pytest and coverage .github/workflows/unittest.yml30-37 |
| Publishing | publish-pypi.yml automates package builds and uploads to PyPI upon release .github/workflows/publish-pypi.yml9-43 |
CI/CD Entity Mapping
Sources: .github/workflows/unittest.yml1-37 .github/workflows/publish-pypi.yml1-43 .github/workflows/pre-commit.yml1-29