Skip to content

CI Summary Integration

aryehcitron@gmail.com edited this page May 17, 2026 · 7 revisions

Overview

Kronikol can automatically surface test results and sequence diagrams in your GitHub Actions job summary or Azure DevOps build summary. This eliminates the need to download artifacts or open the full HTML report to see what happened — failed scenarios, error messages, stack traces, and the corresponding sequence diagrams appear directly in the CI build page.

The feature is opt-in via the WriteCiSummary property on ReportConfigurationOptions.


Quick Start

Add WriteCiSummary = true to your report configuration:

new ReportConfigurationOptions
{
    WriteCiSummary = true,
    // ... your existing configuration
}

That's it. The CI platform is auto-detected. On GitHub Actions, the summary is written to $GITHUB_STEP_SUMMARY. On Azure DevOps, it's uploaded via ##vso[task.uploadsummary].


How It Works

CI Platform Detection

CiEnvironmentDetector.Detect() checks environment variables to identify the CI platform:

Environment Variable Platform
GITHUB_ACTIONS GitHub Actions
TF_BUILD Azure DevOps
Neither No CI detected (summary still written to Reports/CiSummary.md for local use)

When both variables are set, GitHub Actions takes precedence.

Summary Content

The summary is failure-focused to stay concise at scale (1000+ tests):

When tests fail:

  • Summary table with pass/fail/skip counts and duration
  • Up to MaxCiSummaryDiagrams (default 10) failed scenarios, each in a collapsed <details> section with darkred text, containing:
    • Error message
    • Stack trace (expanded by default within the scenario)
    • Sequence diagram(s) — with truncated and full versions when notes exceed 10 lines (see Note Truncation below)
  • No passing scenarios are shown

When all tests pass:

  • Summary table with counts and duration
  • Sequence diagrams for the first MaxCiSummaryDiagrams scenarios (collapsed)
  • Truncation notice if more scenarios exist

Diagrams are rendered as standard Markdown images (![diagram](url)) using the PlantUML server URL by default. When using NodeJs or Local rendering, diagrams are embedded as inline base64 SVGs.


Output Files

File Condition Description
Reports/CiSummary.md Always (when WriteCiSummary = true) Markdown summary — also written to the CI platform

GitHub Actions

Example Workflow

name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5

      - name: Setup .NET
        uses: actions/setup-dotnet@v5
        with:
          dotnet-version: '10.0.x'

      - name: Test
        run: dotnet test

      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v5
        with:
          name: test-reports
          path: '**/Reports/'

No additional workflow configuration is needed — the summary is automatically written to $GITHUB_STEP_SUMMARY during test execution. The summary appears in the Actions tab under the job summary section.

What Appears in the Summary

The summary renders inline with full Markdown support including:

  • Tables for the pass/fail/skip counts
  • Sequence diagram images (server URLs, or inline base64 SVGs with NodeJs/Local rendering)
  • Collapsed scenarios via <details> tags, with darkred text for failures
  • Stack traces expanded by default within their scenario
  • Error messages for each failed scenario

Node.js Rendering

Set PlantUmlRendering = PlantUmlRendering.NodeJs to render diagrams as inline base64 SVGs using a local Node.js process. This eliminates the dependency on an external PlantUML server — diagrams render offline and appear directly in the summary without loading external images.

new ReportConfigurationOptions
{
    WriteCiSummary = true,
    PlantUmlRendering = PlantUmlRendering.NodeJs,
}

How it works:

  1. On first use, the renderer downloads plantuml.js (~7 MB) and viz-global.js (~1.4 MB) from the PlantUML JS CDN and caches them to %LOCALAPPDATA%/Kronikol/plantuml-js/
  2. A small render script (embedded resource) is extracted alongside them
  3. For each diagram, the renderer shells out to node with the PlantUML source on stdin and captures the SVG from stdout
  4. The SVG is embedded as a base64 data URI in the Markdown

Requirements: node must be on PATH. This is pre-installed on all major CI runners (GitHub Actions, Azure DevOps, GitLab CI, CircleCI).

Advantages over server rendering:

  • No network dependency for diagram rendering
  • No broken images if the PlantUML server is unreachable
  • Diagrams are self-contained in the Markdown output
  • Uses the 8,000 character diagram split limit (same as browser JS), not the 2,000 character URL-based limit

Note Truncation

When using NodeJs or Local rendering, the CI summary automatically generates two versions of each diagram if any PlantUML note block exceeds 10 lines:

  1. Truncated Sequence Diagram (expanded by default) — Notes exceeding 10 lines are cut to the first 10 lines plus a ... line. This produces a more compact, readable diagram.
  2. Full Sequence Diagram (collapsed by default) — The original untruncated diagram, available by expanding the <details> section.

If no notes in the diagram exceed 10 lines, only the full diagram is shown (expanded, no wrapper).

This matches the interactive truncation feature in the HTML reports but is applied at the PlantUML source level before rendering.


Azure DevOps

Example Pipeline

trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: UseDotNet@2
    inputs:
      version: '10.0.x'

  - script: dotnet test
    displayName: 'Run tests'

  - task: PublishBuildArtifacts@1
    condition: always()
    inputs:
      pathToPublish: '**/Reports'
      artifactName: 'test-reports'

The summary is automatically uploaded via the ##vso[task.uploadsummary] logging command. It appears in the Extensions tab of the build summary page.


Configuration

Property Type Default Description
WriteCiSummary bool false When true, generates a Markdown CI summary and writes it to the detected CI platform (GitHub Actions or Azure DevOps). The summary is also saved to Reports/CiSummary.md.
MaxCiSummaryDiagrams int 10 Maximum number of sequence diagrams to include in the CI summary. When failing: caps the number of failed scenarios shown. When passing: caps the number of showcase diagrams.
PlantUmlRendering PlantUmlRendering BrowserJs How diagrams in the CI summary are rendered. Server uses PlantUML server URLs. NodeJs renders locally via Node.js and embeds inline base64 SVGs (recommended for CI — see Node.js Rendering). Local uses IKVM (requires LocalDiagramRenderer). BrowserJs falls back to server URL rendering in the summary context.

Example Output

All Tests Passing (1,023 scenarios)

# Diagrammed Test Run Summary

| Metric | Value |
|---|---|
| Status | ✅ Passed |
| Scenarios | 1,023 |
| Passed | 1,023 |
| Failed | 0 |
| Skipped | 0 |
| Duration | 4m 12s |

## Sequence Diagrams

<details><summary><strong>Order Processing — Create order happy path</strong></summary>

![diagram](https://plantuml.com/plantuml/svg/SoWkIImg...)

</details>

<details><summary><strong>Order Processing — Cancel order</strong></summary>

![diagram](https://plantuml.com/plantuml/svg/SoWkIImg...)

</details>

*1,013 more scenario(s) not shown — see full report*

3 Failures Out of 1,023

# Diagrammed Test Run Summary

| Metric | Value |
|---|---|
| Status | ❌ Failed |
| Scenarios | 1,023 |
| Passed | 1,019 |
| Failed | 3 |
| Skipped | 1 |
| Duration | 4m 12s |

## ❌ Failed Scenarios (3)

<details><summary><strong style="color: darkred">Order Processing — Duplicate order returns 409</strong></summary>

<div style="color: darkred">

**Error:** Expected 409 Conflict but received 500 Internal Server Error

<details open><summary>Stack Trace</summary>

at OrderTests.Duplicate_order_returns_409() in OrderTests.cs:line 42

</details>

<details open><summary>Truncated Sequence Diagram</summary>

![diagram](data:image/svg+xml;base64,...)

</details>

<details><summary>Full Sequence Diagram</summary>

![diagram](data:image/svg+xml;base64,...)

</details>

</div>

</details>

Note: The truncated/full diagram split only appears when using NodeJs or Local rendering and when at least one note in the diagram exceeds 10 lines. Otherwise, only the full diagram is shown. With Server rendering, diagrams use PlantUML server URLs instead of inline base64.


Limitations

  • Diagram images require network access (server rendering only) — When using the default Server or BrowserJs rendering, the Markdown summary uses PlantUML server URLs. If the CI runner cannot reach the PlantUML server, images will appear as broken links. Use PlantUmlRendering = PlantUmlRendering.NodeJs for offline rendering.
  • GitHub step summary size limit — GitHub Actions imposes a 1 MB limit per step summary. The failure-focused design keeps summaries small (typically 2–10 KB), but NodeJs/Local rendering with inline base64 SVGs produces larger output (~5–50 KB per diagram). Adjust MaxCiSummaryDiagrams to cap output.
  • Azure DevOps rendering — Azure DevOps renders Markdown summaries with slightly different styling than GitHub. The content is the same but may look different.
  • Node.js required for NodeJs rendering — The node executable must be on PATH. This is pre-installed on all major CI runners.

Troubleshooting

Symptom Cause Fix
Summary not appearing in GitHub Actions GITHUB_STEP_SUMMARY env var not set Ensure you're running on a GitHub Actions runner (not a self-hosted runner that lacks this variable)
Summary not appearing in Azure DevOps ##vso[task.uploadsummary] not processed Check the build log for the ##vso command. Ensure the markdown file path is valid and accessible
Broken diagram images PlantUML server unreachable from CI Use PlantUmlRendering = PlantUmlRendering.NodeJs for offline rendering, or point PlantUmlServerBaseUrl to a reachable server
Summary too large Too many diagrams with inline base64 SVGs Reduce MaxCiSummaryDiagrams or switch to Server rendering
Summary appears but no diagrams No DiagramAsCode entries matched scenario IDs Ensure HTTP tracking is configured correctly — see Tracking Dependencies
node not found error Node.js not installed on CI runner All major CI runners (GitHub Actions, Azure DevOps, GitLab CI) have Node.js pre-installed. For self-hosted runners, install Node.js or use Server rendering instead

Home


Demo


Getting Started

Common Tasks

Integration Guides

Extensions

Configuration

Features

Reference

Clone this wiki locally