-
Notifications
You must be signed in to change notification settings - Fork 1
Integration BigQuery Extension
The Kronikol.Extensions.BigQuery package adds Google BigQuery REST API operation tracking to your test diagrams. Instead of showing raw HTTP requests like POST /bigquery/v2/projects/my-project/queries, your sequence diagrams show classified operations like Query: /query or Insert: /mydataset/table/mytable.
Using a shared library or abstraction layer? If your code doesn't use the BigQuery SDK directly — e.g. it goes through a shared data-access library, wrapper, or custom abstraction — this extension won't be able to intercept the underlying calls. See Tracking Custom Dependencies for alternative approaches including
RequestResponseLogger.LogPair(),TrackingProxy<T>, andMessageTracker.
The Google BigQuery .NET SDK (Google.Cloud.BigQuery.V2) translates all operations into REST HTTP requests. BigQueryTrackingMessageHandler is a DelegatingHandler that intercepts these HTTP requests, classifies each one into a BigQuery operation (Query, Insert, Read, List, Create, Delete, Update, Cancel) using the HTTP method and URL path, then logs it to RequestResponseLogger with a human-readable label.
Because it logs to the same RequestResponseLogger as the standard TestTrackingMessageHandler, BigQuery operations appear alongside your HTTP API calls in the same sequence diagram — showing the complete flow from test → API → BigQuery.
dotnet add package Kronikol.Extensions.BigQueryThe extension supports three verbosity levels that control how much detail appears in the diagrams:
| Level | Method shown | URI shown | Headers | Request body | Response body |
|---|---|---|---|---|---|
| Raw | HTTP method (POST, GET, etc.) |
Full BigQuery REST API URI | All except excluded set | Full JSON | Full JSON |
| Detailed | Classified operation (Query) |
Clean path with dataset/resource context (/mydataset/table/mytable) |
Filtered (noisy Google API headers excluded) | Full JSON | Full JSON |
| Summarised | Classified operation (Query) |
Resource type and name only (/table/mytable) |
None | None | None |
The default is Detailed.
| Operation | Raw | Detailed | Summarised |
|---|---|---|---|
| Run a query | POST: /bigquery/v2/projects/proj/queries |
Query: /query |
Query: /query |
| Get query results | GET: /bigquery/v2/projects/proj/queries/job123 |
Query: /query/job123 |
Query: /query/job123 |
| Streaming insert | POST: /bigquery/v2/projects/proj/datasets/ds/tables/tbl/insertAll |
Insert: /ds/table/tbl |
Insert: /table/tbl |
| Read a table | GET: /bigquery/v2/projects/proj/datasets/ds/tables/tbl |
Read: /ds/table/tbl |
Read: /table/tbl |
| List tables | GET: /bigquery/v2/projects/proj/datasets/ds/tables |
List: /ds/table |
List: /table |
| Create a dataset | POST: /bigquery/v2/projects/proj/datasets |
Create: /dataset |
Create: /dataset |
| Delete a table | DELETE: /bigquery/v2/projects/proj/datasets/ds/tables/tbl |
Delete: /ds/table/tbl |
Delete: /table/tbl |
| Create a job (load) | POST: /bigquery/v2/projects/proj/jobs |
Create: /job |
Create: /job |
| Cancel a job | POST: /bigquery/v2/projects/proj/jobs/job456/cancel |
Cancel: /job/job456 |
Cancel: /job/job456 |
| SDK metadata | GET: /discovery/v1/apis |
Other: ... |
(skipped) |
The classifier recognises these BigQuery operations from the SDK's HTTP traffic:
| Operation | HTTP Pattern |
|---|---|
Query |
POST /projects/{proj}/queries or GET /projects/{proj}/queries/{jobId}
|
Insert |
POST /datasets/{ds}/tables/{tbl}/insertAll |
Read |
GET on a specific resource (table, dataset, job, model, routine) |
List |
GET on a collection (/tables, /datasets, /jobs, /models, /routines, /tables/{tbl}/data) |
Create |
POST on a collection (/datasets, /tables, /jobs) |
Delete |
DELETE on a specific resource, or POST /jobs/{id}/delete
|
Update |
PUT or PATCH on a specific resource |
Cancel |
POST /jobs/{id}/cancel |
Other |
Unrecognised paths (e.g. discovery API, service account metadata) |
In Summarised mode, Other operations are silently skipped.
The classifier extracts and tracks these resource types:
| Resource Type | URL Path Segment |
|---|---|
query |
/queries |
dataset |
/datasets/{id} |
table |
/datasets/{ds}/tables/{id} |
tabledata |
/datasets/{ds}/tables/{id}/data |
job |
/jobs/{id} |
model |
/datasets/{ds}/models/{id} |
routine |
/datasets/{ds}/routines/{id} |
The upload endpoint variant (/upload/bigquery/v2/projects/...) is also recognised for data loading jobs.
Use the WithTestTracking() extension method on BigQueryClientBuilder:
var trackingOptions = new BigQueryTrackingMessageHandlerOptions
{
ServiceName = "BigQuery",
CallerName = "My API",
Verbosity = BigQueryTrackingVerbosity.Detailed,
CurrentTestInfoFetcher = CurrentTestInfo.Fetcher,
HttpContextAccessor = httpContextAccessor // optional: enables dual-resolution from HTTP headers
};
var client = new BigQueryClientBuilder
{
ProjectId = "my-project",
// Credential = ... (for real BigQuery)
}
.WithTestTracking(trackingOptions)
.Build();WithTestTracking sets the builder's HttpClientFactory to a TrackingBigQueryHttpClientFactory that wraps the default HTTP handler with the tracking handler. The pipeline becomes:
BigQueryClient → ConfigurableMessageHandler (auth, retry) → BigQueryTrackingMessageHandler → HttpClientHandler
For WebApplicationFactory-based integration tests where BigQuery clients are registered in DI:
builder.ConfigureTestServices(services =>
{
// Remove existing BigQueryClient registration
var descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(BigQueryClient));
if (descriptor != null) services.Remove(descriptor);
// Register tracked client
services.AddSingleton(sp =>
{
var trackingOptions = new BigQueryTrackingMessageHandlerOptions
{
ServiceName = "BigQuery",
CallerName = "My API",
Verbosity = BigQueryTrackingVerbosity.Detailed,
CurrentTestInfoFetcher = CurrentTestInfo.Fetcher
};
return new BigQueryClientBuilder
{
ProjectId = "test-project",
}
.WithTestTracking(trackingOptions)
.Build();
});
});If you need full control over the HTTP pipeline, use BigQueryTrackingMessageHandler directly:
var trackingOptions = new BigQueryTrackingMessageHandlerOptions
{
ServiceName = "BigQuery",
CallerName = "My API",
Verbosity = BigQueryTrackingVerbosity.Detailed,
CurrentTestInfoFetcher = () => ("My Test", testId)
};
var handler = new BigQueryTrackingMessageHandler(trackingOptions, new HttpClientHandler());
// Use handler with your own HTTP pipeline setup| Property | Type | Default | Description |
|---|---|---|---|
ServiceName |
string |
"BigQuery" |
The participant name shown in the diagram for the BigQuery service |
CallerName |
string |
"Caller" |
The participant name shown for the service making BigQuery calls |
Verbosity |
BigQueryTrackingVerbosity |
Detailed |
Controls how much detail appears in the diagram (Raw, Detailed, Summarised) |
CurrentTestInfoFetcher |
Func<(string Name, string Id)>? |
null |
Returns the current test's name and ID. Required — if null, requests are forwarded but not logged |
CurrentStepTypeFetcher |
Func<string?>? |
null |
Optional — returns the current BDD step type (Given/When/Then) |
HttpContextAccessor |
IHttpContextAccessor? |
null |
Optional — enables dual-resolution of test identity from HTTP headers |
ExcludedHeaders |
HashSet<string> |
See below | Headers to exclude from diagrams in Raw/Detailed mode |
SetupVerbosity |
BigQueryTrackingVerbosity? |
null |
Verbosity override for the Setup phase. See Phase-Aware Tracking |
ActionVerbosity |
BigQueryTrackingVerbosity? |
null |
Verbosity override for the Action phase. See Phase-Aware Tracking |
TrackDuringSetup |
bool |
true |
When false, tracking is suppressed during Setup. See Phase-Aware Tracking
|
TrackDuringAction |
bool |
true |
When false, tracking is suppressed during Action. See Phase-Aware Tracking
|
v2.23.0+ Dual-Resolution:
BigQueryTrackingMessageHandleraccepts an optionalIHttpContextAccessor? httpContextAccessorconstructor parameter for resolving test identity from HTTP request headers when running inside the SUT's request pipeline. See HTTP Tracking Setup#Dual-Resolution Test Identity (v2.23.0+) for details.
The following Google API headers are excluded by default (they add noise without diagnostic value):
AuthorizationUser-Agentx-goog-api-clientx-goog-request-paramsgoogle-cloud-resource-prefixCache-Control
Override ExcludedHeaders to customise:
new BigQueryTrackingMessageHandlerOptions
{
ExcludedHeaders = ["Authorization"] // Only exclude Authorization
}If you don't need HTTP-level interception — for example, you want to log BigQuery operations from a gRPC-based pipeline, a shared data layer, or a custom abstraction — use BigQueryTracker directly:
using Kronikol.Extensions.BigQuery;
var tracker = new BigQueryTracker(new BigQueryTrackingMessageHandlerOptions
{
ServiceName = "BigQuery",
CallerName = "OrdersApi",
Verbosity = BigQueryTrackingVerbosity.Detailed,
CurrentTestInfoFetcher = CurrentTestInfo.Fetcher
});
// Classify and log
var request = new HttpRequestMessage(HttpMethod.Post, "/bigquery/v2/projects/my-project/queries");
var op = BigQueryOperationClassifier.Classify(request);
var (reqId, traceId) = tracker.LogRequest(op, "SELECT * FROM dataset.table");
// ... execute BigQuery operation ...
tracker.LogResponse(op, reqId, traceId, "42 rows returned");BigQueryTracker implements ITrackingComponent, supports phase-aware tracking, and uses the same BigQueryTrackingMessageHandlerOptions as the handler.
Register a BigQueryTracker instance in your DI container:
services.AddBigQueryTestTracking(options =>
{
options.ServiceName = "BigQuery";
options.CallerName = "OrdersApi";
options.Verbosity = BigQueryTrackingVerbosity.Detailed;
options.CurrentTestInfoFetcher = CurrentTestInfo.Fetcher;
});Then inject BigQueryTracker where needed in your test setup or custom middleware.
Note: Options D and E are complementary to Options A–C. The HTTP handler (Options A–C) intercepts all BigQuery SDK traffic automatically. The direct tracker (Options D–E) is for scenarios where you intercept at a different layer or want to log custom BigQuery-related events.
If you already use TestTrackingMessageHandlerOptions for HTTP tracking and want to share the same test info fetchers with BigQuery tracking:
var httpOptions = new TestTrackingMessageHandlerOptions
{
CurrentTestInfoFetcher = () => (testName, testId),
CurrentStepTypeFetcher = () => currentStepType,
CallerName = "My API"
};
var bqOptions = new BigQueryTrackingMessageHandlerOptions
{
ServiceName = "BigQuery",
CallerName = httpOptions.CallerName,
CurrentTestInfoFetcher = httpOptions.CurrentTestInfoFetcher,
CurrentStepTypeFetcher = httpOptions.CurrentStepTypeFetcher,
Verbosity = BigQueryTrackingVerbosity.Detailed
};The extension works with all Kronikol framework adapters:
| Framework | Test Info Fetcher Pattern |
|---|---|
| xUnit3 |
TestContext.Current.Test → (TestDisplayName, UniqueID)
|
| xUnit2 |
ITestOutputHelper + ambient storage |
| NUnit4 | NUnit.Framework.TestContext.CurrentContext |
| MSTest | MSTest.TestContext |
| TUnit | TUnit.Core.TestContext.Current |
| LightBDD |
LightBddTestTrackingMessageHandlerOptions bridge |
| ReqNRoll |
ReqNRollTestTrackingMessageHandlerOptions bridge |
| BDDfy |
BDDfyTestTrackingMessageHandlerOptions bridge |
BigQueryTrackingMessageHandler implements ITrackingComponent and auto-registers with TrackingComponentRegistry on construction. At report generation time, unused components are automatically detected and surfaced as console warnings and in the diagnostic report (when DiagnosticMode=true). This never throws or fails tests.
See Diagnostics and Debugging for full details on the TrackingComponentRegistry API.
Getting Started
Common Tasks
Integration Guides
- Integration xUnit3
- Integration xUnit2
- Integration NUnit
- Integration MSTest
- Integration TUnit
- Integration BDDfy xUnit3
- Integration LightBDD xUnit2
- Integration LightBDD xUnit3
- Integration LightBDD TUnit
- Integration ReqNRoll xUnit2
- Integration ReqNRoll xUnit3
- Integration ReqNRoll TUnit
Extensions
- Integration AtlasDataApi Extension
- Integration BigQuery Extension
- Integration Bigtable Extension
- Integration BlobStorage Extension
- Integration CloudStorage Extension
- Integration CosmosDB Extension
- Integration Dapper Extension
- Integration DynamoDB Extension
- Integration EF Core Relational Extension
- Integration Elasticsearch Extension
- Integration EventBridge Extension
- Integration EventHubs Extension
- Integration Grpc Extension
- Integration Kafka Extension
- Integration MassTransit Extension
- Integration MongoDB Extension
- Integration MySqlConnector Extension
- Integration Npgsql Extension
- Integration Oracle Extension
- Integration PubSub Extension
- Integration Redis Extension
- Integration S3 Extension
- Integration ServiceBus Extension
- Integration SNS Extension
- Integration Spanner Extension
- Integration SqlClient Extension
- Integration Sqlite Extension
- Integration SQS Extension
- Integration StorageQueues Extension
- Integration OpenTelemetry Extension
- Integration DispatchProxy Extension
- Integration MediatR Extension
- Integration PlantUML IKVM
Configuration
- Tracking Dependencies
- Tracking Custom Dependencies
- HTTP Tracking Setup
- Report Configuration
- Diagram Customisation
- Phase-Aware Tracking
- Content Formatting
- PlantUML Server Configuration
Features
- Generated Reports
- Search Syntax
- Component Diagrams
- PlantUML Browser Rendering
- Inline SVG Rendering
- Internal Flow Tracking
- Tags and Attributes
- Excluding Requests
- Excluded Headers
- Multi-Host Test Architectures
- Event-Driven Architecture Testing
- Service Bus Tracking Patterns
- Background Thread Correlation
- Parallel-Safe Background Correlation
- Event & Message Tracking
- Assertion Tracking
- Step Tracking
- Tabular Attributes
- Large Response and Diagram Handling
- Diagnostics and Debugging
- CI Summary Integration
- CI Artifact Upload
Reference