Сreate pyRevit Telemetry API

[dosymep]

• create telemetry api
• create unit and integrations test

---

Telemetry Server for pyRevit

A high-performance .NET 10 Web API service designed to collect and store telemetry data
from pyRevit. It supports multiple database providers and can be easily
deployed using Docker.

Project Structure

The project follows a Clean Architecture approach:

• src/Telemetry.Api: Main application source code.
  • Application: Business logic, DTOs, mappings, and service interfaces.
  • Domain: Core domain models and entities.
  • Infrastructure: Database persistence, Entity Framework Core contexts, and external service implementations.
  • Web: ASP.NET Core controllers, middleware, and API configuration.
• src/Telemetry.Migrations.*: Separate projects for Entity Framework Core migrations (PostgreSQL, SQL Server, SQLite, Oracle).
• tests: Comprehensive test suite using TUnit.
  • Telemetry.Api.UnitTests: Isolated tests for business logic and mappings.
  • Telemetry.Api.IntegrationTests: End-to-end tests for API endpoints and database interactions.
• telemetry-db: Local directory for database persistence (used by Docker).

Prerequisites

• .NET 10 SDK
• Docker and Docker Compose

Compilation

To build the project locally, run the following command from the root directory:

dotnet build telemetry-server.slnx

To run the application:

dotnet run --project src/Telemetry.Api/Telemetry.Api.csproj

Docker Deployment

The project provides several Docker Compose configurations to support different database backends.

General Usage

To start the API using the pre-built image, you can pull the latest version:

docker pull pyrevit-telemetry:latest

To start the API with a specific database, use the following command structure:

docker compose -f docker-compose.yml -f docker-compose.<db-type>.yml up -d

Supported Database Variants

Database	Command
PostgreSQL	docker compose -f docker-compose.yml -f docker-compose.postgres.yml up -d
SQL Server	docker compose -f docker-compose.yml -f docker-compose.mssql.yml up -d
SQLite	docker compose -f docker-compose.yml -f docker-compose.sqlite.yml up -d
Oracle	docker compose -f docker-compose.yml -f docker-compose.oracle.yml up -d
MongoDB	docker compose -f docker-compose.yml -f docker-compose.mongodb.yml up -d

❗IMPORTANT

• MySQL: Support is currently unavailable because the necessary Entity Framework Core libraries have not yet been ported to .NET 10.
• SQL Server & Oracle: While these providers are implemented, they have not been fully tested in a .NET 10 environment. Full verification and ongoing support for these databases are left to the community.

Environment Variables

Each database variant requires specific environment variables (usually defined in a .env file or passed directly). Key
variables include:

• DbProvider: Specifies the database provider (e.g., postgres, mssql, sqlite, oracle, mongodb).
• ConnectionStrings__DefaultConnection: The connection string for the selected database.
• ASPNETCORE_HTTP_PORTS: The port the API listens to on (default: 8080).

API Endpoints

The API is versioned (currently v2).

• POST /api/v2/scripts: Submit pyRevit script execution telemetry.
• POST /api/v2/events: Submit pyRevit event telemetry.
• GET /api/v2/status: Check service and database connection status.
• GET /metrics: Prometheus metrics for monitoring.
• GET /swagger: OpenAPI/Swagger documentation (in Development mode).

API Documentation

When running in Development mode, the application provides interactive API documentation using Swagger (OpenAPI).

To access the Swagger UI:

1. Start the application in Development mode (ASPNETCORE_ENVIRONMENT=Development).
2. Navigate to http://localhost:8080/swagger (or your configured port).

From the Swagger UI, you can explore all available endpoints, view request/response schemas, and test the API directly from your browser.

Monitoring

The application exposes Prometheus metrics at the /metrics endpoint. This can be used to monitor the service's health, request rates, and performance using Prometheus and Grafana.

To view the metrics:

1. Start the application.
2. Navigate to http://localhost:8080/metrics (or your configured port).

Testing

The project uses TUnit, a modern testing framework for .NET, taking advantage of
the latest features in .NET 10.

Unit Tests

Unit tests focus on isolated business logic and data mappings. They do not require any external dependencies.

To run unit tests:

dotnet run --project tests/Telemetry.Api.UnitTests/Telemetry.Api.UnitTests.csproj

Integration Tests

Integration tests verify the end-to-end functionality of the API, including database persistence. These tests
use Testcontainers to spin up actual database instances (except for SQLite) during the
test execution.

Prerequisites: Docker must be running on your machine to execute integration tests for PostgreSQL, SQL Server,
Oracle, and MongoDB.

To run integration tests:

dotnet run --project tests/Telemetry.Api.IntegrationTests/Telemetry.Api.IntegrationTests.csproj

Manual Deployment (Without Docker)

To run the server manually, follow these steps to build, configure, and publish the application.

Build and Publish

1. Publish the project:
   Run the following command to create a self-contained or framework-dependent deployment:

   dotnet publish src/Telemetry.Api/Telemetry.Api.csproj -c Release -o ./publish

2. Navigate to the publish directory:

   cd ./publish

Configuration (appsettings.json)

Before running the server, you must configure the database provider and connection string in appsettings.json.

1. Open appsettings.json and add the following configuration:

   {
     "DbProvider": "postgres",
     "ConnectionStrings": {
       "DefaultConnection": "Host=localhost;Database=telemetry;Username=postgres;Password=password"
     },
     "MongoDbDatabaseName": "telemetry",
     "Logging": {
       "LogLevel": {
         "Default": "Information",
         "Microsoft.AspNetCore": "Warning"
       }
     },
     "AllowedHosts": "*"
   }

2. Configuration Options:

   • DbProvider: Set to one of postgres, mssql, sqlite, oracle, mongodb.
   • ConnectionStrings:DefaultConnection: Provide the appropriate connection string for your database.
   • MongoDbDatabaseName: (Optional) Required only if using mongodb.

Running the Server

Start the application using the dotnet CLI:

dotnet Telemetry.Api.dll

The server will start and listen on the default ports (usually http://localhost:5000 or as configured via environment variables).

Database Migrations

The project uses Entity Framework Core (EF Core) to manage relational database schemas (PostgreSQL, SQL Server, SQLite, Oracle).

Prerequisites

You need the dotnet-ef tool installed globally:

dotnet tool install --global dotnet-ef

Creating a New Migration

If you modify the data models in the Domain or Infrastructure layers, you need to generate a new migration for each supported database provider. Each database has its own project for migrations.

To generate migrations correctly, set the required environment variables for the target database and run the dotnet ef migrations add command from the root directory.

Oracle

DbProvider=oracle ConnectionStrings__DefaultConnection="Data Source=localhost:1521/xe;User Id=test;Password=test" \
dotnet ef migrations add <MigrationName> \
    --project src/Telemetry.Migrations.Oracle \
    --startup-project src/Telemetry.Api \
    --context ApplicationDbContext

PostgreSQL

DbProvider=postgres ConnectionStrings__DefaultConnection="Host=localhost;Database=test;Username=test;Password=test" \
dotnet ef migrations add <MigrationName> \
    --project src/Telemetry.Migrations.Postgres \
    --startup-project src/Telemetry.Api \
    --context ApplicationDbContext

SQLite

DbProvider=sqlite ConnectionStrings__DefaultConnection="Data Source=/app/data/telemetry.db" \
dotnet ef migrations add <MigrationName> \
    --project src/Telemetry.Migrations.Sqlite \
    --startup-project src/Telemetry.Api \
    --context ApplicationDbContext

SQL Server

DbProvider=mssql ConnectionStrings__DefaultConnection="Server=localhost;Database=test;User Id=sa;Password=test" \
dotnet ef migrations add <MigrationName> \
    --project src/Telemetry.Migrations.SqlServer \
    --startup-project src/Telemetry.Api \
    --context ApplicationDbContext

Reviewing the generated files

New migration files will be created in their respective projects under the Migrations directory (e.g., src/Telemetry.Migrations.Postgres/Migrations).

Applying Migrations

• Automatic: The server automatically applies pending migrations on startup for all relational database providers.

• Manual: You can manually update the database using the following command (example for SQLite):

  DbProvider=sqlite ConnectionStrings__DefaultConnection="Data Source=/app/data/telemetry.db" \
  dotnet ef database update \
      --project src/Telemetry.Migrations.Sqlite \
      --startup-project src/Telemetry.Api \
      --context ApplicationDbContext

❗NOTE
MongoDB does not use EF Core migrations. Changes to the MongoDB data structure are handled by the application at runtime (schema-less or through driver-level configuration).

[Copilot]

Pull request overview

This PR introduces a new .NET 10 “Telemetry Server for pyRevit” Web API, including database persistence (multiple providers), Docker deployment assets, and an initial unit/integration test suite.

Changes:

• Added Telemetry.Api Web API with v2 endpoints (/scripts, /events, /status), Swagger, Prometheus metrics, logging, and a DB context abstraction.
• Added EF Core migrations projects for PostgreSQL, SQL Server, SQLite, and Oracle.
• Added TUnit-based unit and integration tests, plus Docker Compose configurations and CI workflows.

Reviewed changes

Copilot reviewed 73 out of 79 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
tests/Telemetry.Api.UnitTests/Web/Controllers/TelemetryControllerTests.cs	Unit tests for controller endpoints
tests/Telemetry.Api.UnitTests/Telemetry.Api.UnitTests.csproj	Unit test project definition
tests/Telemetry.Api.UnitTests/Application/Mappings/TelemetryMappingExtensionsTests.cs	Unit tests for DTO↔model mappings
tests/Telemetry.Api.IntegrationTests/Telemetry.Api.IntegrationTests.csproj	Integration test project definition
tests/Telemetry.Api.IntegrationTests/SqliteIntegrationTests.cs	SQLite end-to-end API tests
tests/Telemetry.Api.IntegrationTests/PostgresIntegrationTests.cs	PostgreSQL end-to-end API tests
tests/Telemetry.Api.IntegrationTests/OracleIntegrationTests.cs	Oracle end-to-end API tests
tests/Telemetry.Api.IntegrationTests/MsSqlIntegrationTests.cs	SQL Server end-to-end API tests
tests/Telemetry.Api.IntegrationTests/MongoDbIntegrationTests.cs	MongoDB end-to-end API tests
tests/Telemetry.Api.IntegrationTests/BaseIntegrationTest.cs	Shared WebApplicationFactory integration test base
telemetry-server.slnx	Solution definition including src/tests projects
src/Telemetry.Migrations.SqlServer/Telemetry.Migrations.SqlServer.csproj	SQL Server migrations project
src/Telemetry.Migrations.SqlServer/Migrations/ApplicationDbContextModelSnapshot.cs	SQL Server EF model snapshot
src/Telemetry.Migrations.SqlServer/Migrations/20260330090733_InitSqlServer.Designer.cs	SQL Server migration designer
src/Telemetry.Migrations.SqlServer/Migrations/20260330090733_InitSqlServer.cs	SQL Server initial migration
src/Telemetry.Migrations.Sqlite/Telemetry.Migrations.Sqlite.csproj	SQLite migrations project
src/Telemetry.Migrations.Sqlite/Migrations/ApplicationDbContextModelSnapshot.cs	SQLite EF model snapshot
src/Telemetry.Migrations.Sqlite/Migrations/20260330090723_InitSqlite.Designer.cs	SQLite migration designer
src/Telemetry.Migrations.Sqlite/Migrations/20260330090723_InitSqlite.cs	SQLite initial migration
src/Telemetry.Migrations.Postgres/Telemetry.Migrations.Postgres.csproj	Postgres migrations project
src/Telemetry.Migrations.Postgres/Migrations/ApplicationDbContextModelSnapshot.cs	Postgres EF model snapshot
src/Telemetry.Migrations.Postgres/Migrations/20260330090711_InitPostgres.Designer.cs	Postgres migration designer
src/Telemetry.Migrations.Postgres/Migrations/20260330090711_InitPostgres.cs	Postgres initial migration
src/Telemetry.Migrations.Oracle/Telemetry.Migrations.Oracle.csproj	Oracle migrations project
src/Telemetry.Migrations.Oracle/Migrations/ApplicationDbContextModelSnapshot.cs	Oracle EF model snapshot
src/Telemetry.Migrations.Oracle/Migrations/20260330090657_InitOracle.Designer.cs	Oracle migration designer
src/Telemetry.Migrations.Oracle/Migrations/20260330090657_InitOracle.cs	Oracle initial migration
src/Telemetry.Api/Web/Middleware/ExceptionHandlingMiddleware.cs	Custom exception-handling middleware
src/Telemetry.Api/Web/Controllers/TelemetryController.cs	v2 telemetry API controller
src/Telemetry.Api/Telemetry.Api.http	Example HTTP requests for API testing
src/Telemetry.Api/Telemetry.Api.csproj	Main Web API project definition
src/Telemetry.Api/Properties/launchSettings.json	Local launch profiles
src/Telemetry.Api/Program.cs	App bootstrap: DI, DB providers, Swagger, metrics, logging, migrations
src/Telemetry.Api/JsonConverters/DynamicDataJsonConverter.cs	JSON converter for “dynamic JSON to string”
src/Telemetry.Api/JsonConverters/DynamicDataBsonSerializer.cs	MongoDB BSON serializer for “dynamic JSON to string”
src/Telemetry.Api/Infrastructure/Persistence/ApplicationDbContext.cs	EF Core DbContext + schema mapping + DB status helpers
src/Telemetry.Api/Domain/Models/TraceInfo.cs	Domain model: trace info
src/Telemetry.Api/Domain/Models/ScriptRecord.cs	Domain model: script telemetry record
src/Telemetry.Api/Domain/Models/MetaRecord.cs	Domain model: metadata record
src/Telemetry.Api/Domain/Models/EventRecord.cs	Domain model: event telemetry record
src/Telemetry.Api/Domain/Models/EngineInfo.cs	Domain model: engine info
src/Telemetry.Api/Dockerfile	Multi-stage Docker build/publish
src/Telemetry.Api/appsettings.json	Base app config
src/Telemetry.Api/appsettings.Development.json	Development config
src/Telemetry.Api/Application/Services/ServiceInfo.cs	Service identity provider
src/Telemetry.Api/Application/Mappings/TelemetryMappingExtensions.cs	DTO↔domain mapping extensions
src/Telemetry.Api/Application/Interfaces/IServiceInfo.cs	Service info interface
src/Telemetry.Api/Application/Interfaces/IApplicationDbContext.cs	DbContext abstraction interface
src/Telemetry.Api/Application/DTOs/TraceInfoDto.cs	API DTO: trace info
src/Telemetry.Api/Application/DTOs/StatusRecordDto.cs	API DTO: status payload
src/Telemetry.Api/Application/DTOs/ScriptRecordDto.cs	API DTO: script record
src/Telemetry.Api/Application/DTOs/MetaDto.cs	API DTO: metadata
src/Telemetry.Api/Application/DTOs/EventRecordDto.cs	API DTO: event record
src/Telemetry.Api/Application/DTOs/EngineInfoDto.cs	API DTO: engine info
src/Telemetry.Api/.dockerignore	Docker ignore for API subdir
SECURITY.md	Security policy / reporting guidance
README.md	Expanded project documentation
global.json	SDK pinning + test runner config
docker-compose.yml	Base compose for API container
docker-compose.sqlite.yml	SQLite compose overlay
docker-compose.postgres.yml	Postgres compose overlay
docker-compose.oracle.yml	Oracle compose overlay
docker-compose.mssql.yml	SQL Server compose overlay
docker-compose.mongodb.yml	MongoDB compose overlay
Directory.Packages.props	Central package version management
Directory.Build.targets	Build targets placeholder
Directory.Build.props	Assembly/package version settings
CREDITS.md	Credits link
CONTRIBUTING.md	Contribution guidance link
CODE_OF_CONDUCT.md	Contributor covenant
.gitignore	Updated ignore rules for .NET tooling/artifacts
.github/workflows/tests.yml	CI workflow to build/test
.github/workflows/release.yml	Release + Docker publish workflow
.github/workflows/build.yml	Build + prerelease Docker publish workflow
.gitattributes	Line ending / diff settings
.env.example	Example environment variables
.editorconfig	C# formatting and conventions
.dockerignore	Root docker ignore rules

Files not reviewed (4)

• src/Telemetry.Migrations.Oracle/Migrations/20260330090657_InitOracle.Designer.cs: Language not supported
• src/Telemetry.Migrations.Postgres/Migrations/20260330090711_InitPostgres.Designer.cs: Language not supported
• src/Telemetry.Migrations.SqlServer/Migrations/20260330090733_InitSqlServer.Designer.cs: Language not supported
• src/Telemetry.Migrations.Sqlite/Migrations/20260330090723_InitSqlite.Designer.cs: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.