API Reference
All endpoints are served from a single Bloop instance. Ingest routes use HMAC authentication (project key + request body signature), while query and management routes accept either a Bearer token or a browser session cookie.
Error Tracking
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /health |
None | Health check |
| POST | /v1/ingest |
HMAC | Submit single error event |
| POST | /v1/ingest/batch |
HMAC | Submit batch (max 50) |
| GET | /v1/errors |
Bearer / Session | List errors (paginated, filterable) |
| GET | /v1/errors/:fingerprint |
Bearer / Session | Error detail + samples |
| GET | /v1/errors/:fingerprint/occurrences |
Bearer / Session | Raw events |
| POST | /v1/errors/:fingerprint/resolve |
Bearer / Session | Mark resolved |
| POST | /v1/errors/:fingerprint/ignore |
Bearer / Session | Mark ignored |
| POST | /v1/errors/:fingerprint/mute |
Bearer / Session | Mute (suppress alerts) |
| POST | /v1/errors/:fingerprint/unresolve |
Bearer / Session | Unresolve |
| GET | /v1/errors/:fingerprint/trend |
Bearer / Session | Per-error hourly counts |
| GET | /v1/errors/:fingerprint/history |
Bearer / Session | Status audit trail |
| GET | /v1/releases/:release/errors |
Bearer / Session | Errors for a release |
| GET | /v1/trends |
Bearer / Session | Global hourly counts |
| GET | /v1/stats |
Bearer / Session | Overview stats |
Alerting
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /v1/alerts |
Bearer / Session | List alert rules |
| POST | /v1/alerts |
Bearer / Session | Create alert rule |
| PUT | /v1/alerts/:id |
Bearer / Session | Update alert rule |
| DELETE | /v1/alerts/:id |
Bearer / Session | Delete alert rule |
| POST | /v1/alerts/:id/channels |
Bearer / Session | Add notification channel |
| POST | /v1/alerts/:id/test |
Bearer / Session | Send test notification |
Projects & Auth
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /v1/projects |
Session | List projects |
| POST | /v1/projects |
Session | Create project |
| GET | /v1/projects/:slug |
Session | Project details |
| POST | /v1/projects/:slug/sourcemaps |
Bearer / Session | Upload source map |
| POST | /v1/tokens |
Session | Create API token |
| GET | /v1/tokens |
Session | List tokens |
| DELETE | /v1/tokens/:id |
Session | Revoke token |
| PUT | /v1/admin/users/:id/role |
Admin Session | Promote/demote user |
| GET | /v1/admin/retention |
Admin Session | Get retention config |
| PUT | /v1/admin/retention |
Admin Session | Update retention config |
| POST | /v1/admin/retention/purge |
Admin Session | Trigger immediate purge |
Analytics (feature: analytics)
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /v1/analytics/spikes |
Bearer / Session | Spike detection (z-score) |
| GET | /v1/analytics/movers |
Bearer / Session | Top movers |
| GET | /v1/analytics/correlations |
Bearer / Session | Correlated error pairs |
| GET | /v1/analytics/releases |
Bearer / Session | Release impact scoring |
| GET | /v1/analytics/environments |
Bearer / Session | Environment breakdown |
LLM Tracing (feature: llm-tracing)
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | /v1/traces |
HMAC | Submit single LLM trace |
| POST | /v1/traces/batch |
HMAC | Submit batch of traces (max 50) |
| PUT | /v1/traces/:id |
HMAC | Update a running trace |
| GET | /v1/llm/overview |
Bearer / Session | Summary: traces, tokens, cost, errors |
| GET | /v1/llm/usage |
Bearer / Session | Hourly token/cost by model |
| GET | /v1/llm/latency |
Bearer / Session | p50/p90/p99 latency by model |
| GET | /v1/llm/models |
Bearer / Session | Per-model breakdown |
| GET | /v1/llm/traces |
Bearer / Session | Paginated trace list |
| GET | /v1/llm/traces/:id |
Bearer / Session | Full trace + span hierarchy |
| GET | /v1/llm/settings |
Bearer / Session | Content storage policy |
| PUT | /v1/llm/settings |
Bearer / Session | Update content storage policy |
| GET | /v1/llm/search |
Bearer / Session | Full-text search over traces |
| POST | /v1/llm/traces/:id/scores |
Bearer / Session | Submit a score for a trace |
| GET | /v1/llm/traces/:id/scores |
Bearer / Session | Get scores for a trace |
| GET | /v1/llm/scores/summary |
Bearer / Session | Score summary across traces |
| GET | /v1/llm/prompts |
Bearer / Session | List prompt versions with metrics |
Query Parameters for GET /v1/errors
| Param | Type | Description |
|---|---|---|
release | string | Filter by release |
environment | string | Filter by environment |
source | string | Filter by source (ios/android/api) |
route | string | Filter by route |
status | string | Filter by status (unresolved/resolved/ignored) |
since | i64 | Epoch ms lower bound on last_seen |
until | i64 | Epoch ms upper bound on last_seen |
sort | string | Sort: last_seen (default), total_count, first_seen |
limit | i64 | Page size (default 50, max 200) |
offset | i64 | Pagination offset |
Event Contract
The IngestEvent schema defines the shape of every error event submitted to Bloop. Required fields are marked; all others are optional.
{
"timestamp": 1700000000,
"source": "ios",
"environment": "production",
"release": "1.2.0",
"app_version": "1.2.0",
"build_number": "42",
"route_or_procedure": "/api/checkout",
"screen": "PaymentScreen",
"error_type": "RuntimeError",
"message": "Connection refused at port 5432",
"stack": "RuntimeError: Connection refused\n at connect (db.swift:42)",
"http_status": 500,
"request_id": "req-abc-123",
"user_id_hash": "sha256:a1b2c3...",
"device_id_hash": "sha256:d4e5f6...",
"fingerprint": "custom-fp-override",
"metadata": {
"os_version": "17.2",
"device_model": "iPhone 15"
}
}
Hard caps on payload sizes:
- Total payload: 32 KB
stack: 8 KBmetadata: 4 KBmessage: 2 KB
Events exceeding these limits are rejected with 400 Bad Request.
API Tokens
API tokens provide scoped, long-lived bearer credentials for CI pipelines, AI agents, and scripts. They are always scoped to a single project and grant fine-grained access based on the scopes assigned at creation time.
Creating Tokens
Create tokens from the dashboard (Settings → API Tokens) or via the API. The plaintext token is SHA-256 hashed before storage and is only shown once at creation time — copy it immediately.
# Create a token (requires session auth)
curl -X POST http://localhost:5332/v1/tokens \
-H "Cookie: session=..." \
-H "Content-Type: application/json" \
-d '{"name":"CI deploy","project_id":"my-project","scopes":["errors:read","sourcemaps:upload"]}'
# Use the token
curl http://localhost:5332/v1/errors \
-H "Authorization: Bearer bloop_abc123..."
Available Scopes
| Scope | Grants access to |
|---|---|
errors:read | List/get errors, occurrences, trends, stats |
errors:write | Resolve, ignore, mute, unresolve errors |
sourcemaps:upload | Upload source maps |
alerts:read | List alert rules and channels |
alerts:write | Create, update, delete rules and channels |
analytics:read | Query analytics endpoints (spikes, movers, correlations) |
llm:read | Query LLM tracing endpoints (overview, usage, traces, search, scores, prompts) |
llm:write | Submit scores for LLM traces |
Bearer tokens are project-scoped — you don't need to pass a project_id query parameter. The project is determined automatically from the token. If a token lacks the required scope, the request returns 403 Forbidden.
Alerting
Bloop supports configurable alert rules that evaluate incoming events and notify your team via webhooks, Slack, or email when conditions are met.
Rule Types
| Type | Fires when |
|---|---|
| New Issue | A never-seen fingerprint appears for the first time |
| Threshold | An error exceeds N occurrences within a configurable time window |
| Spike | Error rate exceeds a z-score threshold compared to the rolling baseline |
| LLM Cost Spike | Hourly LLM cost exceeds a dollar threshold |
| LLM Error Rate | LLM error rate exceeds a percentage threshold |
| LLM Latency | LLM p90 latency exceeds a millisecond threshold (optionally per model) |
LLM Alert Rule Examples
The LLM-specific alert rules use the same /v1/alerts CRUD endpoints with new rule_type values.
# Alert when hourly LLM cost exceeds $10
curl -X POST http://localhost:5332/v1/alerts \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "LLM cost spike",
"rule_type": "llm_cost_spike",
"threshold_dollars": 10.0,
"window_hours": 1,
"project_id": "my-project"
}'
# Alert when LLM error rate exceeds 5%
curl -X POST http://localhost:5332/v1/alerts \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "LLM error rate",
"rule_type": "llm_error_rate",
"threshold_percent": 5.0,
"window_hours": 1,
"project_id": "my-project"
}'
# Alert when gpt-4o p90 latency exceeds 5 seconds
curl -X POST http://localhost:5332/v1/alerts \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "GPT-4o slow responses",
"rule_type": "llm_latency",
"threshold_ms": 5000,
"window_hours": 1,
"model": "gpt-4o",
"project_id": "my-project"
}'
LLM Alert Rule Configuration
| Rule Type | Required Fields | Optional Fields |
|---|---|---|
llm_cost_spike |
threshold_dollars (float), window_hours (int) |
project_id |
llm_error_rate |
threshold_percent (float), window_hours (int) |
project_id |
llm_latency |
threshold_ms (int), window_hours (int) |
model (string), project_id |
Notification Channels
Each alert rule can deliver notifications to multiple channels:
- Slack — Sends a formatted message to a Slack channel via incoming webhook
- Generic Webhook — POSTs a JSON payload to any HTTP endpoint (PagerDuty, Opsgenie, custom)
- Email — Sends an alert email via SMTP (requires SMTP configuration)
Creating a Rule
curl -X POST http://localhost:5332/v1/alerts \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "High error rate",
"rule_type": "threshold",
"threshold": 100,
"window_secs": 3600,
"project_id": "my-project"
}'
Cooldown
After an alert fires, it enters a cooldown period to prevent alert fatigue during error storms. The default cooldown is 900 seconds (15 minutes) between re-fires for the same rule and fingerprint. This is configurable via alerting.cooldown_secs in config.toml.
Email Alerts (SMTP)
Bloop can send alert notifications via email. This requires SMTP configuration in config.toml or via environment variables.
[smtp]
enabled = true
host = "smtp.example.com"
port = 587
username = "[email protected]"
password = "your-password"
from = "[email protected]"
starttls = true
Once SMTP is configured, add an email channel to any alert rule via the dashboard (Settings → Alerts → Add Channel → Email) or via the API:
curl -X POST http://localhost:5332/v1/alerts/1/channels \
-H "Content-Type: application/json" \
-H "Cookie: session=YOUR_SESSION_TOKEN" \
-d '{"config": {"type": "email", "to": "[email protected]"}}'
Email alerts require smtp.enabled = true. If SMTP is not configured, creating an email channel will succeed but emails will not be sent. Use the Test button on the alert rule to verify email delivery.
Projects
Bloop supports multiple projects, each with its own isolated namespace. Every project gets its own API key (used for HMAC-signed ingest), alerts, source maps, and retention settings. Projects are identified by a URL-safe slug and a human-readable name.
# Create project
curl -X POST http://localhost:5332/v1/projects \
-H "Cookie: session=..." \
-H "Content-Type: application/json" \
-d '{"name":"My Mobile App","slug":"my-mobile-app"}'
# List projects
curl http://localhost:5332/v1/projects -H "Cookie: session=..."
The response includes the project's auto-generated HMAC key. Pass this key as X-Project-Key in your SDK configuration for ingestion authentication.
A default project is created automatically on first startup using the legacy hmac_secret from config. SDKs that don't send X-Project-Key are routed to this default project for backward compatibility.
User Management
Bloop has two user roles: Admin and User. The first user to register is automatically promoted to Admin. Admins can manage projects, alert rules, API tokens, source maps, users, invites, and retention settings.
Promote / Demote Users
# Promote user to admin
curl -X PUT http://localhost:5332/v1/admin/users/2/role \
-H "Cookie: session=..." \
-H "Content-Type: application/json" \
-d '{"role":"admin"}'
Self-demotion is not allowed. You cannot remove your own admin role. This ensures there is always at least one admin who can manage the system.
Data Retention
Bloop automatically prunes old data based on configurable retention settings. By default, raw events are pruned after 7 days while aggregates are kept indefinitely. Retention is configurable per-project and managed via admin-only endpoints.
# Get current retention settings
curl http://localhost:5332/v1/admin/retention -H "Cookie: session=..."
# Update retention
curl -X PUT http://localhost:5332/v1/admin/retention \
-H "Cookie: session=..." \
-H "Content-Type: application/json" \
-d '{"raw_events_days": 14}'
# Trigger immediate purge
curl -X POST http://localhost:5332/v1/admin/retention/purge -H "Cookie: session=..."
Automatic pruning runs every hour by default. The interval is configurable via retention.prune_interval_secs in config.toml.
Source Maps
Upload source maps for JavaScript and TypeScript stack trace deobfuscation. Source maps are uploaded per project per release. Once uploaded, stack traces for that release are automatically deobfuscated in the dashboard and API responses.
# Upload a source map
curl -X POST http://localhost:5332/v1/projects/my-app/sourcemaps \
-H "Authorization: Bearer bloop_..." \
-F "release=1.2.3" \
-F "file=@dist/app.js.map"
The dashboard provides a toggle to switch between the original (deobfuscated) and minified (raw) stack trace views when source maps are available for the matching project and release.
Source maps are stored compressed (gzip) and cached in memory for fast lookups. The release field in the upload must exactly match the release field sent with error events for matching to work.
Analytics (DuckDB)
Bloop includes an optional analytics engine powered by DuckDB for advanced error analysis. DuckDB reads the existing SQLite database directly via its sqlite_scanner extension — there is no data migration, no schema changes, and no write contention with the main SQLite writer.
Feature-gated: Analytics requires building with cargo build --features analytics. It is not included in the default build.
Analysis Types
Five analysis endpoints are available:
- Spike Detection — z-score based anomaly detection on hourly error rates. Returns fingerprints where the hourly count deviates significantly from the rolling mean.
- Top Movers — Errors with the biggest absolute rate change between the current and previous time windows. Surfaces rapidly increasing or decreasing errors.
- Correlations — Pairs of errors that tend to co-occur in the same hourly windows (Pearson correlation ≥0.7, requiring ≥6 data points). Useful for root cause discovery.
- Release Impact — Scores each release by error introduction and reduction. Combines new fingerprints and error count deltas into a composite impact score.
- Environment Breakdown — Error distribution and percentiles (P50/P90/P99 hourly rates) broken down by environment.
# Get spike detection results
curl "http://localhost:5332/v1/analytics/spikes?hours=24&threshold=2.5" \
-H "Authorization: Bearer bloop_..."
Analytics data is read-only from DuckDB. There is no write contention with the main SQLite writer. Results are cached in memory (configurable via analytics.cache_ttl_secs, default 60 seconds).
LLM Tracing
Monitor AI and LLM API calls across your application. Track token usage, costs, latency, and errors per model and provider. Hierarchical traces group related LLM calls together with nested spans for generation, tool use, retrieval, and custom operations.
Feature-gated: LLM tracing requires building with cargo build --features llm-tracing AND setting llm_tracing.enabled = true at runtime in config.toml.
Ingest API
Submit traces with nested spans. All ingest endpoints use HMAC authentication (same as error ingest).
| Method | Path | Description |
|---|---|---|
| POST | /v1/traces |
Submit single trace with nested spans |
| POST | /v1/traces/batch |
Submit up to 50 traces per batch |
| PUT | /v1/traces/:id |
Update a running trace (status, output, end time) |
Trace JSON Contract
{
"id": "trace-001",
"session_id": "session-abc",
"user_id": "user-123",
"name": "chat-completion",
"prompt_name": "summarize-v2",
"prompt_version": "1.3.0",
"status": "completed",
"input": "User prompt text",
"output": "AI response text",
"metadata": {"key": "value"},
"started_at": 1700000000000,
"ended_at": 1700000001200,
"spans": [
{
"id": "span-001",
"parent_span_id": null,
"span_type": "generation",
"name": "gpt-4o call",
"model": "gpt-4o",
"provider": "openai",
"input_tokens": 100,
"output_tokens": 50,
"cost": 0.0025,
"latency_ms": 1200,
"time_to_first_token_ms": 300,
"status": "ok",
"error_message": null,
"input": "prompt text",
"output": "completion text",
"metadata": {}
}
]
}
Span types: generation, tool, retrieval, custom.
Validation limits:
- Trace ID: max 128 characters
- Max 100 spans per trace
- Max 50 traces per batch
Cost format: Specify costs in dollars (float). They are stored internally as microdollars (integer) to avoid floating-point precision issues. $0.0025 becomes 2500 micros.
Query Endpoints
| Endpoint | Description |
|---|---|
GET /v1/llm/overview | Summary: total traces, spans, tokens, cost, error rate |
GET /v1/llm/usage | Hourly token/cost breakdown by model |
GET /v1/llm/latency | p50/p90/p99 latency by model, avg TTFT |
GET /v1/llm/models | Per-model: calls, tokens, cost, error rate |
GET /v1/llm/traces | Paginated trace list with filters and sorting |
GET /v1/llm/traces/:id | Full trace + span hierarchy |
GET /v1/llm/search | Full-text search over trace names, sessions, users |
POST /v1/llm/traces/:id/scores | Submit a score for a trace |
GET /v1/llm/traces/:id/scores | Get all scores for a trace |
GET /v1/llm/scores/summary | Aggregated score summary across traces |
GET /v1/llm/prompts | List prompt versions with performance metrics |
Shared Query Parameters
| Param | Type | Default | Description |
|---|---|---|---|
project_id | string | from token | Filter by project |
hours | i64 | 24 | Time window (max 720) |
limit | i64 | 50 | Page size (max 200) |
model | string | — | Filter by model name |
provider | string | — | Filter by provider |
status | string | — | Filter by trace status (completed, error, running) |
session_id | string | — | Filter by session |
sort | string | newest | Sort order: newest, oldest, cost_desc, latency_desc, tokens_desc |
offset | i64 | 0 | Pagination offset |
# Get LLM usage overview
curl "http://localhost:5332/v1/llm/overview?hours=24" \
-H "Authorization: Bearer bloop_..."
# Get traces for a specific model
curl "http://localhost:5332/v1/llm/traces?model=gpt-4o&limit=20" \
-H "Authorization: Bearer bloop_..."
LLM Proxy API
Zero-instrumentation tracing by acting as a reverse proxy to LLM providers. Simply point your LLM client at bloop's proxy endpoint instead of the provider directly. All calls are automatically traced with prompts, completions, token usage, cost, and latency captured server-side.
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | /v1/proxy/openai/{*path} |
Provider API Key | Proxy OpenAI API calls with automatic tracing |
How It Works
- Your app sends request to
/v1/proxy/openai/chat/completionswith your OpenAI API key - bloop forwards request to OpenAI, capturing prompt content
- OpenAI returns response (streaming or non-streaming)
- bloop captures completion, token usage, calculates cost
- Trace is created and sent to pipeline
- Original response returned to your app (unchanged)
Setup
# Point your OpenAI client at bloop
# Python
import openai
client = openai.OpenAI(
api_key="sk-your-openai-key",
base_url="http://localhost:5332/v1/proxy/openai"
)
# TypeScript
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "sk-your-openai-key",
baseURL: "http://localhost:5332/v1/proxy/openai"
});
# Or via environment variable (most SDKs)
export OPENAI_BASE_URL=http://localhost:5332/v1/proxy/openai
Configuration
Proxy behavior is controlled via config.toml:
[llm_tracing.proxy]
enabled = true
providers = ["openai"]
openai_base_url = "https://api.openai.com/v1"
capture_prompts = true
capture_completions = true
capture_streaming = true
Content Storage
LLM prompts and completions can contain sensitive data. Content storage is configurable per-project with three policies:
| Policy | What's stored | Use case |
|---|---|---|
none (default) | Prompts and completions stripped at ingest, never hit disk | Production with sensitive data |
metadata_only | Only metadata JSON kept | Structured tags without prompt content |
full | Everything stored (prompts, completions, metadata) | Development, debugging, audit trails |
# Check current policy
curl "http://localhost:5332/v1/llm/settings?project_id=my-app" \
-H "Authorization: Bearer bloop_..."
# Update policy
curl -X PUT "http://localhost:5332/v1/llm/settings?project_id=my-app" \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{"content_storage": "full"}'
Content storage policy is enforced at ingest time. Changing from none to full only affects future traces — previously ingested traces with stripped content cannot be recovered.
LLM Experiments (A/B Testing)
Create and manage A/B tests for prompt versions. Compare metrics like latency, cost, success rate, and user feedback scores across prompt variants to determine the best performing version.
Experiment Endpoints
| Method | Path | Description |
|---|---|---|
| GET | /v1/llm/experiments | List all experiments for a project |
| POST | /v1/llm/experiments | Create a new experiment with variants |
| GET | /v1/llm/experiments/:id | Get experiment details with metrics |
| GET | /v1/llm/experiments/:id/compare | Compare two variants with statistical analysis |
Creating an Experiment
# Create an A/B test for prompt variants
curl -X POST "http://localhost:5332/v1/llm/experiments" \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "Summarizer Prompt Test",
"prompt_name": "summarizer",
"description": "Testing detailed vs concise prompts",
"variants": [
{
"name": "Baseline",
"version": "v1",
"prompt_template": "Summarize: {text}",
"traffic_percentage": 50
},
{
"name": "Detailed",
"version": "v2",
"prompt_template": "Provide a detailed summary in 3-5 sentences: {text}",
"traffic_percentage": 50
}
]
}'
Comparing Variants
# Compare baseline vs treatment variant
curl "http://localhost:5332/v1/llm/experiments/experiment-123/compare?baseline=v1&treatment=v2" \
-H "Authorization: Bearer bloop_..."
# Response includes:
# - Latency improvement %
# - Cost improvement %
# - Success rate improvement %
# - Statistical significance (p-value, confidence level)
# - Recommendation
Experiment Metrics
Each variant tracks the following metrics automatically:
- Total calls - Number of traces with this prompt version
- Average latency - Mean response time in milliseconds
- Average cost - Mean cost per call in dollars
- Success rate - Percentage of calls with status "ok"
- Average tokens - Mean total tokens per call
- User feedback score - Aggregated feedback (thumbs up/down) if collected
LLM Search
Full-text search over LLM traces. Searches across trace names, session IDs, and user IDs to quickly find specific traces or groups of related traces.
Search Endpoint
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /v1/llm/search |
Bearer / Session | Full-text search over traces |
Query Parameters
| Param | Type | Default | Description |
|---|---|---|---|
q | string | required | Search query (matches trace name, session_id, user_id) |
project_id | string | from token | Filter by project |
hours | i64 | 24 | Time window to search within |
limit | i64 | 50 | Max results to return |
# Search for traces by name
curl "http://localhost:5332/v1/llm/search?q=chat-completion&hours=48" \
-H "Authorization: Bearer bloop_..."
# Search by session ID
curl "http://localhost:5332/v1/llm/search?q=session-abc&limit=20" \
-H "Authorization: Bearer bloop_..."
Response
{
"traces": [
{
"id": "trace-001",
"name": "chat-completion",
"session_id": "session-abc",
"user_id": "user-123",
"status": "completed",
"total_tokens": 150,
"total_cost_micros": 2500,
"latency_ms": 1200,
"started_at": 1700000000000
}
],
"total": 1,
"query": "chat-completion"
}
LLM Scoring
Attach quality scores to LLM traces for evaluation and monitoring. Scores are named (e.g. "quality", "relevance", "safety") and normalized to a 0.0–1.0 range. Use scores to track LLM output quality over time, compare prompt versions, and build evaluation datasets.
Scoring Endpoints
| Method | Path | Auth | Description |
|---|---|---|---|
| POST | /v1/llm/traces/:id/scores |
Bearer / Session | Submit a score for a trace |
| GET | /v1/llm/traces/:id/scores |
Bearer / Session | Get all scores for a trace |
| GET | /v1/llm/scores/summary |
Bearer / Session | Aggregated score summary across traces |
Submit a Score
curl -X POST http://localhost:5332/v1/llm/traces/trace-001/scores \
-H "Authorization: Bearer bloop_..." \
-H "Content-Type: application/json" \
-d '{
"name": "quality",
"value": 0.85,
"comment": "Good response, accurate and well-structured"
}'
Score Body
| Field | Type | Required | Description |
|---|---|---|---|
name | string | Yes | Score name (e.g. "quality", "relevance"). Max 128 characters. |
value | float | Yes | Score value, must be between 0.0 and 1.0 inclusive. |
comment | string | No | Optional comment explaining the score. |
Get Scores for a Trace
curl "http://localhost:5332/v1/llm/traces/trace-001/scores" \
-H "Authorization: Bearer bloop_..."
Response
{
"scores": [
{
"id": 1,
"trace_id": "trace-001",
"name": "quality",
"value": 0.85,
"comment": "Good response, accurate and well-structured",
"created_at": "2024-01-15T10:30:00Z"
},
{
"id": 2,
"trace_id": "trace-001",
"name": "relevance",
"value": 0.92,
"comment": null,
"created_at": "2024-01-15T10:31:00Z"
}
]
}
Score Summary
Get aggregated statistics for all score names across traces in a time window.
curl "http://localhost:5332/v1/llm/scores/summary?hours=48" \
-H "Authorization: Bearer bloop_..."
Summary Query Parameters
| Param | Type | Default | Description |
|---|---|---|---|
project_id | string | from token | Filter by project |
hours | i64 | 24 | Time window |
Response
{
"scores": [
{
"name": "quality",
"count": 142,
"avg_value": 0.78,
"min_value": 0.12,
"max_value": 0.99
},
{
"name": "relevance",
"count": 89,
"avg_value": 0.85,
"min_value": 0.45,
"max_value": 1.0
}
],
"window_hours": 48
}
Multiple scores with the same name can be submitted for a single trace (e.g. from different evaluators). The summary endpoint aggregates across all scores with a given name.
Prompt Version Logging
Track which prompt templates and versions are used across your LLM traces. By tagging traces with prompt_name and prompt_version, you can monitor performance regressions across prompt iterations and compare metrics between versions.
Tagging Traces with Prompt Versions
Include the optional prompt_name and prompt_version fields when submitting traces via POST /v1/traces or POST /v1/traces/batch.
{
"id": "trace-042",
"name": "summarize-article",
"prompt_name": "summarize-v2",
"prompt_version": "1.3.0",
"session_id": "session-xyz",
"status": "completed",
"started_at": 1700000000000,
"ended_at": 1700000001500,
"spans": [...]
}
Prompt Metrics Endpoint
| Method | Path | Auth | Description |
|---|---|---|---|
| GET | /v1/llm/prompts |
Bearer / Session | List prompt versions with aggregated performance metrics |
Query Parameters
| Param | Type | Default | Description |
|---|---|---|---|
project_id | string | from token | Filter by project |
hours | i64 | 24 | Time window |
curl "http://localhost:5332/v1/llm/prompts?hours=72" \
-H "Authorization: Bearer bloop_..."
Response
{
"prompts": [
{
"prompt_name": "summarize-v2",
"prompt_version": "1.3.0",
"trace_count": 482,
"avg_latency_ms": 1350.5,
"avg_tokens": 175.2,
"total_cost_micros": 1205000,
"error_rate": 0.02,
"last_seen": "2024-01-15T14:22:00Z"
},
{
"prompt_name": "summarize-v2",
"prompt_version": "1.2.0",
"trace_count": 1204,
"avg_latency_ms": 1580.3,
"avg_tokens": 210.8,
"total_cost_micros": 3612000,
"error_rate": 0.05,
"last_seen": "2024-01-14T09:15:00Z"
}
]
}
Both prompt_name and prompt_version are optional. Traces without these fields are excluded from the /v1/llm/prompts aggregation. Use consistent naming (e.g. semver for versions) to get meaningful comparisons across prompt iterations.