Technology · Agentic → Observability · Open Core · wiki:deep
Comet Optik
Opik (catalog folder comet-optik) is Comet’s open-source LLM observability and evaluation platform: agent/LLM tracing, datasets and experiments, LLM-as-a-judge metrics, prompt/agent optimization, production dashboards, online evaluation rules, optional guardrails, and PyTest CI hooks. Apache-2.0; free to self-host the full platform, with Comet-hosted options. Folder name keeps the landscape spelling “Optik”; the product/repo is Opik.
Why it matters here
Same cell job as Langfuse/Phoenix: see and score agent behavior. Prefer Opik when Comet MLOps affinity or self-host Apache-2.0 + high-scale production monitoring is the draw. Prefer Langfuse for ClickHouse-backed OSS LLM engineering; Phoenix for OpenInference/Arize; Helicone for gateway-first logging.
How it works
SDK/integrations log traces and spans; UI supports annotation, playground, datasets/experiments, and online eval rules. Production paths emphasize high-volume ingest and dashboards.
- Self-host Opik server or use Comet-hosted.
- Integrate SDK / framework connectors.
- Trace agents; annotate and evaluate.
- Add online rules / CI evals for production guardrails.
Related: arize-phoenix · helicone · langfuse · topics/09-self-monitoring
Scroll inside the canvas to pan
When to reach for it
- Use when: you want self-hostable tracing + evals with Comet ecosystem hooks.
- Skip when: you only need OTEL export (openllmetry), proxy logging (helicone), or LangChain SaaS (langsmith).
- Prefer instead: Langfuse/Phoenix as peer defaults; Weave if W&B is already SoT for experiments.
Limits
- Not company SoR: ops outcomes still live in the ledger.
- Naming drift: “Optik” vs “Opik” confuses search — use repo name.
- Self-host ops: scale claims need real capacity planning.
- Judge metrics: automated scores need human calibration.
Linked from
Catalog backlinks — what points here (wiki “what links here”).
Features
- cost-latency-ops
- observability-auditability
- r2-09-self-monitoring
- r2-14-process-mining
- r2-34-decision-lineage
- r2-38-abm-span-mapping
- r2-47-trace-maturity-gate
- wt-runtime
Stacks
Peers
What we checked
Claims below are backed by science sources on disk.
Product / tracing role
comet-ml/opik · MODERATE
“Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.”
Docs / lifecycle
Opik documentation · MODERATE
“Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.”
Contrast vs Langfuse
Langfuse contrast · MODERATE
“Langfuse is an open source LLM engineering platform. It helps teams collaboratively develop, monitor, evaluate, and debug AI applications.”
In this research stack
Features and peers linked from the catalog map — not a second product surface.
Features it supports
- cost-latency-ops
- observability-auditability
- r2-09-self-monitoring
- r2-14-process-mining
- r2-34-decision-lineage
- r2-38-abm-span-mapping
- r2-47-trace-maturity-gate
- wt-runtime
Same cell
Stack
Document management
No DM vendor crosswalk edges yet.
Source
Primary repo github.com/comet-ml/opik · Open Core
technologies/comet-optik/README.md
Research inventory
9 tags · 14 out · 15 in · 3 artifacts · 1 gaps · 31 corpus docs
Catalog tags
- landscape.layer
- Agentic
- landscape.subcategory
- Observability
- license_tag
- Open Core
- maps.dm
- absent
- maps.features
- 8
- one_liner
- Agentic
- review.depth
- science
- slug
- comet-optik
- title
- Comet Optik
Artifacts
- dm_map · absent
- features_map · present · technologies/comet-optik/features.md
- readme · present · technologies/comet-optik/README.md
Out · alternative_to
- arize-phoenixtechnology
- heliconetechnology
- langfusetechnology
- langsmithtechnology
- openllmetrytechnology
- wandb-weavetechnology
Out · maps_to
- cost-latency-opsfeature
- observability-auditabilityfeature
- r2-09-self-monitoringfeature
- r2-14-process-miningfeature
- r2-34-decision-lineagefeature
- r2-38-abm-span-mappingfeature
- r2-47-trace-maturity-gatefeature
- wt-runtimefeature
In · alternative_to
- arize-phoenixtechnology
- heliconetechnology
- langfusetechnology
- langsmithtechnology
- openllmetrytechnology
- wandb-weavetechnology
In · in_stack
- agentic-observabilitystack · alternative_slot
In · maps_to
- cost-latency-opsfeature
- observability-auditabilityfeature
- r2-09-self-monitoringfeature
- r2-14-process-miningfeature
- r2-34-decision-lineagefeature
- r2-38-abm-span-mappingfeature
- r2-47-trace-maturity-gatefeature
- wt-runtimefeature
Corpus tags
- category
- Agentic → Observability
- dedication
- open-core
- feature
- cost-latency-opsobservability-auditabilityr2-09-self-monitoringr2-14-process-miningr2-34-decision-lineager2-38-abm-span-mappingr2-47-trace-maturity-gatewt-runtime
- kind
- map_edgetech_featurestech_quotetech_readmetech_science_sourcetech_section
- needs_deepen
- false
- quality
- ok
- section
- EvidenceFeatures mapGitHub mapHow it worksLimits & failure modesLinksScientific notesWhat it isWhen to use / skipWhy it matters here
- slug
- comet-optik
- source_id
- langfuse-contrastopik-docsopik-github
- technology
- comet-optik
Corpus documents (31)
map_edge · 8
- comet-optik → cost-latency-ops
- comet-optik → observability-auditability
- comet-optik → r2-09-self-monitoring
- comet-optik → r2-14-process-mining
- comet-optik → r2-34-decision-lineage
- comet-optik → r2-38-abm-span-mapping
- comet-optik → r2-47-trace-maturity-gate
- comet-optik → wt-runtime
tech_features · 1
- Comet Optik · features
tech_quote · 8
- Comet Optik · langfuse-contrast
- Comet Optik · langfuse-contrast
- Comet Optik · langfuse-contrast
- Comet Optik · opik-docs
- Comet Optik · opik-docs
- Comet Optik · opik-docs
- Comet Optik · opik-github
- Comet Optik · opik-github
tech_readme · 1
- Comet Optik
tech_science_source · 3
- Comet Optik · langfuse-contrast
- Comet Optik · opik-docs
- Comet Optik · opik-github
tech_section · 10
- Comet Optik · Evidence
- Comet Optik · Features map
- Comet Optik · GitHub map
- Comet Optik · How it works
- Comet Optik · Limits & failure modes
- Comet Optik · Links
- Comet Optik · Scientific notes
- Comet Optik · What it is
- … +2 more