Radicalbit AI Monitoring

What Radicalbit AI Monitoring is

Radicalbit AI Monitoring is a free, self-hosted platform for monitoring production LLM and machine-learning models. It checks numerical and categorical data for anomalies, missing values, and outliers, and tracks metrics over time. Available model metrics include Precision, Accuracy, Recall, F1, MSE, MAE, Perplexity, and Probability. Drift detection offers methods including Kolmogorov-Smirnov, PSI, Wasserstein, Jensen-Shannon, Chi-square, Kullback-Leibler, and Hellinger. The platform compares reference and current datasets to assess data quality, model quality, and drift. For LLM applications, tracing records requests, prompts, and tools, then displays sessions, traces, and spans with hierarchy, duration, and metadata. Documentation for version 1.4.0 focuses on binary and multiclass classification and regression, with more model types planned. A web interface covers model setup through metric visualization, and a Python SDK implements the REST API functionality. The API requires PostgreSQL, Kubernetes for Spark metric jobs, and distributed storage. Local deployment uses Docker Compose with K3s and supports MinIO or AWS S3. LLM tracing uses OpenLLMetry, an OpenTelemetry collector, and the Traceloop SDK. The project says it collects anonymous usage data, not personally identifiable information, and offers an opt-in or opt-out choice at first use.