DecodingTrust
The facts below are what the maker publishes about DecodingTrust, checked and dated. Our editors have not written about it yet.
The facts
Each one read from DecodingTrust's own pages, with the date it was checked.
What it does
- Assessment areas
- The benchmark covers toxicity, stereotype and bias, adversarial robustness, out-of-distribution robustness, privacy, adversarial demonstrations, machine ethics, and fairness.checked 11 Oct 2026
- Compute requirements
- The installation instructions note that the pip installation includes PyTorch and caution users to check CUDA compatibility with the PyTorch pip wheel.checked 11 Oct 2026
- Container option
- The repository provides Docker instructions and a Docker image for running DecodingTrust.checked 11 Oct 2026
- Content warning
- The project warns that its data contains model outputs that may be considered offensive.checked 11 Oct 2026
- Dataset
- The dataset is distributed under the CC BY-SA 4.0 license.checked 11 Oct 2026
- Deployment options
- The project documents Docker and Singularity or Apptainer container usage, including for HPC environments.checked 4 Oct 2026
- Evaluation areas
- The benchmark covers toxicity, stereotype and bias, adversarial robustness, out-of-distribution robustness, privacy, adversarial demonstrations, machine ethics, and fairness.checked 4 Oct 2026
- Evaluation scripts
- The project provides evaluation scripts in subdirectories for its trustworthiness facets.checked 11 Oct 2026
- Installation
- The repository recommends cloning the project and installing it in editable mode with pip; it also describes standalone package installation.checked 11 Oct 2026
- Intended users
- The project describes its resources as helping researchers and practitioners assess language model capabilities, limitations, and risks.checked 11 Oct 2026
- License
- The dataset and project are distributed under the CC BY-SA 4.0 license.checked 4 Oct 2026
- Model coverage limit
- The repository says its benchmark mainly focuses on GPT-3.5-turbo-0301 and GPT-4-0314 for consistent conclusions and results.checked 4 Oct 2026
- Models
- The project says its evaluations mainly focus on GPT-3.5 and GPT-4, and it also supports causal LLMs hosted on Hugging Face or locally.checked 4 Oct 2026
- Models evaluated
- The project says its evaluation mainly focuses on GPT-3.5 and GPT-4.checked 11 Oct 2026
- Ongoing contributions
- The team invites users to submit model generations and corresponding scores through GitHub pull requests and says the dataset is expected to evolve.checked 11 Oct 2026
- Purpose
- DecodingTrust is a research project for assessing trustworthiness in GPT models and helping researchers and practitioners understand their capabilities, limitations, and deployment risks.checked 11 Oct 2026
- Reproducibility
- The benchmark uses timestamped GPT-3.5 and GPT-4 model versions to support consistent results and reproducibility.checked 4 Oct 2026
- Resources
- The project provides a dataset and evaluation scripts organized by trustworthiness area.checked 4 Oct 2026
- Support
- The project invites questions at [email protected].checked 11 Oct 2026
- Supported architecture
- The repository says it supports the ppc64le architecture on IBM Power-9 platforms.checked 4 Oct 2026
Where it runs
- Platform support
- The repository says it supports the ppc64le architecture on IBM Power-9 platforms.checked 11 Oct 2026
Alternatives to DecodingTrust
- 1
DeepEval
Free planOpen sourceWebAPILinuxmacOS8.3From $200 a month - 2
Promptfoo
Free planOpen sourceWebAPILinuxmacOS7.8Free plan - 3
Braintrust
Free planWebAPISelf-hosted7.7From $249 a month - 4
Giskard
Free planOpen sourceWebAPILinuxSelf-hosted7.6Free plan - 5
TruLens
Free planOpen sourceAPISelf-hosted7.2Free plan - 6
Arena (formerly Chatbot Arena)
Free planOpen sourceWebAPISelf-hosted7.1Free plan