uncertainty-calibration

Uncertainty Toolbox: a Python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

A Library for Uncertainty Quantification.

A collection of research and application papers of (uncertainty) calibration techniques.

A toolkit for visualizations in materials informatics.

Calibration library and code for the paper: Verified Uncertainty Calibration. Ananya Kumar, Percy Liang, Tengyu Ma. NeurIPS 2019 (Spotlight).

Calibrated Explanations is an explanation method for uncertainty-aware explanations of classification and regression models.

(ECCV 2022) BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks

Code for evaluating uncertainty estimation methods for Transformer-based architectures in natural language understanding tasks.

Source code for our paper: "LoGU: Long-form Generation with Uncertainty Expressions".

A project to train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-distribution detection and uncertainty estimation performances. Calibrate your model to produce enhanced uncertainty estimations. Detect out-of-distribution data using the defined score type and threshold.

Scoring rules like the Brier Score (Mean Squared Error, Quadratic Score) and Log Loss (Cross-Entropy, Negative Log-Likelihood, Logarithmic Score) can favor incorrect predictions. To address this limitation, the Probabilistic Brier Score (PBS) and Probabilistic Logarithmic Loss (PLL) have been introduced for probabilistic classifiers.

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

Calibration of Few-Shot Classification Tasks: Mitigating Misconfidence from Distribution Mismatch, IEEE Access vol.10

Truth Discovery Promotes Uncertainty Calibration of DNNs (UAI 2021)

A modular Python workflow for assimilating observed land-subsidence data into an ensemble of geomechanical simulations with the Ensemble Smoother with Multiple Data Assimilation (ES-MDA).

MSc thesis: evaluating & improving uncertainty calibration in retrieval-augmented LLMs. Compares 5 embedding models (BGE beats OpenAI) and cuts ECE ~32% via isotonic regression on real financial filings.

Risk-based sequential stopping: when to invoke an LLM in streaming systems. Code for ECML PKDD 2026 (Research Track). DOI: 10.5281/zenodo.20783298

GranularVAR: Multi-scale video understanding with augmentation-graded contrastive learning and calibrated uncertainty. V-JEPA 2 encoder + granularity-conditioned decoder. GWU MS Data Science Capstone, Spring 2026.

AI skill for evidence calibration, fact verification, source checking, and hallucination-resistant answers.——<用于证据校准、事实核查、信息来源验证，以及生成抗幻觉答案的AI技能>

Reproducible MEDAI deferral simulation (AIRI 2026). Synthetic research code.