🐢 Open-Source Evaluation & Testing library for LLM Agents

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

A Python package to assess and improve fairness of machine learning models.

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

WEFE: The Word Embeddings Fairness Evaluation Framework. WEFE is a framework that standardizes the bias measurement and mitigation in Word Embeddings models. Please feel welcome to open an issue in case you have any questions or a pull request if you want to contribute to the project!

Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems 🔎🤖🧰

Safety Score for Pre-Trained Language Models

Fairness Aware Machine Learning. Bias detection and mitigation for datasets and models.

PyTorch package to train and audit ML models for Individual Fairness

[ACL 2020] Towards Debiasing Sentence Representations

👋 Influenciae is a Tensorflow Toolbox for Influence Functions

[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

[ACM 2024] Jurity: Fairness & Evaluation Library

Credo AI Lens is a comprehensive assessment framework for AI systems. Lens standardizes model and data assessment, and acts as a central gateway to assessments created in the open source community.

A tool for gender bias identification in text. Part of Microsoft's Responsible AI toolbox.

A fairness library in PyTorch.

SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments

Counterfactual Local Explanations of AI systems

PyTorch reimplementation of computing Shapley values via Truncated Monte Carlo sampling from "What is your data worth? Equitable Valuation of Data" by Amirata Ghorbani and James Zou [ICML 2019]

Open-source toolkit to help companies implement responsible AI workflows.