nlp-evaluation

Comprehensive NLP Evaluation System

This repo houses experimental projects inspired by insights from the course 'Building and Evaluating Advanced RAG Applications' offered by DeepLearning.AI

This repository contains the dataset and code used in our paper, “I Am Aligned, But With Whom? Diagnosing Structural Alignment Failures in Multilingual LLMs” It provides tools to evaluate how LLMs represent cultural values across 16 countries, multiple languages, and perspectives.

A structured evaluation pipeline for LLM-generated outputs in financial supervision contexts. Combines PRA-aligned prompts, thread-type detection, and metric-level meta-review to assess relevance, justification, and actionability across 50+ regulatory and conversational metrics.

NLP evaluation tool prepared for Udacity Front End Development Nanodegree. First attempt to write unitary tests with Jest. App not working properly anymore due to the trial period of Aylien API, that reached the end.

Framework for testing Generative Dialog Models (GDMs)

Evaluation toolkit for comparing NLP and language-model outputs using quality metrics and resource benchmarks.

Detecting hallucinations in LLM-generated answers using cross-checking consistency across models. Implements and extends the SAC3 method to smaller open-source models.

A side-by-side comparison of extractive and abstractive text summarization: TF-IDF plus PageRank sentence ranking versus a fine-tuned DistilBART transformer, with ROUGE-based evaluation and visualization.

Systematic RAG evaluation framework — 3 controlled experiments measuring how top-k retrieval, prompt design, and context filtering affect faithfulness, answer correctness, and hallucination rates. Built with OpenAI + FAISS + LLM-as-judge scoring.

Textual Similarity Assessment Suite

Small research tool for repeated balanced pairwise ordinal judgments with uncertainty-aware ranking.