evaluation

Framework for enhancing LLMs for RAG tasks using fine-tuning.

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

🦞 Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more.

Python implementation of the IOU Tracker

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍

Benchmarking long-form factuality in large language models. Original code for our paper "Long-form factuality in large language models".

Open-source benchmark for browser AI agents on daily tasks.

Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表，主要面向基础大模型评测，旨在探求生成式AI的技术边界.

AutoPrompt: Automatic Prompt Construction for Masked Language Models.

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

A Simple Math and Pseudo C# Expression Evaluator in One C# File. Can also execute small C# like scripts

A resource repository for machine unlearning in large language models

TCExam is a CBA (Computer-Based Assessment) system (e-exam, CBT - Computer Based Testing) for universities, schools and companies, that enables educators and trainers to author, schedule, deliver, and report on surveys, quizzes, tests and exams.

Simple Safe Sandboxed Extensible Expression Evaluator for Python

This is a toolbox repository to help evaluate various methods that perform image matching from a pair of images.

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

A collection of datasets that pair questions with SQL queries.

ParseBench - A Document Parsing Benchmark for AI Agents

Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.

Dataset and benchmark for RAG on company internal documents.