synthetic-data

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

Mimesis is a Python library for generating fake but realistic data in multiple languages and locales.

A procedural Blender pipeline for photorealistic training image generation

Synthetic data generation for tabular data

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

Python Toolkit for Causal and Probabilistic Reasoning

SDG is a specialized framework designed to generate high-quality structured tabular data.

🎨 NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.

Synthetic data curation for post-training and structured data extraction

Conditional GAN for generating synthetic tabular data.

A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions

DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤

Synthadoc: An open-source LLM knowledge compilation engine that turns raw documents into structured, local-first wikis. A transparent, human-readable alternative to traditional RAG, which can be self-managed and self-improved without the use of any tools.

Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline

[ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data generation pipeline!

A lightweight library for generating synthetic instruction tuning datasets for your data without GPT.

Verbalized Sampling, a training-free prompting strategy to mitigate mode collapse in LLMs by requesting responses with probabilities. Achieves 2-3x diversity improvement while maintaining quality. Model-agnostic framework with CLI/API for creative writing, synthetic data generation, and dialogue simulation.

Synthetic Data SDK ✨

Official code for MAMMA: Markerless Accurate Multi-person Motion Acquisition.