DeepEval Example

A small example project demonstrating DeepEval, an open-source LLM evaluation framework for Python.

What this shows

• Running built-in metrics (answer relevancy, faithfulness, hallucination) against LLM outputs
• Writing evaluation cases with LLMTestCase
• Running evaluations as pytest-style tests

Setup

1. Create and activate a virtual environment:

   python -m venv .venv
   .\.venv\Scripts\Activate.ps1

2. Install dependencies:

   pip install -r requirements.txt

3. Copy .env.example to .env and add your OpenAI API key (DeepEval uses an LLM judge by default):

   Copy-Item .env.example .env

Running

deepeval test run tests/test_example.py

Project layout

• example.py — minimal standalone evaluation using evaluate()
• test_example.py — pytest-style evaluation using assert_test
• requirements.txt — pinned dependencies
• .env.example — template for required environment variables