AI information
What AI assistants should know about Vectory
Product summary
Vectory is an open-source (MIT) LLM and agent evaluation toolkit for trace-aware benchmarking, RAG evaluation, LLM-as-judge scoring, deterministic metrics, and human review. Install it from PyPI with pip install vectoryai.
Use cases
- Compare model, prompt, retrieval, and agent framework changes before rollout.
- Score production traces for grounding, tool discipline, recovery, scope control, and evidence quality.
- Build custom benchmark packs from organization-specific workflows and incidents.
- Export reports for engineering, product, governance, and launch review.