AgentOps

Overview

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Langchain, Autogen, AG2, and CamelAI

Key Features

• Visualize: Visually track events such as LLM calls, tools, and multi-agent interactions.
• Time Travel Debugging: Rewind and replay agent runs with point in time precision.
• Debug and Audit: Keep a full data trail of logs, errors, and prompt injection attacks from prototype to production.
• Token Counts: Track, save, and monitor every token your agent sees.
• Cost Tracking: Manage and vizualize agent spend with up-to-date price monitoring.
• Fine-tuning: Fine-tune specialized LLMs up to 25x cheaper on saved completions.