The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

Evaluation and Tracking for LLM Experiments and AI Agents

A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying autonomous AI agents

Transparent telemetry collector for agents

AI agent to automatically check grammar and spelling on documentation files

oly turns long-running and interactive CLI workflows into persistent, supervised sessions for humans and AI agents. Close the terminal, keep the process alive, get notified when input is needed, and jump back in from anywhere.

CrewAI agents that gather and analyze YouTube comments to generate insights to inform better content creation.

Geniusrise: Framework for building geniuses

A collection of cookbooks to help developers get started quickly with the Firecrawl API.

Learn AI, ML, and NLP with interactive Jupyter Notebook tutorials.

A radically simple, reliable, and high performance template to enable you to quickly get set up building multi-agent applications

🧱 CrewNews is an AI news generator that delivers an unbiased version of the news for a given topic, using Streamlit for the GUI, Llama 3.1 as the LLM (inferenced via the AIML API), CrewAI for building AI agents, AgentOps for testing AI agents, and Exa & Firecrawl as tools

The open-source MultiAgentOps evaluation and verification harness for any industry business workflow.

GPT4 for windows, macos and ubuntu

Operational patterns for AI agents and infrastructure from solo to enterprise scale. Applies DevOps + SRE principles to the intersection: infrastructure FOR AI + AI FOR infrastructure. Alpha framework extracting meta-patterns from mission critical environments.

This smart assistant uses advanced Multi-Agent AI systems, leveraging over 10+ agents working asynchronously to perform 20+ tasks. From disease research to personalized diet and exercise plans, as well as doctor consultations and patient journeys, it brings all essential healthcare services into a single platform.

A curated list of the best AgentOps tools for 2026 — observability, tracing, evaluation, cost monitoring, and guardrails for LLM agents. Covering open-source and SaaS tools with feature benchmarks and architecture guidance.