What Are Health Checks for AI Agents?
Health checks for AI agents are diagnostic endpoints and internal monitoring mechanisms that verify not just whether a service is running, but whether it can think, reason, and act correctly. Unlike traditional microservices where a simple HTTP 200 response often suffices, AI agents are stateful, model-dependent systems that can degrade in subtle, non-binary ways. A health check for an AI agent must validate the entire cognitive pipeline: the model backend, the tool-calling infrastructure, the memory store, the rate-limit budget, and the quality of responses themselves.
Think of it as the difference between checking if a car engine turns on versus verifying that the steering, brakes, navigation system, and fuel levels all work together under load. An AI agent can be "up" in the HTTP sense while being functionally useless because its LLM provider is returning gibberish, its vector store is corrupted, or its token budget is exhausted.
Why Simple HTTP Pings Fall Short
A basic /health endpoint that returns {"status": "ok"} tells you the web framework is alive. It doesn't tell you any of the following:
- Model availability: Is the LLM API actually responding with coherent completions?
- Token budget health: Are you approaching rate limits that will cause cascading failures?
- Tool integrity: Do all registered tools (search APIs, calculators, database connectors) still function?
- Memory store connectivity: Can the agent read and write to its long-term memory?
- Response quality degradation: Is the model outputting truncated, repetitive, or nonsensical content?
- Latency creep: Has inference time silently drifted beyond acceptable thresholds?
For a traditional REST API, a binary alive/dead signal works because the service is deterministic. For an AI agent, failure modes are probabilistic and layered. A health check must probe each layer independently and report a composite view that operations teams can act on.
Anatomy of an AI Agent Health Check
A robust health check for an AI agent should assess at least four dimensions:
1. Model Backend Connectivity
This is the most obvious layer. Verify that the LLM provider (OpenAI, Anthropic, self-hosted vLLM, etc.) is reachable and returns valid completions. But don't just send a trivial pingâsend a calibrated prompt that tests reasoning capability.
2. Tool & Plugin Health
Many AI agents rely on external tools: web search, code execution, database queries, file system operations. Each tool needs its own health probe. A search tool might return HTTP 200 but serve empty result sets due to expired API keys.
3. Memory & State Integrity
If your agent uses a vector store, a conversation history database, or a key-value cache, those need explicit connectivity and integrity checks. A corrupted embedding index can silently poison every retrieval-augmented generation call.
4. Operational Budgets & Limits
Track token consumption rates, remaining API quota, and cost accumulation. A health check should warn when you're at 85% of your daily budget, not after you've already been throttled.
Implementing a Multi-Layer Health Check Endpoint
Below is a production-ready implementation in Python using FastAPI. It probes an OpenAI-compatible model, a Pinecone vector store, a Redis memory cache, and a mock search tool, then aggregates results into a structured health report.
import asyncio
import time
import os
from datetime import datetime, timedelta
from typing import Dict, List, Optional
from dataclasses import dataclass, field
from enum import Enum
import httpx
from fastapi import FastAPI, Response
from fastapi.responses import JSONResponse
import openai
import redis.asyncio as aioredis
from pinecone import Pinecone
app = FastAPI()
# -----------------------------------------------------------
# Health check data models
# -----------------------------------------------------------
class HealthStatus(str, Enum):
HEALTHY = "healthy"
DEGRADED = "degraded"
UNHEALTHY = "unhealthy"
@dataclass
class ComponentHealth:
component: str
status: HealthStatus
latency_ms: float
message: str
metadata: dict = field(default_factory=dict)
@dataclass
class AgentHealthReport:
overall_status: HealthStatus
timestamp: str
components: List[ComponentHealth]
uptime_seconds: float
token_budget_remaining: Optional[dict] = None
# -----------------------------------------------------------
# Individual health probes
# -----------------------------------------------------------
async def probe_model_backend() -> ComponentHealth:
"""
Probes the LLM backend with a reasoning task.
A simple echo or ping won't catch model collapseâwe send a
prompt that requires coherent multi-step reasoning and validate
the structure of the response.
"""
start = time.perf_counter()
try:
client = openai.AsyncOpenAI(
api_key=os.getenv("OPENAI_API_KEY"),
timeout=httpx.Timeout(15.0, connect=5.0)
)
# Calibrated prompt: requires reasoning but is fast
response = await client.chat.completions.create(
model=os.getenv("MODEL_NAME", "gpt-4o-mini"),
messages=[
{
"role": "system",
"content": "You are a health check probe. "
"Respond with exactly this JSON and nothing else: "
'{"reasoning_complete": true, "steps": 3}'
},
{
"role": "user",
"content": "If Alice has 5 apples and gives 2 to Bob, "
"then buys 3 more, how many does she have? "
"Think step by step, then output the JSON."
}
],
max_tokens=150,
temperature=0.0 # deterministic for health checks
)
elapsed = (time.perf_counter() - start) * 1000
content = response.choices[0].message.content.strip()
# Validate we got structured output, not gibberish
if "reasoning_complete" in content and "steps" in content:
return ComponentHealth(
component="model_backend",
status=HealthStatus.HEALTHY,
latency_ms=round(elapsed, 2),
message="Model responded with valid structured output",
metadata={
"model": os.getenv("MODEL_NAME", "gpt-4o-mini"),
"completion_tokens": response.usage.completion_tokens,
"total_tokens": response.usage.total_tokens
}
)
else:
return ComponentHealth(
component="model_backend",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message="Model response did not match expected structure",
metadata={"raw_content": content[:200]}
)
except openai.RateLimitError:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="model_backend",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message="Rate limited by model provider",
metadata={"error": "rate_limit_exceeded"}
)
except openai.APITimeoutError:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="model_backend",
status=HealthStatus.UNHEALTHY,
latency_ms=round(elapsed, 2),
message="Model backend timed out",
metadata={"error": "timeout", "threshold_s": 15.0}
)
except Exception as e:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="model_backend",
status=HealthStatus.UNHEALTHY,
latency_ms=round(elapsed, 2),
message=f"Model backend probe failed: {str(e)[:200]}",
metadata={"error_type": type(e).__name__}
)
async def probe_vector_store() -> ComponentHealth:
"""
Probes the vector store (Pinecone) by listing indexes and
performing a cheap fetch operation. A corrupted index often
still responds to list operations, so we validate data integrity
by fetching a known vector.
"""
start = time.perf_counter()
try:
pc = Pinecone(api_key=os.getenv("PINECONE_API_KEY"))
# List indexes to verify connectivity
indexes = await asyncio.to_thread(pc.list_indexes)
if not indexes:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="vector_store",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message="No indexes foundâvector store may be empty or misconfigured",
metadata={"index_count": 0}
)
target_index_name = os.getenv("PINECONE_INDEX", indexes[0].get("name", ""))
if not target_index_name:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="vector_store",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message="No target index configured",
metadata={"available_indexes": [idx.get("name") for idx in indexes]}
)
index = pc.Index(target_index_name)
# Fetch a small set of vectors to verify data plane health
# Use a zero vector as a probeâfetch top-1
fetch_result = await asyncio.to_thread(
index.query,
vector=[0.0] * 1536, # assumes OpenAI embedding dimension
top_k=1,
include_metadata=False
)
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="vector_store",
status=HealthStatus.HEALTHY,
latency_ms=round(elapsed, 2),
message=f"Vector store reachable, index '{target_index_name}' operational",
metadata={
"index_name": target_index_name,
"index_count": len(indexes),
"query_successful": True
}
)
except Exception as e:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="vector_store",
status=HealthStatus.UNHEALTHY,
latency_ms=round(elapsed, 2),
message=f"Vector store probe failed: {str(e)[:200]}",
metadata={"error_type": type(e).__name__}
)
async def probe_redis_memory() -> ComponentHealth:
"""
Probes Redis (used for conversation memory / short-term cache).
Performs a SET/GET round-trip with a health-check key.
"""
start = time.perf_counter()
try:
r = aioredis.from_url(
os.getenv("REDIS_URL", "redis://localhost:6379"),
socket_connect_timeout=3.0,
socket_timeout=3.0
)
health_key = f"health_check_probe_{int(time.time())}"
test_value = "agent_memory_ok"
# SET with expiry to avoid littering the store
await r.set(health_key, test_value, ex=60)
retrieved = await r.get(health_key)
elapsed = (time.perf_counter() - start) * 1000
if retrieved:
decoded = retrieved.decode("utf-8") if isinstance(retrieved, bytes) else retrieved
if decoded == test_value:
await r.delete(health_key) # cleanup
return ComponentHealth(
component="redis_memory",
status=HealthStatus.HEALTHY,
latency_ms=round(elapsed, 2),
message="Redis memory store read/write round-trip successful",
metadata={"roundtrip_ok": True}
)
return ComponentHealth(
component="redis_memory",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message="Redis SET/GET mismatchâpossible data corruption",
metadata={"roundtrip_ok": False}
)
except Exception as e:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="redis_memory",
status=HealthStatus.UNHEALTHY,
latency_ms=round(elapsed, 2),
message=f"Redis memory probe failed: {str(e)[:200]}",
metadata={"error_type": type(e).__name__}
)
async def probe_search_tool() -> ComponentHealth:
"""
Probes an external search tool (mock implementation).
In production, this would call SerpAPI, Tavily, or Brave Search.
"""
start = time.perf_counter()
try:
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0)) as client:
response = await client.get(
"https://api.search.example.com/health",
headers={"Authorization": f"Bearer {os.getenv('SEARCH_API_KEY')}"}
)
elapsed = (time.perf_counter() - start) * 1000
if response.status_code == 200:
return ComponentHealth(
component="search_tool",
status=HealthStatus.HEALTHY,
latency_ms=round(elapsed, 2),
message="Search tool API responding normally",
metadata={"status_code": 200}
)
else:
return ComponentHealth(
component="search_tool",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message=f"Search tool returned non-200 status: {response.status_code}",
metadata={"status_code": response.status_code}
)
except httpx.TimeoutException:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="search_tool",
status=HealthStatus.UNHEALTHY,
latency_ms=round(elapsed, 2),
message="Search tool timed out",
metadata={"error": "timeout", "threshold_s": 10.0}
)
except Exception as e:
elapsed = (time.perf_counter() - start) * 1000
return ComponentHealth(
component="search_tool",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message=f"Search tool probe failed: {str(e)[:200]}",
metadata={"error_type": type(e).__name__}
)
# -----------------------------------------------------------
# Token budget tracker (simplified)
# -----------------------------------------------------------
class TokenBudgetTracker:
"""
Tracks token usage against a configurable daily budget.
In production, persist this to Redis or a database.
"""
def __init__(self, daily_budget: int = 1_000_000):
self.daily_budget = daily_budget
self.usage_today = 0
self.last_reset = datetime.now().date()
def record_usage(self, tokens: int):
today = datetime.now().date()
if today != self.last_reset:
self.usage_today = 0
self.last_reset = today
self.usage_today += tokens
def remaining_budget(self) -> int:
return max(0, self.daily_budget - self.usage_today)
def usage_percentage(self) -> float:
return (self.usage_today / self.daily_budget) * 100
token_budget = TokenBudgetTracker(daily_budget=int(os.getenv("DAILY_TOKEN_BUDGET", "1000000")))
# -----------------------------------------------------------
# Composite health check endpoint
# -----------------------------------------------------------
startup_time = datetime.now()
@app.get("/health")
async def basic_health():
"""
Quick liveness probe for Kubernetes. Returns 200 if the process is alive.
Use this for K8s livenessProbe (fast, shallow).
"""
return {"status": "ok", "uptime_s": (datetime.now() - startup_time).total_seconds()}
@app.get("/health/agent")
async def agent_deep_health(response: Response):
"""
Comprehensive readiness probe for the AI agent.
Runs all component checks concurrently and aggregates results.
Use this for K8s readinessProbe or monitoring dashboards.
"""
# Run all probes concurrently
probes = [
probe_model_backend(),
probe_vector_store(),
probe_redis_memory(),
probe_search_tool(),
]
results = await asyncio.gather(*probes, return_exceptions=True)
components = []
for i, result in enumerate(results):
if isinstance(result, Exception):
# A probe crashed entirelyâtreat as unhealthy
components.append(ComponentHealth(
component=f"probe_{i}",
status=HealthStatus.UNHEALTHY,
latency_ms=0,
message=f"Probe crashed: {str(result)[:200]}",
metadata={"error_type": type(result).__name__}
))
else:
components.append(result)
# Determine overall status
statuses = [c.status for c in components]
if any(s == HealthStatus.UNHEALTHY for s in statuses):
overall = HealthStatus.UNHEALTHY
elif any(s == HealthStatus.DEGRADED for s in statuses):
overall = HealthStatus.DEGRADED
else:
overall = HealthStatus.HEALTHY
# Build budget info
budget_info = {
"daily_budget": token_budget.daily_budget,
"used_today": token_budget.usage_today,
"remaining": token_budget.remaining_budget(),
"usage_percentage": round(token_budget.usage_percentage(), 2),
"status": "exhausted" if token_budget.remaining_budget() == 0 else "available"
}
report = AgentHealthReport(
overall_status=overall,
timestamp=datetime.now().isoformat(),
components=components,
uptime_seconds=(datetime.now() - startup_time).total_seconds(),
token_budget_remaining=budget_info
)
# Set HTTP status code based on health
if overall == HealthStatus.UNHEALTHY:
response.status_code = 503
elif overall == HealthStatus.DEGRADED:
response.status_code = 200 # Still serving, but with warnings
else:
response.status_code = 200
return report.__dict__
@app.get("/health/readiness")
async def kubernetes_readiness(response: Response):
"""
Kubernetes-compatible readiness check.
Returns 200 only if the agent is fully healthy, 503 otherwise.
"""
report = await agent_deep_health(response)
return report
Key Design Decisions in This Implementation
- Concurrent probing: All component checks run in parallel via
asyncio.gather, so a slow model backend doesn't block the vector store check. Total health check latency is bounded by the slowest probe, not the sum. - Calibrated prompts: The model probe sends a reasoning task with a deterministic expected output format. This catches cases where the model is "up" but returning incoherent responses due to quantization bugs or corrupted weights.
- Graceful degradation: A degraded search tool doesn't make the entire agent unhealthyâit reports
DEGRADEDwith a 200 status, signaling that the agent can still operate with reduced capability. - Token budget awareness: The health report includes real-time budget consumption, letting operations teams see not just if the agent works but how long it can continue working.
- Separate liveness vs. readiness:
/healthis a fast liveness probe (no dependencies), while/health/agentis a deep readiness probe. This aligns with Kubernetes best practices.
Probing Response Quality: Beyond Binary Checks
The most insidious failure mode for AI agents is silent quality degradation. The model responds, the tools work, memory is intactâbut output quality has dropped from 95% to 70% accuracy. Traditional health checks miss this entirely. Here's how to catch it:
Golden Test Set Regression
Maintain a small set of canonical queries with known correct responses. Periodically run them through the agent and compare outputs using semantic similarity (not exact string match, since LLMs are non-deterministic).
import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
# Pre-computed embeddings of "golden" expected answers
GOLDEN_EMBEDDINGS = {
"capital_of_france": np.array([...]), # embedding of "Paris"
"earth_radius_km": np.array([...]), # embedding of "6371 km"
"python_creator": np.array([...]), # embedding of "Guido van Rossum"
}
async def probe_response_quality(agent, threshold: float = 0.85) -> ComponentHealth:
"""
Runs golden queries through the agent and checks semantic similarity
of responses against expected answers.
"""
start = time.perf_counter()
golden_queries = [
("What is the capital of France?", "capital_of_france"),
("What is the radius of Earth in kilometers?", "earth_radius_km"),
("Who created the Python programming language?", "python_creator"),
]
similarities = []
for query, key in golden_queries:
agent_response = await agent.run(query)
response_embedding = await embed_text(agent_response)
expected_embedding = GOLDEN_EMBEDDINGS[key]
sim = cosine_similarity(
[response_embedding], [expected_embedding]
)[0][0]
similarities.append((key, sim))
avg_similarity = np.mean([s[1] for s in similarities])
elapsed = (time.perf_counter() - start) * 1000
if avg_similarity >= threshold:
return ComponentHealth(
component="response_quality",
status=HealthStatus.HEALTHY,
latency_ms=round(elapsed, 2),
message=f"Golden test set passed, avg similarity: {avg_similarity:.3f}",
metadata={"avg_similarity": round(avg_similarity, 3), "threshold": threshold}
)
else:
failing = [s[0] for s in similarities if s[1] < threshold]
return ComponentHealth(
component="response_quality",
status=HealthStatus.DEGRADED,
latency_ms=round(elapsed, 2),
message=f"Quality degradation detected on: {failing}",
metadata={
"avg_similarity": round(avg_similarity, 3),
"failing_queries": failing,
"threshold": threshold
}
)
Output Sanity Validators
For every agent response in production, run lightweight structural validators. These catch malformed JSON, truncated outputs, repetition loops, and hallucinated URLs.
def validate_agent_output(output: str) -> Dict[str, bool]:
"""
Structural sanity checks on agent output.
Returns a dict of check_name -> passed.
"""
checks = {}
# Check for truncation markers (many models cut off mid-sentence)
truncation_markers = ["...", "[truncated]", "continue", "---"]
checks["no_truncation"] = not any(
marker in output[-50:] for marker in truncation_markers
)
# Check for repetition loops (model stuck in a loop)
lines = output.split("\n")
if len(lines) > 10:
unique_lines = len(set(lines[-10:]))
checks["no_repetition_loop"] = unique_lines > 3
# Check minimum length
checks["minimum_length"] = len(output.strip()) > 10
# Check for hallucinated URLs (common failure mode)
import re
urls = re.findall(r'https?://[^\s<>"]+', output)
if urls:
checks["urls_present"] = True
# In production, you'd verify these URLs actually exist
return checks
Health Check Dashboard & Alerting Integration
The health endpoint data is most valuable when fed into your observability stack. Here's how to wire it up:
# prometheus_metrics.py - Export health data to Prometheus
from prometheus_client import Gauge, Histogram, Info, CollectorRegistry
from prometheus_client import push_to_gateway
agent_health_status = Gauge(
"agent_health_status",
"Overall agent health: 2=healthy, 1=degraded, 0=unhealthy",
["agent_id"]
)
component_health_status = Gauge(
"agent_component_health",
"Per-component health status",
["agent_id", "component"]
)
component_latency_ms = Histogram(
"agent_component_latency_ms",
"Health check latency per component",
["agent_id", "component"],
buckets=[10, 50, 100, 250, 500, 1000, 5000, 15000]
)
token_budget_remaining = Gauge(
"agent_token_budget_remaining",
"Remaining token budget for the current period",
["agent_id"]
)
async def record_health_metrics(report: AgentHealthReport, agent_id: str):
"""Push health check results to Prometheus Pushgateway."""
# Overall status
status_map = {"healthy": 2, "degraded": 1, "unhealthy": 0}
agent_health_status.labels(agent_id=agent_id).set(
status_map.get(report.overall_status.value, 0)
)
# Per-component metrics
for component in report.components:
component_health_status.labels(
agent_id=agent_id, component=component.component
).set(status_map.get(component.status.value, 0))
component_latency_ms.labels(
agent_id=agent_id, component=component.component
).observe(component.latency_ms)
# Token budget
if report.token_budget_remaining:
token_budget_remaining.labels(agent_id=agent_id).set(
report.token_budget_remaining["remaining"]
)
# Push to gateway (for batch jobs; for long-running services use scraping)
push_to_gateway("pushgateway:9091", job="ai_agent_health")
Best Practices for AI Agent Health Checks
1. Separate Liveness from Readiness
Your liveness probe (/health) should be lightweightâcheck only that the process isn't deadlocked. It should never fail because an external dependency is down, or Kubernetes will restart your pod unnecessarily. The readiness probe (/health/agent) runs the full suite and controls traffic routing.
2. Use Calibrated, Deterministic Probes
When probing an LLM, set temperature=0.0 and use a prompt engineered to produce a predictable structure. Validate the structure, not the exact wording. A prompt asking for a specific JSON schema with a reasoning task is far more reliable than "respond with 'pong'".
3. Probe with Realistic Workloads
Don't just check if the model echoes text. Send a prompt that mirrors actual agent workloads: multi-step reasoning, tool calling, or structured output generation. This catches quantization artifacts, attention bugs, and context window corruption that simple pings miss.
4. Implement Graceful Degradation
Design your agent to continue operating with reduced capability when non-critical components fail. The health check should reflect this: a degraded search tool yields DEGRADED with HTTP 200, not UNHEALTHY with 503. This prevents unnecessary failovers while still alerting operators.
5. Track Token Budget as a Health Metric
Running out of tokens is a predictable failure mode. Surface remaining budget in every health check response. Alert at 75%, 85%, and 95% thresholds so teams have time to upgrade plans or switch providers before the agent goes dark.
6. Add Response Quality Regression Probes
Run golden test queries on a schedule (every 5 minutes, every 100th request) and compare semantic similarity against known good answers. Feed the similarity scores into a Prometheus gauge. If the 7-day rolling average drops below a threshold, trigger an investigationâeven if all other checks are green.
7. Timeout and Circuit-Break External Dependencies
Every probe should have a strict timeout (5-15 seconds max). If a dependency fails consistently, implement circuit breaking: stop probing it for a cooldown period and route traffic accordingly. This prevents health checks themselves from exacerbating rate limit issues.
8. Include Metadata for Debugging
Each component health result should carry metadata: error types, raw response snippets (truncated), timing breakdowns, and configuration snapshots. When an on-call engineer sees "model_backend: UNHEALTHY" at 3 AM, they need enough context to know whether it's a rate limit, a timeout, or a model regression without digging through logs.
9. Version Your Health Check Contract
As your agent evolves, the health check format will change. Include a health_check_version field so monitoring dashboards and alerting rules can adapt. Breaking changes to the health check response format should be treated like API breaking changes.
10. Test Health Checks in CI/CD
Run the full health check suite in your deployment pipeline before rolling out to production. If a new model version or tool update causes the golden test similarity to drop, halt the deployment. Health checks aren't just for runtimeâthey're a deployment gate.
Putting It All Together: A Complete Deployment-Ready Configuration
Here's a sample Kubernetes deployment manifest that correctly configures liveness and readiness probes for an AI agent using the endpoints we built:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-agent
spec:
replicas: 3
selector:
matchLabels:
app: ai-agent
template:
metadata:
labels:
app: ai-agent
spec:
containers:
- name: agent
image: myregistry/ai-agent:latest
env:
- name: OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: agent-secrets
key: openai-api-key
- name: PINECONE_API_KEY
valueFrom:
secretKeyRef:
name: agent-secrets
key: pinecone-api-key
- name: REDIS_URL
value: "redis://redis-service:6379"
- name: DAILY_TOKEN_BUDGET
value: "1000000"
# Liveness: fast, no dependencies, checks only process health
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
# Readiness: full agent health check, determines traffic routing
readinessProbe:
httpGet:
path: /health/readiness
port: 8080
initialDelaySeconds: 15
periodSeconds: 30
timeoutSeconds: 20
failureThreshold: 2
# Startup: give the agent time to initialize model connections
startupProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 0
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 12 # up to 60s for cold start
Conclusion
Health checks for AI agents demand a fundamentally different approach than traditional service monitoring. A simple HTTP ping tells you the web server is alive; it says nothing about whether your agent can reason, remember, or act. By implementing layered, concurrent probes that validate model backends, vector stores, memory caches, tool dependencies, token budgets, and response quality, you build an honest picture of agent healthâone that operations teams can trust. The code patterns in this tutorial give you a production-ready foundation. The best practices ensure your health checks themselves are robust, actionable, and evolve gracefully alongside your agents. In a world where AI agents increasingly drive critical business workflows, treating health checks as a first-class engineering concern isn't optionalâit's the difference between catching a degradation at 10 AM when budgets hit 75% and discovering it at 3 AM when everything is down.