Monitoring API Gateway: Metrics, Alarms, and Dashboards

Understanding API Gateway Monitoring

An API gateway acts as the single entry point for client requests, routing them to appropriate backend services. It handles authentication, rate limiting, request transformation, and response aggregation. Monitoring this critical infrastructure component is essential to detect anomalies, maintain performance, control costs, and ensure high availability.

Monitoring an API gateway involves collecting metrics that reflect its health and usage, configuring alarms to alert operators when thresholds are breached, and building dashboards that provide real-time and historical visibility. Together, these practices form an observability pipeline that helps teams respond quickly to incidents and make data-driven decisions about scaling, caching, and architecture improvements.

Core Metrics for API Gateways

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

API gateways emit a wide range of metrics, typically categorized by latency, errors, traffic, and saturation (the "LETS" model). The exact naming and availability depend on the gateway product (AWS API Gateway, Kong, Apigee, etc.), but the concepts are universal. Below are the essential metrics to track.

Latency Metrics

Latency directly impacts user experience. The gateway introduces its own processing time on top of backend response time. Two key latencies are:

IntegrationLatency: time from when the gateway forwards the request to the backend until it receives the response.
Latency (total): time from when the gateway receives the request until it sends the response to the client. This includes integration latency plus gateway overhead (authorization, transformation).

Monitoring the gap between total latency and integration latency helps identify gateway-side bottlenecks like slow authorizers or large payload transformations.

Error Metrics

Errors indicate failures that directly affect clients. They are split into client-side (4xx) and server-side (5xx) categories:

4XXError: counts unauthorized, forbidden, throttled, or malformed requests. A sudden spike can signal an expired token or a misconfigured client.
5XXError: counts internal gateway errors or invalid backend responses. A high rate often points to backend failures, timeouts, or gateway misconfiguration.
IntegrationError: specific failures when the gateway cannot connect to the backend or receives a non-2xx response.

Track error rate (errors per second) rather than absolute count to normalize against traffic variations.

Traffic Metrics

Traffic metrics reflect request volume and throughput:

Count: total number of requests in a period (often available per resource, method, or stage).
RequestRate: requests per second. Helps detect traffic spikes or DDoS patterns.
ConcurrentRequests: number of requests being processed simultaneously. High concurrency can exhaust connection pools.

Cache Metrics (if applicable)

Many gateways support response caching. Monitoring cache efficiency helps reduce backend load:

CacheHitCount / CacheMissCount: raw numbers of cache hits and misses.
CacheHitRate: percentage of requests served from cache. A low rate might indicate short TTLs or cache key collisions.

Setting Up Alarms

Alarms translate metric thresholds into actionable notifications (email, SMS, Slack, PagerDuty). They should cover both service degradation (high latency, high error rate) and cost anomalies (unexpected traffic surge). Below are practical examples for AWS API Gateway and Kong Gateway.

Defining Thresholds

Start with baseline measurements during normal load. For example, if average 5xx error rate is below 0.1%, you might alarm on >1% for 5 minutes. For latency, alarm when p95 exceeds a target SLO (e.g., 500 ms). Avoid overly sensitive alarms that cause alert fatigue.

Example: AWS API Gateway Alarm with CloudFormation

The following CloudFormation snippet creates a CloudWatch alarm on the 5XXError metric for an API Gateway stage. It triggers when the error rate exceeds 1% of total requests over two consecutive evaluation periods.


AWSTemplateFormatVersion: '2010-09-09'
Resources:
  Api5xxAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${ApiName}-${StageName}-High5xxErrors"
      AlarmDescription: "Triggers when 5xx error rate > 1% for 2 periods"
      Namespace: AWS/ApiGateway
      Metrics:
        - Id: errors
          Expression: "5xxErrors / Count * 100"
          Label: "5xxErrorRate"
          ReturnData: true
        - Id: Count
          MetricStat:
            Metric: Count
            Stat: Sum
            Period: 300
        - Id: 5xxErrors
          MetricStat:
            Metric: 5XXError
            Stat: Sum
            Period: 300
      Threshold: 1
      ComparisonOperator: GreaterThanThreshold
      EvaluationPeriods: 2
      AlarmActions:
        - !Ref SnsTopicARN

The alarm uses a metric math expression to compute the error rate. The AlarmActions reference an SNS topic that sends notifications. Adjust Period and EvaluationPeriods based on your tolerance for transient spikes.

Example: Kong Gateway with Prometheus and Alertmanager

Kong exposes metrics via Prometheus. The following Prometheus alert rule fires when the rate of HTTP 5xx responses from Kong exceeds 0.5% of total requests over 5 minutes.


groups:
- name: kong_alerts
  rules:
  - alert: KongHigh5xxRate
    expr: |
      sum(rate(kong_http_status{code=~"5.."}[5m])) 
      / 
      sum(rate(kong_http_status[5m])) 
      > 0.005
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Kong high 5xx error rate"
      description: "Kong 5xx rate is {{ $value | humanizePercentage }} over the last 5 minutes"

This rule uses PromQL to calculate the ratio of 5xx responses to all HTTP responses reported by Kong. The alert fires only if the condition persists for 5 minutes (for: 5m), reducing false positives.

Building Dashboards

Dashboards provide a visual overview of gateway health and performance. They help engineers spot trends, correlate events, and communicate status to stakeholders. Modern observability stacks use tools like Grafana, AWS CloudWatch Dashboards, or Datadog.

Visualizing Key Metrics

A well-designed API gateway dashboard should include:

A request rate graph (count per second) broken down by route or method.
A latency heatmap (p50, p90, p99) with separate lines for total and integration latency.
An error rate panel showing 4xx and 5xx rates as percentage of traffic.
A cache hit ratio gauge (if caching is enabled).
A top-N backend services panel by error count or latency to quickly identify problematic integrations.

Example: Grafana Dashboard Panel for AWS API Gateway

Below is a JSON snippet for a Grafana panel that uses the CloudWatch data source to display the 5xx error rate of an API Gateway stage. The query retrieves both Count and 5XXError metrics and computes the percentage.


{
  "datasource": "CloudWatch",
  "targets": [
    {
      "region": "us-east-1",
      "namespace": "AWS/ApiGateway",
      "metricName": "Count",
      "statistic": "Sum",
      "dimensions": {
        "ApiName": "MyApi",
        "Stage": "prod"
      },
      "period": 300,
      "alias": "TotalRequests"
    },
    {
      "region": "us-east-1",
      "namespace": "AWS/ApiGateway",
      "metricName": "5XXError",
      "statistic": "Sum",
      "dimensions": {
        "ApiName": "MyApi",
        "Stage": "prod"
      },
      "period": 300,
      "alias": "5xxErrors"
    }
  ],
  "fieldConfig": {
    "overrides": [
      {
        "matcher": {"id": "byName", "options": "5xxErrors"},
        "properties": [
          {"id": "custom.transform", "value": "divide($A, $B) * 100"}
        ]
      }
    ]
  },
  "title": "5xx Error Rate (%)",
  "type": "timeseries",
  "gridPos": {"h": 8, "w": 12}
}

In a real dashboard, you would add similar panels for latency, 4xx errors, and cache metrics, arranging them in a logical grid. Use template variables (e.g., $ApiName, $Stage) to make the dashboard reusable across environments.

Best Practices for API Gateway Monitoring

Monitor both gateway and backend integration metrics: A spike in gateway latency may originate from a slow backend. Use distributed tracing (e.g., AWS X-Ray, Jaeger) to correlate gateway and service spans.
Set alarms on anomaly detection: Instead of static thresholds, use CloudWatch anomaly detection or Prometheus forecasting to catch subtle degradations that fixed thresholds miss.
Use dashboards for real-time and historical analysis: Keep a “live” dashboard for on-call engineers and a historical one for weekly performance reviews. Include SLO burn-rate charts.
Automate dashboard deployment: Store dashboard definitions as code (Grafana JSON, CloudFormation, Terraform) and version them alongside your gateway configuration.
Alert on cost and throttling: Unexpected traffic surges can lead to high costs or throttled users. Monitor Throttle metrics and set billing alarms.
Test alarms regularly: Simulate failures in a staging environment to verify that alarms fire and notifications reach the right channels.
Correlate with deployment events: Overlay deployment markers on dashboards to immediately identify if a metric change coincided with a recent release.

Conclusion

Monitoring your API gateway with a comprehensive strategy of metrics, alarms, and dashboards is not optional — it is essential for delivering reliable APIs. By tracking latency, errors, traffic, and cache efficiency, you gain insight into user experience and system health. Alarms transform these metrics into proactive alerts, while dashboards provide the context needed for rapid troubleshooting and capacity planning. The code examples above illustrate how to implement these patterns on AWS and Kong, but the principles apply universally. Invest in continuous refinement of your monitoring setup: adjust thresholds as traffic patterns evolve, add new panels when new services are introduced, and automate everything to reduce manual toil. A well-monitored API gateway becomes a foundation for confident, data-driven operations.