The Silent Killer: Expired API Keys in Production
When an API key expires, the failure mode is uniquely dangerous:
- No error at deploy time. Your application starts fine. The config variable exists. The key is just a string — it doesn't validate itself.
- Delayed discovery. The first request fails with a 401. Maybe it's a background job that runs once an hour. Maybe it's a webhook callback that nobody monitors. The failure is silent until a customer complains.
- Unclear root cause. 401 could mean a lot of things. Wrong scope, wrong endpoint, IP restriction, or an expired key. You check the provider dashboard, find the key was rotated 3 days ago, and realize your production config is stale.
- Manual cleanup. You generate a new key, update the environment variable, redeploy. Then you check every other service that might use the same key. It takes hours.
The fix is simple: audit your credentials before they expire. APIAuth audit does this in one command.
Quick Start: Your First Credential Audit
Step 1: Install APIAuth
pip install apiauth
Step 2: Generate Your First Key
# Generate an API key for a service
apiauth generate api-key --name "Stripe Production" --service "stripe" --expiry-days 90
# Output:
# ✓ Key generated: ak_xYz123abc...
# ID: k_7f3a2b
# Name: Stripe Production
# Service: stripe
# Expires: 2026-08-25
#
# ⚠ Save this key now — it won't be shown again.
Security model: The plaintext key is shown once at creation time. After that, APIAuth stores only the SHA-256 hash. The verify command checks incoming keys against these hashes — no plaintext ever lives in the keystore.
Step 3: Add More Keys
# API key with 30-day expiry
apiauth generate api-key --name "SendGrid" --service "sendgrid" --expiry-days 30
# JWT with custom claims
apiauth generate jwt --name "Auth Service Token" --service "auth" --expiry-days 60 --claim role=admin
# Import an existing key
apiauth import sk_live_abc123 --name "Existing Stripe Key" --service "stripe" --expiry-days 15
Step 4: Run the Audit
apiauth audit
# Healthy output:
# ✓ All 3 keys are healthy
# Problem output:
# ✗ 1 EXPIRED key(s):
# k_a1b2c3 Existing Stripe Key — expired 2026-05-20
#
# ⚠ 1 EXPIRING key(s) (within 7 days):
# k_d4e5f6 SendGrid — expires 2026-05-29
#
# ⊘ 0 REVOKED key(s)
Three categories, three severity levels:
| Status | Meaning | Required Action | Urgency |
|---|---|---|---|
| EXPIRED | Key's expiry date has passed. The provider will reject it. | Rotate immediately | Critical — production impact |
| EXPIRING | Key expires within 7 days | Schedule rotation this week | High — rotate before it breaks |
| REVOKED | Key was manually revoked via apiauth revoke |
Remove from configs, verify not in use | Cleanup — already handled |
| HEALTHY | Key is valid, not expired, not revoked | No action needed | None |
How the Audit Works
The apiauth audit command iterates over every key in your keystore and checks three conditions:
- Revocation check: Has the key been explicitly revoked with
apiauth revoke? Revoked keys are permanently invalidated — they can't be un-revoked. - Expiry check: Is the key's expiry date in the past? If so, the key is expired regardless of whether the provider has actually deactivated it.
- Expiring check: Does the key expire within 7 days? This is your early warning window.
Keys that pass all three checks are marked healthy.
The 7-day window is intentional. Most key rotation workflows require approval, deployment, and verification. A 7-day warning gives your team enough time to schedule the rotation without panic. If you need more lead time, run apiauth list and check the expiry dates manually.
Verifying Incoming API Keys
The audit command checks your own keystore. The verify command does the opposite — it checks an incoming API key against your stored hashes:
# Verify a key from an incoming request
apiauth verify ak_xYz123abc...
# Valid key:
# ✓ Key k_7f3a2b is VALID
# Name: Stripe Production
# Service: stripe
# Version: 2
# Rate limit: 1000 req/s
# Invalid key:
# ✗ Key is INVALID
# Key not found in keystore
# Expired key:
# ✗ Key k_a1b2c3 is EXPIRED
# Revoked key:
# ✗ Key k_d4e5f6 is REVOKED
The verify command checks three things:
- Existence: Is the key's SHA-256 hash in the keystore?
- Revocation: Has the key been revoked?
- Expiry: Has the key's expiry date passed?
If all three pass, the key is valid. The response includes the key's name, service, version, and rate limit (if configured).
JSON Output for Programmatic Verification
apiauth verify ak_xYz123abc... --json-output
{
"status": "valid",
"id": "k_7f3a2b",
"name": "Stripe Production",
"service": "stripe",
"version": 2,
"rate_limit": 1000
}
Use this in your API middleware to verify keys on every request:
# Python FastAPI example
import subprocess, json
def verify_api_key(api_key: str) -> dict:
result = subprocess.run(
["apiauth", "verify", api_key, "--json-output"],
capture_output=True, text=True
)
if result.returncode != 0:
return {"status": "invalid"}
return json.loads(result.stdout)
# In your middleware
@app.middleware("http")
async def verify_key(request: Request, call_next):
api_key = request.headers.get("X-API-Key")
if not api_key:
return JSONResponse(status_code=401, content={"error": "Missing API key"})
result = verify_api_key(api_key)
if result.get("status") != "valid":
return JSONResponse(status_code=401, content={"error": "Invalid API key"})
# Attach key metadata to request state
request.state.key_id = result["id"]
request.state.key_service = result["service"]
request.state.key_rate_limit = result.get("rate_limit")
response = await call_next(request)
return response
Three CI/CD Credential Health Patterns
Pattern 1: Pre-Deploy Audit Gate
Run the audit before every deployment. If any key is expired, the deploy fails:
# .github/workflows/credential-audit.yml
name: Credential Audit
on: [pull_request]
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install apiauth
- name: Audit credentials
run: |
# Restore keystore from encrypted artifact
# (your team's method of sharing the keystore)
mkdir -p ~/.apiauth
# ... restore keystore.enc and master.key ...
# Run audit — check if any keys are expired
OUTPUT=$(apiauth audit 2>&1)
echo "$OUTPUT"
# Fail if any expired keys found
if echo "$OUTPUT" | grep -q "EXPIRED"; then
echo "::error::Expired API keys found — rotate before deploying"
exit 1
fi
Pattern 2: Scheduled Weekly Audit
Run the audit on a schedule and create issues for expiring keys:
# .github/workflows/weekly-credential-check.yml
name: Weekly Credential Check
on:
schedule:
- cron: '0 9 * * 1' # Monday 9 AM
workflow_dispatch:
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install apiauth
- name: Audit and report
run: |
# Restore keystore
# ...
OUTPUT=$(apiauth audit 2>&1)
EXPIRED=$(echo "$OUTPUT" | grep -c "EXPIRED" || true)
EXPIRING=$(echo "$OUTPUT" | grep -c "EXPIRING" || true)
if [ "$EXPIRED" -gt 0 ] || [ "$EXPIRING" -gt 0 ]; then
BODY="## Credential Audit Report\n\n\`\`\`\n$OUTPUT\n\`\`\`"
gh issue create \
--title "Credential audit: $EXPIRED expired, $EXPIRING expiring" \
--body "$BODY" \
--label "security,credentials"
fi
Pattern 3: Key Verification in API Gateway
Use apiauth verify as a lightweight API key validation layer in your gateway:
# nginx + Lua example (OpenResty)
# Verify API key before proxying to backend
location /api/ {
access_by_lua_block {
local api_key = ngx.req.get_headers()["X-API-Key"]
if not api_key then
ngx.exit(401)
end
local handle = io.popen("apiauth verify " .. api_key .. " --json-output")
local result = handle:read("*a")
handle:close()
local data = cjson.decode(result)
if data.status ~= "valid" then
ngx.exit(401)
end
-- Pass key metadata to backend
ngx.req.set_header("X-Key-ID", data.id)
ngx.req.set_header("X-Key-Service", data.service)
}
proxy_pass http://backend;
}
The Encrypted Keystore: How APIAuth Protects Your Keys
APIAuth never stores plaintext keys. Here's the security model:
At Generation Time
- The key is generated with
secrets.token_urlsafe()(cryptographically secure random) - The plaintext key is displayed once — this is your only chance to copy it
- The key is hashed with SHA-256 and stored in the encrypted keystore
- The plaintext is discarded from memory
At Import Time
- You provide the existing key via CLI argument
- The key is hashed with SHA-256
- Only the hash is stored — the original value is never saved
- You can still
verifyincoming keys against the hash
At Verification Time
- You provide the key to verify
- APIAuth hashes it with SHA-256
- The hash is compared against stored hashes in the keystore
- If a match is found, revocation and expiry are checked
- The result is returned — no plaintext ever leaves the keystore
Encryption Layer
The entire keystore file is encrypted with AES-256-GCM:
~/.apiauth/
├── master.key # AES-256-GCM master key (never share this)
├── keystore.enc # Encrypted key-value store
└── config.yaml # User configuration
master.key— 256-bit AES key, generated on first run, never leaves~/.apiauth/keystore.enc— All key metadata (hashes, names, services, expiry dates) encrypted at restconfig.yaml— Non-sensitive user preferences
Never commit master.key to version control. Add it to .gitignore immediately. If the master key is compromised, an attacker can decrypt the entire keystore. If you lose it, you cannot recover stored hashes — you'll need to re-import all keys.
Full Credential Lifecycle with APIAuth
The audit and verify commands are part of a complete key lifecycle:
| Phase | Command | What It Does |
|---|---|---|
| Generate | apiauth generate api-key |
Create a new key, show it once, store the hash |
| Generate JWT | apiauth generate jwt |
Create a JWT with custom claims, store the hash |
| Import | apiauth import |
Hash an existing key and store it |
| Verify | apiauth verify |
Check an incoming key against stored hashes |
| Rotate | apiauth rotate |
Generate a replacement, hash out the old value |
| Revoke | apiauth revoke |
Permanently invalidate a key |
| Audit | apiauth audit |
Scan for expired, expiring, and revoked keys |
| Export | apiauth export |
Output keys in env, dotenv, JSON, or GitHub Actions format |
| List | apiauth list |
Show all keys with expiry status indicators |
| Stats | apiauth stats |
Keystore statistics (key count, services, health) |
Export Formats for CI/CD Integration
The export command outputs your keys in the format your pipeline needs:
Shell Environment Variables
apiauth export --format env --service stripe
# Output:
export STRIPE_API_KEY=ak_xYz123abc...
export STRIPE_WEBHOOK_SECRET=whsec_abc123...
Dotenv File
apiauth export --format dotenv > .env
# .env contents:
STRIPE_API_KEY=ak_xYz123abc...
SENDGRID_API_KEY=sg_abc123...
GitHub Actions
# In your workflow
- name: Export API keys
run: |
apiauth export --format github-actions --service production
# This writes to $GITHUB_ENV so subsequent steps can use:
# ${{ env.STRIPE_API_KEY }}, ${{ env.SENDGRID_API_KEY }}, etc.
JSON (Programmatic)
apiauth export --format json --service stripe
{
"STRIPE_API_KEY": "ak_xYz123abc...",
"STRIPE_WEBHOOK_SECRET": "whsec_abc123..."
}
Real-World Scenario: Multi-Service Startup
Your startup uses 8 API services: Stripe, SendGrid, Twilio, AWS, OpenAI, GitHub, Datadog, and PagerDuty. Each has at least 2 keys (production + staging). That's 16 keys, each with different expiry schedules.
Without APIAuth, you discover expired keys when things break. With APIAuth:
# Generate keys for all services
apiauth generate api-key --name "Stripe Prod" --service "stripe" --expiry-days 90
apiauth generate api-key --name "Stripe Staging" --service "stripe" --expiry-days 180
apiauth generate api-key --name "SendGrid Prod" --service "sendgrid" --expiry-days 90
apiauth generate api-key --name "Twilio Prod" --service "twilio" --expiry-days 365
# ... and so on
# Quick health check
apiauth audit
# List by service
apiauth list --service stripe
# Export production keys for CI
apiauth export --format github-actions --service production
# Weekly cron: audit + create rotation tickets
apiauth audit # → finds 2 expiring keys
apiauth rotate k_expiring1 # → generates replacement, invalidates old
apiauth rotate k_expiring2
Install APIAuth
pip install apiauth
# Generate your first key
apiauth generate api-key --name "My First Key" --service "api-gateway" --expiry-days 90
# Check keystore health
apiauth audit
# View all keys
apiauth list
Star APIAuth on GitHub
Related Reading
- Zero-Downtime API Key Rotation in CI/CD — automate key rotation
- Rotate Keys Across Environments — Envault + APIAuth combo
- Block Deployments on Config Drift — ConfigDrift CI gating
- Fail CI on Dead Code — DeadCode CI gating