Skip to content

Kill Switch Runbook

When to Use

  • Daily PnL loss exceeds circuit breaker threshold (default: -3%)
  • Weekly PnL loss exceeds threshold (default: -5%)
  • Broker API degradation or outage
  • Unexpected market conditions (flash crash, circuit breakers)
  • Data feed corruption detected
  • Any situation where continuing to trade poses unacceptable risk

Automated Kill Switch

The live trading daemon (qgtm_live) automatically triggers kill switch when: 1. Daily loss > CIRCUIT_BREAKER_DAILY_LOSS (default: 3%) 2. Weekly loss > CIRCUIT_BREAKER_WEEKLY_LOSS (default: 5%) 3. Heartbeat from data feed missing > 60 seconds 4. Order rejection rate > 20% in rolling window 5. Latency spike > 5x normal for > 30 seconds

Manual Kill Switch

# Option 1: Via CLI (recommended)
doppler run --config production -- python -m qgtm_live.kill_switch

# Option 2: Via API
curl -X POST https://api.qgtm.ai/admin/kill-switch \
  -H "Authorization: Bearer $ADMIN_TOKEN"

# Option 3: Via Alpaca directly (nuclear option)
# Cancel all orders and close all positions via Alpaca dashboard

What the Kill Switch Does

  1. Cancels all open orders — immediately, all symbols
  2. Flattens all positions — market orders to close everything
  3. Disables live trading flag — sets QGTM_LIVE_TRADING_ENABLED=false
  4. Sends alerts — Discord, Telegram, email to all admins
  5. Logs everything — structured log with full portfolio state at trigger time

Recovery

  1. Investigate root cause
  2. Verify market conditions are stable
  3. Review all positions reconciled correctly
  4. Re-enable trading flag (requires two-person approval)
  5. Start with reduced position sizes (50% for first 24h)