Chapter 10: The Signal Pipeline

29 strategies fire simultaneously. This chapter explains how their outputs become a single, risk-managed portfolio.

Reading time: 22 min | Difficulty: Advanced | Prerequisites: Chapters 5-6, 9

The Pipeline at a Glance

  29 STRATEGIES                AGGREGATION              PORTFOLIO
  ┌─────────────┐            ┌──────────────┐         ┌──────────────┐
  │ TSMOM (3)   │──┐        │              │         │              │
  │ Mean Rev (4)│──┤        │  Inverse-Vol │         │  Regime-     │
  │ Carry (3)   │──┤        │  × Skill     │         │  Conditional │
  │ Vol Arb (4) │──┤───────▶│  × Capacity  │───────▶ │  Optimizer   │
  │ Macro (4)   │──┤        │  × Meta-Label│         │              │
  │ Seasonal (3)│──┤        │              │         │              │
  │ Micro (4)   │──┤        └──────────────┘         └──────┬───────┘
  │ Stat Arb (4)│──┘                                        │
  └─────────────┘                                           ▼
                                                    ┌──────────────┐
  COMPLIANCE              EXECUTION                 │  Vol Target  │
  ┌──────────────┐       ┌──────────────┐          │  10-18%      │
  │ Wash Sale    │       │ Vehicle      │          │  Leverage    │
  │ PDT Check    │◀──────│ Routing      │◀─────────│  Scaling     │
  │ Position Lim │       │ (GLD/GC/XAU) │          │              │
  └──────────────┘       └──────────────┘          └──────────────┘
        │                       │
        ▼                       ▼
  ┌──────────────┐       ┌──────────────┐
  │ REJECT or    │       │ ORDER SENT   │
  │ FLAG         │       │ TO BROKER    │
  └──────────────┘       └──────────────┘

Every market bar (daily close, or intraday for certain strategies), all 29 strategies evaluate simultaneously. Each emits a signal object containing: direction (+1 long, -1 short, 0 flat), raw conviction score, strategy-specific metadata, and a timestamp. These raw signals then pass through four transformation stages before becoming executable orders.

Stage 1: How 29 Strategies Generate Signals

The strategies are grouped into 8 families. Within each family, strategies share a thesis but differ in parameterization, lookback window, or instrument focus.

Family	Count	Core Thesis	Typical Holding Period
Trend / TSMOM	3	Price momentum persists across horizons	1-6 months
Mean Reversion	4	Gold/silver ratio, RSI extremes, basis reversion	3-15 days
Carry	3	Roll yield, convenience yield, lease rate differentials	1-3 months
Volatility Arbitrage	4	IV-RV spread, term structure, skew trades	1-30 days
Macro / Regime	4	Real rates, DXY, central bank flows, risk appetite	1-6 months
Seasonal	3	Calendar effects, Indian wedding season, COMEX delivery	2-6 weeks
Microstructure	4	Order flow, COT positioning, ETF flow imbalance	1-10 days
Statistical Arbitrage	4	Cross-metal pairs, miner-metal spread, inter-market	3-20 days

Why 29? Not because more is always better. Each strategy survived a gauntlet: out-of-sample Sharpe > 0.4, walk-forward validation across 3 regimes, CPCV (combinatorial purged cross-validation) to confirm it is not overfit, and a positive meta-label probability. Strategies that fail any gate are removed. The count fluctuates — it was 24 six months ago, it might be 32 next quarter.

Signal Generation: The Inner Loop

Every strategy follows the same interface:

class BaseStrategy:
    def generate_signal(self, features: DataFrame) -> Signal:
        """
        Input:  Feature matrix (latest bar + history)
        Output: Signal(direction, conviction, metadata)

        direction:  +1 (long), -1 (short), 0 (flat)
        conviction: 0.0 to 1.0 (raw, pre-aggregation)
        metadata:   strategy-specific context
        """

Strategies do NOT know about each other. They do not know the current portfolio. They receive features, they emit signals. The aggregation layer handles everything else.

Stage 2: Signal Aggregation

Raw signals from 29 strategies are combined into a single portfolio view per instrument. The aggregation formula:

W_i = (1/σ_i) × S_i × C_i × M_i

Where for strategy i:

1/σ_i — Inverse volatility weight. Strategies with lower return volatility get higher weight. This is the foundation. A trend strategy with 8% annual vol contributes more per unit of conviction than a stat-arb strategy with 25% vol.
S_i — Skill score. An adaptive weight based on recent out-of-sample performance. Ranges from 0.1 (strategy is cold, barely contributing) to 2.0 (strategy is hot, contribution amplified). Updated weekly using a Bayesian skill estimator with a 63-day half-life.
C_i — Capacity factor. Prevents strategies from requesting more capital than the underlying instrument can absorb. A GLD momentum strategy might request $500K notional easily (GLD trades $1B+/day), but a silver micro-structure strategy on a thinly traded options expiry might cap at $50K.
M_i — Meta-label confidence. The output of a gradient-boosted classifier that predicts whether this specific signal will be profitable, conditioned on the current regime, recent strategy performance, and cross-strategy agreement. Ranges from 0.0 (model says skip) to 1.0 (model says strong conviction). Signals with M_i < 0.3 are discarded entirely.

Aggregation Example

Say three strategies are long gold:

Strategy	Direction	σ_i	S_i	C_i	M_i	W_i
TSMOM-252	+1	0.12	1.4	1.0	0.82	9.57
Macro-RealRate	+1	0.15	1.1	1.0	0.71	5.20
MeanRev-RSI	-1	0.20	0.8	1.0	0.44	-1.76

Net aggregated signal: +9.57 + 5.20 - 1.76 = +12.01 (strongly net long)

The system normalizes this to a target volatility (next section), but the direction is clear: the preponderance of evidence says go long gold, with trend and macro dominating and a weak mean-reversion dissent.

Stage 3: Regime-Conditional Allocation

The aggregated signals feed into an optimizer, but which optimizer depends on the current market regime.

Regime Detection

The system maintains two regime classifiers running in parallel:

HMM (Hidden Markov Model) — 4-state model fitted on gold returns, equity vol (VIX), and real rates. States: risk-on, risk-off, crisis, transition. Updated daily. Provides smooth state probabilities.
BOCPD (Bayesian Online Change-Point Detection) — Monitors for abrupt structural breaks. Does not classify states — it just says "something changed" with a probability. When BOCPD fires (change-point probability > 0.7), the system temporarily increases uncertainty and widens risk bands.

Optimizer Selection

Regime State	Optimizer	Rationale
Risk-On	Mean-variance (Markowitz) with shrinkage	Markets behaving normally. Standard optimization works. Shrinkage covariance (Ledoit-Wolf) prevents extreme weights.
Risk-Off	Hierarchical Risk Parity (HRP)	Correlations unstable. HRP is more robust to estimation error in the covariance matrix. Produces more diversified allocations.
Crisis	Equal risk contribution with vol scaling	Everything correlates to 1 in a crisis. Minimize concentration. Cut overall exposure by 50%.
Transition	Blend of Risk-On and Risk-Off optimizers	Weighted by HMM state probabilities. Smooth transition between regimes.

Why this matters: Running mean-variance in a crisis is how funds blow up. Correlations estimated from calm periods are garbage during stress. The regime-conditional approach means the system gets more conservative exactly when it needs to.

Stage 4: Vol Targeting and Leverage Scaling

The optimizer outputs raw portfolio weights. These are then scaled to hit a target volatility band.

The 10-18% Band

The portfolio targets 14% annualized volatility with tolerance between 10% and 18%.

Below 10%: The system is leaving money on the table. Scale up.
10-18%: Normal operating range. No adjustment needed.
Above 18%: Excessive risk. Scale down positions proportionally.

How It Works

Target vol:        σ_target = 14%
Realized vol:      σ_realized = estimated from trailing 21-day returns
Scaling factor:    λ = σ_target / σ_realized
Leverage applied:  Position sizes multiplied by λ

Constraints: - Maximum leverage: 1.5x (we will never be more than 150% invested) - Minimum leverage: 0.3x (floor — never go below 30% invested even in high-vol) - Adjustment speed: Maximum 20% change per day to avoid whipsaw

Example: If realized vol spikes to 28% (gold flash crash), the scaling factor drops to 14/28 = 0.5x. All positions are halved over 2-3 days (the 20% daily adjustment cap prevents a single-day fire sale).

Why a Band, Not a Point Target

A hard 14% target would cause constant rebalancing. The band creates a "dead zone" where the system does nothing, reducing transaction costs and avoiding churn. The system only acts when vol breaches the 10% or 18% boundaries.

Stage 5: Vehicle Routing

The same "long gold" view can be expressed through multiple instruments. The vehicle router picks the cheapest, most liquid option:

Vehicle	Exchange	Typical Spread	Commission	Min Lot	Best For
GLD (ETF)	NYSE	$0.01	$0/trade	1 share	Small positions, tax lots
IAU (ETF)	NYSE	$0.01	$0/trade	1 share	Slightly lower expense ratio
GC (futures)	COMEX	1 tick ($10)	~$2.50/rt	1 contract ($260K)	Large positions, margin efficiency
MGC (micro futures)	COMEX	1 tick ($1)	~$1.50/rt	1 contract ($26K)	Medium positions
XAU/USD (spot FX)	OTC	0.3-0.5 pip	Spread only	Flexible	24hr access, leverage
GLD options	CBOE	$0.02-0.10	$0.65/contract	1 contract	Defined risk, vol expression

Routing Logic

The router evaluates each vehicle on:

Cost efficiency: Spread + commission + expected slippage for the target size
Capacity: Can the vehicle absorb the order without moving the market? (Measured by average daily volume in dollars)
Margin efficiency: Futures offer 5-10x leverage vs. ETFs at 1x
Tax treatment: Futures get 60/40 long-term/short-term treatment. ETFs are fully short-term if held < 1 year
Market hours: If a signal fires at 2 AM ET, only futures and spot FX are available

For a typical $250K account, the router usually picks GLD for positions under $50K notional and micro gold futures (MGC) for larger positions.

Stage 6: Drawdown Management

The drawdown manager sits between the optimizer and the execution engine. It overrides position sizes when the account is in drawdown.

Four Phases

NORMAL            SOFT CAP           HARD CAP           RE-ENTRY
(DD < 5%)         (DD 5-8%)          (DD 8-12%)         (After hard cap)

Full position     75% sizing         Flatten all         50% sizing
sizing            No new positions   24h cool-off        Scale up 10%/day
All strategies    in losing strats   Full audit          over 5 days
active            Winners can add

Phase 1: Normal (drawdown < 5% from peak) Business as usual. Full position sizing. All strategies active.

Phase 2: Soft Cap (drawdown 5-8%) Position sizes reduced to 75% of target. Strategies that are contributing to the drawdown (negative P&L attribution) cannot add to existing positions. Strategies that are profitable can still initiate new trades. OpA is alerted. No action required unless it persists > 3 days.

Phase 3: Hard Cap (drawdown 8-12%) All positions flattened. Trading halted for 24 hours minimum. Automated incident report generated. OpA + OpB must both confirm re-entry (two-person rule). This is the system protecting you from yourself.

Phase 4: Re-Entry (after hard cap and cool-off) Trading resumes at 50% position sizing. Size increases by 10% of target per day, reaching full size after 5 trading days. If drawdown exceeds 5% again during re-entry, the system drops back to Phase 2.

Gradual Re-Entry: Why It Matters

The worst time to re-enter at full size is right after a drawdown. The market conditions that caused the loss may still be present, your regime classifier may be lagging, and the strategies that just got stopped out are statistically more likely to get stopped out again in the same regime. Gradual re-entry protects against "the second punch."

Stage 7: Compliance Gateway

Every order passes through compliance checks before reaching the broker. These are hard gates — a failure means the order is rejected, not just flagged.

Pre-Trade Checks

Check	Rule	Consequence of Violation
Wash Sale	No buying back a security within 30 days of selling at a loss (for tax lots)	Order rejected. System suggests waiting or using a different vehicle.
Pattern Day Trader	Accounts under $25K cannot execute 4+ day trades in 5 business days	Order rejected. Counter shows remaining day trades.
Position Limits	No single position > 20% of account NAV	Order size capped at 20% limit.
Sector Concentration	No more than 60% in precious metals (PM-only system allows higher)	Warning logged. Hard cap at 80% across all PM instruments.
Order Size	No single order > 5% of average daily volume	Order split across multiple fills.
Margin Check	Sufficient margin for the proposed position	Order rejected if margin insufficient.
Restricted List	Check against manually maintained restricted securities list	Order rejected with reason.

Wash Sale Handling

The wash sale check is the most complex. The system maintains a rolling 61-day window (30 days before and after each sale) and tracks across all vehicles. If you sell GLD at a loss, you cannot buy GLD, IAU, or gold futures within 30 days without triggering a wash sale. The router accounts for this — if GLD is wash-sale restricted, it routes to a non-substantially-identical vehicle or delays the trade.

The Order Lifecycle

Signal Generated
    → Aggregation (combine 29 strategies)
    → Optimization (regime-conditional)
    → Vol scaling (hit target band)
    → Vehicle routing (pick cheapest instrument)
    → Drawdown check (are we in a cap phase?)
    → Compliance gateway (7 pre-trade checks)
    → Order submission to broker
    → Fill confirmation
    → State update (positions, P&L, risk)
    → Monitoring (Grafana, alerts)

Every step is logged with timestamps, inputs, outputs, and the decision made. If something goes wrong six months later, you can reconstruct exactly why every order was placed.

Performance Attribution

After execution, the system continuously attributes P&L back to its sources:

Strategy attribution: Which of the 29 strategies contributed what portion of today's/week's/month's P&L?
Factor attribution: How much came from gold beta, silver beta, momentum factor, carry factor, vol factor?
Execution attribution: How much was lost to slippage, commissions, and timing (signal lag)?
Regime attribution: What was the P&L in each regime state?

This closes the loop. Strategy attribution feeds back into skill scores. Execution attribution feeds back into vehicle routing preferences. The system learns and adapts.

Next: Chapter 11 — Gold/Silver Trading Glossary →