Data Dictionary

Every data source used by the gold/silver strategy suite with module, API, frequency, PIT rules, freshness SLA, fields, and example values.

PIT (Point-in-Time) Rules

All data joins MUST go through qgtm_data/pit.py::pit_join() to prevent lookahead bias.

Core rule: For each row at time $t$, only data with knowledge_time $\leq t$ is matched.

Data Class	PIT Rule	Rationale
FRED macro	T+1 business day	FRED publishes with 1-day lag
CFTC COT	T+3 calendar days (Friday report, Tuesday data)	Published Friday, reflects Tuesday positions
COMEX warehouse	T+1 business day	Published next business day
SGE withdrawals	T+5 calendar days	Weekly with publication lag
ETF flows	T+0 (same day)	Shares outstanding available real-time
Options chain	T+0 (real-time)	Market data, no lag
LBMA fix	T+0 (post-fix)	Published immediately after fixing
Central bank reserves	T+90 days (quarterly)	IMF/WGC publishes with 1-quarter lag

Data Sources

1. FRED Macro Series

Module: qgtm_data/fred.py Provider: Federal Reserve Economic Data (FRED) API: https://api.stlouisfed.org/fred Auth: API key (FRED_API_KEY)

Series ID	Field	Frequency	Description	Example
`DFII10`	`real_10y`	Daily	10Y TIPS real yield	1.85%
`T10YIE`	`breakeven_10y`	Daily	10Y breakeven inflation	2.35%
`T5YIE`	`breakeven_5y`	Daily	5Y breakeven inflation	2.20%
`DTWEXBGS`	`dxy`	Daily	Trade-weighted USD (broad)	104.5
`FEDFUNDS`	`fed_funds`	Monthly	Federal funds rate	5.33%
`VIXCLS`	`vix`	Daily	CBOE VIX	18.5
`GOLDAMGBD228NLBM`	`gold_fixing`	Daily	Gold London AM fixing	$2,350.00
`WALCL`	`fed_balance_sheet`	Weekly	Fed total assets	$7.4T
`TEDRATE`	`ted_spread`	Daily	TED spread	0.25%

PIT rule: T+1 business day. Freshness SLA: < 24 hours for daily series.

Strategies consuming: A.1 (real rate), A.2 (DXY), A.3 (breakeven), A.4 (VIX)

2. CFTC Commitments of Traders

Module: qgtm_data/cftc.py, qgtm_data/cot_reports.py Provider: CFTC via Nasdaq Data Link (Quandl) API: https://data.nasdaq.com/api/v3 Auth: Quandl API key

COT Code	Metal	Report Type	Fields
`CFTC/088691_FO_ALL`	Gold	Disaggregated	MM long/short, commercial long/short, OI
`CFTC/084691_FO_ALL`	Silver	Disaggregated	MM long/short, commercial long/short, OI
`CFTC/076651_FO_ALL`	Platinum	Disaggregated	MM long/short, commercial long/short, OI

Key fields:

Field	Type	Description	Example
`report_date`	date	Tuesday of report week	2026-04-07
`mm_long`	int	Managed-money long contracts	215,432
`mm_short`	int	Managed-money short contracts	45,210
`commercial_long`	int	Commercial long contracts	180,000
`commercial_short`	int	Commercial short contracts	320,000
`open_interest`	int	Total open interest	550,000

PIT rule: T+3 calendar days (data as-of Tuesday, published Friday). Freshness SLA: Updated weekly by Saturday.

Strategies consuming: D.14 (COT extreme), D.15 (hedging pressure)

3. COMEX Warehouse Stocks

Module: qgtm_data/comex.py Provider: CME Group via Nasdaq Data Link Frequency: Daily

Field	Type	Description	Example
`metal`	str	"gold" or "silver"	"silver"
`date`	date	Report date	2026-04-11
`registered_oz`	float	Metal earmarked for delivery	35,200,000 oz
`eligible_oz`	float	Metal stored but not designated	280,000,000 oz
`total_oz`	float	registered + eligible	315,200,000 oz

PIT rule: T+1 business day. Freshness SLA: < 24 hours.

Strategies consuming: D.17 (COMEX warehouse), B.9 (backwardation stress)

4. ETF Flow Data

Module: qgtm_data/etf_flows.py Provider: ETF provider APIs, Yahoo Finance Frequency: Daily

Field	Type	Description	Example
`ticker`	str	ETF symbol	"GLD"
`date`	date	Trade date	2026-04-11
`shares_outstanding`	float	Total shares	236,500,000
`shares_delta`	float	Daily change	+1,200,000
`implied_oz`	float	Physical metal equivalent	+120,000 oz
`nav`	float	Net asset value	$216.50

Tracked ETFs: GLD, SLV, IAU, SIVR, SGOL, GDX, GDXJ

PIT rule: T+0 (same day). Freshness SLA: < 1 hour after market close.

Strategies consuming: D.16 (ETF flow momentum)

5. LBMA Fix & Vault Data

Module: qgtm_data/lbma.py Provider: LBMA via Nasdaq Data Link Frequency: Daily (fix), Monthly (vaults)

Field	Type	Description	Example
`metal`	str	"gold" or "silver"	"gold"
`date`	date	Fix date	2026-04-11
`am_fix_usd`	float	AM London fix	$2,355.50
`pm_fix_usd`	float	PM London fix	$2,358.25
`vault_holdings_oz`	float	London vault gold (monthly)	278,000,000 oz
`clearing_volume_oz`	float	Daily LBMA clearing	18,200,000 oz

PIT rule: T+0 (post-fix). Freshness SLA: < 30 minutes after fix.

Strategies consuming: F.24 (fix dislocation), B.9 (backwardation stress)

6. Central Bank Reserves

Module: qgtm_data/central_banks.py Provider: IMF IFS API, World Gold Council Frequency: Monthly (IMF), Quarterly (WGC)

Field	Type	Description	Example
`country`	str	Country name	"China"
`date`	date	Report month	2026-01-01
`tonnes`	float	Official gold holdings	2,330.0
`change_tonnes`	float	Change from prior month	+15.0
`pct_of_reserves`	float	Gold share of FX reserves	5.2%

PIT rule: T+90 days (quarterly reporting lag). Freshness SLA: Quarterly update.

Strategies consuming: A.5 (central bank accumulation)

7. Options Chain & Volatility

Module: qgtm_data/options.py Provider: Alpaca Options API, FRED (GVZ) Frequency: Real-time (intraday), Daily snapshots

Field	Type	Description	Example
`underlying`	str	ETF symbol	"GLD"
`expiry`	date	Option expiration	2026-05-16
`strike`	float	Strike price	$235.00
`option_type`	str	"call" or "put"	"put"
`iv`	float	Implied volatility	0.185
`delta`	float	Delta	-0.25
`gamma`	float	Gamma	0.018
`theta`	float	Theta	-0.045
`vega`	float	Vega	0.32
`volume`	int	Daily volume	3,200
`open_interest`	int	Open interest	15,800
`iv_percentile`	float	252-day IV percentile	0.72

FRED volatility indices:

Series	Description	Example
`GVZCLS`	CBOE Gold ETF Volatility Index (GVZ)	18.5

PIT rule: T+0 (real-time). Freshness SLA: < 1 second (market data).

Strategies consuming: E.19 (VRP), E.20 (skew), E.21 (gamma scalp), E.22 (vol term structure)

8. SGE Withdrawal Data

Module: qgtm_data/ (web scraper / manual entry) Provider: Shanghai Gold Exchange, BullionStar Frequency: Weekly

Field	Type	Description	Example
`week_ending`	date	Week end date	2026-04-11
`withdrawals_tonnes`	float	Weekly physical withdrawal	42.5
`premium_usd`	float	Shanghai-London premium	$15.20/oz

PIT rule: T+5 calendar days (weekly publication lag). Freshness SLA: Weekly.

Strategies consuming: D.18 (SGE withdrawals)

9. Physical Premiums

Module: qgtm_data/refinery_premiums.py Provider: Various (SGE, Indian bullion dealers, CME) Frequency: Daily

Field	Type	Description	Example
`market`	str	Regional market	"shanghai"
`metal`	str	"gold" or "silver"	"gold"
`date`	date	Observation date	2026-04-11
`premium_usd_oz`	float	Premium over London	$12.50
`premium_pct`	float	Percentage premium	0.53%

PIT rule: T+1 business day. Freshness SLA: < 24 hours.

Strategies consuming: D.18 (SGE), F.25 (overnight)

10. Government Mint Sales

Module: qgtm_data/mint_sales.py Provider: US Mint, Perth Mint, RCM public reports Frequency: Monthly

Field	Type	Description	Example
`mint`	str	Mint identifier	"us_mint"
`metal`	str	"gold" or "silver"	"gold"
`date`	date	Month (first of month)	2026-04-01
`coins_oz`	float	Troy ounces in coin form	85,000
`bars_oz`	float	Troy ounces in bar form	12,500

PIT rule: T+10 days (monthly publication lag). Freshness SLA: Monthly.

Strategies consuming: Contrarian indicator overlay

Data Quality Checks

Every data fetch passes through validation before entering the feature store:

Check	Implementation	Failure Action
Staleness	Compare latest timestamp to SLA	Alert + use stale data with flag
Nulls	Count null fields per fetch	Reject if > 5% null in critical fields
Range	Validate against historical bounds	Flag outliers > 5 sigma
Duplicates	Check for duplicate timestamps	Deduplicate, keep latest
Integrity	Cross-check LBMA fix vs spot price	Alert if divergence > 1%

Entity Relationship

erDiagram
    FRED_MACRO ||--o{ STRATEGY : feeds
    CFTC_COT ||--o{ STRATEGY : feeds
    COMEX_WAREHOUSE ||--o{ STRATEGY : feeds
    ETF_FLOWS ||--o{ STRATEGY : feeds
    LBMA_FIX ||--o{ STRATEGY : feeds
    CB_RESERVES ||--o{ STRATEGY : feeds
    OPTIONS_CHAIN ||--o{ STRATEGY : feeds
    SGE_WITHDRAWALS ||--o{ STRATEGY : feeds
    REFINERY_PREMIUMS ||--o{ STRATEGY : feeds
    MINT_SALES ||--o{ STRATEGY : feeds

    STRATEGY {
        string strategy_id PK
        string category
        string module
        string rebalance_freq
    }
    FRED_MACRO {
        string series_id PK
        date date
        float value
    }
    CFTC_COT {
        string cot_code PK
        date report_date
        int mm_long
        int mm_short
    }
    COMEX_WAREHOUSE {
        string metal PK
        date date
        float registered_oz
        float eligible_oz
    }