openInvest Parameter Tuning Feasibility Diagnosis
Read-only assessment, no code changes. 2026-05-27. Supplementary Document (Coupling Analysis / Do-Not-Tune / Optuna Review / Type B Breakdown): 13-param-tuning-feasibility-addendum.md
1. Parameter Inventory
1.1 Regime Classification Thresholds (core/regime.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
trend_ma_spread_pct | core/regime.py:37 | 3.0% (default) | MA20 vs MA120 deviation of 3% indicates a trend. No commit/ADR record, empirical initial value | Quant prompt strategy hints, Dreaming regime bucketing, verdict_review regime tagging |
crash_atr_pct_min | core/regime.py:39 | 5.0% (default) | 14-day ATR ≥5% considered abnormally high volatility. No clear basis | Crash liability exemption, Dreaming exemption, verdict_review macro_shock |
crash_drawdown_30d_pct | core/regime.py:42 | 20.0% | 30-day drop ≥20% + high volatility = sharp crash. No ADR | Same as above |
crash_deep_drawdown_30d_pct | core/regime.py:46 | 30.0% | 30-day drop ≥30% triggers crash independently (regardless of volatility). Added 2026-05-26, has commit record | Same as above |
recovery_rebound_pct | core/regime.py:48 | 10.0% | Rebound ≥10% from 30-day low + still at low level = recovery. No ADR | Quant prompt "recovery" strategy hint |
recovery_quantile_max | core/regime.py:50 | 0.50 | Price 2y percentile <50% qualifies as "still at low level". No basis | Same as above |
low_quantile_threshold | core/regime.py:52 | 0.20 | 2y percentile ≤20% = "low level". No basis | Deprecated (strategy hint no longer reads it after 2026-06 directional lock refactor; key retained for compatibility) |
high_quantile_threshold | core/regime.py:54 | 0.80 | 2y percentile ≥80% = "high level". No basis | Deprecated (same as above) |
Per-asset overrides (core/regime.py:69-85):
| Asset | Overridden Parameter | Value | Basis |
|---|---|---|---|
| GC=F | trend_ma_spread_pct | 5.0% | "Gold trends are more coherent, 3% is too sensitive" — commit comment, no quantitative validation |
| GC=F | crash_atr_pct_min | 3.5% | "Gold's long-term ATR is ~1%, 5% would never trigger" — empirical judgment |
| NDQ.AX | trend_ma_spread_pct | 4.0% | "ETF is smoother than single stocks" — empirical judgment |
| BTC-USD | trend_ma_spread_pct | 8.0% | No basis |
| BTC-USD | crash_atr_pct_min | 8.0% | "Crypto ATR is 3-5%, 8% prevents false positives" — empirical judgment |
| ETH-USD | trend_ma_spread_pct | 8.0% | No basis |
| ETH-USD | crash_atr_pct_min | 8.0% | No basis |
1.2 Committee Sanity Check Thresholds (core/committee/cio_parse.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
buy_confidence_overdrive | core/committee/cio_parse.py:84 | 0.95 | "60-day sample signal is not significant, >0.95 historical hit rate is actually lower than 0.7-0.8" — commit comment, data-supported | BUY→ACCUMULATE downgrade trigger line |
buy_confidence_downgrade_to | core/committee/cio_parse.py:85 | 0.6 | "Expresses original LLM overconfidence" — design intent, no quantification | Confidence value after downgrade |
alloc_cny_ceiling | core/committee/cio_parse.py:86 | 100,000 | "Monthly investment ~¥10k, a single amount exceeding 10x is almost certainly an LLM error" — empirical | alloc clamp upper limit |
worker_unavailable_confidence_floor | core/committee/cio_parse.py:87 | 0.4 | "Worker incomplete → insufficient evidence" — design intent | Confidence cap when worker fails |
1.3 LLM Call Parameters (core/committee/agent_io.py, max_debate_rounds in debate.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
LLM_MAX_ATTEMPTS | core/committee/agent_io.py:23 | 3 (env) | "3 attempts complete within ~14s" — engineering experience | Retry count, affects latency and cost |
LLM_BASE_DELAY | core/committee/agent_io.py:24 | 2.0s (env) | No basis | Retry interval exponential backoff base |
LLM_MAX_DELAY | core/committee/agent_io.py:25 | 20.0s (env) | No basis | Maximum retry wait time |
temperature | core/committee/agent_io.py:35 | 0.2 | No basis. CIO uses 0.1 | LLM output randomness |
max_tool_iterations | core/committee/agent_io.py:75 | 4 | No basis | Agent tool call round limit |
max_debate_rounds | core/committee/debate.py:153 | 1 (default) / 4 (live) | Engineering experience | Cross-challenge rounds, affects LLM call volume |
1.4 Dreaming System Parameters (jobs/dreaming.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
LOOKBACK_DAYS | jobs/dreaming.py:64 | 90 (env) | "Only learn recent behavior" — design intent | Light Sleep ingestion window |
WINDOWS | jobs/dreaming.py:65 | [7, 30] | "Don't look at 1d (1d barely moves, it's a horizon artifact)" — commit comment | Lookback window days |
MIN_RECALL | jobs/dreaming.py:66 | 3 | No basis. LLM falsification prompt says "count≥6" | Minimum sample size threshold |
MIN_SCORE | jobs/dreaming.py:67 | 0.8 | "Same as OpenClaw" — clear source but no self-validation | Deep Sleep selection threshold |
CAUTION_MIN_BASE_DOWN | jobs/dreaming.py:72 | 0.15 | 2026-05-27 ADR 008, has commit and design rationale | Caution mode downside base rate threshold |
CAUTION_LIFT_FULL | jobs/dreaming.py:73 | 0.20 | 2026-05-27 ADR 008, has commit and design rationale | Caution lift full score threshold |
| VIX buckets (low/mid/high) | jobs/dreaming.py:227-231 | <18 / 18-25 / ≥25 | No basis. Standard empirical values | REM Sleep regime label |
| TNX buckets (low/mid/high) | jobs/dreaming.py:234-238 | <4.0 / 4.0-4.5 / ≥4.5 | No basis. Standard empirical values | REM Sleep regime label |
1.5 Verdict Review Scoring Parameters (jobs/verdict_review.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
K_FLAT | jobs/verdict_review.py:244 | 1.0 | "Don't let the system self-learn the scoring ruler" — clear anti-reward-hacking design | HOLD "no movement" judgment multiplier |
FLAT_CEILING_PCT | jobs/verdict_review.py:245 | 8.0% | "Prevent ruler from going out of control when ATR is abnormal" — design intent | HOLD judgment cap |
DEFAULT_DAILY_VOL_PCT | jobs/verdict_review.py:246 | 2.0% | "Fallback when atr_pct fetch fails" — engineering fault tolerance | Fallback value when ATR fetch fails |
1.6 Backtest Reward Function Weights (core/backtest_reward.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
| max_drawdown weight | core/backtest_reward.py:56 | -0.5 | No basis. Formula: reward = annualized - 0.5*max_dd + 0.5*alpha + 0.2*max(0,sharpe-1) | Optuna optimization objective |
| alpha_vs_yuebao weight | core/backtest_reward.py:57 | 0.5 | No basis | Same as above |
| sharpe_bonus weight | core/backtest_reward.py:58 | 0.2 | No basis | Same as above |
| sharpe_bonus_threshold | core/backtest_reward.py:58 | 1.0 | No basis | Same as above |
lam_mdd (v2 forward) | core/backtest_reward.py:91 | 1.0 | No basis | per-sample reward MDD penalty |
lam_return (v2 forward) | core/backtest_reward.py:92 | 0.05 | "Small weight to prevent double-counting Sharpe" — design intent | per-sample reward return bonus |
1.7 Verdict Oracle Accuracy Thresholds (core/backtest_reward.py:123-180)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
| BUY: +5% / -3% | backtest_reward.py:150-154 | 5.0 / -3.0 | No basis | DSPy training oracle metric |
| ACCUMULATE: +3% / -3% | backtest_reward.py:156-160 | 3.0 / -3.0 | No basis | Same as above |
| HOLD: ±3% / ±8% | backtest_reward.py:162-166 | 3.0 / 8.0 | No basis | Same as above |
| TRIM: -3% / +5% | backtest_reward.py:168-172 | -3.0 / 5.0 | No basis | Same as above |
| SELL: -5% / +3% | backtest_reward.py:174-178 | -5.0 / 3.0 | No basis | Same as above |
1.8 Market Metrics Parameters (utils/market_metrics.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
LOOKBACK_30D | market_metrics.py:28 | 30 | Trading day standard | return_30d, rebound_off_30d_low calculation |
RVOL_WINDOW | market_metrics.py:30 | 20 | No basis | Relative volume average window |
| RSI period | market_metrics.py:43 | 14 | Wilder 1978 original standard | RSI calculation |
| ATR period | market_metrics.py:75 | 14 | Standard | ATR% calculation, regime crash determination |
| MA20 / MA120 / MA250 | market_metrics.py:250-252 | 20 / 120 / 250 | Standard technical analysis periods | Regime trend determination, quantile calculation |
1.9 Strategy Metrics Parameters (core/strategy_metrics.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
risk_free_annual | strategy_metrics.py:72 | 0.013 (1.3%) | "Current Yu'ebao rate" — time-sensitive | Sharpe / Sortino calculation |
trading_days_per_year | strategy_metrics.py:73 | 252 | Standard | Annualized return / volatility |
1.10 Trading Friction Parameters (utils/exchange_fee.py)
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
cn_cable_fee | exchange_fee.py:66 | ¥150 | Bank wire transfer fee. Measured value | Currency exchange friction calculation |
cn_commission_rate | exchange_fee.py:67 | 0.1% | Bank commission rate. Measured value | Same as above |
cn_commission_min | exchange_fee.py:68 | ¥50 | Bank minimum commission. Measured value | Same as above |
cn_commission_max | exchange_fee.py:69 | ¥260 | Bank maximum commission. Measured value | Same as above |
au_inward_fee | exchange_fee.py:70 | ¥15 | Australian inward credit fee. Measured value | Same as above |
| CommSec tier ladder | exchange_fee.py:71-74 | $5/$10/$19.95/0.12% | CommSec official fee schedule | Trading commission |
GOLD_OZ_PER_GRAM | utils/gold_price.py:23 | 31.1035 | Physical constant | Gold price unit conversion |
1.11 Other Scattered Parameters
| Parameter | File:Line | Current Value | Basis | Impact Scope |
|---|---|---|---|---|
max_single_invest_cny default | core/portfolio_manager.py:188 | ¥10,000 | No basis | Single investment upper limit fallback |
max_workers (committee) | core/runner/session.py | 4 | "Prevent LLM API rate limiting" — engineering experience | Concurrency level |
step_days (walk-forward) | experiments/train_config.py:121 | 7 | No basis | Optuna training decision frequency |
| Dreaming LLM falsification temperature | jobs/dreaming.py:440 | 0.1 | "Falsification needs stability" — design intent | LLM falsification randomness |
| pnl_snapshot rendering constants | jobs/pnl_snapshot.py:56-66 | W=800, LINE_H=240 etc. | Visual experience | Chart rendering (non-decision) |
2. Parameter Classification
Type A: Deterministic, Cleanly Calibratable (Pure Arithmetic, No LLM Needed for Fit)
Total 28. These parameters reside in pure mathematical functions and can be directly optimized using historical market data.
| # | Parameter | Module | Calibration Method |
|---|---|---|---|
| 1 | trend_ma_spread_pct (default + 4 per-asset) | regime | Grid search on historical regime accuracy |
| 2 | crash_atr_pct_min (default + 3 per-asset) | regime | Grid search on crash detection precision/recall |
| 3 | crash_drawdown_30d_pct | regime | Historical drawdown distribution analysis |
| 4 | crash_deep_drawdown_30d_pct | regime | Same as above |
| 5 | recovery_rebound_pct | regime | Historical rebound distribution |
| 6 | recovery_quantile_max | regime | Quantile distribution analysis |
| 7 | low_quantile_threshold / high_quantile_threshold | regime | 2y price quantile distribution |
| 8 | K_FLAT | verdict_review | Grid search on HOLD accuracy vs flat band width |
| 9 | FLAT_CEILING_PCT | verdict_review | ATR distribution tail analysis |
| 10 | DEFAULT_DAILY_VOL_PCT | verdict_review | ATR distribution fallback position |
| 11 | VIX buckets 18/25 | dreaming | VIX historical distribution quantiles |
| 12 | TNX buckets 4.0/4.5 | dreaming | TNX historical distribution quantiles |
| 13 | LOOKBACK_DAYS | dreaming | Signal decay curve |
| 14 | WINDOWS [7, 30] | dreaming | Forward return convergence window |
| 15 | MIN_RECALL | dreaming | Sample size vs stability tradeoff |
| 16 | Reward function 4 weights | backtest_reward | Optuna already ran 30 trials |
| 17 | Oracle accuracy 10 thresholds | backtest_reward | Historical return distribution per verdict |
| 18 | risk_free_annual | strategy_metrics | Yu'ebao real-time interest rate |
Type B: LLM-Dependent, Contamination Limited
Total 9. The "optimal value" for these parameters depends on LLM output characteristics (confidence distribution, verdict distribution) and requires LLM involvement for evaluation.
| # | Parameter | Module | Contamination Limitation |
|---|---|---|---|
| 1 | buy_confidence_overdrive (0.95) | committee | Requires LLM confidence distribution + ex-post hit rate. But can be reverse-engineered from verdict_review.jsonl historical data |
| 2 | buy_confidence_downgrade_to (0.6) | committee | Same as above |
| 3 | alloc_cny_ceiling (100k) | committee | Requires LLM alloc output distribution. Can be statistically analyzed from historical transcripts |
| 4 | worker_unavailable_confidence_floor (0.4) | committee | Requires simulating worker failure scenarios |
| 5 | MIN_SCORE (0.8) | dreaming | Requires validating insight quality before/after LLM falsification |
| 6 | temperature (0.2) | committee | Requires multiple LLM sampling for consistency comparison |
| 7 | max_tool_iterations (4) | committee | Requires observing LLM tool call behavior |
| 8 | max_debate_rounds (1/4) | committee | Requires LLM debate output quality assessment |
| 9 | Dreaming score weights (0.7/0.3) | dreaming | Hit rate and sample weights in the score formula |
3. Decoupling Difficulty
3.1 Current Sweep Workflow Infrastructure
Existing:
scripts/backtest_committee.py— Runs historical committee day-by-day, blind to the future ✅scripts/backtest_runner.py— Workspace isolation, does not pollute real memory/ ✅scripts/run_walk_forward.py— Walk-forward + paper trading simulator ✅experiments/train_config.py— Optuna parameter space definition ✅scripts/rl_train.py— Optuna training entry point ✅scripts/holdout_validate.py— Hold-out validation ✅core/backtest_reward.py— Reward function ✅core/strategy_metrics.py— Complete strategy metrics ✅jobs/verdict_review.py— Verdict post-mortem analysis ✅
Missing:
- Regime thresholds have no external injection point —
THRESHOLDSandASSET_OVERRIDESare module-level dict literals; backtest can only monkey-patch, cannot pass via env/CLI - Dreaming parameters have no external injection point —
MIN_SCORE,MIN_RECALL,WINDOWS,LOOKBACK_DAYS; onlyLOOKBACK_DAYShas an env override - Reward function weights have no external injection point — The 4 weights in
compute_strategy_rewardare hardcoded - verdict_review K_FLAT/FLAT_CEILING_PCT have no external injection point — Hardcoded constants
- backtest_committee's run_committee call is not parameterized —
max_debate_roundsis passed via envINVEST_MAX_DEBATE_ROUNDS, but other parameters are not
3.2 Rough Effort Estimate
| Change | Effort | Description |
|---|---|---|
| Parameterize Regime THRESHOLDS | 2-3 hours | Add env override or CLI arg → _per_asset_thresholds() reads external values → backtest automatically applies. Changes concentrated in core/regime.py |
| Parameterize Dreaming parameters | 1-2 hours | Add env overrides for MIN_SCORE/MIN_RECALL/WINDOWS, following the same pattern as LOOKBACK_DAYS |
| Parameterize Reward function weights | 1 hour | Add kwargs to compute_strategy_reward or read from TrainConfig.reward |
| Parameterize verdict_review parameters | 1 hour | Add env overrides for K_FLAT/FLAT_CEILING_PCT |
| Sweep runner (batch grid search) | 4-6 hours | Extend based on existing rl_train.py + backtest_runner.py, support grid/random search for Type A parameters + result aggregation |
| Complete pipeline (param sweep → backtest → metrics → comparison) | 1-2 days | Chain sweep runner + walk-forward + strategy_metrics + report generation |
| Offline evaluation pipeline for Type B parameters | 2-3 days | Requires extracting LLM output distribution + ex-post hit rate from verdict_review.jsonl, building a "quasi A/B test" framework |
Total: Making the sweep workflow usable takes approximately 1-2 days; making both Type A and Type B tunable takes approximately 3-5 days.
4. Data Availability
4.1 Data Available for Type A Parameters
| Parameter | post-cutoff | Full 2018-2023 | Either | Description |
|---|---|---|---|---|
trend_ma_spread_pct | ✅ | ✅ | ✅ | Pure market arithmetic. post-cutoff has ~90 days of NDQ.AX + GC=F; full has 2y DB data |
crash_atr_pct_min | ✅ | ✅ | ✅ | ATR is purely historical calculation. But crash events are rare (2020 COVID, 2022 rate hikes), full data is better |
crash_drawdown_30d_pct | ⚠️ | ✅ | ⚠️ | post-cutoff may have no 30-day 20% drop samples. Full data has 2020.03 |
crash_deep_drawdown_30d_pct | ❌ | ✅ | ❌ | post-cutoff has no 30% drop. Requires full data |
recovery_rebound_pct | ⚠️ | ✅ | ⚠️ | post-cutoff has 2026.04 rebound samples, but few |
recovery_quantile_max | ✅ | ✅ | ✅ | Quantile calculation only needs sufficiently long history |
low/high_quantile_threshold | ✅ | ✅ | ✅ | Same as above |
K_FLAT | ✅ | ✅ | ✅ | HOLD hit rate varies with flat band width, usable with any window containing verdicts |
FLAT_CEILING_PCT | ✅ | ✅ | ✅ | ATR tail analysis |
VIX/TNX buckets | ✅ | ✅ | ✅ | Macro data is sufficient |
LOOKBACK_DAYS | ✅ | ❌ | ⚠️ | post-cutoff has 90 days of verdicts; full 2018-2023 has no verdict_review data |
WINDOWS [7,30] | ✅ | ❌ | ⚠️ | Same as above. But full data can be evaluated after generating verdicts via backtest |
MIN_RECALL | ✅ | ❌ | ⚠️ | Same as above |
| Reward weights | ✅ | ✅ | ✅ | Pure arithmetic, only needs P&L curve |
| Oracle accuracy thresholds | ✅ | ✅ | ✅ | Pure arithmetic, only needs return distribution |
risk_free_annual | ✅ | ✅ | ✅ | Yu'ebao interest rate changes slowly |
4.2 Data Available for Type B Parameters
| Parameter | Contamination Limitation | Workaround |
|---|---|---|
buy_confidence_overdrive | Requires LLM confidence output distribution | Can be reverse-engineered from verdict_review.jsonl historical data — jsonl already records each verdict's confidence + ex-post hit rate. Do confidence bucketing → hit rate curve → find inflection point. No need to re-run LLM |
buy_confidence_downgrade_to | Same as above | Same as above. Find the inflection point where "BUY confidence above which hit rate decreases" |
alloc_cny_ceiling | Requires LLM alloc output distribution | Can be statistically analyzed from backtest transcripts — .backtest/ directory has historical CIO memos, regex extract alloc_cny distribution |
worker_unavailable_confidence_floor | Requires simulating worker failure | Can filter records containing [WORKER_UNAVAILABLE] from verdict_review.jsonl — few in number but exist |
MIN_SCORE | Requires evaluating insight quality | Can do retrospective analysis — look at the difference in ex-post hit rate for insights with scores in the 0.7-0.9 range |
temperature | Requires multiple sampling | Most contaminated — cannot be calibrated with historical data, requires A/B testing |
max_tool_iterations | Requires observing tool call behavior | Can be statistically analyzed from llm_telemetry.jsonl — already records iteration for each tool call |
max_debate_rounds | Requires comparing multi-round debate quality | Can be compared via backtest — has INVEST_MAX_DEBATE_ROUNDS env, run 1 vs 3 vs 4 round backtests and compare |
| Dreaming score weights | Requires insight quality evaluation | Can do retrospective analysis — look at ex-post validation rate of insights under different weights |
5. Priority Recommendations
Based on a three-dimensional score of "Honestly Calibratable + Significant Impact + Low Decoupling Cost":
Priority 1: trend_ma_spread_pct (default + per-asset)
- Why it's worth tuning: This is the most upstream parameter for regime classification, directly affecting the three-way split of uptrend/downtrend/range_bound. Quant prompt strategy hints, Dreaming regime bucketing, and verdict_review regime tagging all depend on it. The current default of 3% + per-asset overrides (GC=F 5%, NDQ.AX 4%) are entirely empirical, with no quantitative validation.
- What data to use: Full DB history (2018-2023 + post-cutoff), calculate the historical distribution of MA20/MA120 spread for each asset.
- What validation method to use: Grid search [2%, 8%] step 0.5% → For each threshold, calculate the "separation of ex-post 30d return bucketed by regime" (average gain after uptrend vs average loss after downtrend). The threshold with the highest separation is optimal.
- Expected output: Optimal MA spread threshold for each asset + quantified improvement in separation.
Priority 2: K_FLAT (HOLD "no movement" judgment multiplier)
- Why it's worth tuning: HOLD is the committee's most common verdict (>70%), and K_FLAT directly determines the criterion for "HOLD hit". Too loose (large K_FLAT) → inflated HOLD hit rate → Dreaming learns bad HOLDs as good patterns; too tight → all HOLDs judged as "missed opportunity" → excessive conservatism. Current K_FLAT=1.0 is unvalidated.
- What data to use: verdict_review.jsonl (already has ex-post return + ATR for each verdict), full data available.
- What validation method to use: Grid search [0.5, 2.0] step 0.25 → Recalculate HOLD hit rate for each K_FLAT → Plot "HOLD hit rate vs K_FLAT" curve → Find the inflection point (where the curve slope changes abruptly).
- Expected output: Optimal K_FLAT value + quantitative diagnosis of whether the "HOLD judgment standard is too loose/tight".
Priority 3: crash_atr_pct_min (default + per-asset)
- Why it's worth tuning: Crash determination is the system's liability exemption mechanism — verdicts during crashes are not fed into Dreaming learning and do not count towards hit rates. If the crash threshold is too loose (frequent false crash positives), many samples are exempted and Dreaming learns nothing; if too tight, the system continues learning during real crashes as if it were normal market conditions. The current default of 5% with a 3.5% override for gold is empirical.
- What data to use: Full DB history ATR distribution + historical crash event annotations (2020.03 COVID, 2022 rate hikes, 2026.04 tariffs).
- What validation method to use: For each asset, plot the ATR historical distribution → annotate known crash events → find the "minimum ATR among crash events" as the lower bound → select the "minimum threshold that does not miss any known crash".
- Expected output: Crash ATR threshold for each asset + historical crash event recall rate.
Priority 4: buy_confidence_overdrive (BUY→ACCUMULATE downgrade line)
- Why it's worth tuning: This is the only defense against LLM overconfidence. The current value of 0.95 means "only BUY with confidence ≥ 0.95 is downgraded" — but the commit comment says "BUY with >0.95 confidence has a lower historical hit rate than 0.7-0.8", suggesting the threshold should probably be lower.
- What data to use: verdict_review.jsonl (already records each verdict's confidence + ex-post hit rate).
- What validation method to use: Confidence bucketing [0.5, 0.6, ..., 0.95, 1.0] → BUY hit rate per bucket → Plot "hit rate vs confidence" curve → Find the inflection point where "hit rate starts to decline". The inflection point is the optimal threshold.
- Expected output: Optimal downgrade threshold for BUY confidence + quantitative profile of "LLM overconfidence".
Priority 5: Oracle accuracy thresholds (BUY +5%/-3% etc., 10 values)
- Why it's worth tuning: This set of thresholds defines "what counts as a correct verdict direction" and is the label standard for DSPy training. If the BUY +5% threshold is too high (a 5% gain in 30 days is uncommon in actual markets), many correct BUYs are labeled as "neutral" → training signal is diluted.
- What data to use: Forward return distribution (7d/30d) from verdict_review.jsonl.
- What validation method to use: For each verdict, plot the forward return distribution → redefine thresholds using quantiles (e.g., BUY +1 threshold = 60th percentile of return distribution, -1 threshold = 20th percentile).
- Expected output: Oracle thresholds based on actual return distribution + quantified improvement in training signal density.
Appendix: Parameters Not Recommended for Priority Tuning
| Parameter | Reason |
|---|---|
temperature | Most contaminated, requires A/B testing, cannot be done retrospectively |
max_debate_rounds | Optuna 30 trials already confirmed it's a placebo (confirmed_effective=False) |
cio_confidence_cap | Optuna 30 trials already confirmed it's a placebo |
regime_uptrend / regime_atr | Optuna 30 trials already confirmed it's a placebo (weak) |
| Trading friction parameters | Measured values, determined by bank/broker fee schedules, not calibration targets |
risk_free_annual | Yu'ebao real-time interest rate, just look it up |
| Rendering constants (pnl_snapshot) | Non-decision parameters |
Appendix: Key File Index
| File | Content |
|---|---|
core/regime.py | Regime classification Single Source of Truth (THRESHOLDS + ASSET_OVERRIDES) |
core/committee/cio_parse.py:84-87 | Committee sanity check thresholds (THRESHOLDS build) |
core/backtest_reward.py | Reward function + Oracle accuracy |
core/strategy_metrics.py | Strategy metric calculation |
jobs/dreaming.py:60-73 | Dreaming system parameters |
jobs/verdict_review.py:244-246 | HOLD judgment parameters |
utils/market_metrics.py | Technical indicator calculation parameters |
utils/exchange_fee.py | Trading friction parameters |
experiments/train_config.py | Optuna parameter space (5 hyperparameters) |
scripts/backtest_committee.py | Historical backtest entry point |
scripts/rl_train.py | Optuna training entry point |