Skip to content

openInvest Parameter Tuning Feasibility Diagnosis ​

Read-only assessment, no code changes. 2026-05-27. Supplementary Document (Coupling Analysis / Do-Not-Tune / Optuna Review / Type B Breakdown): 13-param-tuning-feasibility-addendum.md


1. Parameter Inventory ​

1.1 Regime Classification Thresholds (core/regime.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
trend_ma_spread_pctcore/regime.py:373.0% (default)MA20 vs MA120 deviation of 3% indicates a trend. No commit/ADR record, empirical initial valueQuant prompt strategy hints, Dreaming regime bucketing, verdict_review regime tagging
crash_atr_pct_mincore/regime.py:395.0% (default)14-day ATR ≥5% considered abnormally high volatility. No clear basisCrash liability exemption, Dreaming exemption, verdict_review macro_shock
crash_drawdown_30d_pctcore/regime.py:4220.0%30-day drop ≥20% + high volatility = sharp crash. No ADRSame as above
crash_deep_drawdown_30d_pctcore/regime.py:4630.0%30-day drop ≥30% triggers crash independently (regardless of volatility). Added 2026-05-26, has commit recordSame as above
recovery_rebound_pctcore/regime.py:4810.0%Rebound ≥10% from 30-day low + still at low level = recovery. No ADRQuant prompt "recovery" strategy hint
recovery_quantile_maxcore/regime.py:500.50Price 2y percentile <50% qualifies as "still at low level". No basisSame as above
low_quantile_thresholdcore/regime.py:520.202y percentile ≤20% = "low level". No basisDeprecated (strategy hint no longer reads it after 2026-06 directional lock refactor; key retained for compatibility)
high_quantile_thresholdcore/regime.py:540.802y percentile ≥80% = "high level". No basisDeprecated (same as above)

Per-asset overrides (core/regime.py:69-85):

AssetOverridden ParameterValueBasis
GC=Ftrend_ma_spread_pct5.0%"Gold trends are more coherent, 3% is too sensitive" — commit comment, no quantitative validation
GC=Fcrash_atr_pct_min3.5%"Gold's long-term ATR is ~1%, 5% would never trigger" — empirical judgment
NDQ.AXtrend_ma_spread_pct4.0%"ETF is smoother than single stocks" — empirical judgment
BTC-USDtrend_ma_spread_pct8.0%No basis
BTC-USDcrash_atr_pct_min8.0%"Crypto ATR is 3-5%, 8% prevents false positives" — empirical judgment
ETH-USDtrend_ma_spread_pct8.0%No basis
ETH-USDcrash_atr_pct_min8.0%No basis

1.2 Committee Sanity Check Thresholds (core/committee/cio_parse.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
buy_confidence_overdrivecore/committee/cio_parse.py:840.95"60-day sample signal is not significant, >0.95 historical hit rate is actually lower than 0.7-0.8" — commit comment, data-supportedBUY→ACCUMULATE downgrade trigger line
buy_confidence_downgrade_tocore/committee/cio_parse.py:850.6"Expresses original LLM overconfidence" — design intent, no quantificationConfidence value after downgrade
alloc_cny_ceilingcore/committee/cio_parse.py:86100,000"Monthly investment ~¥10k, a single amount exceeding 10x is almost certainly an LLM error" — empiricalalloc clamp upper limit
worker_unavailable_confidence_floorcore/committee/cio_parse.py:870.4"Worker incomplete → insufficient evidence" — design intentConfidence cap when worker fails

1.3 LLM Call Parameters (core/committee/agent_io.py, max_debate_rounds in debate.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
LLM_MAX_ATTEMPTScore/committee/agent_io.py:233 (env)"3 attempts complete within ~14s" — engineering experienceRetry count, affects latency and cost
LLM_BASE_DELAYcore/committee/agent_io.py:242.0s (env)No basisRetry interval exponential backoff base
LLM_MAX_DELAYcore/committee/agent_io.py:2520.0s (env)No basisMaximum retry wait time
temperaturecore/committee/agent_io.py:350.2No basis. CIO uses 0.1LLM output randomness
max_tool_iterationscore/committee/agent_io.py:754No basisAgent tool call round limit
max_debate_roundscore/committee/debate.py:1531 (default) / 4 (live)Engineering experienceCross-challenge rounds, affects LLM call volume

1.4 Dreaming System Parameters (jobs/dreaming.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
LOOKBACK_DAYSjobs/dreaming.py:6490 (env)"Only learn recent behavior" — design intentLight Sleep ingestion window
WINDOWSjobs/dreaming.py:65[7, 30]"Don't look at 1d (1d barely moves, it's a horizon artifact)" — commit commentLookback window days
MIN_RECALLjobs/dreaming.py:663No basis. LLM falsification prompt says "count≥6"Minimum sample size threshold
MIN_SCOREjobs/dreaming.py:670.8"Same as OpenClaw" — clear source but no self-validationDeep Sleep selection threshold
CAUTION_MIN_BASE_DOWNjobs/dreaming.py:720.152026-05-27 ADR 008, has commit and design rationaleCaution mode downside base rate threshold
CAUTION_LIFT_FULLjobs/dreaming.py:730.202026-05-27 ADR 008, has commit and design rationaleCaution lift full score threshold
VIX buckets (low/mid/high)jobs/dreaming.py:227-231<18 / 18-25 / ≥25No basis. Standard empirical valuesREM Sleep regime label
TNX buckets (low/mid/high)jobs/dreaming.py:234-238<4.0 / 4.0-4.5 / ≥4.5No basis. Standard empirical valuesREM Sleep regime label

1.5 Verdict Review Scoring Parameters (jobs/verdict_review.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
K_FLATjobs/verdict_review.py:2441.0"Don't let the system self-learn the scoring ruler" — clear anti-reward-hacking designHOLD "no movement" judgment multiplier
FLAT_CEILING_PCTjobs/verdict_review.py:2458.0%"Prevent ruler from going out of control when ATR is abnormal" — design intentHOLD judgment cap
DEFAULT_DAILY_VOL_PCTjobs/verdict_review.py:2462.0%"Fallback when atr_pct fetch fails" — engineering fault toleranceFallback value when ATR fetch fails

1.6 Backtest Reward Function Weights (core/backtest_reward.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
max_drawdown weightcore/backtest_reward.py:56-0.5No basis. Formula: reward = annualized - 0.5*max_dd + 0.5*alpha + 0.2*max(0,sharpe-1)Optuna optimization objective
alpha_vs_yuebao weightcore/backtest_reward.py:570.5No basisSame as above
sharpe_bonus weightcore/backtest_reward.py:580.2No basisSame as above
sharpe_bonus_thresholdcore/backtest_reward.py:581.0No basisSame as above
lam_mdd (v2 forward)core/backtest_reward.py:911.0No basisper-sample reward MDD penalty
lam_return (v2 forward)core/backtest_reward.py:920.05"Small weight to prevent double-counting Sharpe" — design intentper-sample reward return bonus

1.7 Verdict Oracle Accuracy Thresholds (core/backtest_reward.py:123-180) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
BUY: +5% / -3%backtest_reward.py:150-1545.0 / -3.0No basisDSPy training oracle metric
ACCUMULATE: +3% / -3%backtest_reward.py:156-1603.0 / -3.0No basisSame as above
HOLD: ±3% / ±8%backtest_reward.py:162-1663.0 / 8.0No basisSame as above
TRIM: -3% / +5%backtest_reward.py:168-172-3.0 / 5.0No basisSame as above
SELL: -5% / +3%backtest_reward.py:174-178-5.0 / 3.0No basisSame as above

1.8 Market Metrics Parameters (utils/market_metrics.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
LOOKBACK_30Dmarket_metrics.py:2830Trading day standardreturn_30d, rebound_off_30d_low calculation
RVOL_WINDOWmarket_metrics.py:3020No basisRelative volume average window
RSI periodmarket_metrics.py:4314Wilder 1978 original standardRSI calculation
ATR periodmarket_metrics.py:7514StandardATR% calculation, regime crash determination
MA20 / MA120 / MA250market_metrics.py:250-25220 / 120 / 250Standard technical analysis periodsRegime trend determination, quantile calculation

1.9 Strategy Metrics Parameters (core/strategy_metrics.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
risk_free_annualstrategy_metrics.py:720.013 (1.3%)"Current Yu'ebao rate" — time-sensitiveSharpe / Sortino calculation
trading_days_per_yearstrategy_metrics.py:73252StandardAnnualized return / volatility

1.10 Trading Friction Parameters (utils/exchange_fee.py) ​

ParameterFile:LineCurrent ValueBasisImpact Scope
cn_cable_feeexchange_fee.py:66¥150Bank wire transfer fee. Measured valueCurrency exchange friction calculation
cn_commission_rateexchange_fee.py:670.1%Bank commission rate. Measured valueSame as above
cn_commission_minexchange_fee.py:68¥50Bank minimum commission. Measured valueSame as above
cn_commission_maxexchange_fee.py:69¥260Bank maximum commission. Measured valueSame as above
au_inward_feeexchange_fee.py:70¥15Australian inward credit fee. Measured valueSame as above
CommSec tier ladderexchange_fee.py:71-74$5/$10/$19.95/0.12%CommSec official fee scheduleTrading commission
GOLD_OZ_PER_GRAMutils/gold_price.py:2331.1035Physical constantGold price unit conversion

1.11 Other Scattered Parameters ​

ParameterFile:LineCurrent ValueBasisImpact Scope
max_single_invest_cny defaultcore/portfolio_manager.py:188¥10,000No basisSingle investment upper limit fallback
max_workers (committee)core/runner/session.py4"Prevent LLM API rate limiting" — engineering experienceConcurrency level
step_days (walk-forward)experiments/train_config.py:1217No basisOptuna training decision frequency
Dreaming LLM falsification temperaturejobs/dreaming.py:4400.1"Falsification needs stability" — design intentLLM falsification randomness
pnl_snapshot rendering constantsjobs/pnl_snapshot.py:56-66W=800, LINE_H=240 etc.Visual experienceChart rendering (non-decision)

2. Parameter Classification ​

Type A: Deterministic, Cleanly Calibratable (Pure Arithmetic, No LLM Needed for Fit) ​

Total 28. These parameters reside in pure mathematical functions and can be directly optimized using historical market data.

#ParameterModuleCalibration Method
1trend_ma_spread_pct (default + 4 per-asset)regimeGrid search on historical regime accuracy
2crash_atr_pct_min (default + 3 per-asset)regimeGrid search on crash detection precision/recall
3crash_drawdown_30d_pctregimeHistorical drawdown distribution analysis
4crash_deep_drawdown_30d_pctregimeSame as above
5recovery_rebound_pctregimeHistorical rebound distribution
6recovery_quantile_maxregimeQuantile distribution analysis
7low_quantile_threshold / high_quantile_thresholdregime2y price quantile distribution
8K_FLATverdict_reviewGrid search on HOLD accuracy vs flat band width
9FLAT_CEILING_PCTverdict_reviewATR distribution tail analysis
10DEFAULT_DAILY_VOL_PCTverdict_reviewATR distribution fallback position
11VIX buckets 18/25dreamingVIX historical distribution quantiles
12TNX buckets 4.0/4.5dreamingTNX historical distribution quantiles
13LOOKBACK_DAYSdreamingSignal decay curve
14WINDOWS [7, 30]dreamingForward return convergence window
15MIN_RECALLdreamingSample size vs stability tradeoff
16Reward function 4 weightsbacktest_rewardOptuna already ran 30 trials
17Oracle accuracy 10 thresholdsbacktest_rewardHistorical return distribution per verdict
18risk_free_annualstrategy_metricsYu'ebao real-time interest rate

Type B: LLM-Dependent, Contamination Limited ​

Total 9. The "optimal value" for these parameters depends on LLM output characteristics (confidence distribution, verdict distribution) and requires LLM involvement for evaluation.

#ParameterModuleContamination Limitation
1buy_confidence_overdrive (0.95)committeeRequires LLM confidence distribution + ex-post hit rate. But can be reverse-engineered from verdict_review.jsonl historical data
2buy_confidence_downgrade_to (0.6)committeeSame as above
3alloc_cny_ceiling (100k)committeeRequires LLM alloc output distribution. Can be statistically analyzed from historical transcripts
4worker_unavailable_confidence_floor (0.4)committeeRequires simulating worker failure scenarios
5MIN_SCORE (0.8)dreamingRequires validating insight quality before/after LLM falsification
6temperature (0.2)committeeRequires multiple LLM sampling for consistency comparison
7max_tool_iterations (4)committeeRequires observing LLM tool call behavior
8max_debate_rounds (1/4)committeeRequires LLM debate output quality assessment
9Dreaming score weights (0.7/0.3)dreamingHit rate and sample weights in the score formula

3. Decoupling Difficulty ​

3.1 Current Sweep Workflow Infrastructure ​

Existing:

  • scripts/backtest_committee.py — Runs historical committee day-by-day, blind to the future ✅
  • scripts/backtest_runner.py — Workspace isolation, does not pollute real memory/ ✅
  • scripts/run_walk_forward.py — Walk-forward + paper trading simulator ✅
  • experiments/train_config.py — Optuna parameter space definition ✅
  • scripts/rl_train.py — Optuna training entry point ✅
  • scripts/holdout_validate.py — Hold-out validation ✅
  • core/backtest_reward.py — Reward function ✅
  • core/strategy_metrics.py — Complete strategy metrics ✅
  • jobs/verdict_review.py — Verdict post-mortem analysis ✅

Missing:

  • Regime thresholds have no external injection point — THRESHOLDS and ASSET_OVERRIDES are module-level dict literals; backtest can only monkey-patch, cannot pass via env/CLI
  • Dreaming parameters have no external injection point — MIN_SCORE, MIN_RECALL, WINDOWS, LOOKBACK_DAYS; only LOOKBACK_DAYS has an env override
  • Reward function weights have no external injection point — The 4 weights in compute_strategy_reward are hardcoded
  • verdict_review K_FLAT/FLAT_CEILING_PCT have no external injection point — Hardcoded constants
  • backtest_committee's run_committee call is not parameterized — max_debate_rounds is passed via env INVEST_MAX_DEBATE_ROUNDS, but other parameters are not

3.2 Rough Effort Estimate ​

ChangeEffortDescription
Parameterize Regime THRESHOLDS2-3 hoursAdd env override or CLI arg → _per_asset_thresholds() reads external values → backtest automatically applies. Changes concentrated in core/regime.py
Parameterize Dreaming parameters1-2 hoursAdd env overrides for MIN_SCORE/MIN_RECALL/WINDOWS, following the same pattern as LOOKBACK_DAYS
Parameterize Reward function weights1 hourAdd kwargs to compute_strategy_reward or read from TrainConfig.reward
Parameterize verdict_review parameters1 hourAdd env overrides for K_FLAT/FLAT_CEILING_PCT
Sweep runner (batch grid search)4-6 hoursExtend based on existing rl_train.py + backtest_runner.py, support grid/random search for Type A parameters + result aggregation
Complete pipeline (param sweep → backtest → metrics → comparison)1-2 daysChain sweep runner + walk-forward + strategy_metrics + report generation
Offline evaluation pipeline for Type B parameters2-3 daysRequires extracting LLM output distribution + ex-post hit rate from verdict_review.jsonl, building a "quasi A/B test" framework

Total: Making the sweep workflow usable takes approximately 1-2 days; making both Type A and Type B tunable takes approximately 3-5 days.


4. Data Availability ​

4.1 Data Available for Type A Parameters ​

Parameterpost-cutoffFull 2018-2023EitherDescription
trend_ma_spread_pct✅✅✅Pure market arithmetic. post-cutoff has ~90 days of NDQ.AX + GC=F; full has 2y DB data
crash_atr_pct_min✅✅✅ATR is purely historical calculation. But crash events are rare (2020 COVID, 2022 rate hikes), full data is better
crash_drawdown_30d_pct⚠️✅⚠️post-cutoff may have no 30-day 20% drop samples. Full data has 2020.03
crash_deep_drawdown_30d_pct❌✅❌post-cutoff has no 30% drop. Requires full data
recovery_rebound_pct⚠️✅⚠️post-cutoff has 2026.04 rebound samples, but few
recovery_quantile_max✅✅✅Quantile calculation only needs sufficiently long history
low/high_quantile_threshold✅✅✅Same as above
K_FLAT✅✅✅HOLD hit rate varies with flat band width, usable with any window containing verdicts
FLAT_CEILING_PCT✅✅✅ATR tail analysis
VIX/TNX buckets✅✅✅Macro data is sufficient
LOOKBACK_DAYS✅❌⚠️post-cutoff has 90 days of verdicts; full 2018-2023 has no verdict_review data
WINDOWS [7,30]✅❌⚠️Same as above. But full data can be evaluated after generating verdicts via backtest
MIN_RECALL✅❌⚠️Same as above
Reward weights✅✅✅Pure arithmetic, only needs P&L curve
Oracle accuracy thresholds✅✅✅Pure arithmetic, only needs return distribution
risk_free_annual✅✅✅Yu'ebao interest rate changes slowly

4.2 Data Available for Type B Parameters ​

ParameterContamination LimitationWorkaround
buy_confidence_overdriveRequires LLM confidence output distributionCan be reverse-engineered from verdict_review.jsonl historical data — jsonl already records each verdict's confidence + ex-post hit rate. Do confidence bucketing → hit rate curve → find inflection point. No need to re-run LLM
buy_confidence_downgrade_toSame as aboveSame as above. Find the inflection point where "BUY confidence above which hit rate decreases"
alloc_cny_ceilingRequires LLM alloc output distributionCan be statistically analyzed from backtest transcripts — .backtest/ directory has historical CIO memos, regex extract alloc_cny distribution
worker_unavailable_confidence_floorRequires simulating worker failureCan filter records containing [WORKER_UNAVAILABLE] from verdict_review.jsonl — few in number but exist
MIN_SCORERequires evaluating insight qualityCan do retrospective analysis — look at the difference in ex-post hit rate for insights with scores in the 0.7-0.9 range
temperatureRequires multiple samplingMost contaminated — cannot be calibrated with historical data, requires A/B testing
max_tool_iterationsRequires observing tool call behaviorCan be statistically analyzed from llm_telemetry.jsonl — already records iteration for each tool call
max_debate_roundsRequires comparing multi-round debate qualityCan be compared via backtest — has INVEST_MAX_DEBATE_ROUNDS env, run 1 vs 3 vs 4 round backtests and compare
Dreaming score weightsRequires insight quality evaluationCan do retrospective analysis — look at ex-post validation rate of insights under different weights

5. Priority Recommendations ​

Based on a three-dimensional score of "Honestly Calibratable + Significant Impact + Low Decoupling Cost":

Priority 1: trend_ma_spread_pct (default + per-asset) ​

  • Why it's worth tuning: This is the most upstream parameter for regime classification, directly affecting the three-way split of uptrend/downtrend/range_bound. Quant prompt strategy hints, Dreaming regime bucketing, and verdict_review regime tagging all depend on it. The current default of 3% + per-asset overrides (GC=F 5%, NDQ.AX 4%) are entirely empirical, with no quantitative validation.
  • What data to use: Full DB history (2018-2023 + post-cutoff), calculate the historical distribution of MA20/MA120 spread for each asset.
  • What validation method to use: Grid search [2%, 8%] step 0.5% → For each threshold, calculate the "separation of ex-post 30d return bucketed by regime" (average gain after uptrend vs average loss after downtrend). The threshold with the highest separation is optimal.
  • Expected output: Optimal MA spread threshold for each asset + quantified improvement in separation.

Priority 2: K_FLAT (HOLD "no movement" judgment multiplier) ​

  • Why it's worth tuning: HOLD is the committee's most common verdict (>70%), and K_FLAT directly determines the criterion for "HOLD hit". Too loose (large K_FLAT) → inflated HOLD hit rate → Dreaming learns bad HOLDs as good patterns; too tight → all HOLDs judged as "missed opportunity" → excessive conservatism. Current K_FLAT=1.0 is unvalidated.
  • What data to use: verdict_review.jsonl (already has ex-post return + ATR for each verdict), full data available.
  • What validation method to use: Grid search [0.5, 2.0] step 0.25 → Recalculate HOLD hit rate for each K_FLAT → Plot "HOLD hit rate vs K_FLAT" curve → Find the inflection point (where the curve slope changes abruptly).
  • Expected output: Optimal K_FLAT value + quantitative diagnosis of whether the "HOLD judgment standard is too loose/tight".

Priority 3: crash_atr_pct_min (default + per-asset) ​

  • Why it's worth tuning: Crash determination is the system's liability exemption mechanism — verdicts during crashes are not fed into Dreaming learning and do not count towards hit rates. If the crash threshold is too loose (frequent false crash positives), many samples are exempted and Dreaming learns nothing; if too tight, the system continues learning during real crashes as if it were normal market conditions. The current default of 5% with a 3.5% override for gold is empirical.
  • What data to use: Full DB history ATR distribution + historical crash event annotations (2020.03 COVID, 2022 rate hikes, 2026.04 tariffs).
  • What validation method to use: For each asset, plot the ATR historical distribution → annotate known crash events → find the "minimum ATR among crash events" as the lower bound → select the "minimum threshold that does not miss any known crash".
  • Expected output: Crash ATR threshold for each asset + historical crash event recall rate.

Priority 4: buy_confidence_overdrive (BUY→ACCUMULATE downgrade line) ​

  • Why it's worth tuning: This is the only defense against LLM overconfidence. The current value of 0.95 means "only BUY with confidence ≥ 0.95 is downgraded" — but the commit comment says "BUY with >0.95 confidence has a lower historical hit rate than 0.7-0.8", suggesting the threshold should probably be lower.
  • What data to use: verdict_review.jsonl (already records each verdict's confidence + ex-post hit rate).
  • What validation method to use: Confidence bucketing [0.5, 0.6, ..., 0.95, 1.0] → BUY hit rate per bucket → Plot "hit rate vs confidence" curve → Find the inflection point where "hit rate starts to decline". The inflection point is the optimal threshold.
  • Expected output: Optimal downgrade threshold for BUY confidence + quantitative profile of "LLM overconfidence".

Priority 5: Oracle accuracy thresholds (BUY +5%/-3% etc., 10 values) ​

  • Why it's worth tuning: This set of thresholds defines "what counts as a correct verdict direction" and is the label standard for DSPy training. If the BUY +5% threshold is too high (a 5% gain in 30 days is uncommon in actual markets), many correct BUYs are labeled as "neutral" → training signal is diluted.
  • What data to use: Forward return distribution (7d/30d) from verdict_review.jsonl.
  • What validation method to use: For each verdict, plot the forward return distribution → redefine thresholds using quantiles (e.g., BUY +1 threshold = 60th percentile of return distribution, -1 threshold = 20th percentile).
  • Expected output: Oracle thresholds based on actual return distribution + quantified improvement in training signal density.

ParameterReason
temperatureMost contaminated, requires A/B testing, cannot be done retrospectively
max_debate_roundsOptuna 30 trials already confirmed it's a placebo (confirmed_effective=False)
cio_confidence_capOptuna 30 trials already confirmed it's a placebo
regime_uptrend / regime_atrOptuna 30 trials already confirmed it's a placebo (weak)
Trading friction parametersMeasured values, determined by bank/broker fee schedules, not calibration targets
risk_free_annualYu'ebao real-time interest rate, just look it up
Rendering constants (pnl_snapshot)Non-decision parameters

Appendix: Key File Index ​

FileContent
core/regime.pyRegime classification Single Source of Truth (THRESHOLDS + ASSET_OVERRIDES)
core/committee/cio_parse.py:84-87Committee sanity check thresholds (THRESHOLDS build)
core/backtest_reward.pyReward function + Oracle accuracy
core/strategy_metrics.pyStrategy metric calculation
jobs/dreaming.py:60-73Dreaming system parameters
jobs/verdict_review.py:244-246HOLD judgment parameters
utils/market_metrics.pyTechnical indicator calculation parameters
utils/exchange_fee.pyTrading friction parameters
experiments/train_config.pyOptuna parameter space (5 hyperparameters)
scripts/backtest_committee.pyHistorical backtest entry point
scripts/rl_train.pyOptuna training entry point