Parameter Tuning Feasibility Diagnosis — Supplementary Document
Read-only, no code changes. 2026-05-27.
1. Parameter Coupling Analysis: 28 Type A Parameters → 9 Independent Tuning Units
Cluster Definition Rules
"Coupling" means: changing the optimal value of one parameter depends on the values of other parameters. If two parameters jointly define a behavioral boundary within the same semantic concept, they belong to the same cluster.
Cluster List
Cluster 1: Crash Trigger (3 parameters, tightly coupled)
| Parameter | Value |
|---|---|
crash_atr_pct_min | 5.0% (default) + per-asset |
crash_drawdown_30d_pct | 20.0% |
crash_deep_drawdown_30d_pct | 30.0% |
Coupling Logic: The three parameters define two crash trigger paths (sharp drop = ATR AND drawdown, deep drop = drawdown alone). Adjusting crash_atr_pct_min changes the trigger rate of the sharp drop path, indirectly affecting the "exclusive sample size" of the deep drop path. All three must be swept jointly.
Reason for Inseparability: If crash_atr_pct_min is tuned in isolation while fixing the other two, the sharp drop path may trigger excessively (low ATR threshold + low drawdown threshold = many false crashes), exempting samples that should have been learned.
Cluster 2: Trend Determination (1 parameter + per-asset override, semi-independent)
| Parameter | Value |
|---|---|
trend_ma_spread_pct | 3.0% (default) + GC=F 5%, NDQ.AX 4%, BTC/ETH 8% |
Coupling Logic: Loosely coupled with Cluster 1 — trend determination and crash determination are different regime branches and do not directly compete for samples. However, trend_ma_spread_pct determines the width of the range_bound, indirectly affecting the priority determination of crash/recovery paths (crash highest, recovery next, trend next).
Reason for Independent Tuning: The per-asset override mechanism already isolates interference between assets. The default value only affects uncovered assets. Coupling with Cluster 1 is weak (priority ordering, not numerical dependency).
Cluster 3: Recovery Determination (2 parameters, tightly coupled)
| Parameter | Value |
|---|---|
recovery_rebound_pct | 10.0% |
recovery_quantile_max | 0.50 |
Coupling Logic: recovery = rebound ≥ X% AND quantile < Y. The two parameters have an AND relationship — adjusting X requires simultaneously adjusting Y, otherwise it either triggers excessively (X low + Y high) or never triggers (X high + Y low).
Cluster 4: Price Quantile Boundaries (2 parameters, symmetric constraint)
| Parameter | Value |
|---|---|
low_quantile_threshold | 0.20 |
high_quantile_threshold | 0.80 |
Coupling Logic: The two define the boundary lines for "low" and "high" zones, with the middle being the "mid-range". Semantically symmetric — if low is adjusted to 0.15, high should be correspondingly adjusted to 0.85 to maintain mid-range width. However, actual coupling is weak (and after the direction lock was removed in 2026-06, both are deprecated — strategy hint no longer reads this set of thresholds).
Can be tuned independently: Weak coupling, can be optimized separately. But if consistent mid-range width is desired, joint sweeping is better.
Cluster 5: HOLD Determination (3 parameters, tightly coupled)
| Parameter | Value |
|---|---|
K_FLAT | 1.0 |
FLAT_CEILING_PCT | 8.0% |
DEFAULT_DAILY_VOL_PCT | 2.0% |
Coupling Logic: HOLD hit = |return| < min(K_FLAT × ATR% × sqrt(days), FLAT_CEILING_PCT). K_FLAT is the multiplier, FLAT_CEILING is the cap, DEFAULT_DAILY_VOL is the fallback when ATR fails. All three jointly define "how much movement counts as no movement".
K_FLAT Specificity: It is a multiplier, linearly affecting the flat band width. But FLAT_CEILING is a min() cap — when K_FLAT × ATR × sqrt(days) > 8%, increasing K_FLAT further has no effect. So the effective range of K_FLAT depends on the actual distribution of ATR.
Cluster 6: VIX/TNX Bucketing (2 groups × 2 thresholds = 4 parameters, coupled within groups)
| Parameter | Value |
|---|---|
| VIX low/mid boundary | 18 |
| VIX mid/high boundary | 25 |
| TNX low/mid boundary | 4.0 |
| TNX mid/high boundary | 4.5 |
Coupling Logic: The two VIX thresholds form one group (defining 3 buckets), and the two TNX thresholds form another group. There is no coupling between VIX and TNX (independent macro dimensions).
Can be split into 2 independent units: VIX bucketing + TNX bucketing.
Cluster 7: Dreaming Entry Thresholds (2 parameters, weakly coupled)
| Parameter | Value |
|---|---|
MIN_RECALL | 3 |
MIN_SCORE | 0.8 |
Coupling Logic: Candidates must simultaneously satisfy count ≥ MIN_RECALL AND score ≥ MIN_SCORE. MIN_SCORE internally includes sample = min(count/10, 1.0), so MIN_RECALL indirectly affects the lower bound of the score. But coupling is weak — MIN_RECALL is a hard threshold, MIN_SCORE is a soft rating.
ADR 008 Note: MIN_SCORE = 0.8 comes from OpenClaw. ADR 008 does not say "intentionally not tuned", but it also does not say it should be tuned.
Cluster 8: Dreaming Lookback Window (2 parameters, independent of other clusters)
| Parameter | Value |
|---|---|
WINDOWS | [7, 30] |
LOOKBACK_DAYS | 90 |
Coupling Logic: LOOKBACK_DAYS determines how much historical verdict data is ingested, WINDOWS determines the post-event market performance window. The two are independent — LOOKBACK_DAYS being too small loses samples, WINDOWS being too large introduces noise, but they do not affect each other.
Can be split into 2 independent units.
Cluster 9: Oracle Accuracy Thresholds (10 parameters, 5 internally coupled pairs)
| Verdict | +1 Threshold | -1 Threshold |
|---|---|---|
| BUY | +5% | -3% |
| ACCUMULATE | +3% | -3% |
| HOLD | |return| ≤ 3% | |return| ≥ 8% |
| TRIM | -3% | +5% |
| SELL | -5% | +3% |
Coupling Logic: Each pair (+1, -1) of thresholds defines the "direction determination space" for a verdict, with the middle being the neutral zone. BUY's +5 and -3 have an AND relationship (changing one alters the neutral zone width). However, there is no coupling between the 5 verdicts — BUY's thresholds do not affect SELL's determination.
Can be split into 5 independent units (one pair per verdict).
Cluster Summary
| # | Cluster Name | Parameter Count | Independent Unit Count | Coupling Strength |
|---|---|---|---|---|
| 1 | Crash Trigger | 3 | 1 | Tight |
| 2 | Trend | 1 (+4 per-asset) | 1 | Weak (with Cluster 1) |
| 3 | Recovery Determination | 2 | 1 | Tight |
| 4 | Price Quantile | 2 | 1-2 | Weak (symmetric constraint) |
| 5 | HOLD Determination | 3 | 1 | Tight |
| 6 | VIX/TNX Bucketing | 4 | 2 | Tight within group, independent between groups |
| 7 | Dreaming Thresholds | 2 | 1 | Weak |
| 8 | Dreaming Lookback | 2 | 2 | Independent |
| 9 | Oracle Thresholds | 10 | 5 | Coupled within pair, independent between pairs |
| Total | 28 (excluding per-asset) | 15-16 |
Conclusion: 28 Type A parameters are split into 15-16 independent tuning units. The truly tightly coupled ones that require joint sweeping are Cluster 1 (crash three parameters), Cluster 3 (recovery two parameters), and Cluster 5 (HOLD three parameters). The rest can be optimized independently.
2. "Do-Not-Tune" List
Parameters Explicitly Not Subject to Calibration (with design intent or ADR constraints)
| Parameter | Basis | Why Not Tune |
|---|---|---|
K_FLAT = 1.0 | jobs/verdict_review.py:241-243 comment: "The rule is fixed (K_FLAT constant) — the system must never be allowed to self-learn this 'ruler for scoring itself', otherwise it would lead to reward hacking" | This is the multiplier for the scoring ruler. Allowing the system to self-tune K_FLAT would let the agent decide "how much counts as no movement", shifting the passing line to its own advantage. This is an anti-reward-hacking design commitment. |
CAUTION_MIN_BASE_DOWN = 0.15 | ADR 008 (2026-05-27 Accepted): "lift-based caution scoring (principle-corrected, not reward hacking)", verified by touchstone | ADR 008 explicitly defines the meaning of 0.15 ("the 30d real downside base rate for this regime must be ≥ this value, otherwise a missed opportunity is an illusion of a one-way upward base rate"). Tuning this would overturn the touchstone verification of ADR 008. |
CAUTION_LIFT_FULL = 0.20 | ADR 008 same as above | Same as above. 0.20 defines "lift reaching this value counts as full quality score". |
Dreaming score weights 0.7 / 0.3 | jobs/dreaming.py:474-476 comment: "Hit rate is already the only credible measure of directional correctness" + "No longer adding points for abs(avg_return) — for HOLD, the more the market moves, the more wrong HOLD is" | The 0.7 hit_rate + 0.3 sample is designed through reasoning, not arbitrary. The dominance of hit_rate has a clear logical argument (HOLD's counter-intuitive nature). Changing the weights = changing the definition of "what counts as a credible pattern". |
FLAT_CEILING_PCT = 8.0% | jobs/verdict_review.py:243 comment: "Prevents the ruler from going out of control when ATR is abnormal" | This is the safety valve for K_FLAT. If K_FLAT is not tuned, FLAT_CEILING should not be tuned in isolation either (they are part of Cluster 5, but K_FLAT is do-not-tune, so the entire Cluster 5 is locked). |
DEFAULT_DAILY_VOL_PCT = 2.0% | jobs/verdict_review.py:246 comment: "Fallback when atr_pct fetch fails" | Engineering fault-tolerance value, not a decision parameter. Tuning it only affects edge cases where ATR fetch fails, not the normal path. |
| CIO prompt v1 cash opportunity cost rule | ADR 007 + training_report: "v1 prompt change was the key to unblocking (reward 0 → 0.4), prompt content is the true ceiling" | This is not a numerical parameter, but prompt text. The training_report explicitly states "prompt is the true ceiling, hyperparameters cannot break through it". Tuning numerical parameters without changing the prompt is the current consensus. |
| CIO zero-shot (no few-shot) | ADR 007 Accepted: "CIO remains zero-shot...the few-shot path has been verified as unviable" | Same as above. ADR locks in not using few-shot. |
Parameters Awaiting User Decision
| Parameter | Current Value | Inclination | Reason for Uncertainty |
|---|---|---|---|
MIN_SCORE = 0.8 | OpenClaw equivalent | Inclined towards do-not-tune | Source is clear (OpenClaw), but no self-verification. If OpenClaw's 0.8 is based on a different architecture (non-LLM committee), this value may not be suitable for openInvest. ADR 008 does not directly constrain it. |
MIN_RECALL = 3 | No basis | Inclined towards tunable | A pattern appearing 3 times is statistically unstable (binomial 95% CI is wide), but the LLM falsification prompt says "count≥6", which contradicts it. Might need to be unified to 5 or 6. |
LOOKBACK_DAYS = 90 | Design intent "learn only recent" | Inclined towards tunable | 90 days is an empirical value. Too short (<60) yields insufficient samples; too long (>180) might learn outdated behavioral patterns. But the principle of "learning only recent" itself is sound. |
WINDOWS = [7, 30] | "1d is a horizon illusion" | Inclined towards do-not-tune | There is a clear argument: within 1d, HOLD barely moves and is automatically counted as a hit, which is an illusion. The choice of 7d/30d is reasonable. However, adding a 60d or 90d window (without replacing existing windows) could be considered. |
VIX buckets 18/25 | Standard empirical values | Tunable | 18/25 are classic VIX boundary lines (panic/extreme panic), with financial consensus. But openInvest's regime labels are for Dreaming aggregation and don't necessarily need to match textbooks. |
TNX buckets 4.0/4.5 | Empirical values | Tunable | 4.0/4.5 are not classic boundary lines (10Y UST has no standard "high/mid/low" definition). They are arbitrary. |
3. Complete Setup of the Previous Optuna 30-Trial Run
3.1 Training Data
| Item | Value |
|---|---|
| Training Period | 2024-01-02 → 2024-04-30 (train_start / train_end) |
| Validation Period | 2024-05-01 → 2024-06-30 (tested on val every 5 trials) |
| Hold-out | 2024-11-18 → 2024-12-31 (non-overlapping, final validation) |
| Assets | NDQ.AX, GC=F |
| Decision Frequency | One decision_date every 7 days (step_days=7) |
| Initial Capital | ¥100,000 |
| LLM | deepseek-v4-flash (non-thinking mode) |
Data Period Issue: The training period 2024-01-02 → 2024-04-30 is before the DeepSeek training data cutoff. This means the LLM may have memory of this period, and the verdicts are not truly out-of-sample. However, the hold-out period 2024-11-18 → 2024-12-31 is also before the cutoff.
Confirmation: The training data for this Optuna run is entirely within the DeepSeek training data, posing a risk of LLM memory contamination.
3.2 Objective Function
reward = compute_strategy_reward(metrics)
= annualized_return_pct / 100
- 0.5 * max_drawdown_pct / 100
+ 0.5 * alpha_vs_yuebao_pct / 100
+ 0.2 * max(0, sharpe_ratio - 1.0)Source: core/backtest_reward.py:compute_strategy_reward
Evaluation Metric: Complete walk-forward paper trading P&L curve → strategy_metrics calculates annualized return/drawdown/Sharpe/excess return → synthesized single numerical reward.
Is it clean?: Yes, the objective function is clean. It is based on the complete P&L curve (daily mark-to-market), not on LLM hit rate. The reward measures "how much the account actually earns if you always follow the AI", and does not involve the correctness of verdict direction determination.
Key Distinction:
- ✅ Reward = Real P&L curve → Not affected by LLM hit rate definition
- ❌ If verdict hit rate were used as the objective → Contaminated by K_FLAT / oracle thresholds / HOLD determination definitions
- Conclusion: Optuna's objective function is clean, the placebo conclusion is valid
3.3 Search Space
| Parameter | Range | Type | Consumption Path |
|---|---|---|---|
regime_uptrend | 3.0 ~ 6.0 | float | monkey-patch regime.THRESHOLDS["trend_ma_spread_pct"] |
regime_atr | 2.5 ~ 5.0 | float | monkey-patch regime.THRESHOLDS["crash_atr_pct_min"] |
max_rounds | 1 ~ 3 | int | env INVEST_MAX_DEBATE_ROUNDS → backtest_committee.py |
cio_confidence_cap | 0.70 ~ 0.95 | float | env INVEST_CIO_CONFIDENCE_CAP → parse_cio_memo() |
alloc_aggressiveness | 0.05 ~ 0.30 | float | env INVEST_ALLOC_AGGRESSIVENESS → parse_cio_memo() |
Sampler: TPE (Tree-structured Parzen Estimator), seed=42 Direction: maximize Number of Trials: 30
3.4 Best Trial Results
Trial #20:
regime_uptrend: 4.625
regime_atr: 3.682
max_rounds: 2
cio_confidence_cap: 0.754
alloc_aggressiveness: 0.245
Reward: 0.4223
Annualized Return: +34.4%
Max Drawdown: 15.4%
Sharpe Ratio: 1.373.5 Basis for "Placebo" Determination
Statistical Basis:
30 trials:
mean: 0.3952
median: 0.4165
stdev: 0.0371
min: 0.2621
max: 0.4223Top 5 Nearly Tied (reward difference < 0.002, i.e., < 0.5%):
| Trial | Reward | regime_uptrend | regime_atr | max_rounds | cio_cap | alloc_agg |
|---|---|---|---|---|---|---|
| #20 | 0.4223 | 4.63 | 3.68 | 2 | 0.754 | 0.245 |
| #4 | 0.4223 | 4.84 | 2.85 | 1 | 0.792 | 0.164 |
| #29 | 0.4210 | 5.59 | 2.72 | 1 | 0.773 | 0.224 |
| #1 | 0.4210 | 3.47 | 2.65 | 3 | 0.850 | 0.227 |
| #9 | 0.4205 | 4.99 | 3.28 | 2 | 0.837 | 0.096 |
Per-Parameter Placebo Determination:
| Parameter | Determination | Basis |
|---|---|---|
regime_uptrend | placebo | In the Top 5, 3.47 and 5.59 both achieve 0.421, spanning 86% of the search range |
regime_atr | placebo | In the Top 5, 2.65 and 3.68 both achieve 0.42, spanning 41% of the search range |
max_rounds | placebo | 1/2/3 all make it into the Top 5, reward difference < 0.001 |
cio_confidence_cap | placebo | 0.754 and 0.850 both achieve 0.42+, spanning 63% of the search range |
alloc_aggressiveness | placebo | 0.096 and 0.245 both achieve 0.42, spanning 60% of the search range |
Bottom 5 Check: The worst trial #25 reward=0.2621, difference from best is 0.16 (38%). But #25's params (4.46, 3.06, 3, 0.73, 0.19) are nearly identical to #27 (4.46, 3.06, 3, 0.73, 0.25), suggesting this region might be a local bad zone sampled by Optuna (noise from LLM stochasticity), not a fault of the parameters themselves.
Core Conclusion: The reward landscape is nearly flat (stdev/mean = 9.4%), and none of the 5 parameters are the main drivers of reward. Prompt content is the true ceiling.
3.6 Important Caveat (Applicability Boundary of the Placebo Conclusion)
- Training data is within the DeepSeek training data — The LLM may have memory, verdict quality does not represent true out-of-sample performance
- Only 2 assets (NDQ.AX + GC=F) — May differ for multi-asset portfolios
- Training window is only 4 months (2024-01 → 2024-04) — Limited sample size
- TPE sampler + 30 trials — Coverage of the 5-dimensional space may be insufficient (average of ~2 effective sampling points per dimension)
- Prompt v1 was changed before Optuna — The baseline was already shifted by the prompt change; Optuna searched the fine-tuning space after prompt v1
Validity Range of the Conclusion: Under the combination of prompt v1 + 2 assets + 4-month training window + DeepSeek v4-flash, the 5 hyperparameters are placebo. Re-verification is needed if the prompt/assets/model are changed.
4. Retrospective Feasibility Breakdown for Type B Parameters
4.1 "Post-hoc Definition" Type (Can be recalculated using existing jsonl after change)
These parameters only affect the evaluation/determination logic, not the LLM's verdict output. After changing them, the existing verdict_review.jsonl + backtest/ data can be used for recalculation without re-running the LLM.
| Parameter | Why "Post-hoc Definition" | Available Data | Retrospective Method |
|---|---|---|---|
buy_confidence_overdrive (0.95) | This parameter performs post-processing in parse_cio_memo() — the LLM has already output confidence, this only decides "whether to downgrade". Changing the threshold does not affect what the LLM sees in the prompt. | verdict_review.jsonl (each entry has confidence + verdict + post-hoc return) | Use existing jsonl, for each BUY verdict, apply downgrade based on different confidence thresholds → recalculate hit rate → find inflection point |
buy_confidence_downgrade_to (0.6) | Same as above. The target value after downgrade is post-processing. | Same as above | Same as above. For each candidate downgrade value, recalculate the verdict distribution + hit rate after downgrade |
alloc_cny_ceiling (100k) | Performs clamping in parse_cio_memo() — the LLM has already output alloc_cny, this only decides "whether to truncate". | backtest transcript (.backtest/<date>/<sym>.md contains the original CIO memo, regex extracts alloc_cny) | Use existing transcripts, for different ceiling values, recalculate "how many times it was clamped" + P&L change after clamping |
worker_unavailable_confidence_floor (0.4) | Performs post-processing in parse_cio_memo() — detects [WORKER_UNAVAILABLE] flag and downgrades. | verdict_review.jsonl (a few records containing worker failures) | Filter records containing worker failures, for different floor values, recalculate P&L |
MAX_DEBATE_ROUNDS | This parameter affects LLM call volume (round 1 vs round 3), but verdict_review.jsonl does not record the round number. | Requires re-running backtest with different rounds | Partially retrospective: Can use Optuna data (trials 1/2/3 correspond to rounds 1/2/3) for direct comparison. But sample size is small (30 trials / 3 rounds ≈ 10 per bucket). |
max_tool_iterations (4) | Affects the number of LLM tool call rounds, indirectly affecting the amount of information the LLM sees. | llm_telemetry.jsonl (records the iteration field) | Partially retrospective: Analyze historical telemetry to see "whether the LLM wanted to continue calling tools when iteration reached 4". But changing the iteration limit will change LLM behavior. |
4.2 "Affects LLM Decision" Type (LLM will output different verdicts after change, must re-run)
These parameters directly affect the prompt the LLM sees or the LLM's behavioral pattern. After changing them, the LLM will output different verdicts, and the historical jsonl verdicts are no longer applicable.
| Parameter | Why "Affects LLM Decision" | Contamination Limit | Can it be bypassed? |
|---|---|---|---|
temperature (0.2) | Directly affects the stochasticity of LLM output. temperature 0.1 vs 0.5 will produce completely different verdict distributions. | Most severe — Cannot be retrospective, requires A/B testing | Cannot be bypassed. Requires running two backtests on the same date with temperature=0.1 and temperature=0.5, comparing P&L. Cost = 2x LLM calls. |
MIN_SCORE (0.8) | Changes which Dreaming insights are written to long-term memory → changes prior_insights injection → changes the LLM's CIO verdict. | Medium — Insight injection is indirect, and the current number of insights is very small | Partially bypassable: If the current number of insights is 0 (caution formally dormant, ADR 008), changing MIN_SCORE has no effect on current verdicts. But it will affect verdicts after insights accumulate. |
Dreaming _score weights (0.7/0.3) | Changes the ranking and selection of insights → changes the prior_insights seen by the CIO. | Medium — Same as MIN_SCORE | Partially bypassable: Same as above. The impact is small when current insights are few. |
LOOKBACK_DAYS (90) | Changes the historical window ingested by Dreaming → changes the quality and quantity of insights → changes CIO behavior. | Medium — Indirect impact | Partially bypassable: If current insights are few, changing LOOKBACK_DAYS has no direct impact on verdicts. But it does affect Dreaming's pattern discovery itself. |
4.3 Summary Matrix
| Parameter | Type | Retrospective Possible? | Sweep Strategy |
|---|---|---|---|
buy_confidence_overdrive | Post-hoc Definition | ✅ Recalculate using existing jsonl | Directly enter sweep space |
buy_confidence_downgrade_to | Post-hoc Definition | ✅ Recalculate using existing jsonl | Directly enter sweep space |
alloc_cny_ceiling | Post-hoc Definition | ✅ Recalculate using existing transcripts | Directly enter sweep space |
worker_unavailable_floor | Post-hoc Definition | ✅ Recalculate using existing jsonl | Directly enter sweep space |
MAX_DEBATE_ROUNDS | Partially Post-hoc | ⚠️ Optuna data available but sample size small | Use existing Optuna data for judgment, do not re-run |
max_tool_iterations | Partially Post-hoc | ⚠️ Telemetry can be analyzed but not complete | Analyze using telemetry, do not re-run |
temperature | Affects LLM Decision | ❌ Must re-run | Classify as do-not-tune or separate A/B test |
MIN_SCORE | Affects LLM Decision (indirect) | ⚠️ Can be ignored when current insights are few | Low priority, re-evaluate after insights accumulate |
| Dreaming score weights | Affects LLM Decision (indirect) | ⚠️ Same as above | Low priority |
Conclusion: Among the 9 Type B parameters, 4 can be cleanly retrospective (confidence_overdrive, confidence_downgrade, alloc_ceiling, worker_floor), 2 can be partially retrospective (max_rounds, max_tool_iterations), and 3 require re-running the LLM or waiting for data accumulation (temperature, MIN_SCORE, score weights).
Sweep Strategy: First sweep the 28 Type A parameters (15-16 independent units), then sweep the 4 "post-hoc definition" Type B parameters. The "affects LLM decision" type will not be touched for now.