ADR-006: Event Awareness Layer (Layer 1) + Lightweight News RAG

Date: 2026-05-13 (v1 default-off); Revised to default-on 2026-05-15Status: Accepted (default-on since 2026-05-15, see Default-on Decision section at the end)

Context

Prior to this commit, openInvest was purely timer-driven: daily_report at 10:00, weekly_review on Sundays, with no event-driven entry point. When intraday sudden events occurred, verdicts were delayed until the next day and were based on market conditions before the event.

Additionally, verification found that services/news.py was an orphan module (0 callers), the 5 tools in agents/tools.py did not include news, and jobs/verdict_review.py:376 explicitly stated that "news/macro narratives are not in the tools… Macro Strategist relies solely on numerical indicators." This means Macro was never looking at the news — "adding news" is a new capability, not an optimization of an existing pipeline.

Decision

Add an event layer, with two paths sharing the same sqlite event store:

1. Trigger Path (intraday real-time) — jobs/event_watch.py cron default */30 0-2,8-23 * * * (Asia/Shanghai timezone, i.e., Beijing 8:00 - next day 2:30; configurable at runtime via config event.watch_schedule, ADR-017, scheduler picks up changes within ≤10 minutes):

   • Pull news from multiple sources (DDGS news / RSS / yfinance.Ticker.news)
   • Flash LLM to normalize into structured events (single batch call)
   • Dimension hit (severity ≥ mid + affects holdings/target_assets + stance ≠ neutral) → send digest email + POST /api/committee/run

   2026-07-03 Window Correction: Since going live, it actually ran during Beijing 0:00-7:30 (the old yml comment mistakenly labeled the cron as UTC semantics, but CronTrigger.from_crontab(..., timezone="Asia/Shanghai") interprets it as Beijing time) — This missed the A-share/ASX daytime and the first half of the US session (ADP/NFP/Fed speeches are all during Beijing 20:15-21:30), causing a gold 10-minute vertical surge to go unalerted until discovered by a user. The window was changed to Beijing 8:00 - next day 2:30 and added to the config whitelist; news from the tail end 2:30-8:00 is picked up by the first scan at 8:00 the next day (delayed, but not lost).

2. RAG Path (committee recall) — core/committee_runner.py before preparing macro:

   • Dimension hard filter (entity ∩ time window ∩ severity) → vector re-ranking → timeliness conflict annotation
   • Structured brief injected only into Macro's prompt (event RAG strictly isolated)
   • Controlled by the master switch env INVEST_EVENT_RAG_ENABLED (default-on since 2026-05-15, unset/empty = treated as true; explicit false/0/off to disable)

Key Design Differences (vs. General RAG)

General RAG assumes static, independent corpora with low semantic duplication. News is the opposite — timeliness is the primary dimension, multi-source reporting is noise, events > documents. We:

• Do not embed raw text upon ingestion, first flash extract event structure + claim text, hash the claim for event_id, merge multiple sources of the same event into a sources sub-table
• Retrieval: dimension filtering precedes vector re-ranking (vectors are only marginal re-ranking)
• Recall results explicitly annotate supersedes — new events for the same entity override old events

Embedding Provider

DeepSeek does not have a public embedding API (an early draft of the plan was incorrect). Current implementation:

• Default: deterministic hash 1024 dimensions (sufficient for coarse deduplication; hard filtering handles the heavy lifting)
• Upgrade path: INVEST_EMBEDDING_PROVIDER=openai + OPENAI_API_KEY → text-embedding-3-small (truncated to 1024 dimensions)

Cross-Agent Isolation

Role	Sees Events?
Macro Strategist	✅ Sees event_brief (structured, includes supersedes)
Quant Analyst	❌ Technical analysis only
Risk Officer	❌ User wealth_context only
CIO	❌ Only sees KEY_TAILWIND / KEY_HEADWIND from Macro's synthesis

Rationale: Quant/Risk inputs should be deterministic and reproducible; once news sentiment permeates, it breaks cross-checking. Macro sees the news, CIO sees Macro's summary — this layer of abstraction is itself a filter.

Implementation Order

See PR / feat/event-layer branch.

Validation

Whether event RAG is worth enabling fully is determined by the root cause ratio from scripts/analyze_news_attribution.py:

• news_blindspot ≥ 30% → Enable INVEST_EVENT_RAG_ENABLED=true
• 10% ~ 30% → Enable only the trigger path, keep RAG disabled
• < 10% → Bottleneck is not the news

The trigger path does not depend on root cause conclusions (valuable in any scenario) and can be enabled unconditionally.

Public Data Red Line

The event layer does not touch docs/accuracy_summary.json / pnl-data branch / outperform feed or any public URL. Emails are the user's private inbox, not public data, so the email body can contain symbols / verdict values / position P&L, etc. The red line in CLAUDE.md only constrains public feeds.

---

Default-on Decision (2026-05-15 Revision)

Context

The author's own testing showed that a 4-step process (dry-run / run 2 weeks to see root cause / modify .env / modify yml) is hard to remember. New features hidden behind a feature flag + default-off = no one uses them = ROI is 0.

Decision

Default-on for RAG + event_watch:

• INVEST_EVENT_RAG_ENABLED unset / empty → treated as true (previously treated as false)
• jobs/event_watch.yml: enabled: true (previously false)
• .env.example explicitly states INVEST_EVENT_RAG_ENABLED=true + comments on how to disable

Safety

When fork users don't have DEEPSEEK_API_KEY:

• _resolve_event_brief gracefully degrades to an empty string on any exception (already implemented), the committee runs normally
• event_watch LLM failure during normalization → scheduler records job_runs failed, does not affect other jobs; users can see the failed log and disable the yml themselves

How to Disable (Fork Users)

1. Add INVEST_EVENT_RAG_ENABLED=false to .env — disables RAG injection
2. Change jobs/event_watch.yml to enabled: false — disables background monitoring