Troubleshooting

Where to check when the system goes down, how to debug a broken SSE connection, what to do when sync_gui_dist fails. Organized by "Symptom → Diagnosis Steps → Fix".

← 08-deployment · Wiki Index · 10-design-system →

General Principles

When you find a problem → Check the logs first, don't guess:

bash

# invest-web backend
sudo journalctl -u invest-web -f --since "1 hour ago"

# Caddy (containerized)
docker logs caddy --tail 100 -f

# scheduler / cron jobs
sudo journalctl -u invest-scheduler -f

# CF Access
# Open CF Zero Trust in browser → Logs → Access

→ 90% of problems will be directly explained by the logs.

1. GUI Completely Unreachable

Symptoms

Browser accessing invest.your-domain.com spins, shows white screen, or 502

Diagnosis Sequence

bash

# 1. Can CF reach the origin?
curl -I https://invest.your-domain.com
# → Expect 302 (CF Access redirect to login)

# 2. Is Caddy running?
docker ps | grep caddy
# → Expect caddy process to be healthy

# 3. Is Caddy configuration correct?
docker exec caddy cat /etc/caddy/Caddyfile | grep invest

# 4. Do static files exist?
ls -la /srv/invest-gui/index.html
# → Expect .html to exist + reasonable last modified time

# 5. Is the backend listening?
curl -I http://127.0.0.1:8765/api/health
# → Expect HTTP 200

# 6. systemd status
sudo systemctl status invest-web

Common Causes + Fixes

Symptom	Cause	Fix
`curl localhost:8765 Connection refused`	invest-web not started	`sudo systemctl restart invest-web`
`502 Bad Gateway`	No process listening on the port Caddy proxies to	Same as above
Caddy config looks correct but doesn't take effect	bind mount + reload doesn't read new inode	`docker restart caddy` (not reload)
`/srv/invest-gui/index.html` doesn't exist	GUI not deployed	`cd invest-gui && pnpm deploy`
CF Access infinite redirect loop	Access policy doesn't include your email	Add email in CF Dashboard

2. Endpoint Returns 404 But Exists in Code

Symptoms

bash

curl http://127.0.0.1:8765/api/portfolio/total_value?base=CNY
# → {"detail":"Not Found"}

But the code in connectors/web_api.py clearly has this endpoint.

Cause

99% of the time, the systemd process is running old code: systemd Restart=on-failure only restarts on crash, it does not auto-reload after a git pull.

Fix

bash

sudo systemctl restart invest-web
sleep 2
curl http://127.0.0.1:8765/api/portfolio/total_value?base=CNY
# → {"base_currency":"CNY","grand_total":...}

→ You must manually restart the backend after every push. See 08-deployment.md#upgrade-procedure.

Long-term Solution

Consider adding a git post-receive hook or solidifying the restart step in the deploy script.

3. Committee Triggered But No Response / SSE Not Streaming

Symptoms

The GUI "Trigger Committee" button was clicked, a task_id appeared, but the status stays queued or running without progressing.

Diagnosis

bash

# 1. Does the task status file exist?
ls ~/openInvest/memory/.committee/<task_id>/

# 2. Is status.json changing? (written upon completion of each stage)
watch -n 1 cat ~/openInvest/memory/.committee/<task_id>/status.json

# 3. Are LLM calls running in the journal?
sudo journalctl -u invest-web --since "5 minutes ago" | grep -i "deepseek\|httpx"

# 4. Can you curl the SSE endpoint directly?
curl -N http://127.0.0.1:8765/api/committee/live/<task_id>
# Expect continuous output of event: progress / data: ...

Common Causes

Symptom	Cause	Fix
status.json stuck at queued	asyncio.create_task didn't start	Check journal for traceback; restart backend
LLM never returns 200	DeepSeek API key wrong / rate limited / insufficient balance	Check `.env` / Check journal for 401 / 429
status.json jumps to done but frontend doesn't receive it	SSE connection cut by CF Access 5min idle timeout	Check if keepalive is running in code (every 25s)
status is running but LLM never called	Locked (another task holds the fcntl lock)	`lsof memory/portfolio.md.lock` to find the holder

v3 Convergence Detection Looks Like a Bug

If you see final_round: 3, converged: true, max_debate_rounds: 4, don't panic— This is an early exit due to convergence, not that round 4 didn't finish. See 02-agents.md#convergence-detection.

4. yfinance Can't Fetch Prices / Stale Market Data

Symptoms

GUI HoldingCard shows "⚠ Stale" indicator, or market data fields are null.

Diagnosis

bash

# Try directly with Python
cd ~/openInvest
uv run python -c "from utils.exchange_fee import get_history_data; df = get_history_data('NDQ.AX', '5d'); print(df.tail())"

Common Causes

Symptom	Cause	Fix
`403 Forbidden` appears repeatedly	yfinance IP rate limited	Wait 5-10min; or change yfinance proxy settings
df is None	Symbol wrong / yfinance doesn't cover it	Verify ticker exists on `https://finance.yahoo.com`
Close column is empty	Weekend / holiday, no new data	This is normal; the DB fallback should handle it
BetaShares 403 NDQ.AX can't be fetched	NDQ's own scraper failed	yfinance fallback kicks in automatically (see `🔄 yfinance fallback` in journal)

Data Source Panoramic Diagnosis

GUI /system → "Data Sources" tab shows the last successful time + is_stale status for all data sources at a glance. Corresponding endpoint: GET /api/data_sources/health.

5. CommSec Email Not Importing Trades

Symptoms

Hours after placing an order, there are no new holdings in portfolio.md.

Diagnosis

bash

# v3 changed: CommSec cron is disabled by default, must be triggered manually
uv run python -m scripts.import_commsec --lookback 30
# Expect to see a list of fetched trades (dry-run)

# To actually write
uv run python -m scripts.import_commsec --lookback 30 --apply

Or use the GUI "Import CommSec" button (if added).

Common Causes

Symptom	Cause	Fix
`IMAP connection failed`	EMAIL_PASSWORD is not an app password	Gmail must use an App Password, not your login password
Fetched 0 records	lookback too short or emails already processed	Retry with `--lookback 365`
Fetched but write failed	Symbol not in strategy.target_assets	Add it to strategy first; or add holding manually

Why Cron is Disabled by Default

Automatic cron sync can silently miss trades during temporary IMAP failures, causing the portfolio to mismatch the real account. The manual mode lets users dry-run first to see clearly before applying. See 05-data-model.md.

6. PnL Chart Data Incorrect / Cliff Drop

Symptoms

/api/pnl_chart.svg line chart has cliffs or future dates.

Diagnosis

bash

# Check pnl_history.jsonl directly
tail -20 ~/openInvest/memory/pnl_history.jsonl
# Look for rows with abnormal timestamps (non-trading hours at midnight, future times)

Common Causes

Symptom	Cause	Fix
Data points at midnight	jobs/pnl_snapshot triggered outside trading hours	`python -m scripts.clean_pnl_history --dry-run` then actually delete
Future dates	Timezone misconfigured (UTC vs Asia/Shanghai)	Sync `.env` `TZ=Asia/Shanghai`
Vertical jump on a certain day	User didn't record a transaction, but scheduler kept running	Use `/api/cash/CNY/deposit

Cleanup script:

bash

python -m scripts.clean_pnl_history --dry-run   # Preview
python -m scripts.clean_pnl_history             # Execute (auto-backup)
python -m jobs.pnl_snapshot --render-only       # Re-render SVG

7. sync_gui_dist Fails

Symptoms

bash

uv run python -m scripts.sync_gui_dist
# ❌ Download failed

Diagnosis

bash

# Try fetching directly
curl -I https://github.com/longsizhuo/invest-gui/releases/download/dist-latest/invest-gui-dist.tar.gz
# Expect 302 → S3 → 200

Common Causes

Symptom	Cause	Fix
`404 dist-latest tag doesn't exist`	Repo just pushed but GHA hasn't finished	Wait 1-2 min, or check Actions
`Connection timeout`	Server network blocked from github.com	Use a proxy or raw.githubusercontent.com mirror
`403`	Tag is not public (private repository)	Use `gh auth login` or an access token

8. Tests Fail

Symptoms

bash

uv run pytest tests/
# FAILED tests/test_xxx.py

Diagnosis

bash

# Run the failing test alone + verbose
uv run pytest tests/test_xxx.py::test_yyy -v --tb=long

Common Causes

Symptom	Cause	Fix
`yfinance data fetch failed`	Network / rate limit	Re-run a few times; or add mock
`ImportError cannot import _render_portfolio_body`	v2 renamed → `_render_portfolio_body_v2`	Fixed in CI; local code might be old
`ValidationError schema_version`	portfolio.md doesn't match v2 schema	Run `python -m scripts.migrate_portfolio_to_holdings`

166 tests is the baseline; new features must keep all tests green.

9. NapCat QQ Bot Doesn't Respond to Commands

Symptoms

Sending /balance in a QQ private chat, the bot doesn't reply.

Diagnosis

bash

# 1. Is NapCat running?
ps aux | grep -i napcat

# 2. Is the invest-napcat connector running?
ps aux | grep "connectors.napcat_bot"

# 3. Check journal for received events
sudo journalctl --since "5 min ago" | grep napcat

Common Causes

Symptom	Cause	Fix
Message received but rejected	`INVEST_WHITELIST_QQ` not set / not your QQ number	Add `INVEST_WHITELIST_QQ=YourQQ` to `.env`
Events not received	NAPCAT_WS_URL wrong / NapCat WS not enabled	Check NapCat startup parameters
Received but reply failed	NAPCAT_HTTP_URL wrong	Check NapCat HTTP API port

See connectors/README.md for details.

10. Emergency Contact / Finding History

Historical decisions: memory/daily/<date>/<symbol>.md
LLM call telemetry: memory/llm_usage.jsonl
Tool call audit: memory/.committee/<task_id>/tool_calls.jsonl
Dreaming events: memory/.dreams/events.jsonl
Committee task status: memory/.committee/<task_id>/status.json

→ Almost all "Why did the AI say BUY this time?" questions can be reviewed from these files.

Next Steps

→ 08-deployment.md — Deployment related issues

→ QUICK_START.md Troubleshooting section — Issues during initial setup for fork users

→ Still have an unresolved problem? Open a GitHub Issue

Troubleshooting ​

General Principles ​

1. GUI Completely Unreachable ​

Symptoms ​

Diagnosis Sequence ​

Common Causes + Fixes ​

2. Endpoint Returns 404 But Exists in Code ​

Symptoms ​

Cause ​

Fix ​

Long-term Solution ​

3. Committee Triggered But No Response / SSE Not Streaming ​

Symptoms ​

Diagnosis ​

Common Causes ​

v3 Convergence Detection Looks Like a Bug ​

4. yfinance Can't Fetch Prices / Stale Market Data ​

Symptoms ​

Diagnosis ​

Common Causes ​

Data Source Panoramic Diagnosis ​

5. CommSec Email Not Importing Trades ​

Symptoms ​

Diagnosis ​

Common Causes ​

Why Cron is Disabled by Default ​

6. PnL Chart Data Incorrect / Cliff Drop ​

Symptoms ​

Diagnosis ​

Common Causes ​

7. sync_gui_dist Fails ​

Symptoms ​

Diagnosis ​

Common Causes ​

8. Tests Fail ​

Symptoms ​

Diagnosis ​

Common Causes ​

9. NapCat QQ Bot Doesn't Respond to Commands ​

Symptoms ​

Diagnosis ​

Common Causes ​

10. Emergency Contact / Finding History ​

Next Steps ​

Troubleshooting

General Principles

1. GUI Completely Unreachable

Symptoms

Diagnosis Sequence

Common Causes + Fixes

2. Endpoint Returns 404 But Exists in Code

Symptoms

Cause

Fix

Long-term Solution

3. Committee Triggered But No Response / SSE Not Streaming

Symptoms

Diagnosis

Common Causes

v3 Convergence Detection Looks Like a Bug

4. yfinance Can't Fetch Prices / Stale Market Data

Symptoms

Diagnosis

Common Causes

Data Source Panoramic Diagnosis

5. CommSec Email Not Importing Trades

Symptoms

Diagnosis

Common Causes

Why Cron is Disabled by Default

6. PnL Chart Data Incorrect / Cliff Drop

Symptoms

Diagnosis

Common Causes

7. sync_gui_dist Fails

Symptoms

Diagnosis

Common Causes

8. Tests Fail

Symptoms

Diagnosis

Common Causes

9. NapCat QQ Bot Doesn't Respond to Commands

Symptoms

Diagnosis

Common Causes

10. Emergency Contact / Finding History

Next Steps