OpenInvest

English

中文

English

中文

Appearance

Production Deployment ​

Caddy + Cloudflare Access + systemd + GitHub Releases dist distribution. This chapter covers the complete chain for "I want to run it on my own server."

← 07-extending · Wiki Index · 09-troubleshooting →

Topology Overview ​

Browser
  ↓ HTTPS
Cloudflare (DNS Orange Cloud + Access JWT Authentication)
  ↓ HTTP (Flexible mode, origin server doesn't need a certificate)
Your Server: Port 80 / 443
  ↓
Caddy Reverse Proxy (caddy-gateway container)
  ├─ /api/*  → reverse_proxy 127.0.0.1:8765   (FastAPI invest-web.service)
  └─ /*      → file_server /srv/invest-gui/   (Static files)

Why this setup:

  • No SSR daemon → A single 1G VPS is sufficient (referencing the mc-website lesson: next start caused the entire machine to hang three times)
  • CF Access handles authentication at the edge → No auth code needed in the backend
  • 127.0.0.1 binding → Port 8765 is not exposed to the public internet
  • Caddy serves static files directly → GUI upgrade = rsync dist/ to /srv/invest-gui/, no reload required

0. One-Click Self-Hosting with Containers (Docker Compose / GHCR) ​

Don't want to manually install uv / configure systemd? Use containers. The image ghcr.io/longsizhuo/openinvest is automatically published by publish-image.yml for each backend version tag (v*). The GUI is baked into static/ during the build phase—run docker compose up and view the complete dashboard directly in your browser. To protect it with CF Access, you can still use the Caddy setup from Sections 1–2 below (reverse proxying the container's 127.0.0.1:8765).

Prerequisites: Docker + Docker Compose v2.

bash
git clone https://github.com/longsizhuo/openInvest.git && cd openInvest
cp .env.example .env && $EDITOR .env       # At minimum, fill in DEEPSEEK_API_KEY (it can start without .env, but the committee won't run)

Onboarding (creating memory/)——invest-agent (scheduler) will refuse to start if memory/user.md is missing. Use a one-time command via invest-web (the entrypoint: ["/bin/sh","-c"] in invest-agent will swallow additional arguments):

bash
docker compose run --rm invest-web python -m scripts.skill init
# Or, in Claude Code, say "help me initialize invest" to go through the 5 questions

Start the service:

bash
docker compose up -d --build                 # Local build (first time takes a few minutes: uv sync + build GUI)
# —— Or pull pre-built images (faster, requires the package to be Public or `docker login ghcr.io` first) ——
docker compose pull && docker compose up -d

Open http://localhost:8765 in your browser → Full GUI.

ServiceRolePort
invest-webFastAPI + GUI (uvicorn bound to 0.0.0.0:8765)Host 127.0.0.1:8765 (default binds only to loopback)
invest-agentScheduler: runs jobs/*.yml (daily_report / pnl_snapshot…)None
  • Expose to LAN/Public Internet: Change invest-web's ports to "8765:8765" and set INVEST_API_TOKEN; or keep it on loopback and put Caddy + CF Access in front (see Sections 1–2 below).
  • Data Persistence: memory/ (ledger, required) / db/ / cache_data/ / logs/ are all bind-mounted to the host, so data survives container recreation.
  • Image Visibility: GHCR packages are private by default—to pull anonymously with docker compose pull, you must set the package to Public on GitHub Packages (or use docker login ghcr.io).

1. One-Time Server Configuration ​

1.1 Install uv + Clone Repository ​

bash
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/longsizhuo/openInvest.git ~/openInvest
cd ~/openInvest
uv sync --frozen --python 3.13
cp .env.example .env
# Edit .env to fill in DEEPSEEK_API_KEY / EMAIL_* / etc.

1.2 Pull Frontend Dist ​

bash
uv run python -m scripts.sync_gui_dist
# = Pulls invest-gui-dist.tar.gz from the invest-gui dist-latest release
# = Extracts to static/ (used by FastAPI's built-in GUI mount)

There are two GUI deployment methods, they coexist without conflict:

PathServed ByUpgrade Method
<repo>/static/FastAPI mountpython -m scripts.sync_gui_dist (pull release)
/srv/invest-gui/Caddy file_servercd invest-gui && pnpm deploy (rsync local build)

Production Caddy prefers the latter; FastAPI mount is for single-machine one-click scenarios.

1.3 systemd Unit ​

Copy the unit file provided in the repository:

bash
sudo cp systemd/invest-web.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now invest-web
sudo systemctl status invest-web

Key fields in the unit:

ini
WorkingDirectory=%h/openInvest
EnvironmentFile=%h/openInvest/.env
ExecStart=%h/.local/bin/uv run --no-sync uvicorn connectors.web_api:app --host 127.0.0.1 --port 8765
Restart=on-failure
ProtectSystem=strict
ReadWritePaths=%h/openInvest

→ Only writes to the invest directory (systemd hardening). → Restart=on-failure: Automatically restarts if the process crashes, but you must manually restart to load new code after changes.

1.4 Caddy Site Configuration ​

Add to caddy-gateway/Caddyfile:

caddyfile
http://invest.your-domain.com {
    encode gzip

    handle /api/* {
        reverse_proxy 127.0.0.1:8765
    }

    handle {
        root * /srv/invest-gui

        @viteAssets path /assets/*
        header @viteAssets Cache-Control "public, max-age=31536000, immutable"
        @html path *.html /
        header @html Cache-Control "no-cache"

        file_server
        try_files {path} /index.html   # SPA route fallback
    }
}

After modifying the Caddyfile, you must restart (not reload)—reload doesn't read new inodes in bind mount mode:

bash
docker restart caddy

See memory feedback_caddy_bind_mount_reload for details.

2. Cloudflare Access Configuration ​

2.1 DNS ​

CF Dashboard → DNS → Add an invest A record pointing to your server IP, enable the orange cloud (route through CF proxy).

2.2 Access Application ​

CF Zero Trust → Access → Applications → Add a self-hosted application:

FieldValue
Application nameinvest
Application domaininvest.your-domain.com
Session duration30 days (or 24h, as you prefer)
Identity providersOne-time PIN (email)

2.3 Access Policy ​

Action: Allow
Include: Emails → your-email@gmail.com

→ Only your email can access.

2.4 Verification ​

  • Log out of all CF email sessions
  • Visit https://invest.your-domain.com in your browser
  • You should be redirected to the CF email verification page → Enter OTP → Access the GUI

3. Upgrade Process ​

Upgrade Backend ​

bash
cd ~/openInvest
git pull origin main
uv sync   # If there are dependency changes
sudo systemctl restart invest-web

Important: The systemd unit does not automatically reload new code. You must restart after every push.

Upgrade Frontend ​

bash
# Option A: Pull from GitHub Releases
cd ~/openInvest
uv run python -m scripts.sync_gui_dist
# Note: Only updates static/, does not touch /srv/invest-gui/

# Option B: Local build + rsync to Caddy serve directory (for production)
cd ~/invest-gui
git pull origin main
pnpm install
pnpm deploy   # = scripts/deploy.sh, rsync dist/ → /srv/invest-gui/

Caddy does not need a reload (static file changes). The browser might cache the HTML (even though the Caddyfile sets no-cache) → Users should hard refresh once.

Upgrading Both Frontend and Backend Simultaneously ​

The safest order:

  1. Push frontend and backend code
  2. Deploy frontend first on the server (pnpm deploy) — Users get the old API with the old frontend, works fine
  3. git pull backend
  4. sudo systemctl restart invest-web — Switch to the new API
  5. Users hard refresh → Frontend loads new hashed JS → Calls new API ✓

Why frontend first: A new frontend calling the old API won't crash (backward compatible fields), and an old frontend calling the new API also won't crash. Conversely: An old frontend calling an endpoint that the new frontend depends on will get a 404.

4. Environment Variables ​

Required ​

bash
DEEPSEEK_API_KEY=sk-xxx
DEEPSEEK_BASE_URL=https://api.deepseek.com
INVEST_WEB_HOST=127.0.0.1
INVEST_WEB_PORT=8765

Optional ​

bash
# CommSec Email Import (for Australian stock users)
EMAIL_SENDER=you@gmail.com
EMAIL_PASSWORD=app-password-16-chars

# Send email after committee runs
SMTP_HOST=smtp.gmail.com
SMTP_USER=you@gmail.com
SMTP_PASS=app-password
SMTP_TO=you@gmail.com

# NapCat QQ Bot
NAPCAT_WS_URL=ws://localhost:6101
NAPCAT_HTTP_URL=http://localhost:6100
INVEST_WHITELIST_QQ=12345678   # Required, otherwise napcat rejects all

# Committee behavior switches (can also be changed at runtime via GUI/API/CLI, ADR-017; env is only the deployment default)
# Concentration lens: false=Single asset/intentional concentration/full investable pool not advised to reduce due to concentration (ADR-019)
INVEST_VERDICT_CONCENTRATION_LENS_ENABLED=true

# Development environment (Vite cross-origin calls)
INVEST_WEB_DEV_CORS=1   # Only use for local dev, do not enable in production

.env.example is the Source of Truth ​

New env vars must be added to .env.example, otherwise fork users won't know about them.

5. Hardening Checklist (Server hardening 2026-05) ​

Refer to memory project_server_2026_05_02_hardening:

  • ✅ rpcbind / portmapper disabled
  • ✅ unattended-upgrades for automatic security patches
  • ✅ All 5 services bound to 127.0.0.1 (ports not exposed to the public internet)
  • ✅ wcpp service stopped (no longer used)
  • ✅ Caddy only allows invest.* longsizhuo.com and mc.involutionhell.com
  • ✅ ufw only allows ports 22 / 80 / 443
  • ⏳ TODO: Caddy CF IP whitelist (prevent bypassing CF to connect directly to the origin server)

6. systemd Service List ​

bash
sudo systemctl list-units --type=service | grep invest
# invest-web.service          uvicorn FastAPI :8765
# invest-scheduler.service    APScheduler runs all jobs/*.yml

invest-scheduler.service (if you run cron) unit is similar:

ini
ExecStart=%h/.local/bin/uv run --no-sync python -m scheduler.runner

See systemd/README.md and scheduler/README.md for details.

7. Backup Strategy ​

Authoritative State is Split Across Two Stores—Don't Just Look at memory/ ​

"Money" isn't only in memory/. There are two pieces of authoritative state that cannot be rebuilt, they must be backed up together:

Authoritative StateWhat it isConsequence of Loss
memory/portfolio.mdCurrent holdings + cashLose "what I currently hold"
db/trades.dbTransaction ledger (planned→executed)Lose 9 months of transaction history
memory/{.committee,insights,daily,.dreams}Historical decisions / insightsTheoretically regenerable, but requires a lot of LLM tokens
.envCredentials + configurationRe-apply / re-fill

⚠️ Common Misconception: Thinking the entire db/ is a disposable market data cache. db/trades.db is a ledger, not a cache—migrating with the assumption that "db/ doesn't need backup" will lose the entire transaction history. Only the databases listed in the "No Backup Needed" section below are disposable.

scripts/snapshot.py packages the above authoritative state into a single tar.gz (WAL-safe sqlite online backup + sha256 checksum). A single command on the new machine restores everything. Use this to migrate the hub to a new machine:

bash
# Old machine: Package (does not include .env secret values, only lists which keys need to be filled in the manifest)
uv run python -m scripts.snapshot snapshot --out ~/invest-snapshot.tar.gz

# New machine: After clone + uv sync, restore (default refuses to overwrite existing ledgers, --force to overwrite)
uv run python -m scripts.snapshot restore --in ~/invest-snapshot.tar.gz

Offsite Backup (Weekly Automatic) ​

~/openinvest-research-archive/refresh.sh (cron 0 3 * * 0) pushes decisions/insights and the ledger (portfolio.md + trades.db, placed in invest_ledger/) to the private repo openinvest-research-archive every week. This is currently the only off-machine copy—don't accidentally delete that cron job.

bash
# Daily cron—directly reuses snapshot.py, one tar containing all authoritative state
0 4 * * *  cd $HOME/openInvest && $HOME/.local/bin/uv run python -m scripts.snapshot snapshot --out /backup/invest-$(date +\%F).tar.gz

Cloudflare Access Configuration ​

The Access policy on the CF Dashboard side doesn't have a git backup. It's recommended to manually take a screenshot of the policy JSON.

No Backup Needed (Auto-rebuilt on first run on new machine) ​

  • db/market_data.db / db/events.db (Market data + news cache, can be re-fetched)
  • db/chroma.sqlite3 / db/jobs.sqlite (Vector store / scheduler job store, can be rebuilt)
  • cache_data/ (HTTP cache)
  • static/ (Frontend dist, scripts.sync_gui_dist re-pulls it)
  • memory/.backtest* (Backtest research artifacts, not a ledger)

8. Monitoring (Optional) ​

Backend logs:

bash
sudo journalctl -u invest-web -f --since "1 hour ago"

CF Access logs: CF Zero Trust → Logs → Access.

GUI data source health: https://invest.your-domain.com/system → "Data Sources" tab.

9. Multi-Device: Hub-and-Spoke Remote Mode (2026-06) ​

One machine (hub) holds the single memory/ and runs web_api; other devices (laptop/another dev machine) set INVEST_API_BASE in their CLI / Claude Code skill, and all subcommands are forwarded to the hub via HTTP for reads and writes—locking is still the single-machine fcntl on the hub, zero distributed complexity. Centralized parameter tuning is automatic: strategy / role prompts are all on the hub, changing one place affects all devices.

Laptop (client)                         Hub (local machine/VPS)
  run.sh status ──HTTP──┐                invest-web.service :8765
  run.sh buy ...        ├──────────────▶   ├─ /api/skill/*      (CLI equivalent endpoints)
  prepare_committee ────┘                  ├─ /api/committee/*  (prepare/save/run)
  (local machine has zero memory/)         └─ memory/  ← Single ledger, fcntl lock

Hub Side (Zero Required Changes for Existing Deployments) ​

If you already have the Caddy + CF Access deployment, you don't need to change anything—the client can use a CF Access Service Token to pass through the edge (recommended, see below). Only for LAN scenarios without CF:

bash
# .env
INVEST_API_TOKEN=<openssl rand -hex 24>   # Enable application-layer authentication
INVEST_WEB_HOST=0.0.0.0                   # Bind outside loopback (for direct LAN connections)

Token semantics: Access to /api/* (excluding /api/health) from non-loopback sources requires Authorization: Bearer; behavior is completely unchanged if the token is not set.

Client Side (2 minutes) ​

bash
# clone + uv sync as usual (run.sh auto-runs on first run), then .env only needs:
INVEST_API_BASE=https://invest.your-domain.com   # or http://10.0.0.x:8765
INVEST_API_TOKEN=...                             # Only needed if hub has it enabled
~/.claude/skills/invest/scripts/run.sh doctor    # Verify: status ready + remote section

The client has no memory/, and doesn't need a DeepSeek key / Gmail credentials. init is disabled in remote mode; run_committee runs on the hub (CLI auto-polls); live_prices / correlate still run locally. Write operations go to the hub's ledger, history is recorded with source: skill_remote.

  1. CF Zero Trust → Access → Service Auth → Create Service Token, get Client ID / Secret
  2. Add a Service Auth include rule to the Access Application (invest domain) Policy
  3. Client .env:
    bash
    INVEST_API_BASE=https://invest.your-domain.com
CF_ACCESS_CLIENT_ID=xxx.access
CF_ACCESS_CLIENT_SECRET=yyy
    The remote dispatch will automatically include the CF-Access-Client-Id/Secret headers; browser users still go through SSO. The hub continues to bind only to 127.0.0.1, zero backend changes.

Known Limitations ​

  • Synchronous committee runs can block the uvicorn event loop for 5-10 minutes (an existing limitation, amplified by remote mode): the hub should use --workers 2 for multi-process deployment; all long tasks should use a "trigger + poll" approach to avoid hitting the CF ~100s proxy timeout.
  • The "same-day cache" between client and hub is based on the hub's date (taken from the /api/health timestamp), so cross-timezone devices won't get out of sync.

Next Steps ​

→ 09-troubleshooting.md — Where to look if the deployment fails

→ adr/ — Why this deployment model (no SSR / no K8s)