Neural Trader Workflows

This repository trains reinforcement-learning agents for both VizDoom and portfolio management tasks. The notes below focus on the trading pipeline that relies on PPO agents and the new orchestration helpers.

Environment Setup

./tools/setup_local_env.sh
source .venv/bin/activate
python -c "import sys; print(sys.executable)"

The setup script always installs into the repository-local .venv and verifies that python resolves to that environment. It installs the pinned runtime and test dependencies from requirements.txt and ops_requirements.txt, including TensorFlow 2.20, Keras 3.14, PyTorch 2.10, and the FastAPI operator console stack.

Tests default to CPU by hiding CUDA in tests/conftest.py; this keeps local smoke runs reliable when a GPU is present but busy. For GPU training, activate the .venv, export the CUDA device you want, and check availability first:

source .venv/bin/activate
nvidia-smi
CUDA_VISIBLE_DEVICES=0 python train_alphazero_portfolio.py --help

Single-Dataset Training

stock_player.py now accepts CLI overrides so you can point PPO training at any CSV and control output locations:

source .venv/bin/activate
python stock_player.py \
  --csv data_sets/LHA.DE.csv \
  --checkpoint-dir ppo_checkpoints/LHA.DE \
  --log-dir ppo_logs/LHA.DE \
  --total-updates 4000 \
  --num-envs 8 \
  --rollout-horizon 128

Checkpoints and TensorBoard logs are created on demand when the script starts.

Batch Training Across Datasets

Use train_all_datasets.py to iterate every CSV under data_sets/ (or a subset) and write per-symbol artifacts:

source .venv/bin/activate
python train_all_datasets.py \
  --datasets-dir data_sets \
  --checkpoint-root ppo_checkpoints \
  --log-root ppo_logs \
  --passes 1                     # 0 = loop indefinitely

Each dataset gets its own folder (for example, ppo_logs/BMW.DE). Add --enable-simulator or --disable-simulator to control synthetic market usage across all runs.

Running Until a Local Time

train_until.py wraps the batch runner with a timeout guard. Example: train continuously until 08:00 local time while looping through all datasets:

source .venv/bin/activate
python train_until.py --until 08:00 --passes 0

Additional flags mirror those in train_all_datasets.py (datasets directory, log roots, simulator toggle, hyper-parameters). Launch inside tmux/screen so the session survives disconnects.

source .venv/bin/activate
python train_until.py --until 08:00   --datasets-dir data_sets \
  --checkpoint-root ppo_checkpoints \
  --log-root ppo_logs --passes 0

Monitoring

TensorBoard logs live under ppo_logs/<dataset>. Start TensorBoard with:

source .venv/bin/activate
tensorboard --logdir ppo_logs

Evaluation

After training, evaluate the latest checkpoint for a dataset using the helper in neural_trader.trading.eval:

source .venv/bin/activate
python - <<'PY'
from neural_trader.market_env import MarketConfig
from neural_trader.trading.eval import EvaluationConfig, evaluate_policy

cfg = EvaluationConfig(
    market=MarketConfig(csv_path="data_sets/LHA.DE.csv"),
    checkpoint_dir="ppo_checkpoints/LHA.DE",
    episodes=10,
)
print(evaluate_policy(cfg))
PY

Detailed Analysis & Reporting

For richer diagnostics (time-series CSV plus PNG charts for net worth, trade counts, and cash), run the dedicated analyser. It fixes the dataset, timeframe, and initial capital while replaying a saved policy:

source .venv/bin/activate
python analyze_policy.py \
  --dataset data_sets/LHA.DE.csv \
  --checkpoint-dir ppo_checkpoints/LHA.DE \
  --timeframe 1y \
  --initial-cash 150000 \
  --output-dir reports/LHA_eval

Outputs include:

• timeseries.csv – per-step actions, rewards, net worth, and cumulative trade counts.
• summary.json – final capital, returns, total/positive trades, and cumulative reward.
• net_worth.png, cash.png, trades.png – quick-look charts for overnight reviews or presentations.

Comparing Multiple Checkpoints

To rank checkpoints after long training runs, generate a consolidated report:

source .venv/bin/activate
python compare_checkpoints.py \
  --dataset data_sets/LHA.DE.csv \
  --checkpoint-dir ppo_checkpoints/LHA.DE \
  --timeframe 1y \
  --initial-cash 100000 \
  --output-dir reports/LHA_compare

The tool evaluates each ckpt-* snapshot, stores per-checkpoint charts/CSV in subfolders, and produces report.md plus summary_table.csv with key metrics and an aggregate net-worth comparison plot.

Live Operator Console

The simplified live app is the operational entry point for daily use:

• /portfolio stores the stocks and share counts currently held.
• /signals/open shows current trade recommendations and can run a check immediately.
• /universe controls the stock symbols considered by refresh and recommendation runs.
• /settings selects the active strategy, Telegram chat id, and cron schedules.

Run it locally with the repository virtualenv:

source .venv/bin/activate
APP_BASE_URL=http://127.0.0.1:8080 \
ALLOWED_OPERATOR_EMAILS=operator@example.com \
uvicorn ops_app.app:create_app --factory --host 127.0.0.1 --port 8080

Run it in Docker with persistent SQLite storage:

cp .env.example .env
# edit .env before running on a server
docker compose up --build -d ops-web

The price refresh job downloads enabled /universe symbols into LIVE_DATASETS_DIR (/data/market_data in Docker). For non-EUR quote currencies it also downloads Yahoo FX pairs such as EURUSD=X, then the live portfolio uses the latest downloaded FX close for EUR valuation. FX_RATES_TO_EUR remains available as an explicit override, with static rates only used as fallback. The portfolio page shows both the EUR valuation and the local quote/currency/source so bad currency metadata is visible.

New databases seed Stock Setup from DEFAULT_WATCHLIST_UNIVERSES, which defaults to global_major_indexes across America, Europe, and Asia. Existing databases can append that same set from Stock Setup with Add global defaults. The recommendation job runs after the refresh schedule and sends Telegram messages when Telegram alerts is enabled and a bot token plus chat id are configured. Email and Telegram notification copy follows the operator's selected UI language. Recommendation identifier cards expose ISIN/WKN values and copy buttons for broker search entry.

For server setup, reverse proxy notes, updates, archive distribution, and backups, see docs/server_deployment.md.

Tests

Run the regression suite before publishing results:

source .venv/bin/activate
python -m pytest -q

The operator console UI is covered by FastAPI/TestClient tests that exercise login, navigation, ticket decisions, fills, settings, and audit visibility.

Market Data Refresh

Refresh the current DAX-style local datasets in place:

source .venv/bin/activate
python tools/refresh_market_data.py \
  --datasets-dir data_sets \
  --refresh-existing \
  --canonicalize-existing-paths \
  --workers 8

Download a new universe without changing the base data_sets/ directory:

source .venv/bin/activate
python tools/refresh_market_data.py \
  --universe global_major_indexes \
  --output-dir data_sets_expanded/global_major_indexes

The other built-in manifests remain available for narrower experiments, for example nasdaq100, dax, and emerging_markets_watchlist.

Reference docs:

• Market Data Workflow
• Asset Universe Recommendation

Expanded-universe portfolio benchmark example:

source .venv/bin/activate
python benchmark_portfolio_strategies.py \
  --datasets-dir data_sets \
  --datasets-dir data_sets_expanded \
  --recursive-datasets \
  --workers 16