Forecast what the evidence allows.

Part I / The question

The project began with a familiar goal: predict whether a stock will rise or fall. Instead of tuning toward a headline accuracy number, every model was held to a baseline it had to beat on data it had never influenced. Across twenty-four trained models, no daily direction forecast, next-day or five-day, cleared that bar. Two other targets did: how far a stock is likely to move, and which stocks are likely to beat the others over the next week.

Those two results became the product. After every trading session the system forecasts the next session's move size with a calibrated range for fifty large caps, ranks one hundred large and mid caps for the following five sessions, writes both as immutable files, and scores them once the outcomes exist. The up/down probability is shown as the historical base rate, labeled as such.

Part I / The question

Collection, feature building, training, FinBERT scoring, and inference need LightGBM, a transformer model, and the full dataset, so they run on GitHub Actions runners rather than in request-time functions. Vercel serves the results: a read-only FastAPI function and a static React dashboard. The private S3-compatible archive is the immutable record; PostgreSQL holds a query copy for the API.

Part II / Data you can audit

The pipeline uses only free data. Every HTTP response, including failures, is wrapped in an envelope with its request, status, selected headers, and body, then stored under the SHA-256 of its content before any parsing happens. Query credentials and any echoed secrets are redacted from the archived bytes. Collectors are rate limited, honor Retry-After, bound response sizes, and resume from checkpoints that only advance after the remote write is confirmed.

Several sources failed in practice: GDELT article search returned HTTP 429 and tripped a circuit breaker, and nineteen investor-relations pages returned HTTP 403. Those failures are retained as missing coverage. They are never converted into zero-news features, because a source that failed is different from a day with no news.

Part II / Data you can audit

Cleaning works on the exchange calendar, not on calendar days. Of 138,210 bars, 138,202 are valid canonical rows with no missing sessions. Seven unsupported corporate-action or identity boundaries and one unexplained price jump are quarantined: their future-label intervals are excluded and feature history restarts after them. Range features also cap erroneous provider wicks, such as a pre-split print, at four times the stock's previous 22-session volatility.

Features use only information available by the session close plus thirty minutes. Labels are generated by separate code from future prices. Blocks are split on whole session dates so every stock for a day stays together, and examples whose label would reach into the next block are purged at each boundary.

The first dataset carries 34 core features; the next-day work extends them to 191 price factors in the style of Alpha158. Later studies add volatility, earnings-event, earnings-surprise, news-volume, sentiment, cross-sectional, and slow-momentum groups, each built with the same cutoff rule and checked by tests that change only future rows and confirm features are unchanged.

Part III / What the models found

Direction was attacked from every reasonable angle: tree ensembles, linear and ARIMA baselines, recurrent and convolutional networks, a residual MLP, a 209M-parameter time-series foundation model, macro vintages, SEC fundamentals, historical headlines, FinBERT sentiment, and FinBERT fine-tuned on the headlines themselves. Each was compared on identical rows with a fixed selection rule.

A diagnostic explains why this was expected. Even with perfect knowledge of the next day's market return, three-class accuracy on the 2025-2026 block would only reach 48.3%. Public headlines available at the close carry no measurable next-day direction signal for these large caps; the market has already priced them.

Part III / What the models found

The two-head design separates what is predictable from what is not. The magnitude head forecasts the expected absolute next-session return with LightGBM, an 80% quantile band with split-conformal scaling, and a calibrated big-move probability. The direction head chooses between the training prior, logistic regression, and LightGBM by inner-validation log loss. Three-class probabilities are derived from both heads for display only.

Lower QLIKE is better, and the model beats the HAR volatility baseline in every year. Sorting forecasts into quintiles gives realized average moves of 0.80%, 1.00%, 1.18%, 1.46%, and 2.31%, so the top quintile moves 2.9 times as much as the bottom. Earnings-date features account for most of the gain; earnings surprise, news volume, and FinBERT sentiment were each promoted only after improving on price-only in at least five of six years.

The production configuration keeps price plus the promoted groups, sets direction to the training prior, and was refit on all labeled history: QLIKE −7.155 against −6.966 for HAR, 80.6% band coverage, and a big-move AUC of 0.591. It scores marginally below the all-groups model because the promotion rule dropped a group that only helps in combination. The rule was not overridden after seeing those test years.

Part III / What the models found

Ranking reframes direction as a relative question: which stocks will beat the others? LightGBM is trained on each row's percentile rank of forward return within its session, over 445 inputs that are rank-normalized per session. Evaluation uses daily Spearman rank IC with t-statistics on non-overlapping sessions, and a long-short portfolio of the top and bottom quintiles, net of costs.

At five sessions the model beats 12-1 month momentum, five-day reversal, and low-volatility baselines. Most of the edge comes from the mid-cap cohort, which also carries the most survivorship bias. The reported Sharpe of 0.97 came from the most favorable rebalance day; averaged across all five start days it is 0.73, so the honest reading is roughly 0.5 to 0.75, with rank IC as the more stable measure.

A GPU comparison later trained eight rankers, including XGBoost, CatBoost, LambdaRank, an MLP, and an ensemble, with the winner chosen in advance by inner-year IC. The pre-selected ensemble failed its gate, and the model with the best test IC had ranked second-last on the inner years. Promoting it would have meant choosing on test results, so the live ranker stayed unchanged.

Part IV / Evaluation discipline

Every study uses expanding yearly walk-forward folds: six test years, each preceded by an inner-validation year used for early stopping, calibration, conformal scaling, and model choice. Training labels end before the inner year, inner labels end before the test year, and rows whose outcome would fall in the reserved prospective block are never touched.

The first trained bundle bound its final test archive to the exact recipe that selected it, so a new recipe could not reuse that holdout for another selection. Recipe hashes cover configuration, input checksums, and source files; an identical command reuses completed folds instead of refitting. A unit test scrambles every test-block label and confirms that predictions do not change.

Part V / The live system

A scheduled workflow runs at 22:45 UTC on weekdays, more than thirty minutes after the close in both EST and EDT. It pulls live state from the archive, refreshes bars, SEC filings, and news, scores only headlines that have no FinBERT score yet, rebuilds every feature group with the exact study code, and forecasts the newest session with the production bundle.

Each forecast row records the expected absolute move, the 80% band, P(big move), P(up), derived class probabilities, origin and target sessions, the origin close, and the model recipe that produced it. A parity check rebuilt the 191 price features through the live path and matched the frozen dataset on all 4,700 overlapping rows.

Part V / The live system

Both live models retrain weekly through a champion/challenger check. The champion stays frozen between retrains so each logged forecast tests the model that made it. A retrain becomes due once five new sessions of outcomes have matured after the champion's training data ends; the trigger is data-driven, so a missed day only delays it.

A passing challenger is refit on all history including the new week, saved as a new checksummed bundle, and the production pointer moves to it. Every evaluation, promoted or not, is appended to a JSON Lines registry, published to the model events table, and shown in the dashboard's track record. A week adds about 250 move-size rows to roughly 120,000, so retraining changes the model gradually; the gate exists to stop a bad data week or a broken recipe from reaching production.

Part V / The live system

After the daily jobs, the workflow exports a dashboard snapshot, publishes forecasts, rankings, scorecards, and model events to PostgreSQL, builds the static dashboard, deploys it to Vercel, and smoke-tests the API. Publishing steps run even when an earlier step fails, so a partial day still leaves an inspectable record and logs retained for ninety days.

The API exposes health, the full dashboard snapshot, latest and per-symbol forecasts and rankings, scorecards, and model events. Responses are CDN-cached for five minutes with stale-while-revalidate, symbols are validated, and the health check reports an unreachable database as a generic 503 without connection details. The dashboard loads the newer of the API response and a snapshot bundled at build time, and says which one it is showing.

Part VI / Verification & tradeoffs

The repository carries tests for causality, lagged volume, exchange-calendar gaps, corporate-action reconciliation, boundary purging, holdout access order, scrambled-label invariance, artifact corruption, cached-run reuse, and interrupted-training resume. Metrics in each report were recomputed independently from saved predictions, and reloaded native-text bundles reproduced in-memory predictions. FinBERT scores computed on a Colab GPU matched a CPU re-score of a random sample exactly.

The scorecard is the only evidence free of retrospective selection, and its samples start small: one session gives fifty rows exposed to the same market day. Band coverage and rank correlation become meaningful after a few weeks; ranking results need months.

• Decide the gate before opening the test, and keep the decision when it is inconvenient.
• Report the failures; they explain why the published forecasts are shaped the way they are.
• Archive raw bytes and pin every input so any number can be traced and rerun.
• Keep heavy compute in a scheduled pipeline and serve its results read-only.
• Log forecasts before outcomes exist; it is the only test that cannot be fitted after the fact.