CatalystAlert

Search CatalystAlert

Search for companies, drugs, and catalysts

Search CatalystAlert

Search for companies, drugs, and catalysts

Important Disclaimer

Past performance is not indicative of future results. These predictions are for informational purposes only and should not be considered financial advice. Always conduct your own research and consult with a qualified financial advisor before making investment decisions.

ML Prediction Track Record — Archive

Archived results from before our May 31, 2026 model recalibration. We reset the public record then — the earlier models were trained partly on contaminated data (earnings / large-caps) that inflated their numbers and would not generalize, so we deliberately do not headline them. Kept here in full for transparency.

Last updated: 10/2/2026 | Period: Before relaunch (archived)

Direction Accuracy
Correct up/down/neutral predictions
67.5%
of 474 predictions
Return Accuracy
Within 10% of actual return
80.0%
magnitude accuracy
vs Naive Baseline
Direction accuracy minus the “always neutral” guess
-23.0 ptsbaseline 90.5%

~80% of catalyst moves are neutral, so a model only adds value if it beats the “always-neutral” guess. We show this so the headline can't flatter itself.

Accuracy by Specialized Model
Performance of our category-specific ML models trained on different catalyst types

Regulatory (PDUFA/NDA/BLA)

83.3%
45/54
Model: impact_pdufa
Types: pdufa, nda_filing, regulatory

Clinical Trials (Phase 1/2/3)

68.7%
270/393
Model: impact_trial
Types: phase_3, phase_2, data_readout

Other Events

18.5%
5/27
Model: impact_other
Types: conference, offering, partnership

Each category uses a specialized model trained on its catalyst type. Categories fill in as their catalysts resolve; ones with nothing resolved yet show “No data yet” rather than a 0% that would read as failure. Every figure is measured against the same actual market outcomes.

Entry-Timing (pre-catalyst run-up)
Entry predictions forecast the run-up — the move from ~30 days before to ~3 days before the catalyst — a different target than the event-day predictions above, so it's tracked separately. The honest yardstick is the hit-rate (did the run-up materialise) and whether higher conviction lifts that hit-rate.
Run-up hit-rate
37.9%
run-up ended positive · 1,441 calls
Meaningful run-up
33.5%
run-up beat +2% (beyond noise)
Median run-up (realized)
-2.6%
median of 1441 calls; mean -1% (outlier-skewed on small n)

Hit-rate by conviction

If the signal carries an edge, higher-conviction calls hit more often.

High conviction (≥0.7)
avg -3.8%0/20.0%
Medium (0.5–0.7)
avg -1%524/136838.3%
Low (<0.5)
avg +0.5%22/7131.0%

Coverage by catalyst type

Entry: Phase 2
784 calls
Entry: Phase 3
481 calls
Entry: Conference
39 calls
Entry: Earnings
33 calls
Entry: Other
31 calls
Entry: Pdufa
19 calls
Entry: Bla Filing
16 calls
Entry: Partnership
15 calls
Entry: Data Readout
11 calls
Entry: Regulatory
4 calls
Phase 2
2 calls
Data Readout
2 calls
Entry: Nda Filing
1 calls
Phase 3
1 calls
Conference
1 calls
Offering
1 calls

For reference, 3-way direction accuracy is 40.9% — but direction isn't the model's objective (it forecasts run-up conviction and almost always expects a positive run-up), so the hit-rate and conviction lift above are the meaningful measures.

Dilution Risk (new model — live tracking)self-graded vs SEC filings
Predicts whether a company prices a share offering within 90 days after its catalyst (SEC 424B filing). Every prediction is timestamped before the event and automatically graded against real SEC filings once the 90-day window elapses — no self-reporting involved.
Live predictions
1,000
upcoming catalysts scored
Graded so far
64
resolved against SEC filings
Validation (held-out history)
11% vs 33%
realized raise rate, LOW vs ELEVATED bucket — backtest, not live results
low bucket — live
12% raised
predicted 16% · 34 graded
elevated bucket — live
20% raised
predicted 29% · 30 graded

Graded against: SEC EDGAR 424B filings (priced offerings), resolved ~95 days after each event. Model: calibrated XGBoost on cash runway, filing cadence (time since last raise, active shelf), recent run-up and market cap — holdout AUC ~0.65. An offering is not always negative (it extends the runway); the exact timing within the window is not predictable.

Prediction vs Actual Distribution
Compare predicted directions against actual outcomes

Predictions Made

Up105
Down30
Neutral339

Actual Outcomes

Up15
Down30
Neutral429
Accuracy by Catalyst Type
Performance breakdown for each catalyst category

Model Performance Varies by Catalyst Type

Our ML model performs best on PDUFA events (FDA decisions) where historical patterns are more predictable. Phase 2/3 clinical trial predictions have limited accuracy due to binary outcomes and market efficiency. We are actively working to improve non-PDUFA predictions.

Nda Filing
1/1 correct
100.0%
Data Readout
Limited Data
18/19 correct
94.7%
Pdufa
44/51 correct
86.3%
Phase 3
Limited Data
86/125 correct
68.8%
Phase 2
Limited Data
166/249 correct
66.7%
Offering
2/3 correct
66.7%
Conference
Limited Data
3/22 correct
13.6%
Partnership
Limited Data
0/2 correct
0.0%
Regulatory
Limited Data
0/2 correct
0.0%
Prediction Confusion Matrix
How predictions compare to actual outcomes (rows = predicted, columns = actual)
Up
Down
Neutral
Predicted Up3993
Predicted Down5322
Predicted Neutral718314

Diagonal values (highlighted) represent correct predictions

Weekly Accuracy Trend
Historical accuracy by week
WeekPredictionsAccurateAccuracy
9/21/202611100.0%
9/14/202616531.3%
8/31/20263266.7%
8/10/20269555.6%
7/27/20268562.5%
7/13/202614642.9%
7/6/202655100.0%
6/29/2026281346.4%
6/22/20268787.5%
6/15/2026683855.9%
Methodology

How This Track Record Works

Real ML Predictions

This track record shows actual predictions made by our ML model on historical biotech catalyst events. Each prediction is compared against the real market outcome to measure accuracy.

Predictions include price direction (up/down/neutral) and expected return percentage. The same ML models power our Entry Timing and Opportunities features.

Prediction Types

  • Impact Prediction: Expected price movement direction and magnitude after catalyst
  • Entry Timing: Optimal days before catalyst to enter a position (Pro tier)
  • LOA Score: Likelihood of Approval probability for drug catalysts

Specialized ML Models

  • PDUFA Model (impact_pdufa): Trained on FDA regulatory decisions (PDUFA, NDA, BLA filings). Achieves highest accuracy due to predictable FDA decision patterns.
  • Trial Model (impact_trial): Trained on clinical trial data readouts (Phase 1/2/3). Uses trial-specific features like phase difficulty, indication complexity, and company track record.
  • Other Model (impact_other): Trained on other catalyst types (conferences, earnings, AdCom). Handles diverse event types with varying prediction difficulty.

Outcome Classification

  • Up: Stock price moved >5% within 5 days after catalyst
  • Down: Stock price moved <-5% within 5 days after catalyst
  • Neutral: Stock price moved between -5% and +5%

Accuracy Metrics

  • Direction Accuracy: Did the model correctly classify up/down/neutral? This validates the Hist. positive rate shown on Opportunities
  • Return Accuracy: Was the modelled move within 10% of actual? This validates the Avg. historical move shown on Opportunities
  • High Accuracy Rate: Percentage of predictions with accuracy score ≥ 80%
How This Validates Opportunities

The metrics you see on the Catalyst Entry Analysis page are powered by the same ML models validated here. The Hist. positive rate corresponds to our Direction Accuracy, while theAvg. historical move corresponds to our Return Accuracy. This track record provides transparency into how well those models have performed historically.

Data Sources

  • Price Data: Daily historical stock prices from our market-data providers
  • Catalyst Data: FDA announcements, clinical trial results, earnings from SEC filings and biotech news sources
  • Total Records: 474 verified predictions