Health Reasoning over Time Series
A unified benchmark for evaluating hierarchical reasoning capabilities of LLMs over general health time series.
Latest from HEARTS
GPT 5.4, MiniMax M2.5, Kimi K2.5, and Nemotron 3 Super 120B evaluated on HEARTS-LIGHTENING — results now on the leaderboard.
June 2026
HEARTS-LIGHTENING released: a cost-efficient evaluation subset for fast, affordable model assessment.
May 2026
HEARTS has been accepted to ICML 2026.
May 2026
GLM 5 and Gemini 3.1 Pro evaluation results added to the HEARTS leaderboard.
April 2026
HEARTS initial release: 16 datasets, 12 health domains, 20 modalities, 110 tasks, and 20K+ test cases are now public.
February 2026
Leaderboard is live with category-level breakdowns across Perception, Inference, Generation, and Deduction.
February 2026
Community submission channel opened for new model results, task proposals, and benchmark extension requests.
February 2026
Leaderboard
—
—
Overall scoreThe first diverse benchmark for health time-series reasoning.
With the broadest coverage of sequence length, sampling frequency, and time span to date, HEARTS tests whether models truly reason over physiological signals — not just match surface patterns.
datasets
domains
modalities
tasks
Distribution across 12 health domains
Broadest coverage to date
20 signal modalities
Organized into four levels of reasoning
Perception
Identify physiological markers that ground downstream analysis.
2 task types- Statistical Calculation
- Feature Extraction
Inference
From precise temporal grounding to holistic subject characterization.
3 task types- Event Localization
- Physiological Classification
- Subject Profiling
Generation
Point-by-point synthesis of complete physiological sequences.
3 task types- Future Forecasting
- Signal Imputation
- Cross-modal Translation
Deduction
Arrow-of-time reasoning over longitudinal, multi-session signals.
2 task types- Temporal Ordering
- Trajectory Analysis
HEARTS-LIGHTENING A cost-efficient evaluation subset — our standard track for future evaluations.
To balance evaluation comprehensiveness with computational efficiency, we introduce HEARTS-LIGHTENING, a cost-efficient subset comprising 5,417 test samples. This streamlined configuration preserves the complete taxonomy of all 110 tasks while reducing the number of test cases per task, thereby substantially lowering computational overhead without compromising the statistical stability of our evaluations.
Re-evaluating the models under this lightweight setting confirms that the overall performance tiers remain stable, with only minor ranking shifts observed among models with closely matched scores. Given its efficiency and strong alignment with the full benchmark, we adopt the HEARTS-LIGHTENING configuration as a standard track for future evaluations.
Genuine reasoning, or surface patterns and priors?
5 key findings
Click any card to see the supporting evidence, redrawn from the results.
Put your model on the board
Submission is handled through the official Google form. Contributors should submit either model weights or a model API endpoint. The HEARTS team runs the benchmark tests internally and publishes validated leaderboard updates.
Submission
Submit model weights or an API endpoint through the official Google form.
Team evaluation
The HEARTS team runs the benchmark internally and verifies submission validity.
Leaderboard update
Accepted runs are published with standardized, comparable reporting.
Before you submit — checklist & details
Checklist
- Use the official Google submission form.
- Submit either model weights or a model API endpoint (one is required).
- Provide clear access instructions so the team can run evaluation.
- Include model name, version, and expected inference configuration.
- Include contact details for follow-up and troubleshooting.
What to include
- Submission type: model weights or API endpoint.
- Model family, checkpoint / version, and provider.
- Access details (download path or endpoint / auth instructions).
- Inference requirements (context window, tool usage, limits).
- Any constraints or caveats to know before running tests.
Contact
Questions, feedback, or interested in contributing? Reach the HEARTS team — we're happy to help with setup, access, or proposing new tasks and datasets.
Citation
@article{hearts2026,
title={HEARTS: Benchmarking LLM Reasoning on Health Time Series},
author={Sirui Li and Shuhan Xiao and Mihir Joshi and Ahmed Metwally and Daniel McDuff and Wei Wang and Yuzhe Yang},
journal={arXiv preprint arXiv:2603.06638},
year={2026}
}