Sensori

The Sensori white paper

Movement is a language of health.

By Hang Yuan, Yong Wang

Where it all began

The term foundation model was only introduced in 2021 by Bommasani and colleagues at Stanford. Although the ML community was already familiar with the benefits of large-scale pretraining, the idea was still new to wearable sensing. Large wearable datasets, particularly movement data, were already available through NHANES and UK Biobank, but, as with many new ideas, the field needed time to catch up. After publishing one of the first large-scale pretrained models for movement, using a simple multi-task objective to predict transformations of the input signal, we came across Apple’s foundation models for PPG and ECG. Trained on the Apple Heart and Movement Study, these models showed a striking ability to distinguish participant characteristics and a range of existing health conditions. At first, the results seemed almost too good to be true. That made us want to test whether the same idea could work with publicly available UK Biobank data and wrist movement as the input modality.

Why movement matters

Passive movement monitoring is universal. Approximately one in three adults who use the internet owns a wrist-worn device capable of measuring movement. If we can recover even a small fraction of the health information carried by these signals, the benefits at population scale could be substantial. This ambition is deeply rooted in the philosophy of Oxford’s Nuffield Department of Population Health, the birthplace of UK Biobank: to understand simple but important modifiable risk factors, such as smoking, so that targeted interventions can deliver meaningful benefits across entire populations.

For movement sensing, our intuition is simple:

A direct way to assess someone’s health is to follow them for a day and observe what they do, and how they do it.

From behavioural traits to learned representations

Sensori takes a different approach. Instead of reducing wrist movement to predefined behavioural traits, such as step count or gait asymmetry, it treats the full 24-hour waveform as a continuous time series. We hypothesised that self-supervised learning could discover health-relevant patterns across timescales without first deciding which behaviours matter. This differs from Apple’s WBM and Google’s SensorFM, which learn from derived behavioural or sensor features. Such summaries remain useful, but choosing in advance what to retain can limit the information available to the model.

The comparison below makes this distinction concrete. Even 450 conventional features—18 daily traits and 432 hourly summaries spanning physical activity, steps and sleep—were substantially less predictive of sex, age and BMI than Sensori embeddings learned directly from the raw signal. The same pattern extended across ten additional health axes tested in the UK Biobank.

From summaries to representation

Sensori captures more than predefined summaries.

Sex

AUROC gain

Age

Pearson r gain

BMI

Pearson r gain
Across 10 health axes

+0.048 Sensori vs +0.026 daily + hourly traits

Gain over acceleration mean and s.d. in held-out UK Biobank; point estimates shown.

How we developed Sensori

Sensori was designed around a simple premise: movement carries health information across multiple temporal scales. A tremor may unfold over seconds, while the relationship between daytime activity and night-time sleep emerges across the full day. Capturing both requires a model that can learn local movement patterns alongside characteristics that remain stable over time.

We therefore combined two complementary pretraining objectives. Masked reconstruction hides selected five-minute segments and asks the model to recover them from the surrounding movement, encouraging it to learn within-day temporal structure. Participant-level contrastive learning brings recordings from different days of the same person closer together while separating those from different people, encouraging the representation to retain stable individual characteristics. Because each participant contributed at most seven complete days, this contrastive objective was prone to overfitting. We therefore introduced it only during the final stage of pretraining, after masked reconstruction had established a robust representation of within-day movement.

What to use Sensori for

Health monitoring

Medicine has traditionally focused on detecting and treating established disease. Preventive care also requires us to measure health before disease emerges. Tracking how health changes over time could reveal early deterioration, help evaluate interventions and extend the years people live in good health.

Movement is conceptually simple but information-rich. Sensori embeddings captured variation spanning participant characteristics, smoking status, chronotype, self-rated health and physical function. Many of these domains are ordinarily assessed through questionnaires or occasional clinical measurements. Movement representations could complement these assessments with low-cost, repeated measurements at population scale, enabling new research into health trajectories and supporting future health and fitness applications.

Disease diagnosis and prognosis

Population-scale disease prediction is difficult because many conditions affect fewer than 1% of participants, creating severe class imbalance and leaving relatively few labelled examples. Sensori separates general representation learning from disease-specific learning: it first learns common movement patterns from a large unlabelled dataset, then adapts to each condition through supervised fine-tuning. In UK Biobank, adding fine-tuned Sensori embeddings to common clinical covariates improved prevalent disease classification for 52 of 102 eligible conditions and six-year incident disease risk prediction for 26 of 87. The first task asks whether current movement reflects existing disease; the second asks whether it helps anticipate future onset.

Sensori was deliberately designed to be general-purpose and disease-agnostic, but no established benchmark covered this breadth of conditions. We therefore compared its embeddings with progressively richer conventional movement inputs, from acceleration statistics to 18 device-measured behavioural traits. These comparisons do not establish a formal upper bound, but they help indicate how much predictive information is available in wrist movement. Conventional traits produced smaller gains for prognosis than for prevalent disease classification, suggesting that many future outcomes are less directly expressed in movement at a single point in time or that much of the cause-effect association from movement was absorbed by body mass index (BMI), which was used as a covariate.

Although we evaluated a broad range of diseases, the largest gains were concentrated among conditions with movement-related phenotypes, particularly neurological and psychiatric disorders. Movement alone will not capture every aspect of disease. Combining Sensori with complementary modalities, such as electrocardiography, electronic health records or proteomic measurements, could provide a more complete view of disease status and future risk.

Digital biomarker discovery

Digital biomarkers are objective measures derived from digital devices that can reflect health status, disease progression or response to treatment. A familiar analogue is BMI: its trajectory can prompt preventive lifestyle changes, while persistently high values may help inform more intensive treatment. A useful digital biomarker could play a similar role by turning passively collected signals into a measure that can be followed over time.

Developing such measures can take years. Researchers must first identify promising candidates among many possible features, then establish that a selected measure tracks the relevant aspect of disease reliably across devices, environments and populations.

A more fundamental question comes first: does the signal contain enough information to distinguish the health states of interest? A carefully designed movement measure may still add little beyond a questionnaire or common clinical variables. Without first establishing the information available in the underlying signal, biomarker development can become a costly search with an uncertain ceiling.

Sensori can help guide this search. When its embeddings discriminate a disease, they provide evidence that the raw movement signal contains relevant information and identify conditions for which biomarker discovery may be most promising. The embedding model can then serve as a benchmark while researchers search for simpler, interpretable measures that recover the same signal. An agentic AI framework could accelerate this process by testing large numbers of candidate measures, but any resulting biomarker would still require independent and prospective validation.

Discovery pathway

From signal to biomarker.

  1. 01 / Screen Is there useful signal?
  2. 02 / Discover Can a simpler measure recover it?
  3. 03 / Validate Is it robust and clinically useful?

The future of movement sensing

Movement sensing is becoming a continuous part of everyday life. Its value should grow as measurements extend over time and are combined with complementary physiological and clinical data. The next challenge is not simply to collect more signals, but to make them accessible to models that can reason across modalities and communicate their findings through language.

There are two promising directions. In the near term, engineered features and structured descriptions of sensor data can provide a practical interface to general-purpose language models; SensorLM illustrates the broader potential of connecting wearable signals with language. Longer term, movement–language models could align rich representations, such as Sensori embeddings, directly with text. Much as vision–language models connect images with language, this approach could make continuous movement easier to query and interpret while preserving more of the information in the original signal.