Why measuring knee movement has always been hard

How do you know whether your knee is genuinely improving — or just having a better week? It is a question that trips up patients and clinicians alike, because the tools traditionally used to answer it have real practical limits.

Labelling movement precisely has long meant fitting reflective markers to a patient's skin, placing them in a purpose-built gait laboratory, and recording a session that can cost hundreds of pounds and requires specialist facilities few NHS trusts have in routine clinical use. The data those sessions produce is valuable, but the barrier to repeating the measurement — to checking whether anything has changed after a course of injections or physiotherapy — is high enough that it rarely happens in practice.

MRI scanning adds a different kind of uncertainty. Reports from the same scan, read by two experienced radiologists, can differ in their description of cartilage condition or structural change. Without a consistent, quantifiable measurement, tracking subtle deterioration or recovery over months is genuinely difficult.

Both gaps — movement measurement and image interpretation — are the focus of work under way at the MSK Computer Vision Lab based in Sleaford, Lincolnshire, which is running two parallel research tracks to address them.

What AI segmentation adds to a knee MRI

The first of those tracks addresses what happens inside the MRI scanner — specifically, how the images are read and measured once they exist.

In a conventional MRI report, a radiologist describes what they see: the degree of cartilage thinning, areas of signal change, structural irregularities. That description is skilled and clinically meaningful, but it is inherently qualitative. Two experienced readers examining the same scan may reach slightly different conclusions, and comparing a scan taken six months apart depends heavily on both reports using consistent language and thresholds.

Deep learning segmentation works differently. The model — in this case an architecture called nnAtrousU-Net, developed at the MSK Computer Vision Lab — is trained to draw precise outlines around each anatomical structure inside the MRI image: the femoral cartilage, tibial cartilage, and the underlying bone. Once those outlines exist, the tissue can be measured numerically: its volume, thickness, and extent become a number rather than a description. Run the same scan through the model twice and the result is the same, removing the reader-to-reader variability that makes sequential MRI comparisons genuinely difficult.

Bone is segmented with near-perfect accuracy — around 98.76% on a standard research benchmark (the OAI-ZIB dataset). Cartilage is harder: thinner, lower in contrast, and variable in how it appears on different scanners. The model achieves approximately 90% accuracy for femoral cartilage and around 86% for tibial cartilage. Those figures are state-of-the-art, but they also reflect a well-recognised challenge across the field — cartilage segmentation remains an active area of research, not a solved problem.

This body of work forms the research foundation for onMRI™, the group's patent-pending AI-driven MRI analysis platform, which applies quantitative cartilage assessment within the clinical setting. The aim is not to replace the reporting radiologist or the surgeon's clinical judgement, but to make MRI findings more reproducible and more comparable as a patient's condition evolves over time.

Tracking knee movement from ordinary video footage

The second track moves away from the scanner entirely. Walk into a consultation, perform a simple sit-to-stand or a squat, and the camera does the rest — no reflective stickers on your skin, no specialist suit, no laboratory booking required.

The underlying method is called VIBE (Video Inference for Body Pose and Shape Estimation). Working from standard video footage, it reconstructs a full three-dimensional body mesh at each moment in time: 23 joints and nearly 7,000 surface points captured in sequence. What that produces is not a rough sketch of how you moved, but a dense, frame-by-frame record of how every major joint — including the knee — changed position and velocity throughout the task.

For in-person assessments, an Azure Kinect depth camera runs alongside VIBE, providing skeleton tracking in real time at 24 frames per second. The two tools are complementary: Kinect gives immediate clinical feedback during the appointment; VIBE supports richer post-analysis of the same movement data.

From these inputs, objective biomechanical metrics are derived — knee flexion smoothness and movement impulse among them — that function as measurable biomarkers of knee pain and treatment response. In a clinical trial comparing joint kinematics before and after injection treatment, sit-to-stand produced considerably more statistically significant biomarker signals than a squat, a finding that has practical implications for how assessments are structured.

This research lineage is the foundation for MAI Motion®, the group's UKCA / MHRA-registered markerless motion capture platform used in clinical consultations. The aim, as with the MRI segmentation work, is reproducible, objective measurement — numbers that can be compared meaningfully across appointments as a patient's condition or recovery evolves.

At a broader scale, the same class of video pose estimation may eventually be applied to CCTV footage in community settings, enabling passive, gait-based screening for musculoskeletal conditions across large populations. That remains a research direction rather than a current clinical offer.

Sit-to-stand versus squat: what a clinical trial found

Choosing the right movement task matters as much as having the right measurement tool — and a clinical trial built around joint injection made that difference concrete.

Patients were filmed performing two standard functional movements — sit-to-stand and squat — before and after receiving an injection to the knee. The same biomechanical analysis pipeline described above processed both. The contrast in what each task revealed was sharp: the squat produced just two statistically significant biomarkers at the more-than-95% confidence level (left and right knee flexion smoothness). Sit-to-stand produced considerably more — multiple significant metrics across joints, including elbow flexion acceleration and smoothness — reflecting the greater coordinative demand the task places on the whole lower limb chain.

That asymmetry has direct design consequences. An assessment protocol built around the squat alone would risk missing a real treatment response, not because the effect was absent but because the movement task was too blunt an instrument to surface it. Knee flexion smoothness and impulse — the metrics that did shift significantly — serve as objective measures of both pain level and how the body responded to treatment, giving clinicians a numbered record of change rather than a verbal pain score.

This is the kind of methodological evidence that moves a technology into clinical use: not just 'the system can measure movement', but 'here is the specific protocol that reliably detects what we need to detect'.

How accurate is video-based motion analysis?

Independent researchers have put markerless motion capture through direct comparison with the laboratory gold standard — and the results at the knee are encouraging.

Song et al. (2023) recorded ten participants across eight movements simultaneously on marker-based and markerless systems. For knee joint angles, the two methods agreed to within a root-mean-square difference of 5.9°, with moment correlations of 0.934 or above. For most clinical decisions — tracking whether a knee bends further after physiotherapy, or whether gait symmetry has improved following an injection — a margin of that size is unlikely to be consequential.

A 2026 study by Chen and colleagues pushed the validation further. A deep learning model trained on markerless video kinematics and musculoskeletal modelling predicted knee contact forces — the loads passing through the joint during movement — with R² values of 0.973 during walking, 0.982 during running, and 0.951 during stair descent, all without force plates. The study population was patients recovering from ACL reconstruction rather than a general knee cohort, and the figures reflect a modelling result rather than direct in-body measurement; even so, the accuracy suggests this class of method can reach beyond surface movement into meaningful joint mechanics.

Hip joint accuracy presents a more complex picture. Angular errors in current markerless systems reach 6.7°–15.9° at the hip — substantially wider than at the knee — and that gap remains an open challenge across the field rather than a limitation of any single tool. The knee, by contrast, is consistently the joint best characterised by current methods, and the convergence of multiple independent research groups towards similar conclusions adds weight to the general approach even as individual tools continue to be refined.

From research lab to your consultation at MSK Doctors

At Sleaford, the Computer Vision Lab, the Open MRI scanner, and the Regeneration Hub occupy the same site, so the research informing the clinical tools and the tools themselves exist within the same scientific and physical context. Equivalent consultant-led assessment is available at Grantham; London-based patients can access the same expertise through the London Cartilage Clinic.

In practice, an assessment can draw on markerless motion analysis via MAI Motion® — already in clinical use as a registered medical device — alongside quantitative cartilage measurement through onMRI™, the group's AI-driven MRI analysis tool (patent-pending). Together they generate an objective baseline of both function and structure: how the joint moves, and what the tissue shows, expressed in reproducible numbers rather than purely descriptive language, which the consultant then weighs alongside clinical examination.

The more advanced video inference pipeline used in the Lab's research still requires post-processing rather than producing real-time output, so it does not yet function as a consultation-room tool in the same way that depth-camera tracking does. Hip joint accuracy in current markerless systems lags behind knee accuracy — a gap that remains open across the field. Questions around data standardisation and regulatory frameworks for AI-driven clinical tools are also actively being worked through. These are the normal boundaries of a science in progress, not reasons to discount what has already been validated.

What the work collectively demonstrates is that meaningful, objective information about how a knee joint behaves can now be gathered in an ordinary clinical setting — no force plates, no gait laboratory. Whether that changes a decision for a specific patient is something a clinical assessment can begin to answer; patients can book without a GP referral at mskdoctors.com.