Opportunity
Traditional fitness tests and physical activity assessments are largely conducted manually by human assessors (e.g., coaches or teachers). This process is time-consuming, labor-intensive, and subjective, especially when assessing large groups such as students in schools or members in gyms. Existing computer vision systems for fitness testing remain complex and still require manual review of the data. Many current systems also require subjects to wear expensive sensors or actigraphy devices. Furthermore, visual assessment alone can be inaccurate, with technique errors often missed. There is a clear need for an automated, objective, and cost-effective system that can identify a physical activity being performed, assess its quality, and provide feedback without requiring specialized wearable equipment or constant human supervision.
Technology
This patent presents a physical activity assessment system that automatically identifies an activity (e.g., push-ups, dance, martial arts) and evaluates how well a subject performs it. The system uses one or more cameras (preferably multiple for stereo depth estimation) to capture video of a subject. A computing apparatus receives the video and applies a 3D pose estimation process using a convolutional neural network. The network identifies keypoints (joints) and limbs in each frame. From this, the system generates a 3D frame model (a wireframe or skeleton representation) of the subject.
The processing unit then determines the 3D spatial positions and movement of keypoints and limbs. It calculates functions between keypoints (e.g., joint angles, limb orientations, posture correctness) both in static poses and during motion. Based on these movements and functions, the system identifies the type of activity being performed. It then automatically scores the activity by comparing the subject's performance against a reference model of an ideal performance stored in a database. The score can be quantitative (e.g., number of correct repetitions) and/or qualitative (e.g., form quality). The 3D frame model and score are displayed on a screen, and the model can be manipulated (rotated, panned) by the user for multi-angle analysis. The system can also generate an avatar from the 3D frame model for virtual world applications.
Advantages
- Fully Automated Assessment: Eliminates manual review by human assessors, saving significant time and labor.
- Objective Scoring: Provides consistent, quantitative/qualitative scores based on comparison with an ideal reference model, reducing human bias.
- No Wearable Sensors: Uses only standard cameras, making it low-cost and easy to deploy.
- Real-Time Feedback: Can display scores, feedback, and highlighted technique errors (e.g., incorrect joint angles) to help subjects improve.
- Scalable for Groups: Ideal for assessing large numbers of students or athletes simultaneously (e.g., in schools or gyms).
- 3D Model Manipulation: Users can rotate and view the 3D frame model from any angle for thorough analysis.
- Gamification & Engagement: Scores and feedback can be used to create competitions and track progress.
Applications
- School Physical Education: Automated fitness testing (push-ups, sit-ups, etc.) aligned with curriculum standards (e.g., Hong Kong Education Bureau).
- Gym & Personal Training: Self-assessment for clients to check exercise form without a trainer present.
- Sports Coaching: Analyzing technique in dance, martial arts, fencing, yoga, or other sports.
- Virtual Fitness (Metaverse): Animating avatars based on real subject movements for remote group workouts.
- Medical Screening: Identifying muscular or skeletal issues by detecting abnormal movement patterns or postural weaknesses.
