Visual Sensing of Human Behavior using Kinect Sensor for Home-based Rehabilitation

Abstract

Knee Osteoarthritis (OA) is an important public health issue because of high costs associated with treatment interventions, the chronic, prolonged course, and ever earlier onset due to obesity epidemic. It is a progressive degenerative disease afflicting a large and growing number of Americans. OA induced pain and instability of knees limits function and interrupts locomotion tasks, and an estimated 24-37% of arthritic adults in the US report reduced capacity for "normal daily activity". The current treatment procedures like knee replacements and joint realignment surgery are invasive, expensive and, require long recoveries. OA bracing is an alternate conservative (non-operative) procedure that mediates pain, and improves function and quality of life. OA braces are thought to mechanically unload affected knees by applying external forces to better align them, thereby slowing progression of the disease. There is a great interest in using this technique and the associated personalized rehabilitation program to enhance patient outcomes.

In our current research, we are interested in developing a Kinect sensor based system for tracking and recording low-level dynamic behaviors of the patients in their homes, with ongoing internet-based monitoring and support from a remote clinician. Such a home-based rehabilitation program offers consideration flexibility in tailoring the schedule, intensity and duration of the rehabilitation regimen.

Our overall cyber-physical framework consists of a Patient Interface (ultimately intended to be home-based) and a Therapist Interface (ultimately intended to be at a remote central hospital location) that are connected through Internet. A knee OA patient at home interacts with the patient interface using the Kinect sensor and the OA brace which serve to quantitatively capture the patient motion and activity characteristics as well as to deploy customized exercises. The remote rehabilitation therapist would then be able to use the therapist interface to monitor the patient's sensorimotor performance. The framework allows a visual and haptic recreation of a customized model of a patient which can be examined by the therapist. Information such as computed performance indices based on the measurements and time histories of the patient exercise would be available to aid the diagnosis and analysis. The therapist would be able to create a patient-specific rehabilitation regimen, selected from a parametric library of exercises, refine the regimen using the virtual patient model and download it back to the patient's home machine.

The Kinect sensor is used in the framework to capture synchronized depth and color information about the human dynamic behavior. The information obtained from such behavioral tracking can be utilized to perform human pose estimation and convert the visual dynamics into a parametric and composable low-dimensional manifold representation. The manifold representation is intended to act as a link space between different levels of modeling. Visual sensing data at the low level and the musculoskeletal model at the middle level. We assume that the human pose and visual dynamics are embedded in a low-dimensional manifold space, in which the nonlinear geometric structure can be estimated and assembled by many adjacent locally Euclidean spaces.

Award ID: 1135660