Vision helps robot perform physiotherapy

Spanish engineers have developed a robot that can treat a variety of medical conditions that cause back pain

Ignacio Riestra

Physiotherapy treatments are widely applied to help restore movement and functions to individuals who have been affected by injury, illness or disability. Traditionally, many physiotherapy treatments have been performed manually by physiotherapists who repeat the treatments regularly - usually on a daily basis - for a prescribed number of weeks to mobilize joints and soft tissues.

Over the past three years, however, a vision-guided robotic system called the CBot has been under development in the Northern part of Spain that can apply several types of such physiotherapy automatically. While the condition of the patient is still diagnosed by a trained physician, the robotic system emulates the role of the physiotherapist, relieving heavily-burdened clinics from their workload.

The robotic physiotherapy machine (Figure 1) was created as a joint effort between two companies - Clinica Ordonez (Oviedo, Asturias, Spain; www.clinicaordonez.com), a physiotherapy center that specializes in sports and orthopedic rehabilitation, and Robotics Special Applications (Oviedo, Asturias, Spain; www.roboticssa.es), a systems integrator that has developed a number of industrial vision-based robotics systems over the past several years.

Figure 1: A robotic physiotherapy machine that can treat backache problems in the cervical, dorsal and lumbar regions of the body has been developed as a joint effort between Clinica Ordonez and Robotics Special Applications.

In use, patents seeking treatment from back pain first enter the clinic where they are provided with an RFID card with their name and personal data. Next, a trained physician performs a face-to-face diagnosis to determine the specific type of pain that requires treatment. A patient is then scanned by a Kinect-based body scanner after which the physician defines what type of treatment, where on the back, and for what length of time, it should be applied by the robot. The data relating to the specific treatment is then stored on the RFID card.

Armed with the RFID card, the patient can enter the clinic at a prescribed time to receive treatment from the CBot robotic physiotherapist. To do so, the RFID card is inserted into a card reader interfaced to the PC-based robotic system which then reads the treatment data from it. After the patient lies down on a treatment couch, the robot then completes a series of physiotherapy treatments that have been prescribed by the physician. Without further human intervention, the patient can return to the clinic for the prescribed number of sessions that are required to complete the treatment.

System design

To bear the weight of the universal robot and ensure the accuracy of the vision system, the CBot system comprises a rigid steel framework which supports the treatment bed onto which the patient lies. A UR5 robotic arm from Universal Robots (Odense, Denmark; www.universal-robots.com) is used to apply the therapy to the patient via a tool held by a Robotiq (Saint-Nicolas, QC, Canada, www.robotiq.com) two-finger robot gripper at the end of the robot arm which emanates from an aperture in the back of the frame. The system itself is controlled by the PC-based control unit housed in a cabinet beneath the bed (Figure 2).

Figure 2: To bear the weight of the universal robot and ensure the accuracy of the vision system, the CBot system itself comprises a rigid steel framework which also supports the treatment bed onto which the patient lies.

A C69 PC-based system from Beckhoff (Verl, Germany; www.beckhoff.com) controls the movement of the arm of the universal robot and the position of the physiotherapy tool held by the gripper through a motion controller housed in the PC. In addition, it controls the position of an HLE-RB electromechanical positioner from Parker Hannifin (Cleveland, OH, USA; www.parker.com) mounted in the top of the steel frame to which a Microsoft Kinect 3D vision sensor and its motorized tilt is attached. This enables the sensor to be positioned at the correct orientation to measure the position of the body of the patient before and during the treatment process. The PC is also interfaced to an RFID system from Kimaldi (Barcelona, Spain; www.kimaldi.com) that identifies the patient via the RFID card and determine the type of treatment that needs to be applied (Figure 3).

Figure 3: A PC running software developed using Robotics Special Applications CSPRO software development and run-time environment is used to control both the image acquisition and processing as well as the movement of the robot.

The control software for the robotic vision system was developed using Robotics Special Applications' CSPRO software development and run-time environment, while Microsoft's Kinect Fusion software vision software was used to reconstruct a surface model from the images captured by the camera.

Robotics Special Applications' CSPRO software enables the PC to perform real-time control functions commonly found in PLC-based systems. Developed in C# under Microsoft Visual Studio's .NET platform, it comprises a development environment which was used to program, diagnose, and configure the system and a run-time system which controls the position of the robotic arm and the orientation of the camera. In addition, the software development environment was used to create a Human Machine Interface that displays a representative image of the area to be treated by the robot onto a touch screen.

Prior to treatment, the body of a patient is scanned by the Kinect sensor to create a 3D model of the surface of the patient's back. The Kinect Fusion software running on the PC reconstructs a surface model of the patient by integrating the depth data from Kinect over time from multiple viewpoints. The camera pose is tracked as the sensor is moved and the multiple viewpoints of the individual on the treatment couch are fused, or averaged together, into a single 3D volume of the back of the patient. The model is then compared to the 3D model captured during the initial scanning process to ensure that areas to be treated have been identified accurately.

During the treatment, the patient is continuously scanned by the Kinect-based system to ensure that treatment is applied to the correct regions of the body, even if the patient should move. Cues from the RGB and depth streams from the Kinect sensor are transferred to the PC over a USB3 interface where they are processed by a skeleton-tracking algorithm in Robotics Special Applications CSProVision software which uses the data from the Kinect sensor to detect the position of the main joints of the patient body in real time (Figure 4).

Figure 4: Prior to treatment, the body of a patient is scanned by the Kinect sensor to create a 3D model of the surface of the patients back and areas and type of treatment identified by a physician. The model is then compared to the 3D model captured during the initial scanning process to ensure that areas to be treated have been identified accurately.

Should the software detect any deviation from the original location of the patient, the new coordinate data is transferred to a Parker Hannifin Compax3 servo drive motion controller housed in the PC enclosure which instructs the robot to move to the new set of coordinates. By continuously monitoring the position of the patient, the system can also be automatically halted if the patient moves out of position.

Because the system is used in a medical environment, the safety of the patient undergoing treatment is of paramount importance. To ensure that the data from the PC is transferred in a deterministic manner to the robot and to enable safety-related data to be transmitted on the same medium, data processed by the Kinect sensor is transferred from the motion controller card in the PC to the robot over an Ethercat network which relies on the secure Safety over EtherCAT protocol. High-level software running on the PC generates a protective stop if the robot strikes an item during the physiotherapy procedure. Low-level software limits the torque generated by the gripper joints, permitting only a small deviation from the expected torque.

Practical applications

Although a range of physiotherapy techniques can be employed to treat many different parts of the body, the CBot was specifically developed to treat back pain, which can be caused by a number of issues including poor posture, bending or twisting awkwardly, standing for long periods or lifting or carrying objects incorrectly.

More specifically, the robot can treat backache problems in the cervical, dorsal and lumbar regions of the body, buttock pain from strains or sprains, shoulder pain created by injury to the rotator cuff tendon, and arm pain due to tennis or golf elbow. To do so, three different types of backache treatments can be applied by heads which are picked up by the gripper at the end of the robot arm prior to treatment. These heads are used to apply an ultrahigh frequency (UHF) treatment, a hot jet stream treatment and a vacuum treatment.

The UHF treatment, which is commonly used for muscle relaxation, applies a 434MHz signal to deliver moderate heat directly to lesions in the deep tissues of the body at depths between 6-8cm. To treat shallower muscular conditions, the hot jet stream delivers a pre-heated pressurized air stream to specific areas of the back. The final treatment uses a vacuum cup to relax muscles in the back. In use, silicone suckers are applied to affected areas and a -0.5 bar negative pressure applied to them, releasing rigid soft tissue, draining excess fluids and toxins and loosening adhesions.

The development of the CBot robot began in 2011 when the concept behind the robotic system was first defined. Between 2012 and 2013, two prototype systems were developed and tested, and in September 2013, the first commercial system was used by Clinica Ordonez to effectively treat a patient. The system is now being marketed by a spin-off from Robotics Special Applications called Fisiobot.

Ignacio Riestra, CEO, Robotics Special Applications, Oviedo, Asturias, Spain www.roboticssa.es