Coronavirus outbreak: Can machine vision and imaging play a part?

Thermal Imaging for Airborne Virus Detection

To slow the spreading of Coronavirus Disease 2019 (COVID-19), many people around the world have turned to machine vision and imaging technologies. Some of these may prove more effective than others.

Before diving into some of the technology used recently for recognizing or diagnosing sick people and reducing airborne virus particles, remember that many machine vision systems currently inspect, identify, gauge, or guide products like ventilators, respirators, and protective gear. Such products assist in treating patients and protecting healthcare professionals and in turn, further spreading of the virus.

For example, General Motors approached JR Automation to help produce face masks to combat the COVID-19 outbreak. In just six days, GM produced its first mask in response to the COVID-19 crisis. Read more here.

Thermal imaging

Numerous airports, for example, have installed thermal cameras to screen for people with elevated body temperature, as fever is a main symptom of COVID-19. Police in China have even used helmets with thermal goggles to identify pedestrians who may be unwell, according to the Shenzhen government. However, a recent video posted by Markus Tarin, President and CEO of MoviTHERM explains why the use of thermal imaging is flawed.

When the immune system detects an infection, the core body temperature raises. Based on recent research, the best spot on the face to detect core body temperature is the inner tear duct. When someone wears glasses, however, this creates a problem since thermal cameras cannot see through glass, thus occluding the important temperature locations in the face, explains Tarin.

A thermal camera also cannot detect a particular virus. It can only identify an elevated body temperature, and for that to work well, a very high-resolution thermal image across somebody’s face is needed.

“In some examples, thermal cameras are just pointed at a large crowd. This will not provide enough resolution over the tear ducts for an exact body temperature measurement.”

Vision-guided robots

Nobody, at this point, likely needs a refresher on what social distancing means, but in some cases, accomplishing this may prove difficult. In hospitals, for example, it is vitally important to keep sick people away from others, including the doctors and nurses. One way to do this involves using vision-guided robots. 3D camera company Orbbec reports that its 3D products are being deployed in three different types of robots in Chinese hospitals today to fight the spread of COVID-19.

In more than 50 hospitals across 10 provinces in China, a food delivery robot minimizes contact between patients and medical staff. In hospitals across Shenzhen, a sanitizing robot helps medical staff clean an entire hospital, preventing interactions with germs. Lastly, a guide robot helps navigate patients between hospital departments, avoiding unwanted contact.

These are not the only examples of the use of robots in China to help prevent the spread of the virus.

Robots are also used for police patrol in China, delivering messages about wearing masks, avoiding large gatherings, and maintaining hygiene in this time of crisis. Vision-guided drones also perform similar tasks, informing people to put masks on and return home, as seen in the video above. Furthermore, vision-guided robots help keep factories operational by monitoring the grounds with cameras and sending alerts to workers should anything go wrong.

Additional examples of robot use to minimize infection in China include those for contactless delivery, medical sample transport, spraying disinfectants, and performing diagnostic functions.

At the Circolo di Varese hospital in northern Italy, six robots also perform tasks such as reading patients vital signs to help keep doctors and nurses away from sick patients.Semi-autonomous robots in Singapore are also being used to quickly disinfect large surfaces.

RoboSense LiDAR cooperated with Neolix, Alibaba’s Cainiao Robotics, Unity Drive Innovation, Zhen Robotics and others on unmanned vehicles and robots to deliver goods, disinfecting and cleaning streets to prevent further spread of the coronavirus. Check it out here:

Deep learning

Researchers from the University of Waterloo and the AI firm DarwinAI developed COVID-Net, a deep convolutional neural network tailored for the detection of COVID-19 cases from chest radiography images. The initial network design prototype aims to make one of the following four predictions: a) no infection (normal), b) bacterial infection, c) non-COVID viral infection, and d) COVID-19 viral infection. These possible predictions were chosen because results can aid clinicians to better decide not only who should be prioritized for PCR testing for COVID-19 case confirmation, but also which treatment strategy to employ, depending on the cause of infection, since each type of infection requires a different treatment plan.

A deeper dive article detailing COVID-Net will be published in the May issue of Vision Systems Design. Read the research paper here, in the meantime.

On the MathWorks blog, Barath Narayanan, University of Dayton Research Institute, also detailed the application of a deep learning technique for detecting COVID-19 on chest radiographs using MATLAB software. The COVID-19 dataset used in the research was curated by Dr. Joseph Cohen, a postdoctoral fellow at the University of Montreal. A classification algorithm using ResNet performed fairly accurately, and performance of transfer learning approaches clearly reiterates that CNN-based classification models perform well in extracting features. Additionally, combining the results with other existing architectures would provide a boost in overall accuracy, according to the author.

Drones

Unmanned aerial vehicle (UAV) company Draganfly has been selected by Australian authorities to provide engineering, integration, and distribution help for immediate deployment of drones capable of detection infections among people at a distance. In addition to detecting fever, which we covered earlier in the article, the company claims the drone can help detect coughing, respiratory rate, heart rate, and blood pressure.

Drones may also provide useful for tasks like aerial monitoring, emergency response, medicine delivery, and more. With that in mind, drone video streaming and fleet management software company FlytNow has decided to make FlytNow Pro available, at no cost until at least May 2020. FlytNow capabilities range from drone live streaming over 4G/LTE/5G networks to remote control of the drone and/or camera gimbal from far away. View more information here.

Other tech in use

Numerous other imaging and or technologies tangentially related to imaging are also currently being deployed. For example, as part of a thermal imaging system in Shenzhen, China, an artificial intelligence program not only measures visitors’ body temperature, but also checks their identity and if they are wearing a mask. Chinese tech firm Baidu also reportedly developed an algorithm for Beijing subways that uses video feeds from subway cameras to single out commuters not wearing masks.

[Editor’s note: How far should technology like this go, however? For example, deep learning systems performing gesture/pose analysis could sound an alarm when two people go to shake hands, but does anyone need this?]

In terms diagnosing COVID-19, researchers at Bar-Ilan University in Israel have reportedly shortened the diagnosis time from about an hour to 15 minutes. The method combines optics and magnetic particles and is able to test 100 samples of patients potentially infected with the virus. Amos Danielli of the Alexander Kofkin Faculty of Engineering at the university is currently looking for an investor to accelerate the development of the Coronavirus kit so that it can be rapidly introduced in hospitals.

Ultraviolet LEDs have reportedly shown an ability to sterilize more than 90% of other coronavirus strains. Testing conducted by Chinese agencies on appliances equipped with violeds UV LED products from Seoul Semiconductor subsidiary Seoul Viosys have demonstrated a 97% reduction in airborne virus particles. As a result, Seoul Viosys has reportedly seen a five-times increase in customer inquiries for its violeds products over the previous month. Several automotive companies in China are also reviewing adoption of the products in sterilization systems for vehicle interiors.

Can machine vision and imaging play a part?

In part, yes, machine vision and imaging technologies have a role to play. First, traditional machine vision systems undoubtedly support the production of products and gear used in hospitals for treatment. Some methods prove to be ineffective or inconsistent, while others are perhaps a bit on the scary or intrusive side. On the other hand, some techniques help to avoid potential spread of the virus by minimizing human contact, while related technologies have proved helpful in both diagnosing and reducing airborne virus particles.

While we ultimately await the arrival of a clinically tested vaccine, such technologies and research aid in the equally important goal of suppressing the disease. If anyone in the Vision Systems Design audience is currently working with vision technologies or researching vision-based techniques related to ending the COVID-19 pandemic, please reach out to us, we’d love to hear from you: [email protected].