Researchers have developed a method that uses the camera on a person’s smartphone or computer to get their pulse and breathing signal from a real-time video of their face.
It is developing at a time when telehealth has become a crucial way for doctors to ensure health care while minimizing face-to-face contact during Covid-19.
The team, led by the University of Washington, uses machine learning to detect subtle changes in light reflectance on a person’s face that correlate with a change in blood flow. Then it converts those changes into both pulse and respiratory rate.
The researchers presented the system in December at the conference on neural information processing systems.
Now the team is proposing a better system for measuring these physiological signals.
According to researchers, who will present these results at the Association for Computing Machinery (ACM) health conference on April 8, this system is less likely to be triggered by different cameras, lighting conditions, or facial features such as skin color. Interference and learning.
“Everyone is different,” said lead study author Xin Liu, a UW graduate student.
“This system must therefore adapt quickly to the individual physiological signature of each person and be able to separate them from other variations, such as how they look and what environment they are in.”
The first version of this system was trained with a data set that contained both videos of people’s faces and information on the “basic truth”: each person’s pulse and respiratory rate, measured with standard on-site instruments.
The system then used spatial and temporal information from the videos to calculate both vital signs.
While the system worked fine on some records, it still had problems with others that included different people, backgrounds, and lighting. This is a common problem known as “overfitting,” the team said.
The researchers improved the system by having a personalized machine learning model created for each individual.
In particular, it helps find important areas in a video frame that are likely to contain physiological features that correlate with changes in blood flow in a face under different contexts, such as: B. different skin tones, lighting conditions and environments.
From there, it can focus on that area and measure the pulse and respiratory rate.
Although this new system outperforms its predecessor when it comes to receiving more sophisticated records, especially for people with darker skin tones, there is still more to be done, the team said.
(With contributions from agencies)
