Heart-Beat Rate From Face Video Estimator


Introduction

This estimator enables you to calculate the user’s heart rate from a short video of their face taken from the smartphone’s front camera. The estimator implements an algorithm described in  and yielded interesting results. Figure 1 compares the performance of this algorithm implemented in Android against the heart rate as provided by an FDA approved blood pressure monitor – Omron.

Figure 1: Performance of Android Heart Rate Estimator with respect to the Omron device

Figure 1: Performance of Android Heart Rate Estimator with respect to the Omron device

This short tutorial will guide you through the steps you need to take to incorporate this heart rate estimator. First, we need to check the prerequisites to make sure our smartphone is capable of performing the necessary computations.

Prerequisites

In order to estimate the user’s heartbeat from a face-video estimator, the device must support a front-facing camera, flash, and flash mode. Before starting the estimator, we check if these prerequisites are satisfied by calling the function  reqSatisfied() . The function will throw an  ExtractorPrerequisiteUnsatisfiedException instead of returning void  if at least one of these prerequisites aren’t met. Otherwise, the function will return null and we can start the estimator. We can accomplish this with a try-catch statement:

Making a Fragment

The camera used by the estimator takes the form of a Fragment  in an Android Activity. For setting it up in your app, make a Fragment  for the camera in a RelativeLayout  or LinearLayout  in your Activity as shown here.

Initializing the Estimator

The Activity must extend the  NoSleepFragmentActivity and the estimator and it’s associated encapsulator can be initialized as follows:

Within the  OnCreate function you need to tie the estimator to the camera fragment. To do this, add:

Don’t forget to add the required permissions in the Android Manifest and ensure that your screen orientation is locked on either portrait or landscape mode for the duration of the activity.

Add a listener to the encapsulator if required.

Starting the Estimator

The last step to start the estimator. This can be done by overriding the OnResume  function. In order to complete the activity life cycle, we stop the estimator by overriding the OnPause  function.

Your heart-rate estimator is now ready to run!

Face Video HR Estimation

Figure 2: An example screenshot of the estimator in action.

 

Running Example

(TODO: complete)

References

Poh, M. Z., D. J. McDuff, and R. W. Picard. 2011. “Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam.” IEEE Transactions on Biomedical Engineering 58 (1): 7–11. https://doi.org/10.1109/TBME.2010.2086456.
Poh, Ming-Zher, Daniel J. McDuff, and Rosalind W. Picard. 2010. “Non-Contact, Automated Cardiac Pulse Measurements Using Video Imaging and Blind Source Separation.” Optics Express 18 (10): 10762. https://doi.org/10.1364/OE.18.010762.