This estimator enables you to calculate the user’s heart rate from a short video of their face taken from the smartphone’s front camera. The estimator implements an algorithm described in and yielded interesting results. Figure 1 compares the performance of this algorithm implemented in Android against the heart rate as provided by an FDA approved blood pressure monitor – Omron.
This short tutorial will guide you through the steps you need to take to incorporate this heart rate estimator. First, we need to check the prerequisites to make sure our smartphone is capable of performing the necessary computations.
Prerequisites
In order to estimate the user’s heartbeat from a face-video estimator, the device must support a front-facing camera, flash, and flash mode. Before starting the estimator, we check if these prerequisites are satisfied by calling the function reqSatisfied() . The function will throw an ExtractorPrerequisiteUnsatisfiedException instead of returning void if at least one of these prerequisites aren’t met. Otherwise, the function will return null and we can start the estimator. We can accomplish this with a try-catch statement:
try {
heartBeatFaceExt.reqSatisfed();
heartBeatFaceExt.start();
} catch (ExtractorPrerequisiteUnsatisfiedException e) {
// The prerequisites of the extractor were not satisfied
}
// Keep collecting until...
heartBeatFaceExt.terminate();
Making a Fragment
The camera used by the estimator takes the form of a Fragment in an Android Activity. For setting it up in your app, make a Fragment for the camera in a RelativeLayout or LinearLayout in your Activity as shown here.
Within the OnCreate function you need to tie the estimator to the camera fragment. To do this, add:
this.mHeartRateCalculator = new HRFaceVideoEstimator(faceHREncapsulator);
FragmentManager manager = getSupportFragmentManager();
FragmentTransaction transaction = manager.beginTransaction();
Fragment cameraFragment = this.mHeartRateCalculator.getCameraFragment();
transaction.add(R.id.activity_face_hr_camera, cameraFragment, "faceHRCameraFragment");
transaction.commit();
Don’t forget to add the required permissions in the Android Manifest and ensure that your screen orientation is locked on either portrait or landscape mode for the duration of the activity.
Add a listener to the encapsulator if required.
Starting the Estimator
The last step to start the estimator. This can be done by overriding the OnResume function. In order to complete the activity life cycle, we stop the estimator by overriding the OnPause function.
Poh, M. Z., D. J. McDuff, and R. W. Picard. 2011. “Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam.” IEEE Transactions on Biomedical Engineering 58 (1): 7–11. https://doi.org/10.1109/TBME.2010.2086456.
Poh, Ming-Zher, Daniel J. McDuff, and Rosalind W. Picard. 2010. “Non-Contact, Automated Cardiac Pulse Measurements Using Video Imaging and Blind Source Separation.” Optics Express 18 (10): 10762. https://doi.org/10.1364/OE.18.010762.