Heart-Beat Rate From Finger Video Estimator

Table of Contents

Introduction

This estimator enables you to calculate the user’s heart rate from a short video of their finger taken from the smartphone’s front camera with the flash turned on. The estimator implements an algorithm described in . Figure 1 shows us the results of the heart rate estimator in the form of an error histogram. ‘Raw’ dataset includes all of the data collected, while the ‘Outlier Removal’ dataset has removed poor data points in which a noisy or invalid signal was captured from the finger image. The ground truth measurement was taken with a Bluetooth blood pressure monitor by A&D (UA-651BLE).

Error Histogram Summarizing Results Obtained
Figure 1: Error Histogram Summarizing Results Obtained

Prerequisites

In order to estimate the user’s heartbeat from a finger-video estimator, the device must support a back-facing camera. Before starting the estimator, we check if this prerequisites are satisfied by calling the function reqSatisfied() . The function will throw an ExtractorPrerequisiteUnsatisfiedException if the prerequisite isn’t met. Otherwise, the function will return void  and we can move onto starting the extractor. We can accomplish this with a try-catch statement:

try {
    heartBeatFingerExt.reqSatisfed();
    heartBeatFingerExt.start();
} catch (ExtractorPrerequisiteUnsatisfiedException e) {
    // The prerequisites of the extractor were not satisfied
}

// Keep collecting until...
heartBeatFingerExt.terminate();

Make A Fragment

The camera used by the estimator takes the form of a Fragment  in an Android Activity. For setting it up in your app, make a Fragment  for the camera in a RelativeLayout  or LinearLayout  in your Activity as shown here.

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingBottom="@dimen/activity_vertical_margin"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    tools:context="edu.yourProject.FingerBPActivity">

    <FrameLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_alignParentTop="true"
        android:layout_centerHorizontal="true"
        android:id="@+id/activity_finger_bp_camera_frame_layout"
        android:layout_marginBottom="60dp"></FrameLayout>

</RelativeLayout>

Initialize The Estimator

The Activity must extend the FragmentActivity and the estimator and it’s associated encapsulator can be initialized as follows:

BPFingerVideoEstimator mBPFingerVideoEstimator;
private static BackableEncapsulator<TimeZonedTimestampedObject<HeartRateContainer>> fingerBPEncapsulator = null;

Within the OnCreate function you need to tie the estimator to the camera fragment. To do this, add:

this.mBPFingerVideoEstimator = new BPFingerVideoEstimator(fingerBPEncapsulator);

FragmentManager manager = getSupportFragmentManager();
FragmentTransaction transaction = manager.beginTransaction();
Fragment cameraFragment = this.mBPFingerVideoEstimator.getCameraFragment();
transaction.add(R.id.activity_finger_bp_camera_frame_layout, cameraFragment, "fingerBPCameraFragment");
transaction.commit();

Don’t forget to add the required permissions in the Android Manifest and ensure that your screen orientation is locked on either portrait or landscape mode for the duration of the activity.

Add a listener to the encapsulator if required.

Start The Estimator

The last step to start the estimator. This can be done by overriding the OnResume  function. In order to complete the activity life cycle, we stop the estimator by overriding the OnPause  function.

@Override
protected void onPause() {
    this.mBPFingerVideoEstimator.terminate();
    super.onPause();
}

@Override
protected void onResume(){
if (!this.mBPFingerVideoEstimator.reqSatisfied()) {
      System.out.println("The requirements aren't satisfied in ACTIVITY");
        }
this.mBPFingerVideoEstimator.start();
super.onResume();
}

Your heart-rate estimator is now ready to run!

Finger Video HR Estimation
Figure 2: An example screenshot of the estimator in action.

Running Example

(TODO: complete)

References