Blending Camera and 77 GHz Radar Sensing for Equitable, Robust Plethysmography

Alexander Vilesov1 Pradyumna Chari1 Adnan Armouti1 Anirudh B. Harish1 Kimaya Kulkarni1 Ananya Deoghare1 Laleh Jalilian1 Achuta Kadambi1

University of California, Los Angeles1

SIGGRAPH 2022, Vancouver, Canada

image To overcome fundamental biases in camera-based remote plethysmography, we propose an adversarial learning-based fair fusion method, using a novel RGB-Radar hardware setup.

Abstract
With the resurgence of non-contact vital sign sensing due to the COVID-19 pandemic, remote heart-rate monitoring has gained significant prominence. Many existing methods use cameras; however previous work shows a performance loss for darker skin tones. In this paper, we show through light transport analysis that the camera modality is fundamentally biased against darker skin tones. We propose to reduce this bias through multi-modal fusion with a complementary and fairer modality – radar. Through a novel debiasing oriented fusion framework, we achieve performance gains over all tested baselines and achieve skin tone fairness improvements over the RGB modality. That is, the associated Pareto frontier between performance and fairness is improved when compared to the RGB modality. In addition, performance improvements are obtained over the radar-based method, with small trade-offs in fairness. We also open-source the largest multi-modal remote heart-rate estimation dataset of paired camera and radar measurements with a focus on skin tone representation.


Files


Citations

@article{vilesov2022blending,
  title={Blending Camera and 77 GHz Radar Sensing for Equitable, Robust Plethysmography},
  author={Vilesov, Alexander and Chari, Pradyumna and Armouti, Adnan and Harish, Anirudh Bindiganavale and Kulkarni, Kimaya and Deoghare, Ananya and Jalilian, Laleh and Kadambi, Achuta},
  journal={ACM Transactions on Graphics (TOG)},
  volume={41},
  number={4},
  pages={1--14},
  year={2022},
  publisher={ACM New York, NY, USA}
}


Contact
Alexander Vilesov
Electrical and Computer Engineering Department
vilesov@ucla.edu