Continuous measurement of physiological functions, like heart rate (HR) and heart rate variability (HRV), using commercially available wearable sensors provides the prospects of improving the healthcare of individuals with a positive impact on society, bringing pervasiveness, lower cost, and broader access. However, common wearable devices use photoplethysmography (PPG) to derive data on HR and HRV, and it is yet unclear to which extent PPG signals can be used as a proxy for data collected using medical-grade devices. To address this challenge, we consider five consumer devices to assess the signal quality of HR and two devices measuring HRV and compare them with a standard electrocardiography (ECG) Holter monitor. We collect data from fourteen participants who followed a 55 minutes protocol for at least two sessions. Using this data set, which we make publicly available to the research community, we show that PPG is a valid proxy for both HR and standard time- and frequency-domain measurements of HRV. Further, we demonstrate that wearable devices are suitable for monitoring both HR and HRV in daily life but might be limited during strenuous exercise. The study indicates that armband-based devices are more reliable than wrist-based wearables for HRV assessment.