Objective: To evaluate the performance of two open-source real-time operating systems (RTOSs), Keil RTX5 and FreeRTOS. Besides, a comparison between them has been also established based on four timing metrics: task switching time, pre-emption time, semaphore shuffling time, and inter-task messaging latency. All the tests have been performed on an STM32F429 discovery board based on Cortex-M4 MCUs. Methods: To measure the timing metrics, the ARM cycle counter register implemented in the DWT unit was used. Findings: The DWT cycle counter allows us to capture the number of cycles that occurred in the execution of a part of the code. Therefore, the time measurements of the metrics selected show that FreeRTOS has good performance with the lowest value of switching time, preemption time, and semaphore shuffling time. Instead, Keil RTX5 has fast message passing. Novelty: The study provides an evaluation and comparison of the latest version of the most used open-source RTOSs, Keil RTX5 and FreeRTOS v10.2.0. Furthermore, the timing metrics have been measured accurately with the ARM cycle counter register without using any other hardware or GPIO pin that may disturb the measurement. The comparison is based on four critical timing metrics that affect mostly the performance of any RTOS and define their time capability. Finally, the tests have been made on a low-power ARM MCU.