Background: Screening and monitoring serum bilirubin (SBR) in neonates is crucial to prevent neonatal hyperbilirubinemia (NH)-associated morbidity and mortality worldwide. A lack of resources is often a barrier for measuring SBR in developing countries. Reliable, cost-effective, easy to use point-of-care (POC) SBR tests are needed. This study aimed to evaluate the technical accuracy and usability of the Bilistick System (BS), a new bilirubin POC test, in a tropical setting. Methods: This was a mixed-methods study, including laboratory validation of the BS, direct observation of technical procedures as performed by the midwives and midwives’ assessment of the device’s easiness of use through focus group discussions (FGD) and a self-administered questionnaire. The study was conducted in a field clinic of the Shoklo Malaria Research Unit along the Thailand-Myanmar border between January and December 2017. Results: A total of 173 samples were tested at a median age of 4 days. BS generated an error message—providing no SBR readout—in 48.6% of the tests performed. For the tests that yielded a result, the correlation coefficient (95% CI) between BS and routine laboratory bilirubinometer SBR was 0.87 (0.77-0.93). The accuracy decreased with increasing haematocrit and at higher humidity (≥75%). Direct observation of the operators using the device and analysis of the focus group discussions and questionnaires indicated that the BS was considered easy to use and required limited training. Conclusions: This evaluation showed that the BS, in its current formulation, does not provide reliable results for measuring SBR in a tropical, low-resource setting but has acceptable usability features.