Falls have been one of the main threats to people’s health, especially for the elderly. Detecting falls in time can prevent the long lying time, which is extremely fatal. This paper intends to show the efficacy of detecting falls using a wearable accelerometer. In the past decade, the fall detection problem has been extensively studied. However, since the hardware resources of wearable devices are limited, designing highly accurate embeddable models with feasible computational cost remains an open research problem. In this paper, different types of shallow and lightweight neural networks, including supervised and unsupervised models are explored to improve the fall detection results. Experiment results on a large open dataset show that the lightweight neural networks proposed have obtained much better results than machine learning methods used in previous work. Moreover, the storage and computation requirements of these lightweight models are only a few hundredths of deep neural networks in literature. In tested lightweight neural networks, the best one is proved to be the supervised convolutional neural network (CNN) that can achieve an accuracy beyond 99.9% with only 441 parameters. Its storage and computation requirements are only 1.2 KB and 0.008 MFLOPs, which make it more suitable to be implemented in wearable devices with restricted memory size and computation power.