Falls are a very dangerous situation especially among elderly people, because they may lead to fractures, concussion, and other injuries. Without timely rescue, falls may even endanger their lives. The existing optical sensor-based fall monitoring systems have some disadvantages, such as limited monitoring range and inconvenience to carry for users. Furthermore, the fall detection system based only on an accelerometer often mistakenly determines some activities of daily living (ADL) as falls, leading to low accuracy in fall detection. We propose a human fall monitoring system consisting of a highly portable sensor unit including a triaxis accelerometer, a triaxis gyroscope, and a triaxis magnetometer, and a mobile phone. With the data from these sensors, we obtain the acceleration and Euler angle (yaw, pitch, and roll), which represents the orientation of the user’s body. Then, a proposed fall detection algorithm was used to detect falls based on the acceleration and Euler angle. With this monitoring system, we design a series of simulated falls and ADL and conduct the experiment by placing the sensors on the shoulder, waist, and foot of the subjects. Through the experiment, we re-identify the threshold of acceleration for accurate fall detection and verify the best body location to place the sensors by comparing the detection performance on different body segments. We also compared this monitoring system with other similar works and found that better fall detection accuracy and portability can be achieved by our system.