Annually, a huge number of pilgrims visit Mecca to perform Al Hajj ritual. Crowd management is critical in this occasion in order to avoid crowd disasters (e.g., stampede and suffocation). Recent studies stated that various factors, such as the environment, fatigue level, health condition and emotional status have a significant effect on crowded events. This calls for a need for an automated data analytics system that feeds event organizers with information about those factors on real-time, at least from a generalizable sample of crowd subjects, in which proactive crowd management decisions are made to reduce overall risks. This paper develops a novel methodology that fuses mobile GPS and physiological data of Hajj pilgrims collected through wearable sensors to train three classification models: (a) current performed Hajj activity, (b) fatigue level, and (c) emotional level. In a pilot experiment conducted against two subjects, promising results of a minimum of 75% accuracy levels were achieved for the activity recognition and fatigue level classifiers, whereas the emotional level classifier still requires further refinements.