In recent years, sensor-based human activity recognition (HAR) has become a hot topic due to the advancement of sensing technologies, wireless communication technologies and nano-technologies. Since the sensor signals are usually non-stationary and quite noisy, both selecting the discriminant feature representations and finding out the optimal parameters for recognition algorithm play an important role for the enhanced performance and robustness of an HAR system. However, most of the previous research focused on one of them ignoring their interactions. Very few studies focused on these two aspects simultaneously. Considering the two factors separately may lead to inferior HAR performance. This paper presents a novel HAR framework which can optimize the feature set and the parameters of recognition algorithm synchronously for robust and optimal system performance. A new hybrid feature selection methodology using game-theory based feature selection (GTFS) and binary firefly algorithm (BFA), called GTFS-BFA, is proposed. GTFS-BFA is a hybrid methodology combining evidence from both filter and wrapper feature selection methods. It consists of two phases, namely pre-selection phase and re-selection phase. Pre-selection phase relies on game-theory-based filter method, while the re-selection phase uses binary firefly algorithm (BFA) as a wrapper method. The popular and efficient algorithm kernel extreme learning machine (KELM) is utilized as a classifier. The experimental results indicate that the proposed method can obtain better comprehensive performance in terms of four performance measures through a comparison to other existing methods on daily activity dataset from five body positions.