Pedestrian trajectories are crucial for self-driving cars to plan their paths effectively. The sensors implanted in these self-driving vehicles, despite being state-of-the-art ones, often face inaccuracies in the perception of surrounding environments due to technical challenges for adverse weather conditions, interference from other vehicles sensors and electronic devices, and signal reception failure, leading to incompleteness in trajectory data. But for real-time decision making for autonomous driving, the trajectory imputation is no less crucial. Previous attempts to address this issue such as statistical inference and machine learning approaches have shown promise. Yet, the landscape of deep learning is rapidly evolving with new and more robust models emerging. In this research, we have proposed an encoder-decoder architecture, Human Trajectory Imputation Model, coined as HTIM, to tackle these challenges. This architecture aims to fill in the missing parts of pedestrian trajectories. The model has been evaluated using the Intersection drone inD dataset, containing trajectory data at suitable altitudes preserving naturalistic pedestrian behavior with varied dataset sizes. To assess the effectiveness of our model, we have utilized L1, MSE, Quantile and ADE loss. Our experiments have demonstrated that HTIM outperforms the majority of the state-of-the-art methods in this field, thus indicating its superior performance in imputing pedestrian trajectories.