This research explores strategies to augment the connectivity among users within Hierarchical Non-Terrestrial Networks (HNTNs) dedicated to Disaster Relief Services (DRS). The primary goal is to optimize radio resources, computing capacities, and the trajectories of Unmanned Aerial Vehicles (UAVs) at each time slot, aiming to maximize the number of satisfactory connections (NSCs). UAVs function as aerial base stations (ABSs), establishing links for reduced capability (RedCap) user equipment (UE) using power domain non-orthogonal multiple access (PD-NOMA). Given the potential inoperability of terrestrial networks during disasters, swift data transmission is critical for mission-critical (MC) UEs. Therefore, end-to-end (E2E) delay is a crucial quality of service (QoS) constraint. The proposed problem is solved using a multi-agent recurrent deterministic policy gradient (MARDPG) algorithm, where the ABSs collaborate to maximize the NSCs and determine their optimal policy by interacting with the environment. Additionally, a sharing experience module (SEM) is incorporated, enabling agents to encode actions and observations using long short-term memory (LSTM), allowing each agent to utilize the historical actions and observations of other agents. To demonstrate the superiority of MARDPG, three algorithmic benchmarks and four different system models are implemented. The numerical results demonstrate the impact of various parameters, such as the number of subcarriers, users, and the maximum tolerable E2E delay on the NSCs. Furthermore, different scenarios indicate that MARDPG outperforms the benchmarks, achieving approximately a 6 percent optimality gap and a 91 percent fairness for achievable rate among users. HNTNs, DRS, UAVs, PD-NOMA, MARDPG I. INTRODUCTION A. Motivations and State-of-the-Art W ireless communication technologies have undergone significant advancements in recent years, emerging as a critical facilitator for upcoming consumer applications and services [2], [3]. The transition from the fourth-generation (4G) to the fifth generation leads to a categorizing of quality of services (QoSs), which are named ultra-reliable low-latency communications (URLLC), massive machine-type communication (mMTC), and enhanced mobile broadband