Recent advancements in Software Defined Networks (SDN), Open Radio Access Network (O-RAN), and 5G technology have significantly expanded the capabilities of wireless networks, extending beyond mere data transmission. This progression has led to the emergence of Virtual Networks (VN) and Network Slicing, enabling industries to enhance their services and applications by establishing virtual networks that utilize shared physical infrastructure. Many works in the literature have considered optimizing the allocation of on-demand slices, assuming the absolute availability of resources and their accurate load. However, accurately allocating future network slices remains challenging due to the error in load prediction, diverse Key Performance Indicators (KPIs), resource price variations, and the potential for overor under-provisioning. This study presents a two-phase intelligent approach to address these challenges. The framework proactively predicts different slice loads while considering prediction errors in optimizing future slices with varied KPIs in a cost-efficient manner. Specifically, our method utilizes historical load data per service and employs AI-based forecasts for service load prediction. Subsequently, it employs a Deep Reinforcement Learning (DRL) agent on O-RAN's virtual Control Unit (vCU) and virtual Distributed unit (vDU) to correct errors in prediction and optimize the cost of slice allocation based on service KPI requirements, ultimately pre-allocating future network slices at reduced costs. Through experimental validation against various baselines and state-of-the-art solutions, we demonstrate the efficacy of our proposed solution, achieving a notable reduction (37-51%) in the average cost of allocated slices while inquiring about (1.5-7%