Heterogeneous cells have been emerged as the dominant design approach for the deployment of 5G wireless networks . In this context, inter-cell interferences are expected to drastically affect the 5G targets, especially in terms of throughput experienced by the mobile users. This work proposes a novel Deep Reinforcement Learning (DRL) scheme, targeting at minimizing the difference between the allocated and requested user throughput through power regulation . The developed algorithm is employed in heterogeneous cells that are controlled in a centralized manner and validated for 5G-compliant channel models. First, the proposed learning framework of the DRL method is presented, mainly including the stabilization of the learning-related hyperparameters. Then, the DRL method is evaluated for several simulation scenarios and compared to well-established optimization methods for power allocation, namely the Water-filling and Weighted Minimum Mean Squared Error (WMMSE) algorithms, as well as a fixed power control scheme. The evaluation outcomes demonstrate the ability of the DRL framework in accurately approaching the user requirements, whereas the Water-filling and WMMSE solutions present large deviations from the user demands since they aim at the total network-wide throughput maximization.