It is the goal of private cloud platforms to optimize the resource allocation process and minimize the expense to process tasks. Essentially, resource allocation in clouds involves two phases: virtual machine selection (VMS) and virtual machine placement (VMP), and they can be jointly considered. However, existing solutions separate VMS and VMP, therefore, they can only get local optimal resource utilization. In this paper, we explore how to optimize the resource allocation globally through considering VMS and VMP jointly. Firstly, we formulate the joint virtual machine selection and placement (JVMSP) problem, and prove its NP hardness. Then, we propose the Resource-Decoupling algorithm that converts the JVMSP problem into two independent sub-problems: Max-Capability and Min-Cost. We prove that the optimal solutions of the two sub-problems guarantees the optimal solution of the JVMSP problem. Furthermore, we design the efficient Max-Balanced-Utility and Extent-Greedy heuristic algorithms to solve Max-Capability and Min-Cost, respectively. We evaluate our proposed algorithms on datasets with different distributions of resources, and the results demonstrate that our algorithms significantly improve the resource utilization efficiency compared with traditional solutions and existing algorithms.