Due to the recent trend towards attribute-based access control (ABAC), several studies have proposed constraints specification languages for ABAC. These formal languages enable security architects to express constraints in a precise mathematical notation. However, since manually formulating constraints involves analyzing multiple natural language policy documents in order to infer constraints-relevant information, constraints specification becomes a repetitive, time-consuming and error-prone task. To bridge the gap between the natural language expression of constraints and formal representations, we propose an automated framework to infer elements forming ABAC constraints from natural language policies. Our proposed approach is built upon recent advancements in natural language processing, specifically, sequence labeling. The experiments, using Bidirectional Long-Short Term Memory (BiLSTM), achieved an F1 score of 0.91 in detecting at least 75% of each constraint expression. The results suggest that the proposed approach holds promise for enabling this automation.