Extracting information from patents using machine learning algorithms in the context of TRIZ still faces a major problem: the very limited amount of annotated data. Therefore, most approaches, whether for parameter, problem or solution extraction, are based on unsupervised learning algorithms such as Latent Dirichlet Analysis (LDA), or very small supervised learning algorithms such as Support Vector Machine (SVM) or Multi-Layer Perceptron (MLP), which are relatively inefficient. The use of Deep Learning is still relatively uncommon in the extraction of knowledge from patents despite the significant capabilities of these algorithms, particularly BERT. The objective is therefore to present a method for specializing Deep Learning's supervised algorithms on patents while using a very low volume of annotated data. The effectiveness of the method will be analyzed in a task of extracting contradictions from patents, which is a complex task that cannot be performed using unsupervised techniques.