This paper deals with the problem of unsupervised domain adaptation that aims to learn a classifier with a slight target risk while labeled samples are only available in the source domain. The proposed approach, called DA-SSL (Domain Adaptation meets Semi-Supervised Learning) attempts to find a joint subspace of the source and target domains using Linear Discriminant Analysis, such that the projections of the data into this latent subspace can be both domain invariant and discriminative. This aim, however, can be rather difficult to accomplish because of the missing labeled data in the target domain. To defeat this challenge, we use an incremental semi-supervised approach based on optimal transport theory, that conducts selective pseudo-labeling for unlabeled target instances. The selected pseudo-labeled target data are then combined with the source data to incrementally learn a robust classifier in a self-training fashion after the subspace alignment. Experiments show the competitiveness of the proposed approach over contemporary state-of-the-art methods on two benchmark domain adaptation datasets. We make our code publicly available (Code is available at: https://github.com/MouradElHamri/DA-SSL ).