With the increasing adoption of electric devices and renewable energy generation, electric load forecasting, especially short-term load forecasting (STLF), has recently attracted more attention. Accurate short-term load forecasting is of significant importance for the safe and efficient operation of power grids. Deep learning-based models have achieved impressive success on several applications, including short-term load forecasting. Yet, most deep learning models do require a large amount of training data. However, in the real world, it may be very difficult or even impossible to collect enough data to train a reliable machine learning model. This makes is hard to adopt deep models for several real-world scenarios. Thus, it will be very helpful if deep learning models can be learned to tackle tasks with limited amount of training data and unseen tasks. In this work, we propose to use the meta-learning framework to train a long short-term memory-based model for short-term residential load forecasting. Specifically, by minimizing the task-level loss (loss over several tasks), the model is trained to perform well on different tasks. We also use domain randomization techniques to further augment the training tasks, which may further improve the generalization ability of the proposed model. Our model is evaluated on real-world data sets and compared against some classic forecasting models.