This paper proposes a pre-training method for neural network-based character recognizers to reduce the required amount of training data, and thus the human labeling effort. The proposed method transfers knowledge about the similarities between graph representations of characters to the recognizer by training to predict the graph edit distance. We show that convolutional neural networks trained with this method outperform traditional supervised learning if only ten or less labeled images per class are available. Furthermore, we show that our approach performs up to 33% better than a graph edit distance based recognition approach, even if only one labeled image per class is available.