Mouse dynamics as behavioral biometrics are under investigation for their effectiveness in computer security systems. Previous state-of-the-art methods relied on heuristic feature engineering for the extraction of features. Our work addresses this issue by learning the features with a convolutional neural network (CNN), thereby eliminating the need for manual feature design. Contrary to time-series-based modeling approaches, we propose to use a two-dimensional CNN with images as inputs. While counterintuitive at first sight, it permits to profit from well-initialized lower-layer kernels obtained via transfer learning. We demonstrate our results on two public datasets, Balabit and TWOS, and compare against a 1D-CNN and a classical baseline relying on hand-crafted features, which are both outperformed. We show that a position-independent variant of the 2D-CNN loses little performance yet we learned that the trained classifier is very sensitive to simulated resolution shifts at test time. In a final step, we analyze and visualize the learned features on single test curves using layer-wise relevance propagation (LRP). This analysis reveals that the 2D-CNN uses curve information only sparsely, with a tendency to assign little relevance to straight segments and artifactual curve crossings.