PDE-constrained optimization problems are often treated using the reduced formulation where the PDE constraints are eliminated. This approach is known to be more computationally feasible than other alternatives at large scales. However, the elimination of the constraints forces the optimization process to fulfill the constraints at all times. In some problems this may lead to a highly nonlinear objective, which is hard to solve. An example of such a problem, which we focus on in this work, is full waveform inversion (FWI), which appears in seismic exploration of oil and gas reservoirs and in medical imaging. In an attempt to relieve the nonlinearity of FWI, several approaches suggested expanding the optimization search space and relaxing the PDE constraints. This comes, however, with severe memory and computational costs, which we aim to reduce. In this work we adopt the expanded search space approach and suggest a new formulation of FWI using extended source functions. To make the source-extended problem more feasible in memory and computations, we couple the source extensions in the form of a low-rank matrix. This way, we have a large-but-manageable additional parameter space, which has a rather low memory footprint and is much more suitable for solving large scale instances of the problem than the full-rank additional space. In addition, we show how our source-extended approach is applied together with the popular simultaneous sources technique---a stochastic optimization technique that significantly reduces the computations needed for FWI inversions. We demonstrate our approaches for solving FWI problems using 2D and 3D models with high-frequency data only.