We analyze the dynamics of sum-frequency generation (SFG) and difference-frequency generation (DFG) for broadband input fields. We demonstrate that amplitude modulation of the input fields significantly affects conversion efficiency of SFG and DFG, but frequency modulation does not affect conversion efficiency. Analytic results for SFG and DFG output-field intensities are thereby available for arbitrary fields. Self- and cross-phase modulation can affect the dynamics in the limit of very high field strengths, but these effects are of higher nonlinearity and therefore usually negligible. The quantum efficiency of DFG is shown to be generally higher than SFG, but the energy efficiency is much lower if the difference frequency is significantly less than the input frequencies. Optimal SFG quantum conversion efficiency for such systems approaches that of second-harmonic generation only if the quantum fluence (total number of photons) in the two input beams are equal. Optimal DFG quantum conversion efficiency occurs when the number of photons of frequency ω2 is small yet sufficient for significant stimulation. A recently developed method for obtaining efficient SFG for multimode input fields is theoretically analyzed. The method involves using an arrangement with two or more nonlinear mixing crystals with a time-delay line situated between the crystals that delays one of the fundamental fields relative to the other. The efficiency in the second crystal depends on the cross-correlation function of the two fundamental fields upon leaving the first nonlinear crystal. The time-delay method is not effective for multimode DFG.