A private-key cryptosystem may be viewed as a means by which a trusted dealer privately conveys a large, shared pseudo-random object to a pair of players, using little communication. Alternatively, the messages distributed by the dealer may be viewed as a secure compression of a pair of large identical random pads (or random functions) into a shorter shared “key" or “seed". We pose the question of extending this compression problem to more general correlation patterns among several players. Unlike the simple case of identical pads, where the main security concern is with respect to external eavesdroppers, in the case of general correlations participants also have to be protected from each other. That is, collusions of computationally-bounded players should gain no additional knowledge about the joint pads of the remaining players from the compressed messages they receive, other than what follows from the pads they generate and from knowing the joint distribution of all pads. While this ideal requirement is inherently impossible to meet using little communication, it turns out that it can be approximated to a satisfactory level, allowing to securely use such compressed correlated pads in a wide class of protocols. We propose a simple and modular replication-based approach for securely compressing any linear correlation pattern, using pseudo-random generators or pseudo-random functions in a black-box manner. Applications include amortizing the communication costs of private multi-party computation and proactive secret-sharing of large secrets.