We present communication efficient secure protocols for a variety of linear algebra problems. Our main building block is a protocol for computing Gaussian Elimination on encrypted data. As input for this protocol, Bob holds a k × k matrix M, encrypted with Alice's key. At the end of the protocol run, Bob holds an encryption of an upper-triangular matrix M′ such that the number of nonzero elements on the diagonal equals the rank of M. The communication complexity of our protocol is roughly O(k2). Building on Oblivious Gaussian elimination, we present secure protocols for several problems: deciding the intersection of linear and affine subspaces, picking a random vector from the intersection, and obliviously solving a set of linear equations. Our protocols match known (insecure) communication complexity lower bounds, and improve the communication complexity of both Yao's garbled circuits and that of specific previously published protocols.