Let P be a set of n points in ℝd, so that each point is colored by one of C given colors. We present algorithms for preprocessing P into a data structure that efficiently supports queries of the form: Given an axis-parallel box Q, count the number of distinct colors of the points of P ∩ Q. We present a general and relatively simple solution that has polylogarithmic query time and worst-case storage about O(nd). It is based on several interesting structural properties of the problem that we derive. We also show that for random inputs, the data structure requires almost linear expected storage. We then present several techniques for achieving space-time tradeoff. In ℝ2, the most efficient solution uses fast matrix multiplication in the preprocessing stage. In higher dimensions we obtain a tradeoff using simpler mechanisms. We give a reduction from matrix multiplication to the offline version of problem, which shows that in ℝ2 our time-space tradeoffs are close to optimal in the sense that improving them substantially would improve the best exponent of matrix multiplication. Finally, we present a generalized matrix multiplication problem and show its intimate relation to counting colors in boxes in any dimension.