The Closest String problem asks to find a string s which is not too far from each string in a set of m input strings, where the distance is taken as the Hamming distance. This well-studied problem has various applications in computational biology and drug design. In this paper, we introduce a new variant of Closest String where the input strings can contain wildcards that can match any letter in the alphabet, and the goal is to find a solution string without wildcards. We call this problem the Closest String with Wildcards problem, and we analyze it in the framework of parameterized complexity. Our study determines for each natural parameterization whether this parameterization yields a fixed-parameter algorithm, or whether such an algorithm is highly unlikely to exist. More specifically, let m denote the number of input strings, each of length n, and let d be the given distance bound for the solution string. Furthermore, let k denote the minimum number of wildcards in any input string. We present fixed-parameter algorithms for the parameters m, n, and k+d, respectively. On the other hand, we then show that such results are unlikely to exist when k and d are taken as single parameters. This is done by showing that the problem is NP-hard already for k=0 and d ≥2. Finally, to complement the latter result, we present a polynomial-time algorithm for the case of d=1. Apart from this last result, all other results hold even when the strings are over a general alphabet.