Following the argument wherever it leads is a piece of well-known and time-honored advice we give to students in philosophy. Using three instances drawn from the history of philosophy, we look at reasons for both adhering to this principle and for sometimes putting it aside in favor of other considerations. We find that the requirement of following the argument where it leads is not a simple demand of logic, but rather a complex norm that is sensitive to various considerations. Some of these have to do with the fact that consistency may be restored to one's system of beliefs only at a price that one may judge, on cognitive or moral grounds, as too high. Following the argument wherever it leads is thus a norm about the place of reason in our intellectual and practical lives and is therefore a norm that must be evaluated in the context of various extra-logical considerations.