This paper addresses the problem of adaptive waveform design for target detection with composite sequential hypothesis testing. We begin with an asymptotic analysis of the generalized sequential probability ratio test (GSPRT). The analysis is based on Bayesian considerations, similar to the ones used for the derivation of the Bayesian information criterion (BIC) for model order selection. Following the analysis, a novel test, named penalized GSPRT (PGSPRT), is proposed on the basis of restraining the exponential growth of the GSPRT with respect to the sequential probability ratio test (SPRT). The performance measures of the PGSPRT in terms of average sample number (ASN) and error probabilities are also investigated. In the proposed waveform design scheme, the transmit spatial waveform (beamforming) is adaptively determined at each step based on observations in the previous steps. The spatial waveform is determined to minimize the ASN of the PGSPRT. Simulations demonstrate the performance measures of the new algorithm for target detection in a multiple input, single output channel.