The profiles of narrow lattice solitons are calculated analytically using perturbation analysis. A stability analysis shows that solitons centred at a lattice (potential) maximum or saddle point are unstable, as they drift towards the nearest lattice minimum. This instability can, however, be so weak that the soliton is 'mathematically unstable' but 'physically stable'. Stability of solitons centred at a lattice minimum depends on the dimension of the problem and on the nonlinearity. In the subcritical and supercritical cases, the lattice does not affect the stability, leaving the solitons stable and unstable, respectively. In contrast, in the critical case (e.g. a cubic nonlinearity in two transverse dimensions), the lattice stabilizes the (previously unstable) solitons. The stability in this case can be so weak, however, that the soliton is 'mathematically stable' but 'physically unstable'.