Obstructive sleep apnea (OSA) is a prevalent sleep disorder, characterized by recurrent episodes of upper airway obstructions during sleep. We hypothesize that breath-by-breath audio analysis of the respiratory cycle (i.e., inspiration and expiration phases) during sleep can reliably estimate the apnea hypopnea index (AHI), a measure of OSA severity. The AHI is calculated as the average number of apnea (A)/hypopnea (H) events per hour of sleep. Audio signals recordings of 186 adults referred to OSA diagnosis were acquired in-laboratory and at-home conditions during polysomnography and WatchPat study, respectively. A/H events were automatically segmented and classified using a binary random forest classifier. Total accuracy rate of 86.3% and an agreement of κ=42.98% were achieved in A/H event detection. Correlation of r=0.87 (r=0.74), diagnostic agreement of 76% (81.7%), and average absolute difference AHI error of 7.4 (7.8) (events/hour) were achieved in in-laboratory (at-home) conditions, respectively. Here we provide evidence that A/H events can be reliably detected at their exact time locations during sleep using non-contact audio approach. This study highlights the potential of this approach to reliably evaluate AHI in at home conditions.