Stroke patients often contend with long-term physical challenges that require treatment and support from both formal and informal caregivers. Socially Assistive Robots (SARs) can assist patients in their physical rehabilitation process and relieve some of the burden on the informal caregivers, such as spouses and family members. We collected and analyzed information from 23 participants (11 stroke patients and 12 informal caregivers) who participated in a total of six focus-group discussions. The participants responded to questions regarding using a SAR to promote physical exercises during the rehabilitation process: (a) the advantages and disadvantages of doing so; (b) specific needs that they wish a SAR would address; (c) patient-specific adaptations they would propose to include; and (d) concerns they had regarding the use of such technology in stroke rehabilitation. We found that the majority of the participants in both groups were interested in experiencing the use of a SAR for rehabilitation, in the clinic and at home. Both groups noted the advantage of having the constant presence of a motivating entity with whom they can practice their rehabilitative exercises. The patients noted how such a device can assist formal caregivers in managing their workload, while the informal caregivers indicated that such a system could ease their own workload and sense of burden. The main disadvantages that participants noted related to the robot not possessing human abilities, such as the ability to hold a conversation, to physically guide the patient's movements, and to express or understand emotions. We anticipate that the data collected in this study-input from the patients and their family members, including the similarities and differences between their points of view-will aid in improving the development of SARs for rehabilitation, so that they can better suit people who have had a stroke, and meet their individual needs.