The Grey Rhebok is endemic to a small region in southern Africa, inhabiting montane and plateau grasslands of South Africa, Swaziland and Lesotho. In South Africa, their distribution is discontinuous and patchy, and they no longer occur north of the Orange River in the Northern Cape, or in parts of the North West Province. They formerly occurred widely in the highveld of western Swaziland, but have disappeared from parts of their former range; they remain fairly common in Malolotja Nature Reserve and still survive locally in unprotected areas. In Lesotho, they probably occurred widely in the past, but have been reduced to a few scattered remnant populations. It responded well to protection in Sehlabathebe National Park in the 1980s (East 1999; Avenant in press).

Although believed to have occurred formerly in hilly country around Gaborone in south-east Botswana, there is no conclusive evidence for their presence in this country (Smithers 1971).