Seem to recall that Yanmar use cone type g/box, and badly fouled prop will be enough for g/box to not engage, as will lack of clearance on prop shaft of prop and/or anode.
If it is an old YS there are clutches in the gearbox. One of these could slip meaning in reverse it would not be turning the prop. There could be cone clutches in later gearboxes, again one of these could be worn, causing it to slip.
Another reason not to engage gears is if it is cable operated, is the cable can stretch so failing to put it in gear.
Can you hear, or feel the gearbox going into gear. Is the prop turning at all?
Secure boat to mooring etc. disconnect the control cable at the gearbox and try to put the shift into reverse gear (with engine running), observe the prop shaft, that will tell you where the problem is, drive or gearbox. Hopfully its not the GB
It could be something as simple as something jamming the gear shift mechanism, I had a similar problem caused by an oar which is kept in my cockpit locker jamming against the lever that connected the linkage to the gearbox.