The Cards striker will incur an immediate three-match ban which took the edge of this victory for boss Glenn Cockerill. "They both went in hard but because Challinor got there first it looked like Justin caught him but he was diving about all over the place," he said.
With Woking clinging to the hope they can make up a nine-point gap in their remaining matches Cockerill was looking for his men to take the game to their opponents in front of a disappointingly low crowd.
Exeter had the best chance in the early exchanges in front of a good travelling support - striker Jake Edwards failing to connect with a close-range volley when he looked as shocked as anyone else not to see an offside flag raised.
The rest of the half was relatively incident-free except for a thunderous drive from Chris Sharpling which fizzed a yard past Paul Jones' post.
Exeter's Andy Taylor had a super chance moments after half time but his shot was finger-tipped away by keeper Shwan Jalal.
Woking had a spell of pressure and were aggrieved not to be given a penalty when Craig McAllister's header appeared to hit defender Chris Todd on the arm.
Exeter's full-back Danny Woodards then cut in from the right before unleashing a wonderful left-footed shot from 20 yards which thudded against Jalal's crossbar - the closest either team had come to scoring.
Todd cleared off the line from Steve Evans, and then when Tom Hutchinson was adjudged to have been impeded in the box, Richards buried Woking's winner from the penalty spot before seeing red in the last minute.