Taylor delivered the free-kick which Adam Stansfield headed into his own net, but then produced a back header which goalkeeper Martin Rice failed to gather, allowing Stansfield to make up for his earlier indiscretion.
But Taylor produced the perfect riposte, stepping up to fire home a free-kick winner five minutes later.
Exeter took the lead on 20 minutes when Jamie Pitman fouled Dean Moxey on the very edge of the Hereford area.
Taylor swung in a perfectly-delivered free-kick and Stansfield, in trying to clear, only succeeded in diverting the ball into his own net.
Moxey should have extended the lead on 29 minutes when he raced on to flick-on from Paul Buckle's cross, but volleyed a powerful effort over the bar.
Hereford produced few clear openings although Ryan Green was unlucky to see a fine cross elude all of the home attack and Mark Robinson then cleared the bar with a well-struck 30-yarder.
Steve Flack had the ball in the net again as Exeter dominated the opening minutes of the second period but his strike was ruled out for offside.
Against the run of play, Hereford were back on terms as Taylor headed back to Rice, but the goalkeeper made a mess of trying to gather the ball, leaving Stansfield the simple task of firing into the net.
Shortly afterwards, though, Exeter were awarded a free-kick 25 yards out and Taylor swerved his kick around the wall and into the net.
Rice parried a David Brown free-kick as Hereford went in search of a point, but the visitors held on for an important double.