They created a host of chances and could have won more comfortably but had to wait until the 80th minute for Phillips to scramble home the winner from close range.
Exeter dominated the first half and created enough chances to put themselves well ahead at the interval.
Phillips and Craig Farrell linked well in attack and the Grecians always looked a threat going forward.
Gravesend never troubled Exeter in the first half and looked happy to sit back and soak up the pressure.
Farrell shot over the bar and Jon Challinor fired wide in the opening minutes.
Andy Taylor then found Farrell inside the box and the striker struck a fierce shot just over the bar.
Dean Moxey went close for the Grecians when he was found by Taylor's dangerous low cross but his close-range shot was deflected past the post.
Exeter had clearly been instructed to shoot on sight and tested the visitors with a series of long-range shots.
Phillips struck wide from the edge of the box and defender Danny Woodards twice ventured forward and went close with long-range efforts.
After the break Taylor was found by Moxey's fine cross but failed to direct his header on target.
Gravesend occasionally looked dangerous on the break. Andy Drury ran the length of the pitch and found Charlie MacDonald who forced Paul Jones to save superbly with a powerful shot from 15 yards.
Phillips won it for City with 10 minutes left when he shot low through a crowd of players from six yards.
City boss Alex Inglethorpe said: "It was always going to be a tight game but we battled well and created a lot of chances and thought we deserved to win it."