Reggie Lambe won all three points for the Stags late into injury time at Huish Park with a tremendous header as the game was heading for a stalemate.
With chances few and far between, Stags appeared to be out of luck after seeing a number of chances cleared by the Yeovil defence, but Lambe popped up right at the death to take all three points home.
Adam Murray was forced into a number of changes for the trip to Yeovil, with defenders Nicky Hunt and Krystian Pearce unavailable through suspension and fellow defender Malvind Benning was missing through the devastating knee injury which Murray revealed prior to kick-off at Huish Park.
New Notts County loanee Blair Adams came straight into the side in place of the injured Benning, Mitch Rose returned the side in place of Hunt and Lee Collins was handed a start, as well as the captain's armband, to partner Ryan Tafazolli at the heart of defence.
Yeovil made two changes to the side which were defeated 2-0 by Notts County at the weekend with Matthew Dolan and Wesley Fogden dropping to the bench and being replaced by Shaun Jeffers and Marc Laird in the starting XI.
The opening ten minutes of the game saw no chances for either side as the game started as a midfield battle with a number of free-kicks being given. Stags enjoyed the lion's share of possession without managing to carve out a chance.
The first chance of the game fell to the home side as Ryan Bird tried his luck from 20 yards out. The forward wriggled free of Jamie McGuire and unleashed a well-hit drive but goalkeeper Brian Jensen watched it go safely over the bar.
As the 20 minute mark approached, Yeovil should have taken the lead. A whipped cross from right-back Connor Roberts found the head of Shaun Jeffers in the six-yard box, but the forward couldn't keep his header on target as it skimmed wide of Jensen's goal.
Stags' first attempt of the evening came after half-an-hour. Reggie Lambe did well to win a throw deep in the Yeovil half and Adam Chapman saw his long throw headed out to Jamie McGuire, but the midfielder couldn't keep his shot from 25 yards down and saw it clear the terrace behind the goal.
Murray's men started to grow into the game as the half progressed with Lambe and Chapman becoming increasingly influential in the final third, without creating a real chance of note.
The Glovers' 'keeper Artur Krysiak almost gifted the Stags an opener after he came out to clear but saw his attempted clearance charged down by Jack Thomas, but the stopper managed to hook the ball away at the second attempt.
The final echelons of the first half consisted of no chances of note for either side, with the game becoming more and more of a midfield battle than it already was.
The two sides emerged for the second half unchanged, however after Stags enjoyed most of the possession in the opening stages of the half, Adam Murray made his first change.
Jack Thomas was replaced by Matty Blair for his second appearance of the season as Murray looked to add more width to his side.
Then came Stags' best chance of the game. From the corner which was won by the departed Thomas, the ball fell for Reggie Lambe in the six yard box but Artur Krysiak was on hand to block his shot at point-blank range.
Murray then introduced Adi Yussuf in place of Craig Westcarr as the Stags altered their formation slightly and went to two up-front. Shortly afterwards, Adam Chapman was replaced by Chris Clements.
As the hour mark passed, Stags started to show some more attacking guile. Glovers' 'keeper Artur Krysiak had to be at his sharpest to clear ahead of the onrushing Adi Yussuf after Reggie Lambe threaded a ball through to the Tanzanian.
Then came the first shot on target of the game. Yussuf found himself in a yard of space on the edge of the box and hit a left-footed shot on the turn, which was held by Krysiak low to his left.
Stags continued to assert their authority as substitute Chris Clements hit a rasping drive from 25 yards, but it was deflected over the bar for a corner.
Yeovil were beginning to tire as the Stags remained well on top in the second half, and were so unlucky not to be ahead. Blair, who impressed after coming on, played a low cross towards Lambe who poked it round the 'keeper and hit it towards goal, but Connor Roberts produced a magnificent goal-line clearance to keep the scores level.
The fourth official indicated there would be a minimum of nine minutes time added on at the end of the game, as Stags searched for a late winner.
As the game entered the 98th minute and everyone inside Huish Park was awaiting the referee's final whistle, Reggie Lambe, who was the Stags' most dangerous player on the night, leapt highest to head home the winner.
Mitch Rose out-muscled Nathan Smith down the right-hand side and whipped a ball towards the penalty spot, where the waiting Lambe powered a header into the corner of the Yeovil goal to send the travelling 105 Stags' supporters into raptures.
Mansfield Town: Jensen, Rose, Collins, Chapman (Clements 59), Green, Westcarr (Yussuf 53), Tafazolli, Lambe, J Thomas (Blair 49), McGuire, Adams.
Substitutes not used: Shearer, Marsden, Beardsley, N Thomas.
Yeovil Town: Krysiak, Smith, Arthurworrey (Allen 77), Sokolik, Laird, Bird, Jeffers (Lacey 77), Dickson, Cornick, Roberts, Sheehan.
Substitutes not used: Dolan, Weale, Fogden, Bassett.
Attendance: 2,954 (105 away)