The game had looked likely to end in a draw, after Chris Cohen's first goal for the Reds had helped them claw their way back from behind, after referee Anthony Taylor had handed the visitors a controversial lead.
Forest were furious after seeing a penalty awarded for an innocuous challenge by Kelvin Wilson - who was also red carded after Malvin Kamara had gone down in the box in the 35th minute.
But substitute Cohen started Forest's fightback with a deflected shot in the 68th minute, before McGugan lashed home a spectacular second in the final minute
Huddersfield were let off the hook in early on when, having charged into the box, Kris Commons appeared to be knocked to ground by a clumsy challenge from David Mirfin, but referee Taylor waved play on.
Andy Booth sent a trademark headed effort narrowly over the bar for Huddersfield, although the striker was punished for a push on Ian Breckin in the build-up.
Otherwise the game was short of opportunity and it was not until the 17th minute, when an Ian Breckin header was smartly held by Matt Glennon, that either keeper was forced into action.
Paul Smith was soon tested at the other end, as Robbie Williams bent a swerving free-kick low around the base of the defensive wall and towards the bottom corner, with the Forest man getting down well to block.
Grant Holt failed to hit the target after Junior Agogo had set up a clear headed chance for his strike partner.
But the game was to turn in a moment of controversy, as referee Taylor handed Huddersfield the lead and reduced Forest to ten men with a bizarre decision.
It seemed an innocuous moment when defender Wilson, under pressure from Kamara, headed the ball back into the arms of keeper Smith.
The Huddersfield winger went down in a heap as all three players came together, but there were no appeals from the Huddersfield players, who seemed as surprised as anyone when Taylor pointed to the spot, before then sending Wilson off as well.
And Phil Jevons made sure Forest's punishment was complete with a decisive finish from the spot, to put the visitors ahead in the 36th minute.
Commons and Sammy Clingan both came close to pulling Forest level before the interval but, with Huddersfield in determined mood, their route back into the match seemed uncertain.
Huddersfield, in fact, could have had Forest dead and buried five minutes into the second half, but Booth sidefooted a weak shot when presented with a golden opportunity, with Smith saving easily.
And the Forest stopper had to produce an outstanding block to deny Huddersfield sub Danny Schofield, when the midfielder seemed certain to score from ten yards.
But Forest were awarded for their perseverance as, within three minutes of coming on, Cohen pulled them level.
The midfielder was quick to try his luck from the edge of the box and with the aid of a significant deflection, saw his shot loop into the far corner of the net.
Commons, who powered a shot over, and Agogo, who saw Glennon save, both came close to edging Forest into the lead.
Huddersfield also came close with a rocket shot from Williams which clattered the foot of the post.
But it was McGugan who was to have the final say with a spectacular injury-time strike to give Forest the points.