Lester only came into the action as a first-half substitute with Forest 2-0 down but was always in the thick of the drama.
On the stroke of half time he escaped a red card when he went in with studs showing on keeper Paul Rachubka and was booked instead.
Then within 18 seconds of the restart Lester threw Forest a lifeline with a superb lob over Rachubka.
But Lester, in his second spell at the City Ground, cost his side the chance to grab something from the game when he was dismissed for diving as he went down in the box under challenge from Andy Holdsworth in the 78th minute.
Before that Forest had already been reduced to 10 men when skipper Ian Breckin was sent off for bringing down Gary Taylor-Fletcher who was clean through on goal.
In the first half Huddersfield played some of their best football of the season and should have had the game sewn up before half time.
Danny Schofield scored a stunning 21st minute opener when he collected Danny Adams' long ball and with keeper Paul Gerrard slow off his line lobbed the ball over his head. The ball landed on the line and bounced up into the roof of the net.
That was Schofield's eighth of a productive campaign and Huddersfield were 2-0 up when Andy Booth also claimed his eighth of the season five minutes later.
Chris Brandon twisted and turned and broke into the box and his cross was flicked on by Taylor-Fletcher only for Booth to nod home on the line.
Just three minutes after that Schofield could have sealed victory as the Forest defence backed off during an intricate Huddersfield move.
Schofield saw his shot come back off the upright and Booth fired the rebound straight at Gerrard.
Forest refused to cave in even when down to 10 men - and then nine - and Huddersfield could not make the advantage count.
Huddersfield had a series of chances but the Forest goal led a charmed life at times, though the Terriers held on for a deserved victory.