The midfielder put the Clarets ahead with a spectacular first-half free-kick that had left Forest fuming and he then made sure of three points from the penalty spot in the 70th minute, after Wes Morgan was harshly punished for handball.
Robert Earnshaw had pulled Forest level with an opportunist strike in the 52nd minute - but it was referee Clive Penton who found himself in the spotlight following the final whistle, following Burnley's first win at the City Ground since 1964.
Paul Smith had to make the first save of the game after a low shot from Burnley midfielder Chris McCann took a wicked deflection although Chris Cohen and Lee Martin had both tested the visiting defence with half-chances at the other end.
The first genuine opportunity of the match fell to Nathan Tyson but the Forest striker lifted his shot over the bar after being sent racing clear by Cohen.
Although a miss at the other end, when Clarke Carlisle headed wide, was equally glaring.
Despite Forest's domination in terms of possession and their superior football, it was Burnley who took the lead in spectacular fashion.
Forest felt aggrieved when Guy Moussi was punished as he stretched to snatch the ball off Martin Paterson's toes in the 25th minute, but Alexander showed little sympathy as he bent an unstoppable 20-yard free-kick around the wall and inside the far post.
Smith had to be agile again, diving low to his left, to pounce on a low, speculative effort from Kevin McDonald, as Forest's threat faded and Burnley's confidence grew.
Earnshaw twice came close to levelling in first-half stoppage time, seeing a shot deflected wide and then, from the resulting corner, being denied by a fine save from Brian Jensen.
It took Forest seven minutes of the second half to pull themselves level, as a Cohen free-kick was powered onto his own bar by defender Steven Caldwell, with Earnshaw reacting the quickest to prod the loose ball home from close range.
And they could have snatched the lead almost immediately as a driven, powerful strike from Guy Moussi took a deflection and forced a superb reaction save from Jensen.
Smith produced an outstanding save to keep Forest level after Carlisle had connected with another corner, this time with a powerful downward header.
Then came a second moment of controversy as Morgan slid in to block a cross from substitute Ade Akinbiyi at close range - and referee Penton awarded a penalty for handball.
Alexander again provided an emphatic finish, this time to lash the spot-kick high into the top left corner of Smith's net in the 70th minute.