Colin Calderwood's side had dominated and were leading 2-1 with three minutes to go - but goals from Richard Stearman and Stephen Clemence saw them snatch a 3-2 win.
Forest had been ahead and cruising after Nathan Tyson's 64th minute strike had restored their advantage following a spectacular free kick equaliser for Leicester from Alan Sheehan.
The last game had ended in a moment of sportsmanship as Forest agreed to abandon the match at half time following the collapse of Foxes defender Clive Clarke when Forest were leading 1-0.
So it was fitting - albeit a little surprising - that last night's replayed tie also began in sporting fashion with Leicester generously allowing Forest keeper Paul Smith to net straight from the kick off to give Forest their lead.
Leicester's gesture ensured that they began with an immediate deficit - but it could quickly have been worse.
Only a goalline clearance from Sheehan denied Forest a two goal lead after Junior Agogo had rounded keeper Marton Fulop.
And Sheehan was soon the hero at the other end as his 31st minute free kick drew Leicester level.
Still Forest could have scored again, with Fulop saving bravely at Tyson's feet and also seeing the striker head a good chance over the bar.
Leicester had their keeper to thank yet again as he thwarted Luke Chambers when he directed a downward header towards the bottom corner.
Forest finally got the lead they deserved as Kris Commons fired an inch perfect pass into the path of Tyson who produced a finish to match.
But just as it seemed Forest would earn a third round tie at Aston Villa, Leicester pounced twice in the final stages to snatch an unlikely victory.