The visitors took the lead through Steven Gillespie in the 20th minute. But Colin Calderwood's side were always on top and it was no surprise when Ghana international Agogo pulled them level with a spectacular volley.
Luke Chambers rifled home to put the Reds in front just before the interval - and Agogo added his second two minutes after half-time to make certain of Forest's second victory on the bounce.
Forest almost made an audacious start to the game when defender Kelvin Wilson struck a spectacular 30-yard drive that forced an equally spectacular save from Cheltenham's Shane Higgs, who palmed the ball away as it threatened to arrow high into the top corner.
And, from the resulting corner, Julian Bennett latched onto a Lewis McGugan cross to glance a header over the bar.
It was only Forest that were providing any attacking menace and they came close again as Garath McCleary - the winger handed his first start for the Reds since signing from Bromley in January - provided a knock-down for Brett Ormerod to strike a low shot goalwards, but Higgs saved well.
McCleary was involved again as he rampaged down the right flank before crossing for Agogo to bend a shot narrowly wide.
But, having not previously mounted a single attack, Cheltenham made the most of their first breakaway when they snatched the lead against the run of play.
Paul Connor controlled a loose ball just inside the Forest half and quickly sent Gillespie racing clear of the defence with an incisive pass, and the midfielder kept his composure to roll the ball into the back of the net off the foot of the post.
This only seemed to fire Forest up further, however, and it took only six minutes for them to equalise, with Sammy Clingan delivering the ball into the box from the left and Agogo providing an emphatic, volleyed finish from 12 yards.
An injury to Bennett ensured he had to be replaced by Kris Commons just before the interval, but there was still time for the home side to take the lead, as Luke Chambers lashed home from close range, after McCleary's header had clattered the bar.
The second half began with a nervous moment for Forest as Gillespie hit the bar with a clever chip.
But, in the 47th minute, the home side extended their advantage as Agogo launched into a solo run through the middle, which culminated in a powerful 20-yard drive that bounced in off the foot of the post.
Cheltenham almost pulled one back as Connor thumped a header against the bar following an enticing deep cross from Jerry Gill, but Ormerod and Commons also went close to adding to Forest's lead as Calderwood's side cantered to an easy win.