The Swans made three changes to the side that beat Plymouth in midweek with Andrea Orlandi, Cedric van der Gun and Craig Beattie all starting.
Wales international Chris Gunter was in the Forest line-up but Rob Earnshaw had to be content with a place on the bench.
Swansea had the better of the opening exchanges but it was seven minutes before van der Gun had the first strike on goal.
He was set up by Orlandi but his shot grazed the far angle and flew to safety.
Forest hit back when Wes Morgan headed into the danger area but Kelvin Wilson's tame effort went straight to Dorus de Vries in the home goal.
Ashley Williams missed a glorious chance to open the scoring after 18 minutes when a van der Gun centre came off the crossbar but from only a yard out he headed high and wide.
Forest went straight down the other end and a back-header from McGoldrick was acrobatically saved on the line by de Vries.
The Swans kept up the pressure and Nathan Dyer had a low shot pushed away by Lee Camp at full-stretch, while van der Gun was prevented from scoring on the half hour by another stop from the overworked Camp.
Forest took the lead totally against the run of play in the 35th minute when McGoldrick's shot took a slight deflection from Williams before nestling in the corner with de Vries unsighted.
Lee Trundle replaced Beattie in the 53rd minute as the Swans looked to get back on level terms, but it was McGoldrick who should have increased his side's lead two minutes later only to scuff his shot straight at de Vries.
Forest threw on Nathan Tyson and Dexter Blackstock to increase their strikeforce and they looked far more dangerous than the Swans, who had Trundle as a lone frontrunner.
Swansea pressed hard for the equaliser but Trundle blasted a couple of chances high and wide when through on goal and Forest held on for the victory.