Diamond's, who were one down at half-time, began their recovery when Magnus Okounghae got an equaliser on 49 minutes. A Neil McCafferty free-kick from 30 yards out on the left was fumbled by goalkeeper Matthew Gilks and Okounghae smashed the ball into the net.
Improving Rushden looked as if they were going go on and collect all three points and they would have done had they not missed a couple of really good chances.
Substitute Daniel Grainger missed the chance of a lifetime and must still be wondering how he failed score after a short cross from the right found him four feet out with the goal at his mercy but he somehow fired over the bar.
To add to Rushden's woes a corner kick in the last minute resulted in a substitute Jon Boardman getting Rochdale's winner.
Rochdale had opened the scoring on the stroke of half-time when Blair Sturrock latched onto a centre from the left and slipped the ball past goalkeeper Jamie Young.
In the second half Rushden produced a lot of good football which resulted in several good chances but there was a great shortage of takers.
Rochdale had their second half moments too when Rickie Lambert and Gary Jones both went close before Boardman produced the winner.
Rushden gave Rochdale a troubled first half and they looked quite comfortable inflicting damage on the home side in the second.
Goalkeeper Young made a number of good first half saves but perhaps was at fault when Rochdale got their winner from a corner he failed to deal with in the dying moments.