Rochdale were much the better side throughout and in the second half in particular laid siege to the Bury goal, but with defenders John Fitzgerald and Paul Scott outstanding, the away side held on to extend their recent unbeaten run.
The home side started much the brighter and took the lead on 11 minutes when leading scorer Chris Dagnall exchanged passes with Keith Barker before firing home his eighth goal of the season from 12 yards.
Bury's response was almost immediate when in a crowd of players defender Fitzgerald swept home a Tom Kennedy corner.
Rochdale then went close through Ernie Cooksey and even closer when a Barker header hit the crossbar.
However, they fell behind on 28 minutes when, following a foul by Cooksey on Dwayne Mattis, Bishop rolled home a penalty.
An already excellent game moved up into an even higher gear when Rochdale boss Steve Parkin introduced second-half substitutes Clive Moyo-Modise and Morike Sako on the hour.
For the next 30 minutes the home side laid siege to the Bury goal with chances coming and going.
The best of them saw giant Frenchman Sako's header from a Gary Brown cross rattle the crossbar.
Deep into time added on, with the home side committing men forward, Bury broke through Mattis and after Boardman had been adjudged to have fouled him and been dismissed, Bishop calmly rolled home his second spot-kick to wrap up the points.