Dale lie 14th in npower League Two, six points shy of the play-off places following their relegation last season, but have endured a dismal run in recent months.
After being dumped out of the FA Cup by Morecambe in a first-round replay, Coleman has overseen a run of eight defeats in 11 league matches.
Defensive problems appear to lie at the root of Rochdale's troubles, with their last clean sheet coming in a goalless draw with Fleetwood on October 27.
Saturday's match with Gillingham was postponed due to a frozen pitch, but Coleman will not be afforded the chance to avoid a fourth successive loss after chairman Chris Dunphy and the Dale board stepped in.
A statement on Rochdale's official website read: "Chairman Christopher Dunphy stated the regrettable decision has been made following the club's recent run of poor form.
"This decision was not taken lightly and was made following extensive discussions at board level.
"The club would like to thank both John and Jimmy for all their hard work and endeavours over the last 12 months and would like to wish them well for the future.
"The club hope to name their replacement before we travel to Cheltenham for our next league match on Friday evening."
Coleman ended a 12-and-a-half year tenure at Accrington when he and Bell moved to Spotland to succeed Steve Eyre a year ago this week.