The South Africans were marginally the stronger side in a first half which ended with them 13-11 ahead, but they were a man down for almost the whole of the second half after their captain Marcell Coetzee was sent off for a dangerous tackle on Pierre Schoeman.
Johan Goosen gave the South Africans an eighth-minute lead with a delicate grubber that Devon Williams got on the end of to touch down. Goosen added the conversion.
Ben Healy hit back for Edinburgh with two penalties in quick succession, before Goosen was on target with two awards of his own to make it 13-6 after nearly half an hour.
Then Healy sent a penalty to the corner and the pack drove from the lineout, with Ewan Ashman finishing off the try. The conversion attempt went wide.
The second half had hardly got under way when Coetzee was shown the red card.
Bulls number eight Cameron Hanekom was sent to the sin-bin when Edinburgh attacked right from the restart. That penalty went to touch too, and from the lineout drive Ashman got try number two.
Healy missed the conversion attempt, but at 16-13 Edinburgh were in the lead for the first time.
Then with the Bulls still down to 13 men, Duhan van der Merwe got his teams third try after Mark Bennett and Kinghorn had made the space for the winger.
Inside the final quarter, Bill Mata finished off a powerful drive for the bonus-point try, and Healy - who had missed the Van der Merwe attempt - added the two points.
Elrigh Louw got one back for the Bulls following a tap penalty, and Jaco van der Walt converted.
Mata was yellow-carded for shoulder-to-head contact on Lions centre Stedman Gans, and Van der Walt scored from the penalty to narrow the gap to 28-23.
On-loan Glasgow Warriors scrum-half Ali Price came off the bench in the second half for his Edinburgh debut.
Peterson was yellow-carded in the dying minutes for a deliberate knock-on. Healy's successful penalty ended the game.