The winning goal came in the second half, when Wandile Ngema teed up Gumede at the edge of the box. The midfielder made no mistake, rifling a low shot past Darren Keet to seal the points for the home side. Gumede had earlier created danger with a pinpoint cross that nearly led to a headed opener.
Richards Bay made three changes from their previous outing against AmaZulu, bringing in Lwandile Mabuya, Somila Ntsundwana, and Ntlonelo Bomelo.
Bomelo was lively in the opening stages, heading just wide in the first ten minutes and later forcing Keet into a save before half-time.
Cape Town City, looking to bounce back from their defeat to Sekhukhune United, made just one change with Haashim Domingo replacing Amadou Soukouna.
Jaedin Rhodes and Darwin Gonzalez were bright sparks in the first half, and Rhodes came closest to levelling after the break, drawing a flying save from Ian Otieno.
The second half began with an unexpected delay due to a floodlight failure, but play resumed shortly after.
Cape Town City introduced Kayden Francis and Kamohelo Mokotjo in a bid to chase the game, but Richards Bay stood firm.
An injury to Nkosikhona Radebe saw Sanele Zulu come on late in the first half for the hosts, who defended resolutely in the closing stages. Despite extended stoppage time, City couldn't find a way past Otieno.