The 23-year-old arrives after making 95 competitive appearances for Karlsruher, where he established himself as a consistent option on the left side of defence.
Herold began his development in the academy system of FC Bayern Munich before continuing his professional career elsewhere in Germany.
"David is a young German player who has developed well, but who is also hungry and eager to improve. He still has further potential and sees Borussia as his perfect next step," said Gladbach sporting director Rouven Schroder.
"He stands out through his high work rate and attacking drive, as well as his very open personality."
Monchengladbach's move adds further depth to their back line ahead of the new campaign, with the club securing a player who already has significant senior experience despite his relatively young age.
"The move to Borussia means a lot to me," said Herold.
"It's my first step into the Bundesliga - which fulfils a dream I have had since I was a small child - and to join such a big traditional club as Borussia Monchengladbach is a huge step for me.
"I am especially looking forward to the Borussia fans - I already experienced in the DFB-Pokal last season what a force this stadium can be."