Union started brightly and controlled possession during the opening stages, but Schalke gradually found their rhythm.
Kenan Karaman went close with a long-range effort before Adil Aouchiche tested Union goalkeeper Frederik Ronnow with a curling shot.
The visitors took the lead after 24 minutes when a poor clearance from Leopold Querfeld fell to Robin Gosens. The former Union player reacted quickly, smashing a half-volley from a tight angle against the inside of the post and into the net.
Gosens immediately apologised to the home supporters, but Schalke continued to threaten. Aouchiche had another chance before half-time after Karaman played him through, although Ronnow got a hand to his low effort.
Union coach Mauro Lustrinelli responded by making two changes at the break, but Schalke struck again almost immediately.
Aouchiche combined with Junior Adamu before finishing past Ronnow. The goal was initially ruled out for offside, but VAR overturned the decision to give Schalke a 2-0 advantage.
A lengthy delay caused by an issue with an official's headset then interrupted the match as Union attempted to fight their way back. The hosts continued to enjoy possession after the restart and Loris Karius was forced into a good save to deny Tom Rothe.
Union pushed forward in search of a way back into the contest and came close when substitute Marin Ljubicic created space before seeing his effort hit the outside of the post.
Tim Skarke finally gave the hosts hope by scoring from close range in stoppage time, but Schalke responded with the decisive goal moments later.
Maximilian Wober finished off a superb counter-attack to make it 3-1 and secure a memorable victory for Schalke.
The result lifts Schalke to seventh place at this early stage of the campaign, while Union remain without a league win under Lustrinelli.