The Wallabies played for more than half an hour with 14 men after Miles Amatosero was sent off, yet they held their nerve in the closing stages to secure the win in temperatures close to 30 degrees.
Amatosero was initially shown a yellow card in the 32nd minute after making dangerous contact with Warner Dearns' head during a cleanout near Japan's try-line. The decision was upgraded to a red card before half-time, with the teams level at 14-14.
Japan had made a strong start, with fly-half Ryunosuke Ito producing an excellent individual effort to open the scoring.
Australia responded when Harry Potter broke through after Hunter Paisami intercepted a loose pass. The Wallabies moved the ball quickly across the field before Potter cut inside to score.
Tom Wright then put Australia ahead after another sharp attacking move created space on the wing, giving the visitors a 14-7 lead.
Japan took advantage of Australia's numerical disadvantage before the break, with number eight Faulua Makisi crossing to bring the hosts level.
Despite being down to 14 players, Australia regained control after half-time. Fraser McReight and Max Jorgensen both crossed the line as the Wallabies opened up a 26-17 advantage.
Japan refused to go away, with Takuro Matsunaga kicking two penalties to bring the hosts back to within three points.
The Wallabies' replacements then played a crucial role in the final stages, helping Australia manage the pace of the game and regain territory.
Taniela Tupou provided the decisive moment with 10 minutes remaining when the powerful prop crashed over from close range to extend Australia's lead to 35-25.
Japan responded through captain Warner Dearns to set up a tense finish.
The hosts had one final opportunity to snatch victory, but Tupou made another important contribution by helping force the ball loose as Japan attacked.
Australia recovered possession and ran down the clock to secure a memorable opening victory for Kiss.