Forest took control of the tie thanks to a second-half penalty from Chris Wood after Villa defender Lucas Digne handled the ball inside the area.
The New Zealand striker confidently fired his effort into the top corner to hand the hosts an important advantage before next week's return leg at Villa Park.
But the main talking point for Emery was a first-half challenge from Forest midfielder Elliot Anderson on Ollie Watkins that escaped punishment from VAR.
The Villa manager was furious after the match and believed Anderson should have been sent off.
"The VAR is so bad. It's a clear red card and I don't understand why they did not call the referee to review it," Emery said.
"The referee handled the match very well, but this decision was a huge mistake. He could have broken Ollie's ankle. It is not fair."
Despite the defeat, Emery remained confident Villa could still turn the tie around in Birmingham.
"The match is not finished," he said. "We must come back next week and keep believing."
Villa had started brightly, with Youri Tielemans and Morgan Rogers forcing saves from Forest goalkeeper Stefan Ortega, while Emiliano MartÃnez also produced an important stop to deny Igor Jesus from close range.
After the break, Forest tightened control of the contest and were rewarded when Digne's handball led to Wood's winning penalty.
Forest manager Vitor Pereira praised his side's discipline and fighting spirit after securing the club's first European semi-final victory in decades.
"We kept our mentality and our spirit," Pereira said. "The supporters helped us a lot and tactically we did exactly what we needed to do."
Pereira, Forest's fourth manager this season, also spoke proudly about guiding the club back onto the European stage.
"It is an honour to manage this club and make these supporters smile," he said. "Now we know the second leg will be a big battle."
Forest will head to Villa Park on May 7 looking to reach their first European final since winning the European Cup in 1980.