Oliver attended Northside High School. He was a three year starter and team captain. He played football and basketball.[2] Oliver was rated as a three-star prospect by 247Sports.com, Rivals.com and Scout.com.[3][1] He helped lead his team to a state championship as a sophomore in 2014 and an appearance in the state semifinals as a senior in 2016.[1] Oliver amassed 3,785 passing yards and 3,218 rushing yards during his time at Northside.[1][4] While at Northside, he was rivals with Jake Fromm, who played at Houston County High School.[5][6]
During his redshirt freshman season in 2018, Oliver appeared in 12 games. On October 25 at Virginia Tech, Oliver started his first game in a 49–28 victory for Georgia Tech.[7][8] Oliver's next start was on August 29 against No. 1 Clemson.[9][10]