Former Rep. Nathan Deal (R-Ga.) held a 2,431 vote lead over Karen Handel, the former secretary of state, with 99 percent of precincts reporting.
The race is too close to call for most news outlets, and an automatic recount is looming because less than 1 percent of the vote separates the two candidates.
Neither candidate declared victory late Tuesday night, according to the Atlanta Journal-Constitution. Handel said there are still "a lot of votes to be counted out there," referring to outstanding absentee ballots.
"Keep the faith," she told supporters, according to the paper.
There are also outstanding votes in Fulton and Gwinnett counties, the state's largest.
The winner will face former Gov. Roy Barnes (D) in November.
Handel came in first in the July 20 GOP primary but her 33 percent was not enough to win the nomination outright. Deal took the second spot with 24 percent. The two advanced to Tuesday's runoff.
Democrats were pleased with the deadlock.
"This is the best possible outcome for Democrats," said Nathan Daschle, who heads the Democratic Governors Association. "Even after a vicious runoff, Georgia Republicans couldn’t decide between two bad options: a corrupt Congressman or a Capitol insider."
The race turned into a battleground for potential 2012 GOP presidential candidates as Newt Gingrich and Mike Huckabee lined up for Deal while Sarah Palin and Mitt Romney supported Handel.
Palin characterized the race in grand terms during her rally with Handel on Monday.
"It's epic. It’s historic. The eyes of the nation are on you, Georgia, to see if you get rid of that good old boy network. People are watching what’s going on in Georgia," she said, according to reports.
During the runoff, Handel kept up a barrage against Deal.
She sought to capitalize on news stories that stated her rival was the target of a federal grand jury investigation.
Deal denied the claim, but Handel went on air with a TV ad that hit the former congressman for being a "corrupt relic of Washington, D.C."
Handel defended the ads in a televised debate against Deal on Aug. 1.
"I stand by these ads, everything in them, [as] 100 percent accurate," she said.
She called it "disingenuous" for Deal to be "squealing" about negative campaigning.
"Facts are facts," she said. "This is a race for governor. Things are tough. Campaigns are tough. It's frankly time to put the big-boy pants on because, candidly, if you can’t handle this, how are you going to handle [Democratic nominee] Roy Barnes?"
In response, Deal said that people were "repulsed" by Handel's tactics.
—Updated at 12:34 a.m.