Parliament speaker Norbert Lambert said the 72-year-old Gauck received 991 votes out of 1,232 ballots cast during a special assembly of national and regional lawmakers on Sunday.
His rival, 73-year-old former Nazi hunter Beate Klarsfeld, received 126 votes.
Gauck is a former pastor who was a vocal opponent of East Germany's communist regime. He later went on to head a federal agency that oversaw the files of the Communist dreaded police force, the Stasi.
His appointment to the largely ceremonial position follows the abrupt resignation of his predecessor Christian Wulff over a corruption scandal.
Gauck, who is not affiliated with any political party, received the backing of Chancellor Angela Merkel.
A recent opinion poll shows that he has the trust of 80 percent of Germany's citizens.
