Horace Mann was the third of five children of Thomas and Rebecca (Stanley) Mann, born in Franklin on May 4, 1796.  Horace had two older brothers, Thomas S. and Stephen Mann and two sisters Rebecca and Lydia.  Thomas Mann Sr. died of tuberculosis, in 1809, at the age of 52, leaving Rebecca a widow with five children.  Just a year later Stephen drowned in a swimming accident at Uncas Pond in the summer of 1810 at the age of 17.  The death of his father and his brother in close proximity left a mark on young Horace who was now just 13 years old.

 
Horace had always worked hard on the farm, leaving little time for a proper education, and now with his father and an older brother gone, the added responsibilities left even less time.  Up to the age of 15 Horace’s only formal education was during the 3 months that school was open, getting no more than eight or ten weeks schooling in any given year. To make up for that, Horace, with a thirst for knowledge, used every spare hour for study by making use of the books made available to the town by Benjamin Franklin.  A collection of 100 volumes was donated as a gift to the town by the great statesman Franklin on the occasion of the town being named in his honor.  
By the age of 18, a teacher by the name of Barrett, saw promise in Horace and urged him to enter college and helped him prepare.  After 6 months of intensive study, his 1816 entrance exams to Brown University placed him as a sophomore in his first year. Three years later he graduated as valedictorian of the class of 1819.

After graduation, Mann taught Latin and Greek at Brown University as well as serving as librarian.  He studied Law at Litchfield Law School in Connecticut.  In 1823 he was admitted to the bar and opened a law office in Dedham.  He based his work upon the principle of not taking any unjust cases, and proceeded to build up a good practice.  In 1827 he was elected to the Massachusetts House of Representatives.  In the fall of 1833 he was chosen as State Senator from Suffolk County.

Having established himself successfully, in 1830 Mann married Charlotte Messer, the daughter of Brown University President, Asa Messer.  There were no children from this marriage and tragically Charlotte Messer Mann died of tuberculosis in 1832 before their second wedding anniversary. The profound sadness and grief he experienced of having his wife die at home in his arms stayed with him for many years.