Montpelier was the lifelong home of James Madison, "Father of the Constitution," sponsor of the Bill of Rights, and fourth president. But Montpelier was also home to three generations of Madisons—from 1723, when Ambrose Madison, the president's grandfather was deeded the land, until 1844 when the widowed Dolley Madison sold the estate. Like so many others who first came to the Piedmont, the Madison family hailed from the long-settled Tidewater of Virginia. John Madison had immigrated to Virginia in 1653. His grandson Ambrose was a well-established, well-connected member of the gentry class. He held several significant public offices, married very well, and owned thousands of acres in both the Tidewater and Piedmont.
     In 1723 Ambrose Madison and brother-in-law, Thomas Chew, patented 4,675 acres in the newly opened Piedmont of Virginia. In order to receive title to a land patent, the owner had three years to make certain improvements to the property, including erecting a house and and clearing land. At the end of three years there would be an inspection and certification by neighbors. When his land was viewed in 1726, the record shows that Ambrose was one of the wealthiest men in the Virginia Piedmont.
     It was not until 1732 that Ambrose and his wife, Frances Taylor Madison, along with their three children, moved to the Montpelier estate, then called Mount Pleasant. Less than six months later, Ambrose was dead. Some time in early summer, probably in June or early July, Ambrose fell ill, apparently the victim of poisoning. As historian Ann L. Miller has written, the poison "did not kill him outright but rather caused sufficient damage to his system to condemn him to slow death over several months." As Ambrose lingered near death, three slaves were arrested and charged with "conspiring" his death. On August 27, Ambrose died—leaving his wife Frances to run the Mount Pleasant plantation. The three slaves were convicted of his murder.
     Frances then ran the plantation with notable success, and continued to co-manage it along with their only son, James Madison, once he came of age in 1741. In 1749 Col. Madison, as he would be known, married Nelly Conway, and in 1751 she gave birth to the first of their 12 children, James Madison, Jr., at her mother's home in Port Conway, Virginia. A few months later young James was brought home to Mount Pleasant, and there he spent the first nine years of his life. It was there that his grandmother Frances began the education of the future president.
     Around 1760, for reasons yet unknown, Col. Madison built a new plantation complex, including a new house, about half a mile southeast of Mount Pleasant. It is possible that James and Nelly, who had been living with his mother, needed more space, since by 1760 the family included four children. When the new house was finished, it was the most elaborate structure in the county. The precise date of construction is unknown, but in writing his memoirs, James, Jr., recollected that he helped move lighter pieces of furniture to the new house when he was nine years old.
     Col. Madison was a man of great talent and enterprise. He not only ran a prosperous farm, but also established several businesses, including a distillery, a contracting business and an ironworks. He, like his father, was a prominent public figure, serving as a vestryman, a justice of the Orange County Court, a road surveyor, and a colonel in the militia.
     Education and books clearly played an important role in the Madison family. When Ambrose died, his 1733 inventory listed 18 books: a "Great Bible," five prayer books and twelve "other books." This is an unusual number for a "frontier" plantation. As witnessed by tutoring her grandson, Frances Taylor Madison was also educated. Later, when her son was acting as a merchant, he ordered books for her including several volumes of the London magazines, the Tatler and the Spectator, religious books such as Stanhope's Epistles and Brevint on Sacrament, and a medical book, Quincey's Dispensatory.
     Madison, Sr., established a library, most likely founded on his father's and later augmented by his mother's. By the time of his death in 1801, the library numbered 91 volumes and sets. Although the lack of titles in the inventory makes it difficult to tell, it is likely that the "1 Large old Bible" and "3 Common Prayer" books were Ambrose's. In the case of his mother, the aforementioned Brevint and Quincey appear in James's inventory. His collection covered a range of topics including law, medicine, history, theology, agriculture, and miscellaneous works such as "Athenian Sports" and "Clark's Farriery."
     It wasn't until 1801 at age 50 that James Madison, Jr., inherited Montpelier, and it wasn't until 1817 that he and Dolley returned there full time. Despite this long delay in inheriting, there was apparently no friction between him and his father and indeed surviving correspondence implies a cordial and respectful relationship.
     Less is known about how he related to his mother, who lived until 1829 (age 98), but again, there is no hint of friction between them. She always lived at Montpelier and in later years kept her own household, including her own staff, kitchen, and garden.
SOURCE: Official Montpelier website, online @ http://www.montpelier.org/explore/community/madisons.php