Richmond was founded in 1798 by Colonel John Miller, a Revolutionary War soldier. In 1798, the Kentucky Legislature authorized the moving of the Madison County seat from nearby Milford to land owned by Colonel Miller.

The move was bitterly opposed by about 300 residents of the Milford area and a fight to retain the seat resulted between Dave Kennedy, who represented the Milford location, and William Kearly, who represented the new site.

Although history does not record the results of the brawl, the county approved the move in March of 1798 and two acres of land were purchased from Colonel Miller. Court was held in his barn.

On July 4, 1798, the site was officially named Richmond in honor of Miller's birthplace, Richmond, Virginia.

Richmond takes pride in being the home place of a "Prime Minister" to Russia and five Governors: William J. Stone, Governor of Missouri; James B. McCreary, twice Governor of Kentucky; David R. Francis, Governor of Missouri; Green Clay Smith, Governor of Montana Territory; Keen Johnson, Governor of Kentucky; and Cassius Marcellus Clay, "Prime Minister" during 1861, 1863, and 1869