Rutledge was established in 1797 and named for General George Rutledge, a prominent citizen in nearby Sullivan County. In 1801, Rutledge became the official county seat of Grainger County, a distinction that had been shared by several communities since the county's formation in 1796. Rutledge prospered in the early 19th century in part due to its situation along the Federal Road (now US-11W), which connected New Orleans and Virginia. The road intersected the Old Kentucky Road (now US-25E) at Bean Station, just east of Rutledge.
In the 1820s, future president Andrew Johnson, who worked as a tailor in Greeneville, briefly operated a tailor shop in Rutledge. In 1976, a replica of Johnson's tailor shop was erected on the Grainger County Courthouse lawn.
During the U.S. Civil War, guerilla warfare largely paralyzed Grainger County. In December 1863, Confederate General James Longstreet, who had earlier failed to wrest Knoxville from Union forces, passed through Rutledge en route to winter quarters at Russellville. On December 14, in what became known as the Battle of Bean's Station, Longstreet attacked a Union detachment that had been pursuing him. The Confederates failed to exploit the element of surprise, and the Union forces were able to hold out until reinforcements arrived. While Longstreet was victorious, Union forces were able to retreat to fortifications at Blaine, and Longstreet subsequently abandoned the assault and continued eastward to Russellville.