In 1957, he played at the Grand Ole Opry, where he met Flatt abd Scruggs. He soon became a permanent member of their Foggy Mountain Boys, initially playing bass, but soon changed to the dobro. After the breakup of of the group he joined Flatt's Nashville Grass and continued sessions work
on the side. In 1971, he played with the Earl Scruggs Review: three years later, he decided to go solo. His first solo effort was "Alone at Last" on Epic CMH Records release a musical documentary about Josh entitled."Josh Graves the King of the Dobro.

The survival of the dobro as an important instrument in country music can largely be attributed to Graves. His banjo-style finger rolls and bluesy lead has made the dobro guitar an essential bluegrass instrument. The Josh Graves signature model dobro is a recreation of his 1928 Model 37, with vintage features such as a solid headstock and a 25-inch scale, which is one half-inch longer than standard.