James B. Frazier, Jr. was born on June 23, 1890 in Chattanooga, Tennessee. In 1914, he graduated from the Chattanooga College of Law and was admitted to the Tennessee bar. He volunteered for service in World War One in 1917 and was discharged at the rank of Major in 1919. After the war he returned to Chattanooga and resumed his legal career.
Frazier was appointed as the United States Attorney for the Eastern District of Tennessee in 1933. He served as the U.S. Attorney until 1948. In 1949, he was elected to the United States House of Representatives where he served until 1963. Frazier died in Chattanooga on October 30, 1978.