Belford taught at the Franklinville and Riverhead Academies on Long Island in New York for a time but then studied law and was admitted to the New York bar in 1889 and began a practice in Riverhead.  He also became active in politics in Suffolk County and served secretary and chairman of the county Republican committee.  This led to his election to the Fifty-fifth Congress from the first district of New York between March 4, 1897 and March 3, 1899.  He was was not a candidate for renomination in 1898 although he was a delegate to the Republican National Convention at Philadelphia in 1900.  He returned to his practice and also became involved in banking.  He served from 1904 to 1910 as surrogate of Suffolk County.

A cousin of James Burns Belford, class of 1859, who also served in Congress, he married Inez Hawkins of Jamesport, New York in December, 1892.  The couple had one son, Donald Hawkins Belford.  Joseph McCrum Belford collapsed and died suddenly in Grand Central Station, New York City on May 3, 1917 and was buried in Riverhead Cemetery.