An all-around athlete for the Red Elephants, Pilgrim played in the 1952 Class A-AA Georgia High School Football All-Star Game before moving on to Georgia, where he lettered in football and track at Georgia in the mid-1950s. He played end and punted for legendary football coach Wally Butts and ran the 100-yard dash and the 440-yard dash in track.

"He loved The University of Georgia and everything about it," said his wife, Elizabeth Pilgrim.

Added his son, Harold "Trip" Pilgrim III: "A lot of what he did there shaped him. He had fun going back there, too. Two or three years ago, they had a letterman’s day, and dad went back and got to go on the field to be recognized."

After he finished playing, Harold Pilgrim kept his hand in football as a graduate assistant for Georgia in 1957. He helped coach the freshman team, which was quarterbacked by future NFL Hall of Famer Fran Tarkenton.

"Dad said coaching that team was the most fun he ever had in football," Trip said.

Pilgrim and his and his family moved to Decatur, Ala., in 1967 when he went to work running a savings and loan bank, and they stayed there ever since. After getting out of banking in 1978, he opened The Light House, a lighting fixture company in Decatur.