Leroy Kelly's first head coach was Blanton Collier. He began his NFL coaching career as an assistant coach under Paul Brown in the Browns first year of existence in 1946. Brown assigned Collier the task of teaching the Browns' greatest quarterback, Hall of Famer Otto Graham, the T-formation. Graham credited Collier with teaching him "everything I know". He remained with the Browns for eight years before leaving to replace Paul "Bear" Bryant at the University of Kentucky in 1954. Collier returned to the Browns as an assistant coach under Paul Brown for the 1962 season. After Art Modell fired Brown at the end of that season Blanton Collier took over as head coach. The Browns players responded positively to this change as Collier was more easygoing than his predecessor. Jim Brown in particular reacted to the new attitude on the team by rushing for the most yards of his career in the 1963 season.

In 1964 Collier drafted two players who would make an immediate and lasting impact for the Cleveland Browns. He picked Paul Warfield of Ohio State in the first round and Leroy Kelly of Morgan State in the eighth round. Warfield's role as a wide receiver and Kelly's talent as a punt and kick returner helped the Browns to win the 1964 NFL championship. Collier was impressed with Kelly's all-around abilities as a special teams player saying, “He immediately showed me that he was a very tough tackler. I knew that he would be a good man for the specialty teams.”

Collier talked about Leroy Kelly after Jim Brown's retirement, “I don't expect anyone to replace Jim. Runners like him come along only once in a lifetime. I do expect someone from this squad to make a name for himself and Leroy is going to get his chance. It's impossible to compare them. Jim played at 222 to 228 and was a fullback type of power runner. Leroy is 200 pounds and a halfback type of runner. Leroy did a lot of things Jim was never asked to do. He played on our special teams. He is a fine team man -- good tackler, blocker and pass receiver. Jim Brown used to say that anyone could run outside, but it took a great one to run inside. Kelly can do that because he has good speed, balance and is quick to sense interior openings and get through.”

Blanton Collier's record as head coach of the Browns was 76 wins, 34 losses, and 2 ties. From 1964-69 Collier coached the team to five division titles and four championship games winning one of them. He retired after the 1970 season because of hearing problems. Dick Schafrath, the Browns All-Pro Tackle, said concerning Collier, “[He] instilled more attitude, togetherness, motivation and confidence into the Browns than anyone can imagine by being totally involved with us. We hungered for [him] to teach us more football each day. None of us ever suspected selfishness or greed on [his] part. The Blanton Collier era I will recall as the best years of my entire football life. In closing, I recall the words that the late Brian Piccolo quoted on his deathbed while talking to Gayle Sayers. He said 'I love you, Gayle Sayers'. Those are my feelings for you.” Blanton Collier died on March 22, 1983.