Leslie James Downing was born on March 8, 1925 as the son of Charles Clayton and Camilla (Wilson) Downing.

He married Fay Elizabeth Dennison and they had five children:

1. Linda Lee (Downing) Cowan

2. Nancy Lou Downing

3. Diane Susan (Downing) Schroeder

4. Robert Leslie Downing

5. Anita Ann (Downing) Lister