Ernest H. Oliver was born on January 7, 1870 in Ridgeway Township, Lenawee County, Michigan. He was the son of George L. and Mary (Wyrill) Oliver.

He was married to Mary Curry and they had eleven children:

1. Alethea Mae "Leta" (Oliver) Chase (1894-1975)

2. Nina (Oliver) Taylor (1895-1976)

3. Ralph Oliver (1897-1945)

4. Mary Oliver (1899-1980)

5. Mabel (Oliver) Nelson (1901-1978)

6. Gladys (Oliver) Philipp (1903-1982)

7. Raynor George "Tip" Oliver (1905-1979)

8. George Lavern Oliver (1907-1993)

9. Ernest Wilford Oliver (1911-1944)

10. Clyde Wayne Oliver (1913-1995)

11. Sheldon James Oliver (1916-1917)

Ernest died on October 16, 1955 at the age of 85. He is buried at the Ridgeway Cemetery in Ridgeway Township. Mary, who was born on September 24, 1875 in Lenawee County passed away on January 6, 1945 at the age of 69 and is also buried in Ridgeway Cemetery.