Joseph Oliver was born in 1852 as the child of Simon, Sr. and Mary (Wright) Oliver

In the 1860 census, he was living with his parents in Tecumseh, Lenawee County, Michigan.