In order to solve the problem of soil-lack and water-shortage in ecological restoration of coal gangue hill and subsidence land in Lu'an mining area, in this study, the modified soil substrates were prepared by orthogonal mixing of 5%, 10% and 20% (volume %) of fly ash, sludge and waste compost, their effects on soil physiochemical properties and nutrient conditions, heavy metal accumulation, and the growth of Festuca elata and Berberis thunbergii were studied, and then the optimal formula of soil substrate was determined by principal component - cluster analyses. The results showed that the addition of waste compost not only improved available nutrient and organic matter contents, but also promoted soil physiochemical properties. Adding sludge only increased available P content. Adding fly ash reduced bulk density, increased total porosity and non-capillary porosity significantly, but deteriorated pH, CEC and available nitrogen. The available N, P, K and organic matter contents of all soil substrates were higher, while bulk density, non-capillary pore, pH, EC and CEC and other indicators of substrates of fly ash∶sludge∶waste compost=5%∶20%∶20%∶55%、10%∶10%∶20%∶60%、20%∶5%∶20%∶55% were better. The contents of heavy metals in all soil substrates were lower than those of the "Greening Planting Soil Standard" (CJ/T340—2016). By comprehensive consideration, substrate L3 has the best effects on soil physiochemical properties, fertility and plant growth, thus, the optimal formula of soil substrate is fly ash∶sludge∶waste compost∶soil = 5%∶20%∶20%∶55%.