Conservation tillage is an important measure to optimize soil structure and microbial diversity, which is of great importance for soil fertility and productivity. This study based on the positional experiment of wheat-maize annual conservation tillage in fluvo-aquic soil area of the Huang-Huai-Hai Plain, soil physiochemical properties, microbial diversity and community composition were analyzed after 16 years. The results showed that: 1) Compared with continuous tillage, reduced/no-tillage significantly reduced aeration pore and water holding pore and the mass proportion of microaggregates, and significantly increased soil bulk density, the mass proportion of macroaggregates and total organic carbon (TOC), liable organic carbon (LOC), total nitrogen (TN) and alkali-hydrolyzed nitrogen (AN). In contrast, compared with straw removal, straw returning significantly reduced soil bulk density and mass proportion of microaggregates, and significantly increased soil aeration pore, water holding pore, mass proportion of macroaggregates, TOC, LOC, TN and AN. 2) The responses of bacterial and fungal community diversities and compositions to tillage and straw managements were significantly different. Tillage significantly changed bacterial community diversity and richness, while straw management had significant effects on both bacterial and fungal community diversities and richness. Compared with the conventional treatment (continuous tillage with straw removing), reduced/no-tillage with straw returning decreased the Shannon and Chao1 indices of bacteria in general, increased Shannon and Chao1 indices of fungi significantly. At phylum level, reduced/no-tillage and straw returning both had significant effects on the community compositions of bacteria and fungi. At genus level, species differences among different treatments were mainly between straw managements. 3) Annual conservation tillage significantly affected the diversity and richness of bacterial community through changing soil chemical properties (TOC, LOC, TN and AN), while changes in fungal community diversity and richness were mainly driven by soil physical properties (bulk density, aeration pore and water holding pore). In conclusion, annual conservation tillage in wheat-maize system can affect the diversity of microbial community in different degrees by changing physiochemical characteristics of fluvo-aquic soil. This study can provide theoretical guidance and technical support for cultivating soil biological fertility in the Huang-Huai-Hai Plain.