In this study, a 5-year field long-term experiment was conducted on typical black soil to compare the effects of maize monoculture (CK), sweet clover (Melilotus officinalis L.)-maize rotation (F1), and crimson clover (Trifolium incarnatum L.)-maize rotation (F2) on soil organic carbon (SOC) (0-20 cm) and its distribution across different size aggregates, and the microbial mechanisms were also explored behind these effects were by analyzing alterations in the composition of bacteria, fungi, and arbuscular mycorrhizal fungi. The results showed that SOC enhancement effects of F1 and F2 were similar, with an average significant increase of approximately 28.9%. SOC enhancement under F1 was primarily driven by the recruitment of Glomerales, Glomeromycota and Glomeromycetes, which contributed to SOC sequestration in > 2 mm and 0.25-2 mm aggregates, accounting for over 63.6% of total SOC. Conversely, SOC enhancement under F2 was mainly achieved by recruiting Glomerales to drive SOC sequestration in 0.25-2 mm aggregates, while inhibiting Glomeromycota to enhance carbon sequestration of < 0.053 mm aggregates, which constituted more than 62.5% of total SOC. The increase in carbon sequestration in each size aggregates mentioned above was achieved by simultaneously increasing the proportion of aggregate mass and organic carbon content of aggregates, leading to stronger soil structural stability under F1 compared to F2, which was primarily manifested by the higher MWD and GMD of F1 compared to F2. The results revealed the significance of arbuscular mycorrhizal fungi in enhancing SOC within crop rotation systems involving cover crops, and indicated that different cover crops employ distinct microbial mechanisms to increase SOC.