Improving soil fertility of abandoned farmland is important for the efficient utilization of land resources. A pot experiment was conducted using soil collected from the abandoned farmland to compare the effects of planting and incorporating Vicia villosa L. and Astragalus sinicus L. on following rice growth with the blank control(non-green manure planting treatment). The results showed that planting and incorporating Vicia villosa L. significantly increased the subsequent rice biomass, comparing with the control, the dry weights of rice grown after Vicia villosa L. and Astragalus sinicus L. were increased by 53% and decreased by 38%, respectively. Compared with the control, soil organic matter content significantly increased by 21%, and soil available phosphorus content significantly decreased by 43% after planting Vicia villosa L.; soil pH and total nitrogen content increased by 5% and 11% after incorporating Vicia villosa L., respectively; and after following rice growth, soil organic matter content significantly increased by 26%. In contrast, no significant difference was observed in different soil fertility indicators at different stages after planting and incorporating Astragalus sinicus L.. A significant positive correlation was observed between rice dry mass and soil organic matter content, while a significant negative correlation was observed between rice dry mass and soil available phosphorus content. Independent with sampling time, soil organic matter content and available phosphorus content contributed mostly to rice dry weight. Moreover, the activities of soil β-glucosidase, β-cellulase, and phosphatase after planting and incorporating Vicia villosa L. increased significantly. With a random forest model analysis, carbon-converting enzymes and phosphatases were found to have the greatest impact on rice dry mass. In summary, the planting and incorporating of Vicia villosa L. in abandoned farmland results in enhanced enzymes activities related to soil carbon and phosphorus transformation, which further increases soil organic matter and available phosphorus content, and ultimately facilitate nutrient absorption and biomass increase in subsequent rice crops.