The pollution and risk of Shimen Realgar Mine are one of the most important environmental problems, which have received much attention from environmental groups and the local government. The pollution status and spatial distributions of heavy metal pollution in agricultural soils around Shimen Arsenic (As) Mine were analyzed. Moreover, the USEPA soil health risk models were adopted to assess human health risks. The assessment results of soil heavy metals pollution and comprehensive pollution index showed that the soils were moderately contaminated by As and Cd. The data showed that the average contents of As and Cd were 80.26 mg/kg and 0.55 mg/kg, respectively, Cu, Zn, and Pb contents were below the soil environmental quality standard of secondary state. The assessment result by USEPA indicated that hand-mouth intake were the dominate pathway for personal daily exposure and non-carcinogenic risks. The value of Cd would not cause any non-carcinogenic and total carcinogenic risks for both adults and children while the total non-carcinogenic index of As for children (3.36) indicated a significant health threat. Cd would not cause carcinogenic risk of inhalation intake for adults and children while the average carcinogenic risk indexes of As for adults and children were 3.13×10^-4, and 5.58×10^-4, respectively, which would pose carcinogenic risk. Children were more vulnerable than adults facing As and Cd threats. Thus, the management and prevention for As and Cd contaminated soil around the mining area should be strengthened.