In view of the increasing soil metal contamination in China, it is urgent to develop fast and accurate analytical method for heavy metal detection. In this paper, portable X-ray fluorescence (PXRF) was employed for determination of As, Pb, Cu and Zn concentrations in agricultural soils around a mining area in Huize, Yunnan Province. The results indicated that: 1) The accuracy and precision of PXRF were able to meet with the national procedural regulations regarding the environment quality monitoring of soil. 2) Concentrations of heavy metals in soils measured by PXRF were consistent with those measured by the conventional laboratory method and the coefficients of Pb, Cu, and Zn concentration measured by PXRF and laboratory analysis were generally above 0.70. 3) Spatial distribution patterns of As, Pb, Cu and Zn in soils in study area were similar based on the detection results by in situ PXRF, ex situ PXRF and conventional laboratory method. In general, PXRF is a valuable tool for rapid detection and identification of “hotspots” of As, Pb, Cu, and Zn pollution in agricultural soils around mining areas.