Abstract: The traditional Linux kernel protocol stack can no longer meet the higher performance requirements of large-scale data processing systems for network transmission. A lot of researches have been done to move the protocols and interfaces from the kernel state to the user state, but there are few studies on kernel state optimization. Based on the analysis of the Linux kernel state file sending interface sendfile( ) processing flow and management mechanism, this paper proposes a kernel state file sending optimization method, using automatic load balancing fixed-length memory pool management, CPU affinity technologies, the kernel state interface sendfile( ) has been optimized and modified. It solves the problems of memory fragmentation, memory exhaust and CPU jitter under high load conditions, and improves data transmission performance effectively. Experimental results show that in the high concurrent and high throughput scenario, the system using this optimization method runs more stably, and the kernel state CPU occupancy rate drops by 50%.

Key words: large-scale data processing, user state protocol stack； load balance； memory pool management； CPU affinity