Abstract: Hand gesture segmentation based on fully convolutional networks excessively dependents on the accurate per-pixel annotations of training data. At the same time, the features lack enough context information, which often leads to misclassification with intra-class inconsistency. In order to solve the above issues, a hand gesture segmentation method based on style transfer is proposed. Firstly, the first five layers of hand gesture segmentation network in HGR-Net are selected as the backbone network, and the context information enhancement layer is added to each layer of backbone network. In the context information enhancement layer, global average pool operation and channel attention mechanism are adopted to enhance the weight of the discrimination feature and ensure the continuity of context information in features, so as to solve the intra-class inconsistency. Secondly, in order to improve the generalization ability of the hand gesture segmentation module proposed by this paper, and address the cross-domain samples segmentation problem, a domain adaptive method based on style transfer is proposed. The pre-trained VGG model is used to transfer the source domain testing sample, so as to make the source domain testing sample have both its content and the style of the target domain training sample. Testing on the OUHANDS dataset, the mIoU and MPA values of the proposed method are 0.9143 and 0.9363 respectively, and they are 3.2 and 1.8 percentage points higher than those of HGR-Net. Testing on the self-collection dataset with the style transfer method, the mIoU and MPA values are respectively 19 and 23 percentage points higher than without this method. The domain adaptive method based on style transfer provides a new idea for cross-domain segmentation of unlabeled samples.

Key words: hand gesture segmentation, HGR-Net, context, style transfer