Abstract:  In order to promote the rapid fusion and application of big data technology in oilfield, a multi-functional big data processing platform covering the whole life cycle of big data processing is proposed. The platform combines various big data analysis frameworks and machine learning frameworks to design data mining functions that can support real-time and off-line processing in the oilfield. Based on Docker containers, it encapsulates all kinds of computing frameworks and algorithms, and  based on Kubernetes framework, it completes container arrangement and scheduling. In order to ensure the reliability and extensibility of the business system services, the system adopts the micro-service-based architecture, which decomposes the application of different technology stacks into a single service module independently. This allows enterprise data analysts to focus on business data analysis issues without spending a lot of time learning the details of framework deployment and other big data mining technologies.

Key words: micro-services, big data mining, Docker