标题:
基于Spark的流程对象并行数据挖掘的研究与实现Research and Implementation of Parallel Data Mining of Process Object Based on Spark
作者:
郑雅飞, 杜韬, 朱连江, 曲守宁
关键字:
数据挖掘, 并行计算, 流程对象, Spark, MapReduceData Mining, Parallel Computing, Process Object, Spark, MapReduce
期刊名称:
《Hans Journal of Data Mining》, Vol.6 No.4, 2016-10-24
摘要:
本文研究了基于Spark的并行数据挖掘，并将其应用到了流程对象数据分析中。文章通过对串行的流程对象数据挖掘算法流的研究，提出了一种基于Spark并行计算框架的并行化算法流解决方案，并通过编程实现、并行效率测试、算法调优，最终得出一个并行效果良好的并行数据挖掘方案。该并行方案明显提高了计算效率。
In this paper, we study the parallel data mining based on Spark, and apply it to the data analysis of process object. We propose some parallel algorithm flow solutions based on Spark by studying the algorithm flow of stand-alone process object data mining. Through programming, parallel efficiency testing and algorithm tuning, we conclude an optimized parallel algorithm flow. These solutions improve the computational efficiency.