基于遗传模拟退火算法的Hadoop系统性能配置优化

Performance Configuration Optimization of Hadoop System Based on Genetic Simulated Annealing Algorithm

摘要: 为提高Hadoop在开源云计算平台上的性能，提出了基于遗传模拟退火算法的Hadoop系统性能配置优化方法。基于遗传算法，将配置方案作为染色体进行选择，交叉和变异。结合模拟退火原理，控制新染色体的存活率和整个算法的迭代次数，找到系统配置的最优方案。根据遗传模拟退火算法得到的整体性能较好，在长期优化中优化速度更快，可以用来解决在全局空间内通过随机搜索找出系统的近似最优分配方案的问题。实验结果表明，该方法可以有效提高寻找最优配置的效率。提出的配置方法提高了操作运行速度，充分利用了资源，增加了系统的吞吐量。

Abstract: In order to improve the performance of Hadoop on open-source cloud computing platform, a per-formance configuration optimization method based on genetic simulated annealing algorithm is proposed. Based on genetic algorithm, the configuration scheme is selected, crossed and mutated as chromosome. Combined with the principle of simulated annealing, the survival rate of new chromosome and the number of iterations of the whole algorithm are controlled to find the optimal scheme of system configuration. According to the genetic simulated annealing algorithm, the overall performance is better, and the optimization speed is faster in the long-term optimization. It can be used to solve the problem of finding the approximate optimal allocation scheme of the system through random search in the global space. Experimental results show that this method can effectively improve the efficiency of finding the optimal configuration. The proposed configuration method improves the operation speed, makes full use of resources and increases the throughput of the system.