关于MapReduce程序运行时间的可靠估算

来源 :中国通信 | 被引量 : 0次 | 上传用户:pengwei000
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
As data volume grows, many enterprises are considering using MapReduce for its simplicity. However, how to evaluate the performance improvement before deployment is still an issue. Current researches of MapReduce performance are mainly based on monitoring and simulation, and lack mathematical models. In this paper, we present a simple but powerful performance model for the prediction of the execution time of a MapReduce program with limited resources. We study each component of MapReduce framework, and analyze the relation between the overall performance and the number of mappers and reducers based on our model. Two typical MapReduce programs are evaluated in a small cluster with 13 nodes. Experimental results show that the mathematical performance model can estimate the execution time of MapReduce programs reliably. According to our model, number of mappers and reducers can be tuned to form a better execution pipeline and lead to better performance. The model also points out potential bottlenecks of the framework and future improvement.
其他文献
One of the fundamental design challenges in designing a Wireless Sensor Network (WSN) is to maximize the network lifetime,as each sensor node of the network is
[目的]探索一种避免床单中部过度磨损、延长床单使用寿命的方法.[方法]制作链接式床单给住院病人使用,与传统床单做比较,两种床单各投入40条,外科住院病房开放床位30张,15张
娇态花容百媚生,聪明伶俐兽中精。身藏三窟避狼犬,耳警八方防鹘鹰。丽影常遭宠猫妒,轻功更让狡狐抨。从来不食窝边草,与世无争祈太平。
期刊
在北宋词史上,晏欧齐名,代表着宋初小令的最高成就。在其词境上大多委婉蕴藉,清丽深远,深受冯延巳的影响。同时,他们的词显示出不同的个性特征,在词境上,欧词比晏词进行了更
期刊
Humanoid robots can walk stably on flat ground, regular slopes, and stairs. However, because of their rigid and flat soles, adapting to unknown rough terrains i
The recent research about cavitation jet mainly focuses on the organ-pipe nozzle and triangular nozzle.The research content mainly includes the optimized design
经济学越来越重视对快乐的研究,并已取得了许多有益的理论和技术成果。快乐经济学强调快乐是人类行为的终极目的,在财富达到一定水平后就不会带来快乐的同比增加,而其他因素
当年年少,十三岁,读初一。初冬时节,我和两个同班同学在北京大学的校园里拾到一张纸,上边印有许多感叹号。这种纸当时称“传单”,该单披露重庆发生的某一事件。我看了挺激动,十分向