Proceedings of UNISCON 2008, The 2nd International United Information Systems Conference
An investigation into improving the load balance for term-based partitioning
作者:
A. Abusukhon， M. Talib and M. Oakes
关键词:
Term-partitioning schemes ；Term-frequency partitioning；Term-length partitioning；Node utilization
摘要:
In Parallel (IR) systems the query response time is limited by the time of the slowest node in the system, thus distributing the load equally across the nodes is very important issue. In this paper, we propose improving the load balance for term-based partitioning by classifying the terms based on their length then distribute them equally across nodes. The motivation for term length partitioning comes from the observation that the Excite-97 queries have a very skewed distribution of term lengths with some predominant lengths. We also propose the term-frequency partitioning scheme in which the terms are classified based on the total term frequency (F) and then distribute equally across the nodes.
在线下载