Mining and Harvesting High Quality Topical Resources from the Web

来源 :Chinese Journal of Electronics | 被引量 : 0次 | 上传用户:liongliong525
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
Focused crawlers aim to effectively prioritize uncrawled URLs to harvest relevant pages while avoiding irrelevant ones. In practice, harvesting high quality topical Web resources is more important due to the explosion of Web information. Our study shows that the popular focused crawling strategy cannot achieve this goal.In this paper we develop a new focused crawler, namely On-line topical quality estimation(OTQE), which intelligently evaluates the topical quality of uncrawled pages by the observed link and content evidences and prioritize their URLs accordingly. The new crawler is scalable and requires fewer additional resources to do link-based analysis. The experimental results on crawling 3.6 million Web pages demonstrate the advantages of our proposed method over traditional focused crawlers. Focused crawlers aim to effectively prioritize uncrawled URLs to harvest relevant pages while avoiding irrelevant ones. In practice, harvesting high quality topical Web resources is more important due to the explosion of Web information. Our study shows that popularly crawling strategy can not achieve this goal .In this paper we develop a new focused crawler, namely On-line topical quality estimation (OTQE), which intelligently evaluates the topical quality of uncrawled pages by the observed link and content evidences and prioritize their URLs accordingly. The new crawler is scalable and requires less additional resources to do link-based analysis. The experimental results on crawling 3.6 million Web pages demonstrates the advantages of our proposed method over traditional focused crawlers.
其他文献
In order to further analyze the dynamical behavior of co-evolutionary populations in minimum attribute reduction, an Enhanced attribute co-evolutionary game red
期刊
该文从挂篮荷载计算、施工流程、支座及临时固结施工、挂篮安装及试验、合拢段施工、模板制作安装、钢筋安装、混凝土的浇筑及养生、测量监控等方面人手,介绍了S226海滨大桥
该文从挂篮荷载计算、施工流程、支座及临时固结施工、挂篮安装及试验、合拢段施工、模板制作安装、钢筋安装、混凝土的浇筑及养生、测量监控等方面人手,介绍了S226海滨大桥
As one of the essential information, passenger Origin-destination(OD) matrix is crucial for transit system planning. Automated fare collection(AFC) system is wi
从古至今,人类遭遇了非典、鼠疫、天花、流感等无数次的瘟疫侵袭,其中大部分瘟疫尽管流行时态势猛烈,一旦找到合适的防治措施后,就变得“一蹶不振”,比如已被人类彻底灭绝的
期刊
低密度三聚磷酸钠的生产工艺分为两步法和附聚法。如果能在一个装置中 ,以不同的工艺条件生产不同密度的产品 ,显然是最经济 ,最有效的方法。笔者经过摸索 ,结合转炉一步法生
Design space exploration(DSE) is an important step before the physical level design of Field programmable gate arrays(FPGA). An optimum architecture is usually
硅是一种良好的近红外材料.硅光栅的发展迄今已有20多年历史,在制作方法和应用上都有了较大的发展.硅光栅的微加工工艺可以分为体硅工艺和面硅工艺,这些微加工方法在技术上与
给出了 NOVASCOPE 40 0 0型超声测厚仪对某长空心轴壁厚的实测结果 ,分析了测量精度 ,提出了获得最佳精度的方法。 The measured results of NOVASCOPE 40 0 ​​0 ultrason