论文部分内容阅读
广播式自动相关监视系统(Automatic Dependent Surveillance-Broadcast,ADS-B)是国际民航组织(ICAO)推荐使用的集数据通信、卫星导航和监视技术于一体的新一代航空器运行监视系统,可以自动的接收和发送飞机及其周围的信息。随着监视区域内航班数量的增加,对于以秒为单位进行收发信息的ADSB而言,单机环境已经无法满足海量ADS-B数据的解析、存储与分析,本文利用Mapreduce模型提供的高效分布式编程和运行框架对ADS-B数据进行解析,将解析后的数据存储到基于Hive的ADS-B数据仓库,并通过Mysql建立的索引表联合Hive中的分桶操作对信息种类进行划分,有效提高了数据解析效率并避免了Hive中索引不完善引起的查询效率低的问题。实验表明对于海量的ADS-B数据,利用Mapreduce进行解析并利用Hive进行存储分析的效率明显提升。
Automatic Dependent Surveillance-Broadcast (ADS-B) is a new generation of aircraft operation monitoring system recommended by the International Civil Aviation Organization (ICAO) that integrates data communications, satellite navigation and surveillance technologies and can automatically receive And send information about the plane and its surroundings. With the increase of the number of flights in the surveillance area, for the ADSB sending and receiving information in seconds, the stand-alone environment can not meet the analysis, storage and analysis of massive ADS-B data. In this paper, efficient distributed programming provided by the Mapreduce model And run the framework to analyze the ADS-B data, and store the parsed data in the Hive-based ADS-B data warehouse, and divide the information types by using the index table established by Mysql and the sub-bucket operation in Hive, thereby effectively improving Data parsing efficiency and avoid the inefficiency of the query caused by the index is not perfect in Hive. Experiments show that for mass ADS-B data, the efficiency of using Mapreduce to parse and use Hive for storage analysis is obviously improved.