高性能计算的海量存储系统新型访问策略分析

Research on the new access policy of storage unit under HPC mass storage system

摘要:

为解决海量信息处理中实时访问的"I/O墙"问题,提高海量信息分布式存储系统的性能,提出了一种基于HPC的存储部件新型访问策略. 首先分析了传统访问模型存在的问题；其次研究了存储部件直通路模式的工作机理, 建立了存储系统的多层次、分布式模型,根据不同层次和映射策略实现存储空间物理地址、缓存地址、存储系统逻辑空间地址的连续映射；继而分析了直通路访问模式下的存储路径时间开销；最后在模拟环境下进行存储部件访问的性能测试,并在实际应用系统中对该策略进行验证. 验证测试结果表明,该方法能够有效提高存储系统性能,满足海量信息处理的实时性需要.

Abstract:

To solve the "I/O wall" problem in the case of real-time accessing about mass information processing and to improve performance of distributed mass storage systems, an access policy based on storage unit pass-through is proposed and the problem of traditional access models is analyzed. Then the mechanism of pass-through pattern is studied, and a multi-level and distributed model is built up. Next, the continuous mapping of physical address, cache address of storage space and logical space address of storage system are realized depend on the different levels and mapping strategies. The time consuming of pass-through storage path in pass-through pattern is analyzed. Last, the performance of the storage unit in the simulated environment is tested. The results show that the method can improve the performance of storage system effectively, and can meet the needs of real-time accessing about massive information processing.