A Scalable Segment-Parallel Architecture for High-Efficiency Lossless Data Compression
A Scalable Segment-Parallel Architecture for High-Efficiency Lossless Data Compression
Abstract:
The exponential growth of data volume puts sig-nificant pressure on the throughput and CPU resource usage of conventional software-based compression systems, driving the research focus of LZ4 algorithm towards its parallel hard-ware implementations. However, existing parallel architectures have to make a trade-off between throughput and compression ratio. To address the challenge, this paper presents a novel segment-parallel architecture that simultaneously delivers both high throughput and good compression ratio. The proposed architecture first introduces an interconnected dictionary scheme to preserve compression ratio while enabling parallel processing, limiting compression ratio degradation to less than 8% compared with software benchmarks across various parallelization levels. Second, a priority-based arbitration mechanism for data memory and a hierarchical depth scheduling strategy for hash table are proposed to enhance memory efficiency of multi-port memories. Additionally, the bit-width of hash table entries is reduced by exploiting deterministic address mapping relationships between hash values and input strings. Implemented on FPGA platforms, the proposed architecture achieves a state-of-the-art performance of 17.71 Gbps throughput, matching the performance of our previous work while delivering a 2.91 ∼ 3.85× improvement over other designs. It also demonstrates superior memory efficiency exceeding existing parallel implementations by 1.57 ∼ 5.57×.
” Thanks for Visit this project Pages – Register This Project and Buy soon with Novelty “
A Scalable Segment-Parallel Architecture for High-Efficiency Lossless Data Compression
