||This project is to develop an efficient lossy compression code to significantly reduce the memory footprint for extreme-scale deep neural networks (DNNs) on GPU-based high-performance computing (HPC) systems. DNNs have rapidly evolved as a state-of-the-art technique in many science and technology domains. The scales of DNNs are becoming larger because of increasing demand on the analysis quality for more complex applications to resolve, leading to extreme-scale DNNs. However, the ever-increasing scales of DNNs require a large amount of resources, such as memory, bringing more challenges to heterogeneous HPC systems. One important reason is the huge gap between the memory required by extreme-scale DNNs and the memory available in graphic processing units (GPUs). This gap compels researchers to use multiple GPUs, which would result in significant performance degradation due to expensive communications. We have identified that error-bounded lossy compression can offer high data reduction capability and precise error controllability for large-scale scientific applications. In this project, we will optimize the compression quality of our error-bounded lossy compressor SZ for different DNN intermediate data based on their features, such as sparsity and correlation. We will also conduct a performance optimization of the lossy compression code on state-of-the-art GPUs. Finally, we will integrate our lossy compressor into a distributed deep learning framework and evaluate its performance scalability with interdisciplinary applications on large-scale GPU-based HPC systems. We are requesting a Startup account to develop our lossy compression code and test its supported deep learning framework on the state-of-the-art GPU-based HPC system, i.e., Bridges GPU-AI at PSC.