SemCache++: Semantics-Aware Caching for Efficient Multi-GPU Offloading

Nabeel AlSaber and Milind Kulkarni

ICS 2015 · International Conference on Supercomputing · June 2015 · pp. 79-88

Acceptance rate 25%

Abstract

Offloading computations to multiple GPUs is not an easy task. It requires decomposing data, distributing computations and handling communication manually. GPU drop-in libraries (which require no program rewrite) have made it easy to offload computations to multiple GPUs by hiding this complexity inside library calls. Such encapsulation prevents the reuse of data between successive kernel invocations resulting in redundant communication. This limitation exists in multi-GPU libraries like CUBLASXT.

In this paper, we introduce SemCache++, a semantics-aware GPU cache that automatically manages communication between the CPU and multiple GPUs in addition to optimizing communication by eliminating redundant transfers using caching. SemCache++ is used to build the first multi-GPU drop-in replacement library that (a) uses the virtual memory to automatically manage and optimize multi-GPU communication and (b) requires no program rewriting or annotations. Our caching technique is efficient; it uses a two level caching directory to track matrices and sub-matrices. Experimental results show that our system can eliminate redundant communication and deliver performance improvements over multi-GPU libraries like StarPU and CUBLASXT.

BibTeX

@inproceedings{alsaber15ics,
  author    = {Nabeel AlSaber and Milind Kulkarni},
  title     = {{SemCache}++: Semantics-Aware Caching for Efficient {Multi-GPU} Offloading},
  booktitle = {Proceedings of the International Conference on Supercomputing (ICS)},
  pages     = {79--88},
  publisher = {ACM},
  address   = {Newport Beach, California, USA},
  year      = {2015},
  month     = jun,
  doi       = {10.1145/2751205.2751210}
}

Group members