Language and Compiler Issues in Scalable High Performance Scientific Libraries

Abstract

Library functions for scalable architectures must be designed to correctly and efficiently support any distributed data structure that can be created with the supported languages and associated compiler directives. Libraries must be designed also to support concurrency in each function evaluation, as well as the concurrent application of the functions to disjoint array segments, known as multiple-instance computation. Control over the data distribution is often critical for locality of reference, and so is the control over the interprocessor data motion. Scalability, while preserving efficiency, implies that the data distribution, the data motion, and the scheduling is adapted to the object shapes, the machine configuration, and the size of the objects relative to the machine size. The Connection Machine Scientific Software Library is a scalable library for distributed data structures. The library is designed for languages with an array syntax. It is accessible from all supported languages (Lisp, C, CM-Fortran, and Paris (PARallel Instruction Set) in combination with Lisp, C, and Fortran 77). Single library calls can manage both concurrent application of a function to disjoint array segments, as well as concurrency in each application of a function. The control of the concurrency is independent of the control constructs provided in the high{level languages. Library functions operate efficiently on any distributed data structure that can be defined in the high-level languages and associated directives. Routines may use their own internal data distribution for efficiency reasons. The algorithm invoked by a call to a library function depends upon the shapes of the objects involved, their sizes and distribution, and upon the machine shape and size.