GXP : A Parallel Shell that Makes Distributed Environments Comfortable

GXP is a parallel shell designed for supporting a range of purposes from system managements to parallel processing with a small installation effort. Its early prototype was developed at the end of 2003 and, after twice of complete rewrites, ver.3 is currently published as an open source software. While it shares the basic function with other parallel shells—executing the same command line on many hosts, it achieves a superior speed (response time) and scalability. Its design is also significantly more extended than similar parallel shells for the purpose of supporting parallel processing in distributed environments. For example, (1) it flexibly works in the presence of firewall and NATs, (2) it works on top of various kinds of underlying remote accessing protocols such as SSH, Sun Grid Engine, TORQUE, and mixture thereof, (3) it needs to be installed only one host and can be used on all hosts, and (4) it supports essential features for “interactive” sessions such as setting environment variables and current directories on remote hosts, and choosing execution hosts flexibly. In addition, it has a built-in support for parallel and distributed execution of make. Thus the user is able to run coarse-grained tasks with dependencies with a minimal effort of writing a Makefile. With all these features, it significantly extends the area of applications compared to existing parallel shells, which mainly target system management of a single cluster. All in all, it is a software to use clusters and distributed environments “comfortably.”