Downloads & Free Reading Options - Results

How do we get programmers to routinely write parallel software? We have been working in earnest on this problem for over 25 years; actually much longer if you consider that the first multi-threaded machine appeared in 1958 (the Gamma 60 by Bull). But at this point, I’m not sure we’re really getting any closer to solving it. If we study the history of parallel programming, it is clear that in order to solve the parallel programming problem we need to: (1) understand how people write parallel software (mine the key design patterns), (2) agree on how to discuss programmability, and (3) stop scaring away our software developers. In this talk, I will describe these issues and how my (our?) collaborations at the UC Berkeley’s ParLab are addressing them. ©2009 Microsoft Corporation. All rights reserved.

International Journal of Parallel Programming 1990: Volume 19 , Issue CONTENTS. Digitized from IA1631414-07 . Previous issue: sim_international-journal-of-parallel-programming_1989-12_18_6 . Next issue: sim_international-journal-of-parallel-programming_1990-02_19_1 .

With rapidly evolving technology, multicore and manycore processors have emerged as promising architectures to benefit from increasing transistor numbers. The transition towards these parallel architectures makes today an exciting time to investigate challenges in parallel computing. The TILEPro64 is a manycore accelerator, composed of 64 tiles interconnected via multiple 8x8 mesh networks. It contains per-tile caches and supports cache-coherent shared memory by default. In this paper we present a programming technique to take advantages of distributed caching facilities in manycore processors. However, unlike other work in this area, our approach does not use architecture-specific libraries. Instead, we provide the programmer with a novel technique on how to program future Non-Uniform Cache Architecture (NUCA) manycore systems, bearing in mind their caching organisation. We show that our localised programming approach can result in a significant improvement of the parallelisation efficiency (speed-up).

Programming multiprocessor architectures is a critical research issue. An overview is given of the various approaches to programming these architectures that are currently being explored. It is argued that two of these approaches, interactive programming environments and functional parallel languages, are particularly attractive since they remove much of the burden of exploiting parallel architectures from the user. Also described is recent work by the author in the design of parallel languages. Research on languages for both shared and nonshared memory multiprocessors is described, as well as the relations of this work to other current language research projects.

The Program Development Environment (PDE) is a tool for massively parallel programming of distributed-memory architectures. Adopting a knowledge-based approach, the PDE eliminates the complexity introduced by parallel hardware with distributed memory and offers complete transparency in respect of parallelism exploitation. The knowledge-based part of the PDE is realized in CLIPS. Its principal task is to find an efficient parallel realization of the application specified by the user in a comfortable, abstract, domain-oriented formalism. A large collection of fine-grain parallel algorithmic skeletons, represented as COOL objects in a tree hierarchy, contains the algorithmic knowledge. A hybrid knowledge base with rule modules and procedural parts, encoding expertise about application domain, parallel programming, software engineering, and parallel hardware, enables a high degree of automation in the software development process. In this paper, important aspects of the implementation of the PDE using CLIPS and COOL are shown, including the embedding of CLIPS with C++-based parts of the PDE. The appropriateness of the chosen approach and of the CLIPS language for knowledge-based software engineering are discussed.

The Program Development Environment (PDE) is a tool for massively parallel programming of distributed-memory architectures. Adopting a knowledge-based approach, the PDE eliminates the complexity introduced by parallel hardware with distributed memory and offers complete transparency in respect of parallelism exploitation. The knowledge-based part of the PDE is realized in CLIPS. Its principal task is to find an efficient parallel realization of the application specified by the user in a comfortable, abstract, domain-oriented formalism. A large collection of fine-grain parallel algorithmic skeletons, represented as COOL objects in a tree hierarchy, contains the algorithmic knowledge. A hybrid knowledge base with rule modules and procedural parts, encoding expertise about application domain, parallel programming, software engineering, and parallel hardware, enables a high degree of automation in the software development process. In this paper, important aspects of the implementation of the PDE using CLIPS and COOL are shown, including the embedding of CLIPS with C++-based parts of the PDE. The appropriateness of the chosen approach and of the CLIPS language for knowledge-based software engineering are discussed.

With the industry-wide shift to multi-core processors and the rise of large-scale clusters as the engines for web services, it is clear that the future is all about exploiting parallelism. Unfortunately, writing parallel or concurrent software has long been a notoriously difficult task for programmers. In particular, the goals of maintaining correctness while also achieving high performance are far more challenging in a parallel or concurrent environment compared with a single-threaded system. Meld is a logic-based programming language that has proven effective for programming massively distributed systems. In this report, we describe Meld and how it can also be used to program more traditional parallel machines. We evaluate Meld on a small set of benchmarks and describe some of the methods we use to make it an effective tool for parallel programming. While more work needs to be done, it is clear that the use of a single language for describing both computation and coordination can lead to clear, concise, and efficient implementations.

I present Chorus, a high-level parallel programming model targeting irregular, heap-manipulating applications like mesh refinement and triangulation, and Jchorus, an implementation of the model on top of sequential Java. One focus of Chorus is dynamism: the fact that in irregularly parallel applications, ownership and isolation are usually instance-dependent and highly dynamic. The other is locality: the property that in many of the same applications, typical imperative commands only affect small, local regions in the heap. Chorus addresses dynamism and locality through the unifying abstraction of an *object assembly*, a short-lived, speculative task that owns a local region in a shared data structure. The task can only access objects that it owns; while changes in ownership are frequent, they are local – i.e., objects only move from one assembly to a neighboring one – and linear – i.e., aliasing and copying are tightly controlled. Programming primitives include a merge-operation, by which an assembly merges with an adjacent assembly, and a split-operation, which linearly splits an assembly into smaller ones. Our abstractions are race and deadlock-free, and inherently data-centric. We demonstrate that Chorus and Jchorus allow natural programming of several important applications exhibiting irregular data-parallelism. From preliminary investigations, it demonstrates competitive performance as well. I finish with a discussion of current work on fractional ownership in Chorus, and on the integration of Chorus with the Habanero-Java language from Rice University. ©2009 Microsoft Corporation. All rights reserved.

We present a library of generic software components for parallel and distributed computations on graphs, based on the Boost Graph Library (BGL). The BGL consists of a rich set of generic graph algorithms and supporting data structures, but it was not originally designed with parallelism in mind. In this talk, we revisit the abstractions comprising the BGL in the context of distributed-memory parallelism, lifting away the implicit requirements of sequential execution and a single shared address space. We illustrate our approach by describing the process as applied to one of the core algorithms in the BGL, breadth-first search. The result is a generic algorithm that is unchanged from the sequential algorithm, requiring only the introduction of external (distributed) data structures for parallel execution. More importantly, the generic implementation retains its interface and semantics, such that other distributed algorithms can be built upon it, just as algorithms are layered in the sequential case. By characterizing these extensions as well as the extension process, we develop general principles and patterns for using (and reusing) generic parallel software libraries. We demonstrate that the resulting algorithm implementations are both efficient and scalable with performance results for several algorithms implemented in the open-source Parallel Boost Graph Library. ©2007 Microsoft Corporation. All rights reserved.

Contents: OpenMP Fortran Application Program Interface; OpenMP constructs; The OpenMP run-time library; The environment variables. Lecture Notes Collection FreeScience.info ID2852 Obtained from http://www.openmp.org/presentations/miguel/F95_OpenMPv1_v2.pdf http://www.freescience.info/go.php?pagename=books&id=2852

Here, it is explained why the FORCE parallel programming language is easily portable among six different shared-memory microprocessors, and how a two-level macro preprocessor makes it possible to hide low level machine dependencies and to build machine-independent high level constructs on top of them. These FORCE constructs make it possible to write portable parallel programs largely independent of the number of processes and the specific shared memory multiprocessor executing them.

The objective of this effort was to investigate methods for distributing a knowledge base over multiple processors using the logic programming paradigm. A three-fold approach was employed and is discussed in this report. The first phase involved the testing of the Parallel Knowledge- based Execution System (PKES). Secondly, memory allocation techniques were investigated and evaluated. Lastly, a study of the implementation of data-typing in a logic programming paradigm is discussed. (CP)

As the complexity of machines and architectures has increased, performance tuning has become more challenging, leading to the failure of general compilers to generate the best possible optimized code. Expert performance programmers can often hand-write code that outperforms compiler-optimized low-level code by an order of magnitude. At the same time, the complexity of programs has also increased, with modern programs built on a variety of abstraction layers to manage complexity, yet these layers hinder efforts at optimization. In fact, it is common to lose one or two additional orders of magnitude in performance when going from a low-level language such as Fortran or C to a high-level language like Python, Ruby, or Matlab.

There is a lot more to parallel programming in Python than multiprocessing.Pool().map. In this talk I will share some hard-learned knowledge gained in several years of parallel programming. Covered topics will include performance, ways to measure the performance, memory occupation, data transfer and ways to reduce the data transfer, how to debug parallel programs and useful libraries. I will give some practical examples, both in enterprise programming (importing CSV files in a database) and in scientific programming (numerical simulations). The initial part of the talk will be pedagogical, advocating the convenience of parallel programming in the small (i.e. in single machine environment); the second part will be more advanced and will touch a few things to know when writing parallel programs for medium-sized clusters. I will also briefly discuss the compatibility layer that we have developed at GEM to be independent from the underlying parallelization technology (multiprocessing, concurrent.futures, celery, ipyparallel, grid engine...). Please see our speaker release agreement for details: https://ep2017.europython.eu/en/speaker-release-agreement/

One important feature of a parallel programming model is deterministic semantics: that is, a program produces the same output on every execution with a given input, regardless of the parallel schedule chosen. Determinism makes parallel programs much easier to write, understand, debug, and maintain. Further, many (though not all) parallel programs are, in fact, intended to have deterministic semantics. However, general-purpose languages, particularly those that allow arbitrary updates to shared data, typically do not guarantee determinism, leaving the burden on the programmer to ensure that the program is correctly synchronized. In this talk, I will present Deterministic Parallel Java (DPJ), an extension of the Java programming language that guarantees deterministic semantics in the presence of aliasing and updates to shared mutable objects. An important consequence of deterministic semantics is that a working sequential program needs no further testing after parallelization with DPJ. In DPJ, the programmer partitions the heap into regions and annotates method definitions with read and write effects on those regions. The programmer also explicitly marks parallel tasks and synchronization points. The compiler checks that the method effects are correctly summarized, and that the program is correctly synchronized to ensure determinism. I will describe the core language features, illustrate them with examples, and describe the results of an initial evaluation of the language. I will also describe ongoing work, including features we plan to add to the language. ©2009 Microsoft Corporation. All rights reserved.

Recently, networked and cluster computation have become very popular. This paper is an introduction to a new C based parallel language for architecture-adaptive programming, aCe C. The primary purpose of aCe (Architecture-adaptive Computing Environment) is to encourage programmers to implement applications on parallel architectures by providing them the assurance that future architectures will be able to run their applications with a minimum of modification. A secondary purpose is to encourage computer architects to develop new types of architectures by providing an easily implemented software development environment and a library of test applications. This new language should be an ideal tool to teach parallel programming. In this paper, we will focus on some fundamental features of aCe C.

In many fields, real-world applications for High Performance Computing have already been developed. For these applications to stay up-to-date, new parallel strategies must be explored to yield the best performance; however, restructuring or modifying a real-world application may be daunting depending on the size of the code. In this case, a mini-app may be employed to quickly explore such options without modifying the entire code. In this work, several mini-apps have been created to enhance a real-world application performance, namely the VULCAN code for complex flow analysis developed at the NASA Langley Research Center. These mini-apps explore hybrid parallel programming paradigms with Message Passing Interface (MPI) for distributed memory access and either Shared MPI (SMPI) or OpenMP for shared memory accesses. Performance testing shows that MPI+SMPI yields the best execution performance, while requiring the largest number of code changes. A maximum speedup of 23 was measured for MPI+SMPI, but only 11 was measured for MPI+OpenMP.

The Naval Research Laboratory's (NRL) Code 7440 Production Enhancement Team at the Stennis Space Center has been tasked to develop ways to speedup hydrographic data processing at the Naval Oceanographic Office (NAVOCEANO) [Depn02]. This paper presents the final development of a parallelized version of the Pfm_loader application customized to run on a Beowulf cluster.

The Naval Research Laboratory's (NRL) Code 7440 Production Enhancement Team at the Stennis Space Center has been tasked to develop ways to speedup hydrographic data processing at the Naval Oceanographic Office (NAVOCEANO) [Depn02]. This paper presents the final development of a parallelized version of the Pfm_loader application customized to run on a Beowulf cluster.

Threads provide a useful programming model for asynchronous behavior because of their ability to encapsulate units of work that can then be scheduled for execution at runtime, based on the dynamic state of a system. Recently, the threaded model has been applied to the domain of data parallel scientific codes, and initial reports indicate that the threaded model can produce performance gains over non-threaded approaches, primarily through the use of overlapping useful computation with communication latency. However, overlapping computation with communication is possible without the benefit of threads if the communication system supports asynchronous primitives, and this comparison has not been made in previous papers. This paper provides a critical look at the utility of lightweight threads as applied to data parallel scientific programming.

Threads provide a useful programming model for asynchronous behavior because of their ability to encapsulate units of work that can then be scheduled for execution at runtime, based on the dynamic state of a system. Recently, the threaded model has been applied to the domain of data parallel scientific codes, and initial reports indicate that the threaded model can produce performance gains over non-threaded approaches, primarily through the use of overlapping useful computation with communication latency. However, overlapping computation with communication is possible without the benefit of threads if the communication system supports asynchronous primitives, and this comparison has not been made in previous papers. This paper provides a critical look at the utility of lightweight threads as applied to data parallel scientific programming.

Threads provide a useful programming model for asynchronous behavior because of their ability to encapsulate units of work that can then be scheduled for execution at runtime, based on the dynamic state of a system. Recently, the threaded model has been applied to the domain of data parallel scientific codes, and initial reports indicate that the threaded model can produce performance gains over non-threaded approaches, primarily through the use of overlapping useful computation with communication latency. However, overlapping computation with communication is possible without the benefit of threads if the communication system supports asynchronous primitives, and this comparison has not been made in previous papers. This paper provides a critical look at the utility of lightweight threads as applied to data parallel scientific programming.

The manual application of formal methods in system specification has produced successes, but in the end, despite any claims and assertions by practitioners, there is no provable relationship between a manually derived system specification or formal model and the customer's original requirements. Complex parallel and distributed system present the worst case implications for today s dearth of viable approaches for achieving system dependability. No avenue other than formal methods constitutes a serious contender for resolving the problem, and so recognition of requirements-based programming has come at a critical juncture. We describe a new, NASA-developed automated requirement-based programming method that can be applied to certain classes of systems, including complex parallel and distributed systems, to achieve a high degree of dependability.

International Journal of Parallel Programming 1992: Volume 21 , Issue CONTENTS. Digitized from IA1631414-07 . Previous issue: sim_international-journal-of-parallel-programming_1991-12_20_6 . Next issue: sim_international-journal-of-parallel-programming_1992-02_21_1 .

"This book, now in is second edition, is the premier resource to learn SYCL 2020 and is the ONLY book you need to become part of this community." Erik Lindahl, GROMACS and Stockholm University Learn how to accelerate C++ programs using data parallelism and SYCL. This open access book enables C++ programmers to be at the forefront of this exciting and important development that is helping to push computing to new levels. This updated second edition is full of practical advice, detailed explanations, and code examples to illustrate key topics. SYCL enables access to parallel resources in modern accelerated heterogeneous systems. Now, a single C++ application can use any combination of devices–including GPUs, CPUs, FPGAs, and ASICs–that are suitable to the problems at hand. This book teaches data-parallel programming using C++ with SYCL and walks through everything needed to program accelerated systems. The book begins by introducing data parallelism and foundational topics for effective use of SYCL. Later chapters cover advanced topics, including error handling, hardware-specific programming, communication and synchronization, and memory model considerations. All source code for the examples used in this book is freely available on GitHub. The examples are written in modern SYCL and are regularly updated to ensure compatibility with multiple compilers. What You Will Learn Accelerate C++ programs using data-parallel programming Use SYCL and C++ compilers that support SYCL Write portable code for accelerators that is vendor and device agnostic Optimize code to improve performance for specific accelerators Be poised to benefit as new accelerators appear from many vendors Who This Book Is For New data-parallel programming and computer programmers interested in data-parallel programming using C++ This is an open access book.

strand-new-concepts-in-parallel-programming-by-foster-ian-taylor-stephen_202201

The computational demands of software continue to outpace the capacities of processor and memory technologies, especially in scientific and engineering programs. One option to improve performance is parallel processing. However, despite decades of research and development writing parallel programs continues to be difficult. This is especially the case for scientists and engineers who have limited backgrounds in computer science. MATLAB?, due to its ease of use compared to other programming languages like C and Fortran, is one of the most popular languages for implementing numerical computations, thus making it an excellent platform for developing an accessible parallel computing framework. The MIT Lincoln Laboratory has developed two libraries, pMatlab and MatlabMPI, that not only enables parallel programming with MATLAB in a simple fashion, accessible to non-computer scientists. This document will overview basic concepts in parallel programming and introduce pMatlab.

The computational demands of software continue to outpace the capacities of processor and memory technologies, especially in scientific and engineering programs. One option to improve performance is parallel processing. However, despite decades of research and development writing parallel programs continues to be difficult. This is especially the case for scientists and engineers who have limited backgrounds in computer science. MATLAB?, due to its ease of use compared to other programming languages like C and Fortran, is one of the most popular languages for implementing numerical computations, thus making it an excellent platform for developing an accessible parallel computing framework. The MIT Lincoln Laboratory has developed two libraries, pMatlab and MatlabMPI, that not only enables parallel programming with MATLAB in a simple fashion, accessible to non-computer scientists. This document will overview basic concepts in parallel programming and introduce pMatlab.

The computational demands of software continue to outpace the capacities of processor and memory technologies, especially in scientific and engineering programs. One option to improve performance is parallel processing. However, despite decades of research and development writing parallel programs continues to be difficult. This is especially the case for scientists and engineers who have limited backgrounds in computer science. MATLAB?, due to its ease of use compared to other programming languages like C and Fortran, is one of the most popular languages for implementing numerical computations, thus making it an excellent platform for developing an accessible parallel computing framework. The MIT Lincoln Laboratory has developed two libraries, pMatlab and MatlabMPI, that not only enables parallel programming with MATLAB in a simple fashion, accessible to non-computer scientists. This document will overview basic concepts in parallel programming and introduce pMatlab.

International Journal of Parallel Programming 1995: Volume 23 , Issue CONTENTS. Digitized from IA1631414-07 . Previous issue: sim_international-journal-of-parallel-programming_1994-12_22_6 . Next issue: sim_international-journal-of-parallel-programming_1995-02_23_1 .

Modern distributed systems consisting of powerful workstations and high-speed interconnection networks are an economical alternative to special- purpose super computers. The technical issues that need to be addressed in exploiting the parallelism inherent in a distributed system include heterogeneity, high-latency communication, fault tolerance and dynamic load balancing. Current software systems for parallel programming provide little or no automatic support towards these issues and require users to be experts in fault-tolerant distributed computing. The Paralex system is aimed at exploring the extent to which the parallel application programmer can be liberated from the complexities of distributed systems. Paralex is a complete programming environment and makes extensive use of graphics to define, edit, execute and debug parallel scientific applications. All of the necessary code for distributing the computation across a network and replicating it to achieve fault tolerance and dynamic load balancing is automatically generated by the system. In this paper we give an overview of Paralex and present our experiences with a prototype implementation.

From the bitsavers.org collection, a scanned-in computer-related document. hp :: 9000 hpux :: 1991-200x :: 200003 B3909-90003 Parallel Programming Guide for HP-UX Systems Ed2

We introduce process-oriented programming as a natural extension of object-oriented programming for parallel computing. It is based on the observation that every class of an object-oriented language can be instantiated as a process, accessible via a remote pointer. The introduction of process pointers requires no syntax extension, identifies processes with programming objects, and enables processes to exchange information simply by executing remote methods. Process-oriented programming is a high-level language alternative to multithreading, MPI and many other languages, environments and tools currently used for parallel computations. It implements natural object-based parallelism using only minimal syntax extension of existing languages, such as C++ and Python, and has therefore the potential to lead to widespread adoption of parallel programming. We implemented a prototype system for running processes using C++ with MPI and used it to compute a large three-dimensional Fourier transform on a computer cluster built of commodity hardware components. Three-dimensional Fourier transform is a prototype of a data-intensive application with a complex data-access pattern. The process-oriented code is only a few hundred lines long, and attains very high data throughput by achieving massive parallelism and maximizing hardware utilization.

Threads are typically not the way to take advantage of multiple CPUs for CPU-bound problems. The Global Interpreter Lock (GIL) allows the use of only one CPU at the time when using threads. However, the GIL is released for IO task The use case is a scientific simulation model that has to run 18,000 different simulations. The input data for these simulations need to be extracted from a common database, re-assembled and translated into several input files per simulation. After each simulation that is run with an external, standalone executable, the output data needs to be gathered and rearranged in a output database. The implementation scaled up to 50 threads. On a eight-core machine more than 90 % usage of all CPUs could be achieved, bringing the total run time down to about two hours from about 15 hours. Depending on the use case, threading can help to speedup a program and even take advantage of multiple CPUs. This talk presents such a use case. The approach can be translated to problems from other domains if the sub-tasks can be turned into IO tasks. Asynchronous programming could have been used here. However using a thread per task and using class that represents a task, is likely conceptually simpler for programmers not used to asynchronous programming. Please see our speaker release agreement for details: https://ep2017.europython.eu/en/speaker-release-agreement/

Threads are typically not the way to take advantage of multiple CPUs for CPU-bound problems. The Global Interpreter Lock (GIL) allows the use of only one CPU at the time when using threads. However, the GIL is released for IO task The use case is a scientific simulation model that has to run 18,000 different simulations. The input data for these simulations need to be extracted from a common database, re-assembled and translated into several input files per simulation. After each simulation that is run with an external, standalone executable, the output data needs to be gathered and rearranged in a output database. The implementation scaled up to 50 threads. On a eight-core machine more than 90 % usage of all CPUs could be achieved, bringing the total run time down to about two hours from about 15 hours. Depending on the use case, threading can help to speedup a program and even take advantage of multiple CPUs. This talk presents such a use case. The approach can be translated to problems from other domains if the sub-tasks can be turned into IO tasks. Asynchronous programming could have been used here. However using a thread per task and using class that represents a task, is likely conceptually simpler for programmers not used to asynchronous programming. Please see our speaker release agreement for details: https://ep2017.europython.eu/en/speaker-release-agreement/

Threads are typically not the way to take advantage of multiple CPUs for CPU-bound problems. The Global Interpreter Lock (GIL) allows the use of only one CPU at the time when using threads. However, the GIL is released for IO task The use case is a scientific simulation model that has to run 18,000 different simulations. The input data for these simulations need to be extracted from a common database, re-assembled and translated into several input files per simulation. After each simulation that is run with an external, standalone executable, the output data needs to be gathered and rearranged in a output database. The implementation scaled up to 50 threads. On a eight-core machine more than 90 % usage of all CPUs could be achieved, bringing the total run time down to about two hours from about 15 hours. Depending on the use case, threading can help to speedup a program and even take advantage of multiple CPUs. This talk presents such a use case. The approach can be translated to problems from other domains if the sub-tasks can be turned into IO tasks. Asynchronous programming could have been used here. However using a thread per task and using class that represents a task, is likely conceptually simpler for programmers not used to asynchronous programming. Please see our speaker release agreement for details: https://ep2017.europython.eu/en/speaker-release-agreement/

Includes bibliographical references (p. [709]-736) and indexes

Includes bibliographical references (p. [709]-736) and indexes

International Journal of Parallel Programming 1998: Volume 26 , Issue CONTENTS. Digitized from IA1631414-07 . Previous issue: sim_international-journal-of-parallel-programming_1997-12_25_6 . Next issue: sim_international-journal-of-parallel-programming_1998-02_26_1 .

DINO (Distributed Numerically Oriented language) is a language for writing parallel programs for distributed memory (MIND) multiprocessors. It is oriented towards expressing data parallel algorithms, which predominate in parallel numerical computation. Its goal is to make programming such algorithms natural and easy, without hindering their run-time efficiency. DINO consists of standard C augmented by several high level parallel constructs that are intended to allow the parallel program to conform to the way an algorithm designer naturally thinks about parallel algorithms. The key constructs are the ability to declare a virtual parallel computer that is best suited to the parallel computation, the ability to map distributed data structures onto this virtual machine, and the ability to define procedures that will run on each processor of the virtual machine concurrently. Most of the remaining details of distributed parallel computation, including process management and interprocessor communication, result implicitly from these high level constructs and are handled automatically by the compiler. This paper describes the syntax and semantics of the DINO language, gives examples of DINO programs, presents a critique of the DINO language features, and discusses the performance of code generated by the DINO compiler.

The Sync Model, a parallel execution method for logic programming, is preposed. The Sync Model is a multiple-solution data-driven model that realizes AND parallelism and OR-parallelism in a logic program assuming a message-passing multiprocessor system. AND parallelism is implemented by constructing a dynamic data flow graph of the literals in the clause body with an ordering algorithm. OR parallelism is achieved by adding special Synchronization signals to the stream of partial solutions and synchronizing the multiple streams with a merge algorithm. The ordering algorithm and the merge algorithm are described. The merge algorithm is proved to be correct and therefore, the Sync Model is proved complete, i.e., the execution of a logic program under the Sync Model generates all the solutions.

The paper develops some ideas expounded in a previous paper. It distinguishes a number of ways of using parallelism, including disjoint processes, competition, cooperation, communication and colluding. In each case an axiomatic proof rule is given. Some light is thrown on traps or ON conditions. The program structuring methods described in the report are not suitable for the construction of operating systems.

Threads provide a useful programming model for asynchronous behavior because of their ability to encapsulate units of work that can then be scheduled for execution at runtime, based on the dynamic state of a system. Recently, the threaded model has been applied to the domain of data parallel scientific codes, and initial reports indicate that the threaded model can produce performance gains over non-threaded approaches, primarily through the use of overlapping useful computation with communication latency. However, overlapping computation with communication is possible without the benefit of threads if the communication system supports asynchronous primitives, and this comparison has not been made in previous papers. This paper provides a critical look at the utility of lightweight threads as applied to data parallel scientific programming. (KAR) P. 2

Threads provide a useful programming model for asynchronous behavior because of their ability to encapsulate units of work that can then be scheduled for execution at runtime, based on the dynamic state of a system. Recently, the threaded model has been applied to the domain of data parallel scientific codes, and initial reports indicate that the threaded model can produce performance gains over non-threaded approaches, primarily through the use of overlapping useful computation with communication latency. However, overlapping computation with communication is possible without the benefit of threads if the communication system supports asynchronous primitives, and this comparison has not been made in previous papers. This paper provides a critical look at the utility of lightweight threads as applied to data parallel scientific programming. (KAR) P. 2

Parallel programming is still being based upon antiquated sequence-based definitions of the terms "algorithm" and "computation", resulting in programs which are architecture dependent and difficult to design and analyze. By focusing on obstacles inherent in existing practice, a more portable model is derived here, which is then formalized into a model called Soviets which utilizes a combination of imperative and functional styles. This formalization suggests more general notions of algorithm and computation, as well as insights into the meaning of structured programming in a parallel setting. To illustrate how these principles can be applied, a very-high-level graphical architecture-independent parallel language, called Software Cabling, is described, with many of the features normally expected from today's computer languages (e.g. data abstraction, data parallelism, and object-based programming constructs).

Achieving scalable performance for dynamic irregular applications is eminently challenging. Traditional message-passing approaches have been making steady progress towards this goal; however, they suffer from complex implementation requirements. The use of a global address space greatly simplifies the programming task, but can degrade the performance for such computations. In this work, we examine two typical irregular adaptive applications, Dynamic Remeshing and N-Body, under competing programming methodologies and across various parallel architectures. The Dynamic Remeshing application simulates flow over an airfoil, and refines localized regions of the underlying unstructured mesh. The N-Body experiment models two neighboring Plummer galaxies that are about to undergo a merger. Both problems demonstrate dramatic changes in processor workloads and interprocessor communication with time; thus, dynamic load balancing is a required component.

The purpose of this letter is to discuss our research into parallel systems software and hardware, and parallel problem solving. Under the current grant, a detailed design and partial implementation of a parallel problem solving system, PRISM (parallel inference system), based on logic was achieved. The PRISM requires that the ZMOB parallel processor be available for use. In addition, systems software and hardware have been developed. It is estimated that ZMOB will become available for use during the Fall of 1983. Hence, a full test and debugging of PRISM cannot be achieved under the current grant. At the end of the current grant we expect to have accomplished, as a minimum, all of the objectives proposed. That is, in the area of parallel problem solving, the initial PRISM has been fully designed; individual programs have been implemented and tested in a non-parallel environment; and investigations have been made into extensions to the initial design.

Animate vision systems couple computer vision and robotics to achieve robust and accurate vision, as well as other complex behavior. These systems combine low-level sensory processing and effector output with high-level cognitive planning - all computationally intensive tasks that can benefit from parallel processing. No single model of parallel programming is likely to serve for all tasks, however. Early vision algorithms are intensely data parallel, often utilizing fine-grain parallel computations that share an image, while cognition algorithms decompose naturally by function, often consisting of loosely-coupled, coarse-grain parallel units. A typical animate vision application will likely consist of many tasks, each of which may require a different parallel programming model, and all of which must cooperate to achieve the desired behavior. These multi-model programs require an underlying software system that not only supports several different models of parallel computation simultaneously, but which also allows tasks implemented in different models to interact.

Parallel machines provide a wonderful opportunity for applications with large computational requirements. Effective use of these machines, requires a keen understanding of how they work. Each chapter provides an overview.