Downloads & Free Reading Options - Results

Scalable Parallel Computing on Many/Multicore Systems This set of lectures will review the application and programming model issues that will one must address when one gets chips with 32-1024 cores and “scalable” approaches will be needed to make good use of such systems. We will not discuss bit-level and instruction-level parallelism i.e. what happens on a possibly special purpose core, even though this is clearly important. We will use science and engineering applications to drive the discussion as we have substantial experience in these cases but we are interested in the lessons for commodity client and server applications that will be broadly used 5-10 years from now. We start with simple applications and algorithms from a variety of fields and identify features that makes it “obvious” that “all” science and engineering run well and scalably in parallel. We explain why unfortunately it is equally obvious that there is no straightforward way of expressing this parallelism. Parallel hardware architectures are described in enough detail to understand performance and algorithm issues and need for cross-architecture compatibility; however in these lectures we will just be users of hardware. We must understand what features of multicore chips we can and should exploit. We can explicitly use the shared memory between cores or just enjoy its implications for very fast inter-core control and communication linkage. We note that parallel algorithm research is hugely successful although this success has reduced activity in an area that deserves new attention for the next generation of architectures. The parallel software environment is discussed at several levels including programming paradigm, runtime and operating system. The importance of libraries, templates, kernels (dwarfs) and benchmarks is stressed. The programming environment has various tools including compilers with possible parallelism hints like OpenMP; tuners like Atlas; messaging models; parallelism and distribution support as in HPF, HPCS Languages, co-array Fortran, Global arrays and UPC. We also discuss the relevance of important general ideas like object-oriented paradigms (as in for example Charm++), functional languages and Software Transactional Memories. Streaming, pipelining, co-ordination, services and workflow are placed in context. Examples discussed in the last category include CCR/DSS from Microsoft and the Common Component Architecture CCA from DoE. Domain Specific environments like Matlab and Mathematica are important as there is no universal silver programming bullet; one will need interoperable focused environments. We discuss performance analysis including speedup, efficiency, scaled speedup and Amdahl's law. We show how to relate performance to algorithm/application structure and the hardware characteristics. Applications will not get scalable parallelism accidentally but only if there is an understandable performance model. We review some of the many pitfalls for both performance and correctness; these include deadlocks, race conditions, nodes that are busy doing something else and the difficulty of second-guessing automatic parallelization methods. We describe the formal sources of overhead; load imbalance and the communication/control overhead and the ways to reduce them. We relate the blocking used in caching to that used in parallel computation. We note that load balancing was the one part of parallel computing that was easier than expected. We will mix both simple idealized applications with “real problems” noting that usually it is simple problems that are the hardest as they have poor computation to control/communication ratios. We will explain the parallelism in several application classes including for science and engineering: Finite Difference, Finite Elements, FFT/Spectral, Meshes of all sorts, Particle Dynamics, Particle-Mesh, and Monte Carlo methods. Some applications like Image Processing, Graphics, Cryptography and Media coding/decoding have features similar to well understood science and engineering problems. We emphasize that nearly all applications are built hierarchically from more “basic applications” with a variety of different structures and natural programming models. Such application composition (co-ordination, workflow) must be supported with a common run-time. We contrast the difference between decomposition needed in most “basic parallel applications” to the composition supported in workflow and Web 2.0 mashups. Looking at broader application classes we should cover; Internet applications and services; artificial intelligence, optimization, machine learning; divide and conquer algorithms; tree structured searches like computer chess; applications generated by the data deluge including access, search, and the assimilation of data into simulations. There has been a growing interest in Complex Systems whether it be for critical infrastructure like energy and transportation, the Internet itself, commodity gaming, computational epidemiology or the original war games. We expect Discrete Event Simulations (such as DoD’s High Level Architecture HLA) to grow in importance as they naturally describe complex systems and because they can clearly benefit from multicore architectures. We will discuss the innovative Sequential Dynamical Systems approach used in the EpiSims, TransSims and other critical infrastructure simulation environments. In all applications we need to identify the intrinsic parallelism and the degrees of freedom that can be parallelized and distinguish small parallelism (local to core) compared to large parallelism (scalable across many cores) We will not forget many critical non-technical Issues including “Who programs – everybody or just the marine corps?”; “The market for science and engineering was small but it will be large for general multicore”; “The program exists and can’t be changed”; “What features will next hardware/software release support and how should I protect myself from change?” We will summarize lessons and relate them to application and programming model categories. In last lecture or at end of all lectures we encourage the audience to bring their own multicore application or programming model so we can discuss examples that interest you. ©2007 Microsoft Corporation. All rights reserved.

Scalable Parallel Computing on Many/Multicore Systems This set of lectures will review the application and programming model issues that will one must address when one gets chips with 32-1024 cores and “scalable” approaches will be needed to make good use of such systems. We will not discuss bit-level and instruction-level parallelism i.e. what happens on a possibly special purpose core, even though this is clearly important. We will use science and engineering applications to drive the discussion as we have substantial experience in these cases but we are interested in the lessons for commodity client and server applications that will be broadly used 5-10 years from now. We start with simple applications and algorithms from a variety of fields and identify features that makes it “obvious” that “all” science and engineering run well and scalably in parallel. We explain why unfortunately it is equally obvious that there is no straightforward way of expressing this parallelism. Parallel hardware architectures are described in enough detail to understand performance and algorithm issues and need for cross-architecture compatibility; however in these lectures we will just be users of hardware. We must understand what features of multicore chips we can and should exploit. We can explicitly use the shared memory between cores or just enjoy its implications for very fast inter-core control and communication linkage. We note that parallel algorithm research is hugely successful although this success has reduced activity in an area that deserves new attention for the next generation of architectures. The parallel software environment is discussed at several levels including programming paradigm, runtime and operating system. The importance of libraries, templates, kernels (dwarfs) and benchmarks is stressed. The programming environment has various tools including compilers with possible parallelism hints like OpenMP; tuners like Atlas; messaging models; parallelism and distribution support as in HPF, HPCS Languages, co-array Fortran, Global arrays and UPC. We also discuss the relevance of important general ideas like object-oriented paradigms (as in for example Charm++), functional languages and Software Transactional Memories. Streaming, pipelining, co-ordination, services and workflow are placed in context. Examples discussed in the last category include CCR/DSS from Microsoft and the Common Component Architecture CCA from DoE. Domain Specific environments like Matlab and Mathematica are important as there is no universal silver programming bullet; one will need interoperable focused environments. We discuss performance analysis including speedup, efficiency, scaled speedup and Amdahl's law. We show how to relate performance to algorithm/application structure and the hardware characteristics. Applications will not get scalable parallelism accidentally but only if there is an understandable performance model. We review some of the many pitfalls for both performance and correctness; these include deadlocks, race conditions, nodes that are busy doing something else and the difficulty of second-guessing automatic parallelization methods. We describe the formal sources of overhead; load imbalance and the communication/control overhead and the ways to reduce them. We relate the blocking used in caching to that used in parallel computation. We note that load balancing was the one part of parallel computing that was easier than expected. We will mix both simple idealized applications with “real problems” noting that usually it is simple problems that are the hardest as they have poor computation to control/communication ratios. We will explain the parallelism in several application classes including for science and engineering: Finite Difference, Finite Elements, FFT/Spectral, Meshes of all sorts, Particle Dynamics, Particle-Mesh, and Monte Carlo methods. Some applications like Image Processing, Graphics, Cryptography and Media coding/decoding have features similar to well understood science and engineering problems. We emphasize that nearly all applications are built hierarchically from more “basic applications” with a variety of different structures and natural programming models. Such application composition (co-ordination, workflow) must be supported with a common run-time. We contrast the difference between decomposition needed in most “basic parallel applications” to the composition supported in workflow and Web 2.0 mashups. Looking at broader application classes we should cover; Internet applications and services; artificial intelligence, optimization, machine learning; divide and conquer algorithms; tree structured searches like computer chess; applications generated by the data deluge including access, search, and the assimilation of data into simulations. There has been a growing interest in Complex Systems whether it be for critical infrastructure like energy and transportation, the Internet itself, commodity gaming, computational epidemiology or the original war games. We expect Discrete Event Simulations (such as DoD’s High Level Architecture HLA) to grow in importance as they naturally describe complex systems and because they can clearly benefit from multicore architectures. We will discuss the innovative Sequential Dynamical Systems approach used in the EpiSims, TransSims and other critical infrastructure simulation environments. In all applications we need to identify the intrinsic parallelism and the degrees of freedom that can be parallelized and distinguish small parallelism (local to core) compared to large parallelism (scalable across many cores) We will not forget many critical non-technical Issues including “Who programs – everybody or just the marine corps?”; “The market for science and engineering was small but it will be large for general multicore”; “The program exists and can’t be changed”; “What features will next hardware/software release support and how should I protect myself from change?” We will summarize lessons and relate them to application and programming model categories. In last lecture or at end of all lectures we encourage the audience to bring their own multicore application or programming model so we can discuss examples that interest you. ©2007 Microsoft Corporation. All rights reserved.

In this paper, we implemented both sequential and parallel version of fractal image compression algorithms using CUDA (Compute Unified Device Architecture) programming model for parallelizing the program in Graphics Processing Unit for medical images, as they are highly similar within the image itself. There are several improvement in the implementation of the algorithm as well. Fractal image compression is based on the self similarity of an image, meaning an image having similarity in majority of the regions. We take this opportunity to implement the compression algorithm and monitor the effect of it using both parallel and sequential implementation. Fractal compression has the property of high compression rate and the dimensionless scheme. Compression scheme for fractal image is of two kind, one is encoding and another is decoding. Encoding is very much computational expensive. On the other hand decoding is less computational. The application of fractal compression to medical images would allow obtaining much higher compression ratios. While the fractal magnification an inseparable feature of the fractal compression would be very useful in presenting the reconstructed image in a highly readable form. However, like all irreversible methods, the fractal compression is connected with the problem of information loss, which is especially troublesome in the medical imaging. A very time consuming encoding pro- cess, which can last even several hours, is another bothersome drawback of the fractal compression.

In this paper, we implemented both sequential and parallel version of fractal image compression algorithms using CUDA (Compute Unified Device Architecture) programming model for parallelizing the program in Graphics Processing Unit for medical images, as they are highly similar within the image itself. There are several improvement in the implementation of the algorithm as well. Fractal image compression is based on the self similarity of an image, meaning an image having similarity in majority of the regions. We take this opportunity to implement the compression algorithm and monitor the effect of it using both parallel and sequential implementation. Fractal compression has the property of high compression rate and the dimensionless scheme. Compression scheme for fractal image is of two kind, one is encoding and another is decoding. Encoding is very much computational expensive. On the other hand decoding is less computational. The application of fractal compression to medical images would allow obtaining much higher compression ratios. While the fractal magnification an inseparable feature of the fractal compression would be very useful in presenting the reconstructed image in a highly readable form. However, like all irreversible methods, the fractal compression is connected with the problem of information loss, which is especially troublesome in the medical imaging. A very time consuming encoding pro- cess, which can last even several hours, is another bothersome drawback of the fractal compression.

Parallel Computing 1988: Volume 6 , Issue Index. Digitized from IA1652917-03 . Previous issue: sim_parallel-computing_1987-11_5_3 . Next issue: sim_parallel-computing_1988-01_6_1 .

Parallel Computing 1988: Volume 7 , Issue Index. Digitized from IA1652917-03 . Previous issue: sim_parallel-computing_1988-03_6_3 . Next issue: sim_parallel-computing_1988-04_7_1 .

Parallel Computing 1988: Volume 7 , Issue Index. Digitized from IA1652917-03 . Previous issue: sim_parallel-computing_1988-03_6_3 . Next issue: sim_parallel-computing_1988-04_7_1 .

We propose a parallel quantum computing mode for ensemble quantum computer. In this mode, some qubits can be in pure states while other qubits in mixed states. It enables a single ensemble quantum computer to perform $"$single-instruction-multi-data" type of parallel computation. In Grover's algorithm and Shor's algorithm, parallel quantum computing can provide additional speedup. In addition, it also makes a fuller use of qubit resources in an ensemble quantum computer. As a result, some qubits discarded in the preparation of an effective pure state in the Schulman-Varizani, and the Cleve-DiVincenzo algorithms can be re-utilized.

Over the past decade there have been significant advances in bringing parallel computing and new database management systems to a wider audience. Through a number of efforts such as the National Strategic Computing Initiative (NSCI), there has been a push to merge these Big Data and Scientific Computing communities to a single computational platform. At the Massachusetts Institute of Technology, Lincoln Laboratory, we have been developing HPC and database technologies to address a number of scientific problems including biomedical processing. In this article, we briefly describe these technologies and how we have used them in the past. We are interested in learning more about the needs of clinical pathologists as we continue to develop these technologies.

Over the past decade there have been significant advances in bringing parallel computing and new database management systems to a wider audience. Through a number of efforts such as the National Strategic Computing Initiative (NSCI), there has been a push to merge these Big Data and Scientific Computing communities to a single computational platform. At the Massachusetts Institute of Technology, Lincoln Laboratory, we have been developing HPC and database technologies to address a number of scientific problems including biomedical processing. In this article, we briefly describe these technologies and how we have used them in the past. We are interested in learning more about the needs of clinical pathologists as we continue to develop these technologies.

The objective of this DURIP project was to acquire an HPC cluster and Mini CAVE to support research involving simultaneous parallel computing and parallel visualization at the University of Kansas (KU). We acquired HPC and visualization infrastructure that form a system for simultaneous parallel computing and parallel visualization research. The HPC cluster is composed of 17 compute nodes with 340 cores, 20 NVIDIA GPGPUs, and 14 Intel Co-Phi processors. The visualization infrastructure is a next generation tiled Mini CAVE for semi immersive visualization. Our infrastructure leverages and extends existing KU infrastructure available to researchers through the KU Advanced Computing Facility. The HPC and visualization system has supported research in five Department of Defense (DoD) mission critical thematic areas involving simultaneous parallel computing and parallel visualization: unstructured meshing, scientific visualization, computational fluid dynamics, elastodynamics, and materials chemistry. The infrastructure has enhanced the research supported by and of interest to the DoD. It has enabled research projects involving: serial and parallel adaptive and moving mesh simulations, scientific visualizations, large eddy simulations of turbulent flows in gas engines, long-range acoustic propagation simulations, numerical modeling of nonlinear nanophotonic devices, and molecular dynamics simulations of solidification.

http://uf.catalog.fcla.edu/uf.jsp?st=UF030003142&ix=pm&I=0&V=D&pm=1

In this grant, we have proposed a three-year research effort focused on developing High Performance Computation and Communication (HPCC) methodologies for structural analysis on parallel processors and clusters of workstations, with emphasis on reducing the structural design cycle time. Besides consolidating and further improving the FETI solver technology to address plate and shell structures, we have proposed to tackle the following design related issues: (a) parallel coupling and assembly of independently designed and analyzed three-dimensional substructures with non-matching interfaces, (b) fast and smart parallel re-analysis of a given structure after it has undergone design modifications, (c) parallel evaluation of sensitivity operators (derivatives) for design optimization, and (d) fast parallel analysis of mildly nonlinear structures. While our proposal was accepted, support was provided only for one year.

In this paper, we propose algorithms and software for parallel implementation of conversion of VLSI Layout Data from CAD format Gerber and MEBES into an internal representation and then into format of automatic mask inspection system T29. The software is developed on basis of OpenMP technology to work on PC with 4-core processor. It is shown that using parallel computing speeds up a conversion process. In the future we are planning to develop similar tools for data conversion from CIF and DFX formats into the internal format, and further into format GDS-II. These tools allow developing a program complex of topological data processing to work with automatic mask inspection system made by R&D Company “KBTEM-OMO” of “Planar” Corporatio

This report summarizes the progress and results of the first of a three-year study whose goal is the use of fault tolerant distributed and parallel processing techniques to decrease the cost and improve the performance and reliability of large, disposable sonar arrays. In this first phase, tasks have concentrated on the study, design, and analysis of the fundamental components in the design and analysis of parallel and distributed computing architectures and algorithms for fault tolerant sonar arrays as well as models for their baseline counterparts. An interactive investigation of the interdependent areas of topology, architecture, protocol, and algorithms has been conducted. The results of this first phase have helped to identify the optimum candidate network topology, architecture, hardware components, and interface protocols for the system architecture and a set of fundamental decomposition techniques and parallel algorithms for a representative set of time domain and frequency domain beamforming methods.

We present a simple way to add an arbitrary equation of state to a automaton gas modelled in the lattice Boltzmann limit. As a way of interpreting the lattice Boltzmann equation we present a new derivation based on an automaton Hamiltonian and the Liouville equation. A convective-gradient term added to the LBE is shown to be a sufficient route for modeling hydrodynamic flow with a general equation of state. The method generalizes to multi-speed gases in three-dimensions.

Summaries of the presentations by invited speakers to this conference, on Vector and Parallel Computing are given along with the author's abstracts of the contributed and student scholarship papers. In all, summaries of 16 papers and the abstracts of 91 other papers are included. Keywords: Vector computing, Parallel programming, Parallel processing, Vectorization, Supercomputers, Parallelization.

The first methods for tolerating more than a small number of worst- case faults in commonly-Lised networks such as the butterfly, the mesh of trees, and other hypercubic networks. Previously, work on the fault-tolerance properties of these networks was limited to showing how to overcome a single fault by adding an extra stage of hardware to the networks.

The supervised relaxation operator combines the information from multiple ancillary data sources with the information from multispectral remote sensing image data and spatial context. Iterative calculation integrate information from the various sources, reaching a balance in consistency between these sources of information. The supervised relaxation operator is shown to produce substantial improvements in classification accuracy compared to the accuracy produced by the conventional maximum likelihood classifier using spectral data only. The convergence property of the supervised relaxation algorithm is also described. Improvement in classification accuracy by means of supervised relaxation comes at a high price in terms of computation. In order to overcome the computation-intensive problem, a distributed/parallel implementation is adopted to take advantage of a high degree of inherent parallelism in the algorithm.

This presentation consists of three presentations related to model- based software synthesis. The first presentation by Professor Janos Sztipanovits of Vanderbilt University provides an overview of the MULTIGRAPH Architecture (MGA), which is used as a generic framework for model-based programming. Evolution of the architecture has been driven by the requirements of specific applications. Characteristics of these application domains and their impact on the basic design of MGA are discussed. In the second presentation, Ben Abbott of Vanderbilt University discusses Model-Based Program Synthesis for Parallel Computing, Automatic program synthesis is one of the prime disciplines that can contribute to the advancement of the software engineering of reactive systems. To illustrate, Mr. Abbott presents a large, high-performance parallel instrumentation system used for analysis of turbine engine strain gauge signals produced during altitude testing. The system is called the Computer Assisted Dynamic Data Analysis and Monitoring System (CADDMAS). In the third presentation Dr. Hubertus Franke of the T.J.Watson Research Center presents Programming Environment for Model-Based Program Synthesis. The development of model-based programming environments is driven by two opposite forces: specialization and Standardization. Mr. Franke's presentation addresses the design and implementation of tools which satisfy both above-mentioned forces. In order to overcome this dilemma, the MULTIGRAPH Architecture uses meta-tools. This presentation focuses on the design and implementation of meta-tools and their coordination to form a harmonic environment.

This presentation consists of three presentations related to model- based software synthesis. The first presentation by Professor Janos Sztipanovits of Vanderbilt University provides an overview of the MULTIGRAPH Architecture (MGA), which is used as a generic framework for model-based programming. Evolution of the architecture has been driven by the requirements of specific applications. Characteristics of these application domains and their impact on the basic design of MGA are discussed. In the second presentation, Ben Abbott of Vanderbilt University discusses Model-Based Program Synthesis for Parallel Computing, Automatic program synthesis is one of the prime disciplines that can contribute to the advancement of the software engineering of reactive systems. To illustrate, Mr. Abbott presents a large, high-performance parallel instrumentation system used for analysis of turbine engine strain gauge signals produced during altitude testing. The system is called the Computer Assisted Dynamic Data Analysis and Monitoring System (CADDMAS). In the third presentation Dr. Hubertus Franke of the T.J.Watson Research Center presents Programming Environment for Model-Based Program Synthesis. The development of model-based programming environments is driven by two opposite forces: specialization and Standardization. Mr. Franke's presentation addresses the design and implementation of tools which satisfy both above-mentioned forces. In order to overcome this dilemma, the MULTIGRAPH Architecture uses meta-tools. This presentation focuses on the design and implementation of meta-tools and their coordination to form a harmonic environment.

This presentation consists of three presentations related to model- based software synthesis. The first presentation by Professor Janos Sztipanovits of Vanderbilt University provides an overview of the MULTIGRAPH Architecture (MGA), which is used as a generic framework for model-based programming. Evolution of the architecture has been driven by the requirements of specific applications. Characteristics of these application domains and their impact on the basic design of MGA are discussed. In the second presentation, Ben Abbott of Vanderbilt University discusses Model-Based Program Synthesis for Parallel Computing, Automatic program synthesis is one of the prime disciplines that can contribute to the advancement of the software engineering of reactive systems. To illustrate, Mr. Abbott presents a large, high-performance parallel instrumentation system used for analysis of turbine engine strain gauge signals produced during altitude testing. The system is called the Computer Assisted Dynamic Data Analysis and Monitoring System (CADDMAS). In the third presentation Dr. Hubertus Franke of the T.J.Watson Research Center presents Programming Environment for Model-Based Program Synthesis. The development of model-based programming environments is driven by two opposite forces: specialization and Standardization. Mr. Franke's presentation addresses the design and implementation of tools which satisfy both above-mentioned forces. In order to overcome this dilemma, the MULTIGRAPH Architecture uses meta-tools. This presentation focuses on the design and implementation of meta-tools and their coordination to form a harmonic environment.

This article is from PLoS ONE , volume 9 . Abstract The metagenomic method directly sequences and analyses genome information from microbial communities. The main computational tasks for metagenomic analyses include taxonomical and functional structure analysis for all genomes in a microbial community (also referred to as a metagenomic sample). With the advancement of Next Generation Sequencing (NGS) techniques, the number of metagenomic samples and the data size for each sample are increasing rapidly. Current metagenomic analysis is both data- and computation- intensive, especially when there are many species in a metagenomic sample, and each has a large number of sequences. As such, metagenomic analyses require extensive computational power. The increasing analytical requirements further augment the challenges for computation analysis. In this work, we have proposed Parallel-META 2.0, a metagenomic analysis software package, to cope with such needs for efficient and fast analyses of taxonomical and functional structures for microbial communities. Parallel-META 2.0 is an extended and improved version of Parallel-META 1.0, which enhances the taxonomical analysis using multiple databases, improves computation efficiency by optimized parallel computing, and supports interactive visualization of results in multiple views. Furthermore, it enables functional analysis for metagenomic samples including short-reads assembly, gene prediction and functional annotation. Therefore, it could provide accurate taxonomical and functional analyses of the metagenomic samples in high-throughput manner and on large scale.

The objective of this paper is to accelerate the existing incremental and parallel subgradient methods for constrained nonsmooth convex optimization to solve the problem of minimizing the sum of nonsmooth, convex functionals over a constraint set in a Hilbert space. Our approach to accelerate the existing algorithms is to develop line search methods that can determine a more adequate step-size than the ones used in the existing methods at each iteration. The proposed line search method can determine step-sizes to satisfy weaker conditions than the ones used in the existing methods. This implies that the incremental and parallel subgradient methods based on the line search approach are generalizations of the existing ones. We show that the whole sequence generated by each of the two proposed methods weakly converges to a solution to the problem. The main contribution of this paper is provision of numerical examples showing that the two proposed methods perform better than the existing ones. The numerical results are evaluated with a multi-core computer and show that our parallel method reduces the running time and iterations needed to find an optimal solution compared with other ones.

The objective of this paper is to accelerate the existing incremental and parallel subgradient methods for constrained nonsmooth convex optimization to solve the problem of minimizing the sum of nonsmooth, convex functionals over a constraint set in a Hilbert space. Our approach to accelerate the existing algorithms is to develop line search methods that can determine a more adequate step-size than the ones used in the existing methods at each iteration. The proposed line search method can determine step-sizes to satisfy weaker conditions than the ones used in the existing methods. This implies that the incremental and parallel subgradient methods based on the line search approach are generalizations of the existing ones. We show that the whole sequence generated by each of the two proposed methods weakly converges to a solution to the problem. The main contribution of this paper is provision of numerical examples showing that the two proposed methods perform better than the existing ones. The numerical results are evaluated with a multi-core computer and show that our parallel method reduces the running time and iterations needed to find an optimal solution compared with other ones.

Parallel Computing 1990: Volume 15 , Issue Index. Digitized from IA1652917-03 . Previous issue: sim_parallel-computing_1990-08_14_3 . Next issue: sim_parallel-computing_1990_15_contents .

The 3D two-photon memory was found to be well suited for very large databases with low write requirements because of its potential capacity. raw bandwidth and random access capability. It was determined that performance can be significantly improved by utilizing and accessing feature termed bi-orthogonal access. a feature which allows both record and field parallel access to be supported simultaneously A metric termed bit retrieval efficiency was devised to evaluate the optimality of relational database operations performed with a bi-orthogonally accessed 3D optical memory. The results of this analysis I indicate that the throughput and the random access capability of a memory both affect the performance of relational database machines. A shortcoming that all massively parallel access optical memories have. in terms of this applications is that an entire page of records has to be retrieved even if only one record is desired. From this point they appear to have reduced random access capabilities. We have focused on the theoretical feasibility of obtaining 2-D photonic bandgap from a two-dimensional honeycomb dielectric lattice structure. with a particular emphasis on the effect of having a finite rod length. We have also started investigating the impact of the inclusion of active device structures (e.g.. VCSELs) into the photonic crystals.

The objective of the program is to develop a Memory Distribution Network (MDM) for parallel access to secondary storage (devices) for high performance computing. The secondary storage was a large page oriented memory. It was to be connected to a smaller array of electronic processors for information retrieval. We have designed and constructed a novel reconfigurable interconnection scheme that allows a 4x4 arrays of 32x32 bits page oriented optical memory be parallel accessed with five different interconnection topologies using wavelength tuning. We have also designed, fabricated and tested the MOSIS chips that would process pages of 32x32 bits information received from the large memory before transferring the processed information to the smaller array of electronic processors. The system was capable of operating at 10 MHz with a memory retrieval throughput of 10(4) Mbits/sec. We have also demonstrated the application of parallel database search on the MDN system.

The objective of the program is to develop a Memory Distribution Network (MDM) for parallel access to secondary storage (devices) for high performance computing. The secondary storage was a large page oriented memory. It was to be connected to a smaller array of electronic processors for information retrieval. We have designed and constructed a novel reconfigurable interconnection scheme that allows a 4x4 arrays of 32x32 bits page oriented optical memory be parallel accessed with five different interconnection topologies using wavelength tuning. We have also designed, fabricated and tested the MOSIS chips that would process pages of 32x32 bits information received from the large memory before transferring the processed information to the smaller array of electronic processors. The system was capable of operating at 10 MHz with a memory retrieval throughput of 10(4) Mbits/sec. We have also demonstrated the application of parallel database search on the MDN system.

SciPy 2010 Day 1 Cloud Track 6 Parallel Computing with IPython: an Application to Air Pollution Modeling B.E. Granger and Josh Hemann

This report summarizes the progress and results of the second of a three-year study whose goal is the use of fault-tolerant distributed and parallel processing techniques to decrease the cost and improve the performance and reliability of large, disposable sonar arrays. In this second phase, tasks have concentrated on the design, development, analysis and evaluation of conventional and basic split-aperture beamforming algorithms. A broad assortment of parallel algorithms and programs for FFT beamformers have been completed, and their performance evaluated via a cluster testbed and via rapid virtual prototyping capabilities derived from new network architecture models. These models include a wide variety of network protocols centered around unidirectional, ring, and bidirectional topologies. In addition, new emphasis has begun with split-aperture conventional beamforming and initial results indicate a significant potential for performance improvement through parallel processing. Finally, the architecture for the hardware prototype has been developed, and work has begun on its construction and that of its software system.

A general consensus is developing that highly parallel computers are the only practical near-term route to the teraflops of computing power required by many scientific and engineering problems. While current conventional supercomputers already utilize low parallelism, highly parallel computers are still in the experimental stage, although they are increasingly visible at the front line of grand challenge computations. While it has been clearly demonstrated that such systems can be built, and are reliable, there has been only limited success with developing appropriate user-level software. The authors survey recent and upcoming developments in the area of parallel supercomputing, focusing on the underlying trends that have proved successful. The paper begins with discussions of how parallel computers are classified; recent or current parallel computer projects worldwide; node design, including the individual computational processor, along with its associated communications hardware and local memory; interprocessor communication facilities; and software for currently available parallel computers. The next section examines in more depth the parallel systems that are currently in development: INTEL iPSC, CMU iWarp, NCUBE2, GF-11, Connection Machine CM-1, AMT DAP, MasPar MP-1, Myrias SPS-2, and the SUPRENUM. The final section discusses the likely next-generation computing environment: a heterogeneous ensemble of scalar, vector, parallel, and graphics computers along with disk farms and network interfaces to users.

A general consensus is developing that highly parallel computers are the only practical near-term route to the teraflops of computing power required by many scientific and engineering problems. While current conventional supercomputers already utilize low parallelism, highly parallel computers are still in the experimental stage, although they are increasingly visible at the front line of grand challenge computations. While it has been clearly demonstrated that such systems can be built, and are reliable, there has been only limited success with developing appropriate user-level software. The authors survey recent and upcoming developments in the area of parallel supercomputing, focusing on the underlying trends that have proved successful. The paper begins with discussions of how parallel computers are classified; recent or current parallel computer projects worldwide; node design, including the individual computational processor, along with its associated communications hardware and local memory; interprocessor communication facilities; and software for currently available parallel computers. The next section examines in more depth the parallel systems that are currently in development: INTEL iPSC, CMU iWarp, NCUBE2, GF-11, Connection Machine CM-1, AMT DAP, MasPar MP-1, Myrias SPS-2, and the SUPRENUM. The final section discusses the likely next-generation computing environment: a heterogeneous ensemble of scalar, vector, parallel, and graphics computers along with disk farms and network interfaces to users.

A general consensus is developing that highly parallel computers are the only practical near-term route to the teraflops of computing power required by many scientific and engineering problems. While current conventional supercomputers already utilize low parallelism, highly parallel computers are still in the experimental stage, although they are increasingly visible at the front line of grand challenge computations. While it has been clearly demonstrated that such systems can be built, and are reliable, there has been only limited success with developing appropriate user-level software. The authors survey recent and upcoming developments in the area of parallel supercomputing, focusing on the underlying trends that have proved successful. The paper begins with discussions of how parallel computers are classified; recent or current parallel computer projects worldwide; node design, including the individual computational processor, along with its associated communications hardware and local memory; interprocessor communication facilities; and software for currently available parallel computers. The next section examines in more depth the parallel systems that are currently in development: INTEL iPSC, CMU iWarp, NCUBE2, GF-11, Connection Machine CM-1, AMT DAP, MasPar MP-1, Myrias SPS-2, and the SUPRENUM. The final section discusses the likely next-generation computing environment: a heterogeneous ensemble of scalar, vector, parallel, and graphics computers along with disk farms and network interfaces to users.

A general consensus is developing that highly parallel computers are the only practical near-term route to the teraflops of computing power required by many scientific and engineering problems. While current conventional supercomputers already utilize low parallelism, highly parallel computers are still in the experimental stage, although they are increasingly visible at the front line of grand challenge computations. While it has been clearly demonstrated that such systems can be built, and are reliable, there has been only limited success with developing appropriate user-level software. The authors survey recent and upcoming developments in the area of parallel supercomputing, focusing on the underlying trends that have proved successful. The paper begins with discussions of how parallel computers are classified; recent or current parallel computer projects worldwide; node design, including the individual computational processor, along with its associated communications hardware and local memory; interprocessor communication facilities; and software for currently available parallel computers. The next section examines in more depth the parallel systems that are currently in development: INTEL iPSC, CMU iWarp, NCUBE2, GF-11, Connection Machine CM-1, AMT DAP, MasPar MP-1, Myrias SPS-2, and the SUPRENUM. The final section discusses the likely next-generation computing environment: a heterogeneous ensemble of scalar, vector, parallel, and graphics computers along with disk farms and network interfaces to users.

A general consensus is developing that highly parallel computers are the only practical near-term route to the teraflops of computing power required by many scientific and engineering problems. While current conventional supercomputers already utilize low parallelism, highly parallel computers are still in the experimental stage, although they are increasingly visible at the front line of grand challenge computations. While it has been clearly demonstrated that such systems can be built, and are reliable, there has been only limited success with developing appropriate user-level software. The authors survey recent and upcoming developments in the area of parallel supercomputing, focusing on the underlying trends that have proved successful. The paper begins with discussions of how parallel computers are classified; recent or current parallel computer projects worldwide; node design, including the individual computational processor, along with its associated communications hardware and local memory; interprocessor communication facilities; and software for currently available parallel computers. The next section examines in more depth the parallel systems that are currently in development: INTEL iPSC, CMU iWarp, NCUBE2, GF-11, Connection Machine CM-1, AMT DAP, MasPar MP-1, Myrias SPS-2, and the SUPRENUM. The final section discusses the likely next-generation computing environment: a heterogeneous ensemble of scalar, vector, parallel, and graphics computers along with disk farms and network interfaces to users.

The idea of upgrading performance and utility of computer systems by incorporating parallel processing has been around since at least the 1940s. A significant investment in parallel processing in the 1980s and 1990s has led to an abundance of parallel architectures that due to technical constraints at the time had to rely on multi-chip multi-processing. Unfortunately, their impact on mainstream computing was quite limited. 'Tribal lore' suggests the following reason: while programming for parallelism tends to be easy, turning this parallelism into the 'coarse-grained' type, needed to optimize performance for multi-chip multi-processing (with their high coordination overhead), has been quite hard. The mainstream computing paradigm has always relied on serial code. However, commercially successful processors are entering a second decade of near stagnation in the maximum number of instructions they can issue towards a single computational task in one clock. Alas, the multi-decade reliance on advancing the clock is also coming to an end. The PRAM-On-Chip research project at UMD is reviewed. The project is guided by a fact, an old insight, a recent one and a premise. The fact: Billion transistor chips are here, up from less than 30,000 circa 1980. The older insight: Using a very simple parallel computation model, the parallel random access model (PRAM), the parallel algorithms research community succeeded in developing a general-purpose theory of parallel algorithms that was much richer than any competing approach and is second in magnitude only to serial algorithmics. However, since it did not offer an effective abstraction for multi-chip multi-processors this elegant algorithmic theory remained in the ivory towers of theorists. The PRAM-On-Chip insight: The Billion transistor chip era allows for the first time low-overhead on-chip multi-processors so that the PRAM abstraction becomes effective. The premise: Were the architecture component of PRAM-On-Chip feasible in the 1990s, its parallel programming component would become the mainstream standard. In 1988-90 standard algorithms textbooks chose to include significant PRAM chapters (some still have them). Arguably nothing could stand in the way of teaching them to every student at every computer science program. Programming for concurrency/parallelism is quickly becoming an integral part of mainstream computing. Yet, industry and academia leaders in system software and general-purpose application software have maintained a passive posture: their attention tends to focus too much on getting the best performance out of architectures that originated from a hardware centric approach, or very specific applications, and too little on trying to impact the evolving generation of multi-core and/or multi-threaded general-purpose architectures. We argue, perhaps provocatively, that: (i) limiting programming for parallelism to fit hardware-centric architectures imports the epidemic of programmability problems that has plagued parallel computing into mainstream computing, (ii) it is only a matter of time until the industry will seek convergence based on parallel programmability: the difference in the bottom line between a successful and a less successful strategy on parallel programmability will be too big to ignore, (iii) a more assertive position by such leaders is necessary, and (iv) the PRAM-On-Chip approach offers a more balanced way that avoids these problems. URL: http://www.umiacs.umd.edu/~vishkin/XMT ©2005 Microsoft Corporation. All rights reserved.

Research in the area of distributed computing systems for digital signal processing applications is described. The work involves the modeling of asynchronous parallel processes and computer systems for executing these processes. The objective of the work is to develop techniques by which the compatibility of an architecture and an algorithm can be evaluated. The three part effort addresses: 1. Modeling of asynchronous parallel computer system architectures; 2. Modeling of asynchronous parallel computational processes; 3. Evaluation of alternative architectures relative to classes of computational the approach to the modeling of parallel processes and architectures is to examined the parallelism in a variety of one- and two-dimensional signal processing tasks. This includes a study of the ways in which different types of digital signal processing tasks can be executed on different types of architectures. The goal is to develop one set of features by which processes can be characterized, and another set of features by which parallel architectures can be characterized: and to use these features to obtain measures for the evaluation of process/architecture compatibility. This research will contribute to the understanding both of how distributed computer systems can be designed for the execution of a class of tasks, and of how signal processing tasks can be decomposed for execution on a distributed computing system. (Author)

A workshop on the complexity of parallel and distributed computation in May 1986. It has 21 speakers and 141 participants; their interests ranged from practical questions about the architecture of parallel and distributed systems to highly theoretical questions about the complexity of parallel computation. A panel discussion was held on the topic 'Bridging the Gap Between the Theory and Practice of Parallel and Distributed Computing'. The main focus of the workshop was a set of mathematical and algorithmic issues that underlie the efficient use of the massively parallel computers that are just beginning to come into use. Several of the lectures were concerned with efficient algorithms for such computers. Other lectures were concerned with the problems of synchronization, load-sharing, and communication between processors in such systems. A third major theme was the reliable operation of such systems in the presence of faulty processors.

As the capabilities of computing machinery grow, so does the diverse variety of their applications. The feasibility of many approaches to these applications depends solely upon the existence of computing machinery capable of performing these tasks within a given time constraint. Because the majority of the available computing machinery is general purpose in nature, tasks that do not require purpose facilities, but that do require high throughput, are condemned to execution on expensive general purpose hardware. This research describes several tasks that require fast computing machinery. These tasks do not require general purpose facilities in the sense that the computing machinery used will only perform a fixed set of tasks. Some of the tasks are simple in nature, but are required to execute on very large data sets. Other tasks are computationally intensive in addition to possibly involving large data sets. Both simple and complex algorithms are considered. The discussion includes a description of the tasks. All of the above tasks are useful; however, their value is determined in part by the time required to perform them. This work discusses three architectures for performing remote sensing tasks. These architectures can execute the described tasks more quickly than conventionally available hardware.

One of the most important computer architecture innovations to appear in the market place during the last ten years is parallel processing on a shared memory multicomputer. This report presents new algorithms for a variety of network models along with empirical analysis on both sequential and parallel computers. An empirical study on the AT and T KORBX system is also presented. This system uses eight processors each of which has vector capability. Keywords: Military airlift applications, One to one shortest path problem, Parallel algorithms.

An O(log sup 2 N) parallel algorithm is presented for computing the eigenvalues of a symmetric tridiagonal matrix using a parallel algorithm for computing the zeros of the characteristic polynomial. The method is based on a quadratic recurrence in which the characteristic polynomial is constructed on a binary tree from polynomials whose degree doubles at each level. Intervals that contain exactly one zero are determined by the zeros of polynomials at the previous level which ensures that different processors compute different zeros. The exact behavior of the polynomials at the interval endpoints is used to eliminate the usual problems induced by finite precision arithmetic.

The capability for massively parallel programming (MPP) using a message passing interface (MPI) has been implemented into a three-dimensional version of the Goddard Cumulus Ensemble (GCE) model. The design for the MPP with MPI uses the concept of maintaining similar code structure between the whole domain as well as the portions after decomposition. Hence the model follows the same integration for single and multiple tasks (CPUs). Also, it provides for minimal changes to the original code, so it is easily modified and/or managed by the model developers and users who have little knowledge of MPP. The entire model domain could be sliced into one- or two-dimensional decomposition with a halo regime, which is overlaid on partial domains. The halo regime requires that no data be fetched across tasks during the computational stage, but it must be updated before the next computational stage through data exchange via MPI. For reproducible purposes, transposing data among tasks is required for spectral transform (Fast Fourier Transform, FFT), which is used in the anelastic version of the model for solving the pressure equation. The performance of the MPI-implemented codes (i.e., the compressible and anelastic versions) was tested on three different computing platforms. The major results are: 1) both versions have speedups of about 99% up to 256 tasks but not for 512 tasks; 2) the anelastic version has better speedup and efficiency because it requires more computations than that of the compressible version; 3) equal or approximately-equal numbers of slices between the x- and y- directions provide the fastest integration due to fewer data exchanges; and 4) one-dimensional slices in the x-direction result in the slowest integration due to the need for more memory relocation for computation.

The capability for massively parallel programming (MPP) using a message passing interface (MPI) has been implemented into a three-dimensional version of the Goddard Cumulus Ensemble (GCE) model. The design for the MPP with MPI uses the concept of maintaining similar code structure between the whole domain as well as the portions after decomposition. Hence the model follows the same integration for single and multiple tasks (CPUs). Also, it provides for minimal changes to the original code, so it is easily modified and/or managed by the model developers and users who have little knowledge of MPP. The entire model domain could be sliced into one- or two-dimensional decomposition with a halo regime, which is overlaid on partial domains. The halo regime requires that no data be fetched across tasks during the computational stage, but it must be updated before the next computational stage through data exchange via MPI. For reproducible purposes, transposing data among tasks is required for spectral transform (Fast Fourier Transform, FFT), which is used in the anelastic version of the model for solving the pressure equation. The performance of the MPI-implemented codes (i.e., the compressible and anelastic versions) was tested on three different computing platforms. The major results are: 1) both versions have speedups of about 99% up to 256 tasks but not for 512 tasks; 2) the anelastic version has better speedup and efficiency because it requires more computations than that of the compressible version; 3) equal or approximately-equal numbers of slices between the x- and y- directions provide the fastest integration due to fewer data exchanges; and 4) one-dimensional slices in the x-direction result in the slowest integration due to the need for more memory relocation for computation.

{\sc top-c} (Task Oriented Parallel C) is a freely available package for parallel computing. It is designed to be easy to learn and to have good tolerance for the high latencies that are common in commodity networks of computers. It has been successfully used in a wide range of examples, providing linear speedup with the number of computers. A brief overview of {\sc top-c} is provided, along with recent experience with cosmic ray physics simulations.

We present a computational algorithm for computing short range forces between particles. The algorithm has two distinguishing features. First, it is optimized for multi-processor computers, and will use as many processors as are available. Second, it is optimized for inhomogeneous, dynamic particle distributions; for any distribution the computational load is distributed evenly to all processors, and the communication time is less than 15\% of the total run time. In this talk we present our new algorithm. We developed the program for a grand-challenge problem in cosmology, simulation of the formation of large-scale structure in the universe. This simulation, run on the Thinking Machines Corporation CM-5, uses the particle-particle/particle-mesh (PPPM) \cite{hock_east} algorithm. The particle-particle phase is computed using the algorithm we describe in this paper. We discuss this and other applications.

The Microsoft .NET Framework 4 and Visual Studio 2010 include new technologies for expressing, debugging, and tuning parallelism in managed applications. Dive into key areas of support, including Parallel Language Integrated Query (PLINQ), cutting-edge concurrency views in the Visual Studio profiler, and debugger tool windows for analyzing the state of concurrent code. In addition to exploring such features, we will examine some common parallel patterns prevalent in technical computing and how these features can be used to best implement such patterns. ©2010 Microsoft Corporation. All rights reserved.

The Microsoft .NET Framework 4 and Visual Studio 2010 include new technologies for expressing, debugging, and tuning parallelism in managed applications. Dive into key areas of support, including Parallel Language Integrated Query (PLINQ), cutting-edge concurrency views in the Visual Studio profiler, and debugger tool windows for analyzing the state of concurrent code. In addition to exploring such features, we will examine some common parallel patterns prevalent in technical computing and how these features can be used to best implement such patterns. ©2010 Microsoft Corporation. All rights reserved.

Scalable Parallel Computing on Many/Multicore Systems This set of lectures will review the application and programming model issues that will one must address when one gets chips with 32-1024 cores and “scalable” approaches will be needed to make good use of such systems. We will not discuss bit-level and instruction-level parallelism i.e. what happens on a possibly special purpose core, even though this is clearly important. We will use science and engineering applications to drive the discussion as we have substantial experience in these cases but we are interested in the lessons for commodity client and server applications that will be broadly used 5-10 years from now. We start with simple applications and algorithms from a variety of fields and identify features that makes it “obvious” that “all” science and engineering run well and scalably in parallel. We explain why unfortunately it is equally obvious that there is no straightforward way of expressing this parallelism. Parallel hardware architectures are described in enough detail to understand performance and algorithm issues and need for cross-architecture compatibility; however in these lectures we will just be users of hardware. We must understand what features of multicore chips we can and should exploit. We can explicitly use the shared memory between cores or just enjoy its implications for very fast inter-core control and communication linkage. We note that parallel algorithm research is hugely successful although this success has reduced activity in an area that deserves new attention for the next generation of architectures. The parallel software environment is discussed at several levels including programming paradigm, runtime and operating system. The importance of libraries, templates, kernels (dwarfs) and benchmarks is stressed. The programming environment has various tools including compilers with possible parallelism hints like OpenMP; tuners like Atlas; messaging models; parallelism and distribution support as in HPF, HPCS Languages, co-array Fortran, Global arrays and UPC. We also discuss the relevance of important general ideas like object-oriented paradigms (as in for example Charm++), functional languages and Software Transactional Memories. Streaming, pipelining, co-ordination, services and workflow are placed in context. Examples discussed in the last category include CCR/DSS from Microsoft and the Common Component Architecture CCA from DoE. Domain Specific environments like Matlab and Mathematica are important as there is no universal silver programming bullet; one will need interoperable focused environments. We discuss performance analysis including speedup, efficiency, scaled speedup and Amdahl's law. We show how to relate performance to algorithm/application structure and the hardware characteristics. Applications will not get scalable parallelism accidentally but only if there is an understandable performance model. We review some of the many pitfalls for both performance and correctness; these include deadlocks, race conditions, nodes that are busy doing something else and the difficulty of second-guessing automatic parallelization methods. We describe the formal sources of overhead; load imbalance and the communication/control overhead and the ways to reduce them. We relate the blocking used in caching to that used in parallel computation. We note that load balancing was the one part of parallel computing that was easier than expected. We will mix both simple idealized applications with “real problems” noting that usually it is simple problems that are the hardest as they have poor computation to control/communication ratios. We will explain the parallelism in several application classes including for science and engineering: Finite Difference, Finite Elements, FFT/Spectral, Meshes of all sorts, Particle Dynamics, Particle-Mesh, and Monte Carlo methods. Some applications like Image Processing, Graphics, Cryptography and Media coding/decoding have features similar to well understood science and engineering problems. We emphasize that nearly all applications are built hierarchically from more “basic applications” with a variety of different structures and natural programming models. Such application composition (co-ordination, workflow) must be supported with a common run-time. We contrast the difference between decomposition needed in most “basic parallel applications” to the composition supported in workflow and Web 2.0 mashups. Looking at broader application classes we should cover; Internet applications and services; artificial intelligence, optimization, machine learning; divide and conquer algorithms; tree structured searches like computer chess; applications generated by the data deluge including access, search, and the assimilation of data into simulations. There has been a growing interest in Complex Systems whether it be for critical infrastructure like energy and transportation, the Internet itself, commodity gaming, computational epidemiology or the original war games. We expect Discrete Event Simulations (such as DoD’s High Level Architecture HLA) to grow in importance as they naturally describe complex systems and because they can clearly benefit from multicore architectures. We will discuss the innovative Sequential Dynamical Systems approach used in the EpiSims, TransSims and other critical infrastructure simulation environments. In all applications we need to identify the intrinsic parallelism and the degrees of freedom that can be parallelized and distinguish small parallelism (local to core) compared to large parallelism (scalable across many cores) We will not forget many critical non-technical Issues including “Who programs – everybody or just the marine corps?”; “The market for science and engineering was small but it will be large for general multicore”; “The program exists and can’t be changed”; “What features will next hardware/software release support and how should I protect myself from change?” We will summarize lessons and relate them to application and programming model categories. In last lecture or at end of all lectures we encourage the audience to bring their own multicore application or programming model so we can discuss examples that interest you. ©2007 Microsoft Corporation. All rights reserved.

The use of object-oriented programming techniques in development of parallel, finite element analysis software enhances software reused and makes application development more efficient. In this report, an object-oriented programming framework for developing parallel finite element software is described. All required steps, from date file parsing and equation solving to post processing and graphical user interfaces, are discussed. After development of the framework, a sample parallel finite element code, namely COMPOSE, is extracted from its original functional programming paradigm and implemented in the new object-oriented programming framework. Besides ease of development, the use of generic visualization and interface tools for software using the framework speeds delivery of research codes to end users.