Downloads & Free Reading Options - Results

This article is from Journal of the American Medical Informatics Association : JAMIA , volume 21 . Abstract Background and objective: Integrating data from multiple sources is a crucial and challenging problem. Even though there exist numerous algorithms for record linkage or deduplication, they suffer from either large time needs or restrictions on the number of datasets that they can integrate. In this paper we report efficient sequential and parallel algorithms for record linkage which handle any number of datasets and outperform previous algorithms. Methods: Our algorithms employ hierarchical clustering algorithms as the basis. A key idea that we use is radix sorting on certain attributes to eliminate identical records before any further processing. Another novel idea is to form a graph that links similar records and find the connected components. Results: Our sequential and parallel algorithms have been tested on a real dataset of 1 083 878 records and synthetic datasets ranging in size from 50 000 to 9 000 000 records. Our sequential algorithm runs at least two times faster, for any dataset, than the previous best-known algorithm, the two-phase algorithm using faster computation of the edit distance (TPA (FCED)). The speedups obtained by our parallel algorithm are almost linear. For example, we get a speedup of 7.5 with 8 cores (residing in a single node), 14.1 with 16 cores (residing in two nodes), and 26.4 with 32 cores (residing in four nodes). Conclusions: We have compared the performance of our sequential algorithm with TPA (FCED) and found that our algorithm outperforms the previous one. The accuracy is the same as that of this previous best-known algorithm.

Using hashing techniques, this paper develops a family of space-efficient Las Vegas randomized algorithms for $k$-SUM problems. This family includes an algorithm that can solve 3-SUM in $O(n^2)$ time and $O(\sqrt{n})$ space. It also establishes a new time-space upper bound for SUBSET-SUM, which can be solved by a Las Vegas algorithm in $O^*(2^{(1-\sqrt{\8/9\beta})n})$ time and $O^*(2^{\beta n})$ space, for any $\beta \in [0, \9/32]$.

We initiate the systematic study of the energy complexity of algorithms (in addition to time and space complexity) based on Landauer's Principle in physics, which gives a lower bound on the amount of energy a system must dissipate if it destroys information. We propose energy-aware variations of three standard models of computation: circuit RAM, word RAM, and transdichotomous RAM. On top of these models, we build familiar high-level primitives such as control logic, memory allocation, and garbage collection with zero energy complexity and only constant-factor overheads in space and time complexity, enabling simple expression of energy-efficient algorithms. We analyze several classic algorithms in our models and develop low-energy variations: comparison sort, insertion sort, counting sort, breadth-first search, Bellman-Ford, Floyd-Warshall, matrix all-pairs shortest paths, AVL trees, binary heaps, and dynamic arrays. We explore the time/space/energy trade-off and develop several general techniques for analyzing algorithms and reducing their energy complexity. These results lay a theoretical foundation for a new field of semi-reversible computing and provide a new framework for the investigation of algorithms.

We propose efficient algorithms for two key tasks in the analysis of large nonuniform networks: uniform node sampling and cluster detection. Our sampling technique is based on augmenting a simple, but slowly mixing uniform MCMC sampler with a regular random walk in order to speed up its convergence; however the combined MCMC chain is then only sampled when it is in its "uniform sampling" mode.Our clustering algorithm determines the relevant neighbourhood of a given node u in the network by first estimating the Fiedler vector of a Dirichlet matrix with u fixed at zero potential, and then finding the neighbourhood of u that yields a minimal weighted Cheeger ratio, where the edge weights are determined by differences in the estimated node potentials. Both of our algorithms are based on local computations, i.e. operations on the full adjacency matrix of the network are not used. The algorithms are evaluated experimentally using three types of nonuniform networks: Dorogovtsev-Goltsev-Mendes "pseudofractal graphs", scientific collaboration networks, and randomised "caveman graphs".

We present efficient algorithms to calculate trajectories for periodic Lorentz gases consisting of square lattices of circular obstacles in two dimensions, and simple cubic lattices of spheres in three dimensions; these become increasingly efficient as the radius of the obstacles tends to 0, the so-called Boltzmann-Grad limit. The 2D algorithm applies continued fractions to obtain the exact disc with which a particle will collide at each step, instead of using periodic boundary conditions as in the classical algorithm. The 3D version incorporates the 2D algorithm by projecting to the three coordinate planes. As an application, we calculate distributions of free path lengths close to the Boltzmann-Grad limit for certain Lorentz gases. We also show how the algorithms may be applied to deal with general crystal lattices.

We present an extension to the time-dependent photo-ionization code C$^2$-Ray to calculate photo-heating in an efficient and accurate way. In C$^2$-Ray, the thermal calculation demands relatively small time-steps for accurate results. We describe two novel methods to reduce the computational cost associated with small time-steps, namely, an adaptive time-step algorithm and an asynchronous evolution approach. The adaptive time-step algorithm determines an optimal time-step for the next computational step. It uses a fast ray-tracing scheme to quickly locate the relevant cells for this determination and only use these cells for the calculation of the time-step. Asynchronous evolution allows different cells to evolve with different time-steps. The asynchronized clocks of the cells are synchronized at the times where outputs are produced. By only evolving cells which may require short time-steps with these short time-steps instead of imposing them to the whole grid, the computational cost of the calculation can be substantially reduced. We show that our methods work well for several cosmologically relevant test problems and validate our results by comparing to the results of another time-dependent photo-ionization code.

The problem of attaining energy efficiency in distributed systems is of importance, but a general, non-domain-specific theory of energy-minimal scheduling is far from developed. In this paper, we classify the problems of energy-minimal scheduling and present theoretical foundations of the same. We derive results concerning energy-minimal scheduling of independent jobs in a distributed system with functionally similar machines with different working and idle power ratings. The machines considered in our system can have identical as well as different speeds. If the jobs can be divided into arbitrary parts, we show that the minimum-energy schedule can be generated in linear time and give exact scheduling algorithms. For the cases where jobs are non-divisible, we prove that the scheduling problems are NP-hard and also give approximation algorithms for the same along with their bounds.

ai how i fail

Finding shortest paths and determining visibilities are problems encountered every day. Formal versions of these problems are important in computational geometry. Both kinds of problems specify a space and a set of opaque, impenetrable obstacles in the space. Visibility problems ask what an observer would see if placed in the space; shortest path problems seek the minimum length route for an object moving among the obstacles. This thesis provides algorithms for several visibility and shortest path problems, illuminating the relationship between the two classes. The first part of the thesis considers problems in which the space is the plane and the obstacles are non-intersecting line segments. It presents a worst-case-optimal algorithm to find the visibility graph of the set of segments; that is, it computes what would be seen by observers standing at all the segment endpoints. It then uses this information to find shortest paths for a non-rotating convex body moving among the segments. The second part of the thesis provides several optimal algorithms for shortest path and visibility problems inside simple polygons that have already been triangulated. In this setting, the polygon walls are the only obstacles. The most basic problem considered is that of finding all shortest paths from a particular vertex to other vertices. The solution to this problem can be applied to solve several visibility problems, including that of finding the visibility graph of a simple polygon in time proportional to its size.

In this study, we describe the algebraic computations required to implement the stochastic finite element method for solving problems in which uncertainty is restricted to right hand side data coming from forcing functions or boundary conditions. We show that the solution can be represented in a compact outer product form which leads to efficiencies in both work and sorage, and we demonstrate that block iterative methods for algebraic systems with multiple right hand sides can be used to advantage to compute this solution. Finally, we examine the behavior of these statistical quantities in one setting derived from a model of acoustic scattering.

We study efficient deterministic executions of parallel algorithms on restartable fail-stop CRCW PRAMs. We allow the PRAM processors to be subject to arbitrary stop failures and restarts, that are determined by on-line adversary, and that result in loss of private memory but do not affect shared memory. For this model, we define and justify the complexity measures of: completed work, where processors are charged for completed fixed-size update cycles, and overhead ratio, which amortizes the work over necessary work and failures. We observe that P = N restartable fail-stop processors, the Write-All problem requires omega(N log N) completed work, and this lower bound holds even under the additional assumption that processors can read and locally process the entire shared memory at unit cost. Under this unrealistic assumption we have a matching upperbound. The lower bound also applies to the expected completed work of randomized algorithms that are subject to on line adversaries. Finally, we describe a simple on-line adversary that causes inefficiency in may randomized algorithms.

In this paper we consider the wavelet synopsis construction problem without the restriction that we only choose a subset of coefficients of the original data. We provide the first near optimal algorithm. We arrive at the above algorithm by considering space efficient algorithms for the restricted version of the problem. In this context we improve previous algorithms by almost a linear factor and reduce the required space to almost linear. Our techniques also extend to histogram construction, and improve the space-running time tradeoffs for V-Opt and range query histograms. We believe the idea applies to a broad range of dynamic programs and demonstrate it by showing improvements in a knapsack-like setting seen in construction of Extended Wavelets.

Cloud computing is a technology that provides a platform for the sharing of resources such as software, infrastructure, application and other information. It brings a revolution in Information Technology industry by offering on-demand of resources. Clouds are basically virtualized datacenters and applications offered as services. Data center hosts hundreds or thousands of servers which comprised of software and hardware to respond the client request. A large amount of energy requires to perform the operation.. Cloud Computing is facing lot of challenges like Security of Data, Consumption of energy, Server Consolidation, etc. The research work focuses on the study of task scheduling management in a cloud environment. The main goal is to improve the performance (resource utilization and redeem the consumption of energy) in data centers. Energy-efficient scheduling of workloads helps to redeem the consumption of energy in data centers, thus helps in better usage of resource. This is further reducing operational costs and provides benefits to the clients and also to cloud service provider. In this abstract of paper, the task scheduling in data centers have been compared. Cloudsim a toolkit for modeling and simulation of cloud computing environment has been used to implement and demonstrate the experimental results. The results aimed at analyzing the energy consumed in data centers and shows that by having reduce the consumption of energy the cloud productivity can be improved.

Cloud computing is a technology that provides a platform for the sharing of resources such as software, infrastructure, application and other information. It brings a revolution in Information Technology industry by offering on-demand of resources. Clouds are basically virtualized datacenters and applications offered as services. Data center hosts hundreds or thousands of servers which comprised of software and hardware to respond the client request. A large amount of energy requires to perform the operation.. Cloud Computing is facing lot of challenges like Security of Data, Consumption of energy, Server Consolidation, etc. The research work focuses on the study of task scheduling management in a cloud environment. The main goal is to improve the performance (resource utilization and redeem the consumption of energy) in data centers. Energy-efficient scheduling of workloads helps to redeem the consumption of energy in data centers, thus helps in better usage of resource. This is further reducing operational costs and provides benefits to the clients and also to cloud service provider. In this abstract of paper, the task scheduling in data centers have been compared. Cloudsim a toolkit for modeling and simulation of cloud computing environment has been used to implement and demonstrate the experimental results. The results aimed at analyzing the energy consumed in data centers and shows that by having reduce the consumption of energy the cloud productivity can be improved.

The research efforts supported by AFOSR Grant AFOSR-84-0381 were directed towards development and analysis of robust estimation techniques for autoregressive (AR) and autoregressive-moving average (ARMA) models. Work on related system theoretic problems associated with parameter estimation problems for times series models and on square-root filtering for least squares state estimation applications was also carried out. Finally, an adaptive estimation technique for a class of piecewise (in time) stationary signals was developed. The motivation for our research arises from applications in signal processing including linear predictive singal modeling, signal detection, dynamic state estimation (Kalman filtering), and spectral analysis. The general goal of this research has been to put together ideas and techniques from statistics, signal processing, and system theory to bring new perspectives to such problems. Our research on various autoregressive modeling problems resulted from a desire to relax some of the assumptions made by previous researchers, in order to broaden the domain of application of the basic technique which has proved to be useful in a range of signal processing tasks. In particular, our efforts have been directed at the goal of obtaining allowing robust estimates in the presence of outliers in the observed signal and in modeling of signals whose spectral characteristics change abruptly from time to time.

The proposed greater reliance upon airlifting military forces demands that cargo loading time be minimized while utilization of aircraft cargo compartment space is maximized. Two loading algorithms have been developed with these goals in mind - a two dimensional one for loading cargo where all items must be placed on the floor, and a three-dimensional one for cargo which can be stacked. The three-dimensional algorithm consists of the two-dimensional algorithm and a special stacking algorithm, all designed for either hand calculations or computer calculations.

Consider a matroid where each element has a real-valued cost and a color, red or green; a base is sought that contains q red elements and has smallest possible cost An algorithm for the problem in general matroids is presented, along with a number of variations. its efficiency is demonstrated by implementations on specific matroids. In all cases but one, the running time matches the best-known algorithm for the problem without the red element constraint. On graphic matroids, a smallest spanning tree with q red edges can be found in time O(n log n) more than what is needed to find a minimum spanning tree. A special case is finding a smallest spanning tree with a degree constraint; here the time is only O(m+n) more than that needed to find one minimum spanning tree. On transversal and matching matroids, the time is the same as the best-known algorithms for a minimum cost base. This also holds for transversal matroids for convex graphs, which model a scheduling problem on unit-length jobs with release times and deadlines. On partition matroids, a linear-time algorithm is presented. Finally an algorithm related to our general approach finds a smallest spanning tree on a directed graph, where the given root has a degree constraint. Again the time matches the best-known algorithm for the problem without the red element (i.e., degree) constraint.

8 p. 28 cm

State of the art software has been created for the nonparametric estimation and contour display of densities of dimensions through five. A nonparametric mode finding algorithm has been developed for densities of dimension up to one hundred. An algorithm for data based simulation from high dimensional distributions has been created and employed on behind armour data. In the related area of parameter estimation of stochastic processes, simulation based algorithms have been developed which enable the user to proceed directly from the axioms of the process plus data to the estimation of parameters. Additional keywords: histograms, maximum likelihood estimation, computerized simulation, stochastic processes, remote sensing, army research. (Author)

We propose two new and enhanced algorithms for greedy sampling of high-dimensional functions. While the techniques have a substantial degree of generality, we frame the discussion in the context of methods for empirical interpolation and the development of reduced basis techniques for high-dimensional parametrized functions. The first algorithm, based on a assumption of saturation of error in the greedy algorithm, is shown to result in a significant reduction of the workload over the standard greedy algorithm. In an improved approach, this is combined with an algorithm in which the train set for the greedy approach is adaptively sparsefied and enriched. A safety check step is added at the end of the algorithm to certify the quality of the basis set. Both these techniques are applicable to high-dimensional problems and we shall demonstrate their performance on a number of numerical examples.

Knowing the distribution of the sizes of traffic flows passing through a network link helps a network operator to characterize network resource usage, infer traffic demands, detect traffic anomalies, and accommodate new traffic demands through better traffic engineering. Previous work on estimating the flow size distribution has been focused on making inferences from sampled network traffic. Its accuracy is limited by the (typically) low sampling rate required to make the sampling operation affordable. In this paper we present a novel data streaming algorithm to provide much more accurate estimates of flow distribution, using a “lossy data structure” which consists of an array of counters fitted well into SRAM. For each incoming packet, our algorithm only needs to increment one underlying counter, making the algorithm fast enough even for 40 Gbps (OC-768) links. The data structure is lossy in the sense that sizes of multiple flows may collide into the same counter. Our algorithm uses Bayesian statistical methods such as Expectation Maximization to infer the most likely flow size distribution that results in the observed counter values after collision. Evaluations of this algorithm on large Internet traces obtained from several sources (including a tier-1 ISP) demonstrate that it has very high measurement accuracy (within 2%). Our algorithm not only dramatically improves the accuracy of flow distribution measurement, but also contributes to the field of data streaming by formalizing an existing methodology and applying it to the context of estimating the flow-distribution. ©2004 Microsoft Corporation. All rights reserved.

Knowing the distribution of the sizes of traffic flows passing through a network link helps a network operator to characterize network resource usage, infer traffic demands, detect traffic anomalies, and accommodate new traffic demands through better traffic engineering. Previous work on estimating the flow size distribution has been focused on making inferences from sampled network traffic. Its accuracy is limited by the (typically) low sampling rate required to make the sampling operation affordable. In this paper we present a novel data streaming algorithm to provide much more accurate estimates of flow distribution, using a “lossy data structure” which consists of an array of counters fitted well into SRAM. For each incoming packet, our algorithm only needs to increment one underlying counter, making the algorithm fast enough even for 40 Gbps (OC-768) links. The data structure is lossy in the sense that sizes of multiple flows may collide into the same counter. Our algorithm uses Bayesian statistical methods such as Expectation Maximization to infer the most likely flow size distribution that results in the observed counter values after collision. Evaluations of this algorithm on large Internet traces obtained from several sources (including a tier-1 ISP) demonstrate that it has very high measurement accuracy (within 2%). Our algorithm not only dramatically improves the accuracy of flow distribution measurement, but also contributes to the field of data streaming by formalizing an existing methodology and applying it to the context of estimating the flow-distribution. ©2004 Microsoft Corporation. All rights reserved.

http://uf.catalog.fcla.edu/uf.jsp?st=UF030545002&ix=pm&I=0&V=D&pm=1

Let $G$ be a finite abelian group $G$ with $N$ elements. In this paper we give a O(N) time algorithm for computing a basis of $G$. Furthermore, we obtain an algorithm for computing a basis from a generating system of $G$ with $M$ elements having time complexity $O(M\sum_{p|N} e(p)\lceil p^{1/2}\rceil^{\mu(p)})$, where $p$ runs over all the prime divisors of $N$, and $p^{e(p)}$, $\mu(p)$ are the exponent and the number of cyclic groups which are direct factors of the $p$-primary component of $G$, respectively. In case where $G$ is a cyclic group having a generating system with $M$ elements, a $O(MN^{\epsilon})$ time algorithm for the computation of a basis of $G$ is obtained.

This article is from BMC Medical Informatics and Decision Making , volume 12 . Abstract Background: Recent large scale deployments of health information technology have created opportunities for the integration of patient medical records with disparate public health, human service, and educational databases to provide comprehensive information related to health and development. Data integration techniques, which identify records belonging to the same individual that reside in multiple data sets, are essential to these efforts. Several algorithms have been proposed in the literatures that are adept in integrating records from two different datasets. Our algorithms are aimed at integrating multiple (in particular more than two) datasets efficiently. Methods: Hierarchical clustering based solutions are used to integrate multiple (in particular more than two) datasets. Edit distance is used as the basic distance calculation, while distance calculation of common input errors is also studied. Several techniques have been applied to improve the algorithms in terms of both time and space: 1) Partial Construction of the Dendrogram (PCD) that ignores the level above the threshold; 2) Ignoring the Dendrogram Structure (IDS); 3) Faster Computation of the Edit Distance (FCED) that predicts the distance with the threshold by upper bounds on edit distance; and 4) A pre-processing blocking phase that limits dynamic computation within each block. Results: We have experimentally validated our algorithms on large simulated as well as real data. Accuracy and completeness are defined stringently to show the performance of our algorithms. In addition, we employ a four-category analysis. Comparison with FEBRL shows the robustness of our approach. Conclusions: In the experiments we conducted, the accuracy we observed exceeded 90% for the simulated data in most cases. 97.7% and 98.1% accuracy were achieved for the constant and proportional threshold, respectively, in a real dataset of 1,083,878 records.

Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm performs well in experiments, but little is known about its theoretical properties. Implementing expected improvement requires a choice of Gaussian process prior, which determines an associated space of functions, its reproducing-kernel Hilbert space (RKHS). When the prior is fixed, expected improvement is known to converge on the minimum of any function in the RKHS. We begin by providing convergence rates for this procedure. The rates are optimal for functions of low smoothness, and we modify the algorithm to attain optimal rates for smoother functions. For practitioners, however, these results are somewhat misleading. Priors are typically not held fixed, but depend on parameters estimated from the data. For standard estimators, we show this procedure may never discover the minimum of f. We then propose alternative estimators, chosen to minimize the constants in the rate of convergence, and show these estimators retain the convergence rates of a fixed prior.

We consider the design of efficient algorithms for a multicore computing environment with a global shared memory and p cores, each having a cache of size M, and with data organized in blocks of size B. We characterize the class of `Hierarchical Balanced Parallel (HBP)' multithreaded computations for multicores. HBP computations are similar to the hierarchical divide & conquer algorithms considered in recent work, but have some additional features that guarantee good performance even when accounting for the cache misses due to false sharing. Most of our HBP algorithms are derived from known cache-oblivious algorithms with high parallelism, however we incorporate new techniques that reduce the effect of false-sharing. Our approach to addressing false sharing costs (or more generally, block misses) is to ensure that any task that can be stolen shares O(1) blocks with other tasks. We use a gapping technique for computations that have larger than O(1) block sharing. We also incorporate the property of limited access writes analyzed in a companion paper, and we bound the cost of accessing shared blocks on the execution stacks of tasks. We present the Priority Work Stealing (PWS) scheduler, and we establish that, given a sufficiently `tall' cache, PWS deterministically schedules several highly parallel HBP algorithms, including those for scans, matrix computations and FFT, with cache misses bounded by the sequential complexity, when accounting for both traditional cache misses and for false sharing. We also present a list ranking algorithm with almost optimal bounds. PWS schedules without using cache or block size information, and uses knowledge of processors only to the extent of determining the available locations from which tasks may be stolen; thus it schedules resource-obliviously.

Virtual Content Delivery Network (vCDN) migration is necessary to optimize the use of resources and improve the performance of the overall SDN/NFV-based CDN function in terms of network operator cost reduction and high streaming quality. It requires intelligent and enticed joint SDN/NFV migration algorithms due to the evident huge amount of traffic to be delivered to end customers of the network. In this paper, two approaches for finding the optimal and near optimal path placement(s) and vCDN migration(s) are proposed (OPAC and HPAC). Moreover, several scenarios are considered to quantify the OPAC and HPAC behaviors and to compare their efficiency in terms of migration cost, migration time, vCDN replication number, and other cost factors. Then, they are implemented and evaluated under different network scales. Finally, the proposed algorithms are integrated in an SDN/NFV framework. Index Terms: vCDN; SDN/NFV Optimization; Migration Algorithms; Scalability Algorithms.

In this study, energy-efficient deterministic adaptive beamforming algorithms are proposed for distributed sensor/relay networks. Specifically, DBSA, D-QESA, D-QESA-E, and a hybrid algorithm, hybrid-QESA, that combines the benefits of both deterministic and random adaptive beamforming algorithms, are proposed. Rigorous convergence analyses are provided for all our proposed algorithms and convergence to the global optimal solution is shown for all our proposed algorithms. Through extensive numerical simulations, we demonstrate that superior performance is achieved by our proposed DBSA and D-QESA over random adaptive beamforming algorithms for static channels. Surprisingly, D-QESA is also more robust against random node removal than random adaptive beamforming algorithms. For time-varying channels, hybrid-QESA indeed achieves the best performance since it combines the benefits of both types of adaptive beamforming algorithms. In summary, our proposed deterministic algorithms demonstrate superior performance both in terms of convergence time and robustness against channel and network uncertainties.

The majority of research on efficient and scalable algorithms in computational science and engineering has focused on the forward problem: given parameter inputs, solve the governing equations to determine output quantities of interest. In contrast, here we consider the broader question: given a (large-scale) model containing uncertain parameters, (possibly) noisy observational data, and a prediction quantity of interest, how do we construct efficient and scalable algorithms to (1) infer the model parameters from the data (the deterministic inverse problem), (2) quantify the uncertainty in the inferred parameters (the Bayesian inference problem), and (3) propagate the resulting uncertain parameters through the model to issue predictions with quantified uncertainties (the forward uncertainty propagation problem)? We present efficient and scalable algorithms for this end-to-end, data-to-prediction process under the Gaussian approximation and in the context of modeling the flow of the Antarctic ice sheet and its effect on sea level. The ice is modeled as a viscous, incompressible, creeping, shear-thinning fluid. The observational data come from InSAR satellite measurements of surface ice flow velocity, and the uncertain parameter field to be inferred is the basal sliding parameter. The prediction quantity of interest is the present-day ice mass flux from the Antarctic continent to the ocean. We show that the work required for executing this data-to-prediction process is independent of the state dimension, parameter dimension, data dimension, and number of processor cores. The key to achieving this dimension independence is to exploit the fact that the observational data typically provide only sparse information on model parameters. This property can be exploited to construct a low rank approximation of the linearized parameter-to-observable map.

There is a large literature devoted to the problem of finding an optimal (min-cost) prefix-free code with an unequal letter-cost encoding alphabet of size. While there is no known polynomial time algorithm for solving it optimally there are many good heuristics that all provide additive errors to optimal. The additive error in these algorithms usually depends linearly upon the largest encoding letter size. This paper was motivated by the problem of finding optimal codes when the encoding alphabet is infinite. Because the largest letter cost is infinite, the previous analyses could give infinite error bounds. We provide a new algorithm that works with infinite encoding alphabets. When restricted to the finite alphabet case, our algorithm often provides better error bounds than the best previous ones known.

In this paper we show that certain special cases of the hidden subgroup problem can be solved in polynomial time by a quantum algorithm. These special cases involve finding hidden normal subgroups of solvable groups and permutation groups, finding hidden subgroups of groups with small commutator subgroup and of groups admitting an elementary Abelian normal 2-subgroup of small index or with cyclic factor group.

We present here a new algorithm for the fast computation of N-point correlation functions in large astronomical data sets. The algorithm is based on kdtrees which are decorated with cached sufficient statistics thus allowing for orders of magnitude speed-ups over the naive non-tree-based implementation of correlation functions. We further discuss the use of controlled approximations within the computation which allows for further acceleration. In summary, our algorithm now makes it possible to compute exact, all-pairs, measurements of the 2, 3 and 4-point correlation functions for cosmological data sets like the Sloan Digital Sky Survey (SDSS; York et al. 2000) and the next generation of Cosmic Microwave Background experiments (see Szapudi et al. 2000).

Complex queries are becoming commonplace, with the growing use of decision support systems. These complex queries often have a lot of common sub-expressions, either within a single query, or across multiple such queries run as a batch. Multi-query optimization aims at exploiting common sub-expressions to reduce evaluation cost. Multi-query optimization has hither-to been viewed as impractical, since earlier algorithms were exhaustive, and explore a doubly exponential search space. In this paper we demonstrate that multi-query optimization using heuristics is practical, and provides significant benefits. We propose three cost-based heuristic algorithms: Volcano-SH and Volcano-RU, which are based on simple modifications to the Volcano search strategy, and a greedy heuristic. Our greedy heuristic incorporates novel optimizations that improve efficiency greatly. Our algorithms are designed to be easily added to existing optimizers. We present a performance study comparing the algorithms, using workloads consisting of queries from the TPC-D benchmark. The study shows that our algorithms provide significant benefits over traditional optimization, at a very acceptable overhead in optimization time.

Quantitative games are two-player zero-sum games played on directed weighted graphs. Total-payoff games (that can be seen as a refinement of the well-studied mean-payoff games) are the variant where the payoff of a play is computed as the sum of the weights. Our aim is to describe the first pseudo-polynomial time algorithm for total-payoff games in the presence of arbitrary weights. It consists of a non-trivial application of the value iteration paradigm. Indeed, it requires to study, as a milestone, a refinement of these games, called min-cost reachability games, where we add a reachability objective to one of the players. For these games, we give an efficient value iteration algorithm to compute the values and optimal strategies (when they exist), that runs in pseudo-polynomial time. We also propose heuristics allowing one to possibly speed up the computations in both cases.

We study the following generalized matrix rank estimation problem: given an $n \times n$ matrix and a constant $c \geq 0$, estimate the number of eigenvalues that are greater than $c$. In the distributed setting, the matrix of interest is the sum of $m$ matrices held by separate machines. We show that any deterministic algorithm solving this problem must communicate $\Omega(n^2)$ bits, which is order-equivalent to transmitting the whole matrix. In contrast, we propose a randomized algorithm that communicates only $\widetilde O(n)$ bits. The upper bound is matched by an $\Omega(n)$ lower bound on the randomized communication complexity. We demonstrate the practical effectiveness of the proposed algorithm with some numerical experiments.

Network applications, such as multimedia streaming and video conferencing, impose growing requirements over Quality of Service (QoS), including bandwidth, delay, jitter, etc. Meanwhile, networks are expected to be load-balanced, energy-efficient, and resilient to some degree of failures. It is observed that the above requirements could be better met with multiple disjoint QoS paths than a single one. Let $G=(V,\, E)$ be a digraph with nonnegative integral cost and delay on every edge, $s,\, t\in V$ be two specified vertices, and $D\in\mathbb{Z}_{0}^{+}$ be a delay bound (or some other constraint), the \emph{$k$ Disjoint Restricted Shortest Path} ($k$\emph{RSP})\emph{ Problem} is computing $k$ disjoint paths between $s$ and $t$ with total cost minimized and total delay bounded by $D$. Few efficient algorithms have been developed because of the hardness of the problem. In this paper, we propose efficient algorithms with provable performance guarantees for the $k$RSP problem. We first present a pseudo-polynomial-time approximation algorithm with a bifactor approximation ratio of $(1,\,2)$, then improve the algorithm to polynomial time with a bifactor ratio of $(1+\epsilon,\,2+\epsilon)$ for any fixed $\epsilon>0$, which is better than the current best approximation ratio $(O(1+\gamma),\, O(1+\frac{1}{\gamma})\})$ for any fixed $\gamma>0$ \cite{orda2004efficient}. To the best of our knowledge, this is the first constant-factor algorithm that almost strictly obeys the constraint for the $k$RSP problem.

We describe two implementations of the optimal error correction algorithm known as the maximum likelihood decoder (MLD) for the 2D surface code with a noiseless syndrome extraction. First, we show how to implement MLD exactly in time $O(n^2)$, where $n$ is the number of code qubits. Our implementation uses a reduction from MLD to simulation of matchgate quantum circuits. This reduction however requires a special noise model with independent bit-flip and phase-flip errors. Secondly, we show how to implement MLD approximately for more general noise models using matrix product states (MPS). Our implementation has running time $O(n\chi^3)$ where $\chi$ is a parameter that controls the approximation precision. The key step of our algorithm, borrowed from the DMRG method, is a subroutine for contracting a tensor network on the two-dimensional grid. The subroutine uses MPS with a bond dimension $\chi$ to approximate the sequence of tensors arising in the course of contraction. We benchmark the MPS-based decoder against the standard minimum weight matching decoder observing a significant reduction of the logical error probability for $\chi\ge 4$.

Computer implementation of sophisticated algorithms for physics applications is greatly facilitated by the new generation of component-oriented SD technologies pioneered by Wirth's Oberon and embraced by the software industry (Sun's Java and Microsoft's C#). In a stark contrast with the deeply flawed C++, the Oberon technologies (specifically, the Component Pascal and the BlackBox development environment) offer an unparalleled SD platform for building scientific applications from sophisticated data processing to demanding symbolic calculations. Specific examples are discussed.

Cross-validation is a useful and generally applicable technique often employed in machine learning, including decision tree induction. An important disadvantage of straightforward implementation of the technique is its computational overhead. In this paper we show that, for decision trees, the computational overhead of cross-validation can be reduced significantly by integrating the cross-validation with the normal decision tree induction process. We discuss how existing decision tree algorithms can be adapted to this aim, and provide an analysis of the speedups these adaptations may yield. The analysis is supported by experimental results.

The k-center problem is one of several classic NP-hard clustering questions. For contemporary massive data sets, RAM-based algorithms become impractical. And although there exist good sequential algorithms for k-center, they are not easily parallelizable. In this paper, we design and implement parallel approximation algorithms for this problem. We observe that Gonzalez's greedy algorithm can be efficiently parallelized in several MapReduce rounds; in practice, we find that two rounds are sufficient, leading to a 4-approximation. We contrast this with an existing parallel algorithm for k-center that runs in a constant number of rounds, and offers a 10-approximation. In depth runtime analysis reveals that this scheme is often slow, and that its sampling procedure only runs if k is sufficiently small, relative to the input size. To trade off runtime for approximation guarantee, we parameterize this sampling algorithm, and find in our experiments that the algorithm is not only faster, but sometimes more effective. Yet the parallel version of Gonzalez is about 100 times faster than both its sequential version and the parallel sampling algorithm, barely compromising solution quality.

The Generalized Traveling Salesman Problem (GTSP) is a well-known combinatorial optimization problem with a host of applications. It is an extension of the Traveling Salesman Problem (TSP) where the set of cities is partitioned into so-called clusters, and the salesman has to visit every cluster exactly once. While the GTSP is a very important combinatorial optimization problem and is well studied in many aspects, the local search algorithms used in the literature are mostly basic adaptations of simple TSP heuristics. Hence, a thorough and deep research of the neighborhoods and local search algorithms specific to the GTSP is required. We formalize the procedure of adaptation of a TSP neighborhood for the GTSP and classify all other existing and some new GTSP neighborhoods. For every neighborhood, we provide efficient exploration algorithms that are often significantly faster than the ones known from the literature. Finally, we compare different local search implementations empirically.

Recently, great efforts have been dedicated to researches on the management of large scale graph based data such as WWW, social networks, biological networks. In the study of graph based data management, node disjoint subgraph homeomorphism relation between graphs is more suitable than (sub)graph isomorphism in many cases, especially in those cases that node skipping and node mismatching are allowed. However, no efficient node disjoint subgraph homeomorphism determination (ndSHD) algorithms have been available. In this paper, we propose two computationally efficient ndSHD algorithms based on state spaces searching with backtracking, which employ many heuristics to prune the search spaces. Experimental results on synthetic data sets show that the proposed algorithms are efficient, require relative little time in most of the testing cases, can scale to large or dense graphs, and can accommodate to more complex fuzzy matching cases.

This paper continues the 2012 STACS contribution by Diekert, Ushakov, and the author. We extend the results published in the proceedings in two ways. First, we show that the data structure of power circuits can be generalized to work with arbitrary bases q>=2. This results in a data structure that can hold huge integers, arising by iteratively forming powers of q. We show that the properties of power circuits known for q=2 translate to the general case. This generalization is non-trivial and additional techniques are required to preserve the time bounds of arithmetic operations that were shown for the case q=2. The extended power circuit model permits us to conduct operations in the Baumslag-Solitar group BS(1,q) as efficiently as in BS(1,2). This allows us to solve the word problem in the generalization H_4(1,q) of Higman's group, which is an amalgamated product of four copies of the Baumslag-Solitar group BS(1,q) rather than BS(1,2) in the original form. As a second result, we allow arbitrary numbers f>=4 of copies of BS(1,q), leading to an even more generalized notion of Higman groups H_f(1,q). We prove that the word problem of the latter can still be solved within the O(n^6) time bound that was shown for H_4(1,2).

Applications involving telecommunication call data records, web pages, online transactions, medical records, stock markets, climate warning systems, etc., necessitate efficient management and processing of such massively exponential amount of data from diverse sources. De-duplication or Intelligent Compression in streaming scenarios for approximate identification and elimination of duplicates from such unbounded data stream is a greater challenge given the real-time nature of data arrival. Stable Bloom Filters (SBF) addresses this problem to a certain extent. . In this work, we present several novel algorithms for the problem of approximate detection of duplicates in data streams. We propose the Reservoir Sampling based Bloom Filter (RSBF) combining the working principle of reservoir sampling and Bloom Filters. We also present variants of the novel Biased Sampling based Bloom Filter (BSBF) based on biased sampling concepts. We also propose a randomized load balanced variant of the sampling Bloom Filter approach to efficiently tackle the duplicate detection. In this work, we thus provide a generic framework for de-duplication using Bloom Filters. Using detailed theoretical analysis we prove analytical bounds on the false positive rate, false negative rate and convergence rate of the proposed structures. We exhibit that our models clearly outperform the existing methods. We also demonstrate empirical analysis of the structures using real-world datasets (3 million records) and also with synthetic datasets (1 billion records) capturing various input distributions.

We introduce a new programming language for expressing reversibility, Energy-Efficient Language (Eel), geared toward algorithm design and implementation. Eel is the first language to take advantage of a partially reversible computation model, where programs can be composed of both reversible and irreversible operations. In this model, irreversible operations cost energy for every bit of information created or destroyed. To handle programs of varying degrees of reversibility, Eel supports a log stack to automatically trade energy costs for space costs, and introduces many powerful control logic operators including protected conditional, general conditional, protected loops, and general loops. In this paper, we present the design and compiler for the three language levels of Eel along with an interpreter to simulate and annotate incurred energy costs of a program.

In phylogenetics, a central problem is to infer the evolutionary relationships between a set of species $X$; these relationships are often depicted via a phylogenetic tree -- a tree having its leaves univocally labeled by elements of $X$ and without degree-2 nodes -- called the "species tree". One common approach for reconstructing a species tree consists in first constructing several phylogenetic trees from primary data (e.g. DNA sequences originating from some species in $X$), and then constructing a single phylogenetic tree maximizing the "concordance" with the input trees. The so-obtained tree is our estimation of the species tree and, when the input trees are defined on overlapping -- but not identical -- sets of labels, is called "supertree". In this paper, we focus on two problems that are central when combining phylogenetic trees into a supertree: the compatibility and the strict compatibility problems for unrooted phylogenetic trees. These problems are strongly related, respectively, to the notions of "containing as a minor" and "containing as a topological minor" in the graph community. Both problems are known to be fixed-parameter tractable in the number of input trees $k$, by using their expressibility in Monadic Second Order Logic and a reduction to graphs of bounded treewidth. Motivated by the fact that the dependency on $k$ of these algorithms is prohibitively large, we give the first explicit dynamic programming algorithms for solving these problems, both running in time $2^{O(k^2)} \cdot n$, where $n$ is the total size of the input.

In the area of network performance and discovery, network tomography focuses on reconstructing network properties using only end-to-end measurements at the application layer. One challenging problem in network tomography is reconstructing available bandwidth along all links during multiple source/multiple destination transmissions. The traditional measurement procedures used for bandwidth tomography are extremely time consuming. We propose a novel solution to this problem. Our method counts the fragments exchanged during a BitTorrent broadcast. While this measurement has a high level of randomness, it can be obtained very efficiently, and aggregated into a reliable metric. This data is then analyzed with state-of-the-art algorithms, which reliably reconstruct logical clusters of nodes inter-connected by high bandwidth, as well as bottlenecks between these logical clusters. Our experiments demonstrate that the proposed two-phase approach efficiently solves the presented problem for a number of settings on a complex grid infrastructure.

16 p. 28 cm

16 p. 28 cm