Downloads & Free Reading Options - Results

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.

One of the classical problems in machine learning and data mining is feature selection. A feature selection algorithm is expected to be quick, and at the same time it should show high performance. MeLiF algorithm effectively solves this problem using ensembles of ranking filters. This article describes two different ways to improve MeLiF algorithm performance with parallelization. Experiments show that proposed schemes significantly improves algorithm performance and increase feature selection quality.

The Ethiopian Commodity Exchange (ECX) faces significant challenges, including manual trading processes, market inefficiencies, and data fragmentation, which hinder its ability to operate effectively in a volatile and dynamic environment. This research develops an intelligent trading model leveraging Deep Reinforcement Learning (DRL) algorithms, specifically Deep Q-Networks (DQN), Double Deep Q-Networks (DDQN), and Advantage Actor-Critic (A2C), to address these issues. The proposed framework utilizes DRL to enable agents to learn optimal trading policies through interactions with simulated ECX market environments. The model employs historical market data, representing state features such as price trends, trading volumes, and external economic indicators. Actions are defined as buy, sell, or hold decisions, while reward structures are designed to incentivize profit and penalize excessive risk. The research integrates techniques such as experience replay and target networks in DQN, action evaluation in DDQN, and advantage functions in A2C to enhance model performance and stability. Experimental results demonstrate that the DRL models significantly improve trading efficiency and decision-making accuracy compared to manual processes. DDQN outperforms DQN in managing noisy and volatile market conditions, while A2C excels in handling continuous decision variables, such as dynamic trade volumes. The results highlight the scalability and adaptability of the proposed system in addressing ECX-specific challenges, including risk management and market transparency. The study concludes that the DRL-based trading model offers transformative potential for the ECX by automating decision-making, optimizing trade execution, and promoting equitable participation among stakeholders. This research provides a foundation for integrating advanced machine learning techniques into emerging commodity markets, ensuring their efficiency and competitiveness in a global context.

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov decision processes.

Hierarchical Reinforcement Learning (HRL) exploits temporal abstraction to solve large Markov Decision Processes (MDP) and provide transferable subtask policies. In this paper, we introduce an off-policy HRL algorithm: Hierarchical Q-value Iteration (HQI). We show that it is possible to effectively learn recursive optimal policies for any valid hierarchical decomposition of the original MDP, given a fixed dataset collected from a flat stochastic behavioral policy. We first formally prove the convergence of the algorithm for tabular MDP. Then our experiments on the Taxi domain show that HQI converges faster than a flat Q-value Iteration and enjoys easy state abstraction. Also, we demonstrate that our algorithm is able to learn optimal policies for different hierarchical structures from the same fixed dataset, which enables model comparison without recollecting data.

A recent goal in the Reinforcement Learning (RL) framework is to choose a sequence of actions or a policy to maximize the reward collected or minimize the regret incurred in a finite time horizon. For several RL problems in operation research and optimal control, the optimal policy of the underlying Markov Decision Process (MDP) is characterized by a known structure. The current state of the art algorithms do not utilize this known structure of the optimal policy while minimizing regret. In this work, we develop new RL algorithms that exploit the structure of the optimal policy to minimize regret. Numerical experiments on MDPs with structured optimal policies show that our algorithms have better performance, are easy to implement, have a smaller run-time and require less number of random number generations.