Reinforcement Learning on financial data

A Project of INF581 Advanced Machine Learning and Autonomous Agents course at Ecole Polytechnique

In this project, we implemented a Reinforcement Learning algorithm for financial data analysis.

Acces to report :

Data Structure

  • We sourced our data from Yahoo Finance via their API, specifically focusing on the closing prices for our modeling purposes.

Model Architecture

Environment

  • Our environment:
    • A dataframe containing stock price data,
    • A specified time range for action consideration,
    • A balance representing investable funds,
    • A record of the number of shares held at each time step.

Actions

  • Our action space encompassed three distinct types:
    1. Buying a certain proportion of stocks based on the model’s confidence in a buy strategy.
    2. Selling a certain proportion of stocks based on the model’s confidence in a sell strategy.
    3. Taking no action.
Representation of the actions taken and their proportion.

Deep Q-Network (DQN)

  • Our DQN architecture was a blend of various layers, notably incorporating LSTM due to the sequential nature of trading data. Additionally, it considered other relevant factors such as the number of stocks held and the current account balance.
Two models using combination of RNN and Transformers to predict the next price value.

Results

Action taken over time and evolution of the value of the account.