This is a repository containing code for my Bachelor Thesis made in 2021. The code is based on this repo.

We investigate whether feeding data structure to the Transformer improves its performance on integration and solving ordinary differential equations (ODEs). We study recently developed tree-based model modifications and compare them. In our experience, the use of these alterations provides no benefit over the base approach. We assume this is due to an uncommonly large amount of data.

📝 Thesis, 👨‍🏫 Presentation (gdocs)

Raw data for training and validation can be found here or generated. Data preprocessing is done in notebooks/preprocess_notebook.ipynb and notebooks/ODE_preprocess_notebook.ipynb, including:

1. Deleting found repeating samples
2. Creating adjacency matrices (to a file)
3. Generating paths from root to node (to a file)

(Also notebooks/ODE_preprocess_notebook-ADJ_MAT.ipynb is for generating adjacency matrices for ODEs separately)

Notebooks by reg *my_metrics*.ipynb are for plotting metrics.

The project was done using a server with Slurm. Scripts for training and evaluation can be found in sbatch_scripts/ and sbatch_scripts_eval/ folders respectively. Arguments descriptions can be found in main.py or in this repo.

Any additional information on running can be found here