This project is an excuse for me to play with some tools, namely database management (setting up pipelines, SQL, Postgres), and to play with some optimization techniques. The first part of the project is the simulation, in which I simulate guests randomly walking through a fictional amusement part. During this process, transactions are recorded into a Postgres database. I then aggregate this data by day and attraction, and compute quantities like total revenue by day and attraction, average duration in the park, and standard paths taken by guests. Finally, I feed that information into an online Bayesian optimization agent, which optimizes the total revenue by tweaking the prices of the attractions in the park.

The park is represented as a collection of attractions, each having a fixed physical location $(x_i, y_i) \in [0,1] \times [0, 1]$, a price to ride $c_i$, and a duration $t_i$ that a guest needs to spend on the attraction. Guests attend the park by entering at a random time between 9AM and 1PM through the entrance, which is a special attraction located at [0.5, 0.0], and then performing a biased random walk to different attractions. The guest chooses to walk from attraction $i$ to attraction $j$ with probability $$p_{ij} = e^{-H_{ij}}/\sum_{j \in \partial i} e^{-H_{ij}}, \quad H_{ij} = H^{wealth}{ij} + H^{distance}{ij}$$ where $H_{ij}$ is a utility function which includes two terms. First, guests have a 'wealth level'. Guests with low wealth prefer to ride cheap rides, while guests with high wealth prefer to ride expensive rides. For each wealth level $w$, a cash target $c_w$ is set, and we set $H^{wealth}{ij} = -\alpha_c |c_j - c_w|/c_w$, where $\alpha_c$ represents the price sensitivity of guests in the park. Second, guests prefer to attend rides nearby. This is enforced by setting $H^{distance}{ij} = -\alpha_d \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}$. Finally, guests also prefer not to attend attractions that they have attended recently, so $p_{ij} = 0$ for any attraction $j$ already visited. Every step on the graph takes $t_j$ time, as well as some travel time depending on the distance between the nodes.

Intending to optimize the prices of the park to maximize revenue, we need one more ingredient. Note that by sending the prices $c_i \rightarrow \infty$, the guests have no option to leave early and are forced to pathologically spend infinite money. To prevent this, I add one more ingredient: guest dissatisfaction. At the start of the random walk, guests have dissatisfaction $d = 0$. After every step, the guests' dissatisfaction increases by $\alpha_d |c_j - c_w| / c_w$. If the dissatisfaction becomes $d > 1$, the guest leaves immediately, ending the random walk.

Every time a guest visits an attraction, the transaction is recorded in a Postgres database. This data includes the transaction amount, the attraction identity, the guest identity, and the transaction time. This data is automatically aggregated into summary statistics by the database, recording the total revenue of each attraction for each day. This data is then fed into a reinforcement learning agent, which attempts to optimize the total revenue of the park by modifying the prices of the park, which it is allowed to modify in between days.

For the optimization agent, I decided to go with a Bayesian optimizer with a random forest regressor. I used sklearn-optimize as my chosen framework to utilize these algorithms. I chose a Bayesian optimizer since the samples in this system are quite sparse; I have access to one sample per day, and simulating a day is relatively computationally expensive on a computer and cannot be replicated in real-life. Additionally, the output is quite noisy, due to the stochastic nature of the simulation. This means that the gradients of the fitness function (total revenue) are not accessible by finite difference. Bayesian optimization seemed like a strong way to handle these constraints, although I'm certain there are better approaches. For more specific analysis of data, see analsysis.ipynb.

Some TODOs: 1.) I would like to try to compare to other optimization agents; perhaps reinforcement learning? 2.) I would like to increase the complexity of the simulation, so that guests have preferred types of rides, the seasons have a greater impact on guest behavior, and more. These could produce interesting behaviors for a reinforcement learning agent to learn. 3.) I would like to observe slightly stronger