MySQL Support & Data classes serialisations - #8872

🔍 Triage Checklist

Type

• bug
• feature
• enhancement

Project

• 2.16 Releases
• 2.17 Releases

• ✅ Mark as triaged

great that you are interested :)

• for inventories, are you going to handle things like furnaces where the inventory has to be tied to a specific inventory holder? what happens if the block no longer exists after the variable is loaded again? what about player inventories?
• it does not seem like a wise architectural choice to hardcode exemptions for the serialisation rules all in one place; perhaps you could focus on storage for those variables which can be serialised and then maybe expand on those once the storage is working?
• it would be a good idea to document the general architecture of the storage including javadocs
• is this intended to support multiple servers interacting with the same database? if not, does it lock the database? do you plan on serialising, say, trigger sections as transactions to prevent users having to worry about race conditions?
• did you see skNetwork? do you think it is better to focus on adding minimal database connection support to skript or contribute to the addon instead?

I had a small conversation regarding inventories with some members of the team on Skript's discord, safe to say i will not be working on inventories for the time being.

Yes the current architecture is not perfect, its just a placeholder rn used to make sure the result is achievable, the pull request was made to let others know that i am indeed working on this.
I will probably only focus on fixing yggdrasil for CSV color serialization, and make the sql database follow some regulations (i will try as much as possible to make it data-driven and compact, with no need for specific serialization per data class), but my current end goal changed to extend the save-able values by csv, then implementing them in sql. Not sure if i would work on it again after this.

Documentation and comments will be added in due time.

The implementation is basic, all depends on how the user configures it, the config allows for the use of various tables so the same hosted database could be used by multiple servers
I made it that way as my testing server does not allow me to create databases for some reason, and i think keeping it like this would be benefitial for some rare cases, personally i'd use this.

I have not seen skNetwork.

Ok so i believe im done, i stepped back a bit and ended up only making mysql support & fixing a yggdrasil issue so now colors and probably(?) some other classes are also serialisable and saved in both csv & sql

Were the previous MySQL implementations actually shipped? I don't necessarily think it's important to support variables saved by the old SQLibrary format since it hasn't been supported in years.

I assume you marked AI assistance as none erroneously? It seems at least the documentation is written by an LLM. If you describe the system here in Lebanese Arabic or whichever language is most comfortable for you, it will help us trust that you actually understand the content of the pull request. This feature is not really the kind of thing that state-of-the-art LLMs can implement correctly yet.

I didnt use AI when creating the pull request, i did for the documentation (i got lazy) and i dont think i can change the original message

And im sorry, what are you referring to with "old mysql implementation" as i could think of 3 things rn
And sure, do you want me to fully explain everything i have done? I could do it in english as i cant really explain it code that well in my language (since ykno, coding is in english and i lowk use it more than arabic atp)

Missformed sentence, i meant to say "i didnt use ai PRIOR to creating the pull request"

I didnt use AI when creating the pull request, i did for the documentation (i got lazy) and i dont think i can change the original message

And im sorry, what are you referring to with "old mysql implementation" as i could think of 3 things rn And sure, do you want me to fully explain everything i have done? I could do it in english as i cant really explain it code that well in my language (since ykno, coding is in english and i lowk use it more than arabic atp)

Okay :)

Re.: MySQL, for practical purposes all three of them. We don't need migrations for database schemas that have only ever been used by you while making this pull request, and we also don't need migrations for the old SQLStorage schema that currently exists in mainline Skript either. This is because no Skript users actually have data with these schemas; you already migrated your own database, and Skript's old MySQL schema hasn't been supported since SQLibrary 7.1 broke in Spigot 1.17.

Ideally, the javadocs for each class would contain extensive documentation about the purpose of that class and how it relates to the SQL storage system as a whole; it doesn't need to document every implementation detail, but it should be written so that a person who is reading the code gets an idea of the purpose of that class and—after reading enough javadocs—understands the SQL storage altogether.

English is excellent, and it's okay to use AI to fix spelling mistakes and grammatical errors :) I recommend you write the content yourself though, as AI is very bad at writing prudent javadocs and often includes unnecessary detail about what had been wrong previously and omits crucial information about the design.

MySQL, for practical purposes all three of them. We don't need migrations for database schemas that have only ever been used by you while making this pull request, and we also don't need migrations for the old SQLStorage schema that currently exists in mainline Skript either. This is because no Skript users actually have data with these schemas; you already migrated your own database, and Skript's old MySQL schema hasn't been supported since SQLibrary 7.1 broke in Spigot 1.17.

Ideally, the javadocs for each class would contain extensive documentation about the purpose of that class and how it relates to the SQL storage system as a whole; it doesn't need to document every implementation detail, but it should be written so that a person who is reading the code gets an idea of the purpose of that class and—after reading enough javadocs—understands the SQL storage altogether.

I will get these done then get back to you

I assume you marked AI assistance as none erroneously? It seems at least the documentation is written by an LLM. If you describe the system here in Lebanese Arabic or whichever language is most comfortable for you, it will help us trust that you actually understand the content of the pull request. This feature is not really the kind of thing that state-of-the-art LLMs can implement correctly yet.

The SQL implementation uses Skript’s existing serialization system. Skript turns each variable into a name, type, and bytes before passing it to the backend. SQL doesn’t need to know how types work.

The MySQL backend saves changes in batches on a background thread. Before writing a batch, it records the pending changes in a recovery file, named mysql-pending.bin. If the database becomes unavailable for whatever reason, those changes can be retried. It also attempts to flush pending changes during shutdown. The same file handles deletions of variables periodically.

For colors, the main problem was that 'Color' covers two different representations: 'SkriptColor' and 'ColorRGB' which are different classes (enum & object). I registered them separately so Skript uses the correct serialization format for each. Named colors still go through Yggdrasil’s built-in enum handling, and ColorRGB stores its RGBA value.

I also fixed a separate bug in Yggdrasil’s superclass lookup: it was checking the map entry’s class instead of the registered class. The color registrations fixes the enum/object mismatch, the lookup correction fixes superclass resolution generally. These changes apply to shared serialization, so they benefit both CSV and SQL.

I hope this is what you wanted :>

Also, i will be reworking the whole yggdrasil & colors handling in either a separate PR or continue working on this one, depending on the feedback for this, as sovde requested
If this pr would be accepted, i will work on the rework here, otherwise ill work on a different branch