ZippySQL v1.0

[PrinceOfKenshin]

I am posting this on behalf of iMagic the creator of this script / system.

Hello Some folks from the Graalians Discord had asked me to post this in one of the forums for posterity.

Introduction

ZippySQL is a robust tool for managing backups in a Graal server. It allows server administrators to create, manage, and automate backups of critical server files such as logs, images, ganis, and/or databases and to easily download them in the form of .zip files. The tool is built with scalability and flood control in mind, ensuring that backups can be performed efficiently without triggering server flood alerts (which unfortunately sometimes can be difficult to avoid!).

(P.S. I know reading a wall of text can't be that much fun, so I made a quick video to summarize the following as best as I could.)

Features

1. Flexible Backup Configurations: Use bitwise flags to specify which folders or files to include in the backup.
2. Dynamic Path Handling: Supports wildcard patterns and folder contents for comprehensive backups.
3. SQLite Integration: Store the backup contents in an SQLite database for easy management, safety, and retrieval.
4. Flood Control: Built-in mechanisms to prevent server flood alerts during backup operations and/or to avoid high CPU usage during backups.
5. Error Handling and Logging: Extensive error handling and logging to ensure reliable operation.

---
Setup

Server Access: Ensure that the npcserver has read/write access to the folders and files you want to back up to avoid errors.
The npcserver should have the following right:

NPC Code:

---

rw backups/

---

SQLite Database: Add the following database to your server options to create the SQLite database:

NPC Code:

---

database=zippy,zippy

---

Folder Config: You must add the backups folder to the server's folder configuration. Make sure to add the following:

NPC Code:

---

file      backups/

---

Source Code: A `.txt` file for the source code was apparently too big to upload to the forums. You can click this link to pastebin instead to get the source code
https://pastebin.com/CaXqkady


---

Caveats

ZippySQL is limited quite meaningfully by GS2 itself.

Flood Alerts: The script may cause flood alerts. Flood alerts CAN KEEP OTHER SCRIPTS FROM RUNNING APPROPRIATELY! Flood alerts occur when the server exceeds a certain amount of read-write operations. These boundaries are not well-known and therefore difficult to navigate. The `FLOOD_CONTROL` enum contains properties that can be tweaked (if you know what you're doing!) to reduce the risk of flood alerts.
Impossible Reads: In what is arguably a good thing (and clearly a safety precaution), the GS2 functions `TGraalVar.loadstring()` and `TGraalVar.loadlines()` (which are used to read the contents of files) cannot read files in various folders such as `weapons/`, `scripts/`, and `accounts/` among a few others. Therefore, this tool cannot backup files located in these folders.
High CPU Usage: Because of the nature of the tool, it needs to perform A LOT of operations across possibly hundreds if not thousands of files. Most specifically, the `onFinishZip()` function, which compiles the contents of all of the backed-up files into the singular zip file. If high CPU usage is a problem, I recommend changing the configuration accordingly at the expense of backups which take longer to compile.
Exponential File Size Growth: ZippySQL can theoretically create infinite-many backups. In the lack of infinite storage, this is clearly a problem. It is recommended that as few files as possible are backed up, that larger files such as images and gifs are avoided if possible from being backed up, and that if possible, lifespans for backups are as short as possible (permanent backups are highly discouraged!).



---

Examples

To start a backup, we call the

onStartBackup()

function with the desired configuration. The example functions all have a function modifier of

'public'

to show that these functions can be called externally by other scripts like, say, a cron scheduler.

Example 01: Full Backup

NPC Code:

---

public function onFullBackupExample() {

  // In this THEORETICAL scenario, we set a configuration

  // bit that toggles all of the folders except

  // the levels folder because of how big it can

  // be (not that all of the others combined aren't!).

  //

  // Although this tool can theoretically

  // backup all of the files on the server,

  // it should never be used for that purpose.

  

  temp.config = 

    SAVE_OPTIONS.NPCS |

    SAVE_OPTIONS.DATABASES |

    SAVE_OPTIONS.WEAPONS |

    SAVE_OPTIONS.LOGS |

    SAVE_OPTIONS.STORAGE |

    SAVE_OPTIONS.SCRIPT_FILES |

    SAVE_OPTIONS.SCRIPTS;



  // Which can also be written like this

  //temp.config = SAVE_OPTIONS.LEVELS & ~SAVE_OPTIONS_ALL;



  // This initiates the backup process

  onStartBackup(null, temp.config);

}

---

Example 02: Backup Logs folder

NPC Code:

---

public function onLogsBackupExample() {

  /**

    In this example we create a backup of JUST the logs folder that expires

    every month. It also immediately outputs the log into the backups folder as a zip

    file.



    - The first argument is an identifier of sorts that is added

      to the zip file's filename so that you can tell them apart

      easier.

    

    - The second argument is the bit for the LOGS folder.



    - The third argument is a boolean that determines whether or not

      the backup is written as a zip file to the backups folder



    With this configuration we can expect a similar filename

    to be created for our backup file



      backups/2025_03_14_logs_backup_69d8e420.zip

  */



  // This initiates the backup process

  onStartBackup("logs_backup", SAVE_OPTIONS.LOGS, true);

}

---

Example 03: Custom Backup

NPC Code:

---

public function onCustomBackupExample() {

  // In this example we want to create a more complex

  // backup file that only lasts 3 days. The additional_paths

  // argument gives us a lot of flexibility when choosing what

  // is backed up.



  // For example's sake, the following paths exist exclusively

  // for the purposes of demonstration



  temp.custom_backup_paths = {

    // Backup all of the files that may be

    // on the favorite books folder

    "books/favorites/",

    // Add a singular file to the backup

    "books/scifi/snow_crash.pdf", // Neal Stephenson's cyberpunk novel that coined the term "Metaverse"

    // Backup just the '.png' files matching a pattern

    // '*' here is a wildcard including anything

    // that may come before the filename

    "books/covers/cover_*.png"

  };



  // With this configuration we can expect a filename similar

  // to this one to be created for our backup:

  // backups/2025_03_09_books-images_69d8e420.zip



  onStartBackup("books-images", null, true, TIME.DAY * 3, temp.custom_backup_paths);

}

---

Bugs

If you encounter any issues, find any bugs, have any ideas, or have any questions, feel free to message me on Discord: imimagic

I hope this tool is able to help you in some way. Until the next time!

[Jakov_the_Jakovasaur]

thank you!