Releasing a new, much improved version. It now drops lines which have been written out of memory, which can be very important on populated servers. It also only checks the line count once, rather than before when it was checked each time a file was written (this was actually causing the most slowdown).
It also now provides better progress indications (such as the percent it is finished with writing) in echoes.
PHP Code:
// default setups
enum {
FOLDER_NPCS = 10 // npcs/* minus npclocalnpc*
}
function onCreated() {
this.version = "1.2"; // eventually used in expander?
temp.mb = 1048576;
this.maxWrite = (mb * 4); // max to write at once; I find mb*9 is the absolute limit
// but below that is usually better to avoid flood alert
this.loopLimit = 5000; // the number of loops before a sleep -- must be less than
// this.maxlooplimit -- 5000 is good but don't expect
// the server to be playable while the files are added to
// the "to write" list (the first part which takes very little
// time, but will lock up the server)
this.delay = 60 * 6; // delay between trying to write, default 60*6 -- this seems not to
// cause flood alert, which, while protected against, can prevent
// other scripts on the server from functioning
}
/*
The way folders work is that if I were to compress
only "weapons/*", there would be a folder called
"weapons" with all of the contents inside.
If I were to compress "weapons/*" and "levels/world/*",
there would be two folders: "weapons" and "world"
*/
public function GBall(description, outputName, folders) {
temp.lines = null;
temp.c2 = 0;
for (temp.folderName : folders) {
c2 ++;
temp.defSetup = null;
if (folderName == FOLDER_NPCS) {
folderName = {"npcs/*", false};
defSetup = FOLDER_NPCS;
}
temp.folder.loadFolder(folderName[0], folderName[1]);
temp.p = folderName[0].positions("/");
temp.pathToFolder = folderName[0].substring(0, p[p.size() - 1]);
temp.tokens = pathToFolder.tokenize("/");
temp.bottomFolderName = tokens[tokens.size() - 1];
temp.c = 0;
temp.fSize = folder.size();
for (temp.file : folder) {
temp.c ++;
if (this.loop()) {
echo("GBall: Building file cache... " @ int(c / fSize * 100) @ "% (folder " @ temp.c2 @ "/" @ folders.size() @ ")");
}
if (defSetup == FOLDER_NPCS) {
if (file.starts("npclocalnpc")) {
continue;
}
}
// add it to the package
temp.str.loadString(pathToFolder @ "/" @ file);
str = " " @ base64encode(bottomFolderName @ "/" @ file) @ " " @ base64encode(str);
lines.add(str);
}
this.loop();
}
// don't save it all at once to save resources + avoid flood alert
temp.line = 0;
temp.c = 0;
temp.lineSize = lines.size();
while (line < lineSize) {
temp.chars = 0;
temp.toWrite = {format("GBALL v%s (%s): %s", this.version, base64encode(description), int(timevar2))};
while (chars < this.maxWrite && line < lineSize) {
toWrite.add(lines[0]);
chars += lines[0].length();
lines.delete(0);
line ++;
this.loop();
}
temp.fileName = outputName @ c @ ".arc";
temp.hasTried = false;
while (! (fileExists(fileName))) {
if (hasTried) {
echo("GBall: Unable to write '" @ fileName @ "'; flood alert or bad permissions. Trying again after delay...");
sleep(this.delay);
}
toWrite.saveLines(outputName @ c @ ".arc", false);
echo("GBall: Writing... " @ int(line / lineSize * 100) @ "%");
hasTried = true;
sleep(1); // sometimes it takes a minute for the file to "exist"
}
// sleep to avoid flood alert
if (line < lineSize) {
sleep(this.delay);
}
c ++;
}
echo("GBall: Saved file (" @ outputName @ "x.arc)!");
}
function loop() {
this.count ++;
if (this.count >= this.loopLimit) {
this.count = 0;
sleep(1);
return true;
}
}
I've also added "folder configurations". This lets you do custom things for including files. I've added one, which is
FOLDER_NPCS. This will back up all DB NPCs but not backup local NPCs (putnpc2).
Example usage:
PHP Code:
GBall("Backup of NC 7/28/10", "data/2010_07_28_scripts_", {{"scripts/*", false}, {"weapons/*", false}, FOLDER_NPCS});
In addition, I'm releasing a new version of the expander (attached). It includes a bug fix for files which don't have an underscore before the number. It's also up to 30% faster due to some optimizations for folders with a lot of files.
The source code for the expander
(approved by Skyld) can be found
here.
(tip: combine this with Dylan's
GraalCron for making daily NC backups)