1. When making a thread, please tag your thread accordingly using the menu to the left of the textfield where you name your thread where applicable. Server Advertisements and Mod Releases should be contained to their respective subforums.

Server Help Developed Worlds Randomly Wiping and (Sometimes) Returning

Discussion in 'Multiplayer' started by Eberict, Feb 21, 2014.

  1. ArchGaden

    ArchGaden Big Damn Hero

    It can only happen on worlds that are actually loaded. After all, if starbound isn't touching a file, it can't very easily corrupt that file. Higher traffic worlds are loaded more, so it stands to reason that they would be affected more often. There are, perhaps, other contributing factors to, such as the number of block edits. I doubt there is any sort of hard capacity limit, but rather, more players = more problems.
     
  2. The Grand Mugwump

    The Grand Mugwump Subatomic Cosmonaut

    If there is high player traffic, the planet should be getting loaded and unloaded by the server less because there are higher odds of someone being there at any given time. It is also worth pointing out there are reported wipes from small two player servers as well. So whatever is breaking could be tied to something unrelated to player activity.

    Based on this, it is possible a planet remaining loaded for long periods of time causes a variable, most likely a string somewhere, to grow to larger than accepted sizes and possibly break or overflow on a server. There is one verified case of buffer overflow in the game in their player chat box due to an unrestricted string length, so it is within reason to say there are other possible buffer overflows they are not checking for, most likely unrestricted string lengths as well. Perhaps something becomes too big for their format, gets saved improperly when the planet is unloaded, which then causes the break on load which leads to a fresh planet? This is hypothesis, but it certainly fits the symptoms and deserves investigation.

    EDIT: I''d be curious to hear from the small player servers who reported wipes if they played on the planet for a long period of time without unloading it / leaving it before their wipes occurred.

    Double EDIT TL;DR version: It's possible they're not placing upper limits on all of their strings, which may cause issues if they copy it from a loaded planet to an unloaded planet file expecting a smaller size.
     
    Last edited: Apr 24, 2014
    Kazyyk likes this.
  3. ArchGaden

    ArchGaden Big Damn Hero

    I should clarify that by 'loaded more' I meant 'more time spent being active on the server', not actually a count of how many world loads it experiences. As changes are made to a world, the server writes those changes out to the file. No doubt when you have more players changing more things there is a greater chance of those things going wrong. I was never actually able to pin world corruption on the act of crashing itself... I even ran a test where I did periodic backups of a world using a cron job, and if the server reset, it would automatically revert to those backups. The idea being that if crashing corrupted the world, I'd just make sure to use a version of the world saved off prior to the crash. That didn't work! The corruption was happening while the server was up and seemingly running fine.

    About the only thing I can clearly note is that more popular worlds are corrupted more often. On the server I help run, the RP hub worlds where players gather (mainly spawn and one other world) experience a reset several times a day. I have those worlds frozen now to prevent that (ie, relead a pristine developed copy of the world on reset). Of course, those worlds can no longer be further developed without some effort replacing the pristine copy. Other less popular worlds that experience a moderate amount of traffic... such as guild bases... experience resets about weekly. Personal worlds are hardly ever affected, but due to their great numbers, I see a few of those.

    Seemingly, the amount of crashing is related to world corruption, but it probably isn't a cause/effect relationship between the two. More likely, something is going wrong (like the buffer overflow theory you have) and that causes world corruption immediatelly and a crash later on. They might not actually be related at all though. You aren't going to notice the world corruption until a world unloads and then loads up again from the file (when the world resets). A crashes forces that event to happen, which can lead to the perception of the crash being the cause. This whole business is a complicated mess and I can see why the devs have trouble pinning it down.
     
  4. Dunto

    Dunto Guest

    I would like to mention the fact that (and this has been confirmed by Kyren) if you copy out a world file while the game is running you'll most likely end up with a corrupted file. Gotta let the game flush out the write buffer and close the file before you copy it out, which is usually achieved by exiting the game first (I've had some success by monitoring the log and waiting for the "closed world x due to inactivity" message, usually at this point it seems the game has finished writing and closed the file, I was also able to "hot restore" a world at least once this way, if it's loaded and you replace the file you'll end up crashing).

    If you're restarting every so often for memory leaks and whatnot anyways, just make a start script that copies out your problematic worlds and then launches the server and call that from your restart script instead of the binary directly (or do it all from the restart script, much cleaner, stop > backup > start). You only need to really do this for the problematic worlds, the others can wait until your normal scheduled backups, this should keep your downtime minimal, and as a bonus, give you plenty of backup versions of your popular planets.
     
    Last edited by a moderator: Apr 25, 2014
  5. EkaWindra86

    EkaWindra86 Yeah, You!

    Hey All! I just recently started playing Starbound with my family on our own server and we haven't had any issues with any world wipes before the most recent update on the 25th. I have a private server through Multiplay that we play on. The first occurrence was with my dad and son playing on our server which we used an Alpha planet as our home planet. They were playing on different computers and both experienced lag, not being able to open doors/objects and then the appearance of parts of the world completely disappearing before the whole screen went black. They eventually had to exit the game and when they came back the world was reset to what it had been before. My husband who also plays has a separate home planet is intact on the same server so we are sure it was not a server wipe. The same thing happened today when I tried to start a new home planet on a Gamma planet. I was on my ship warping down to fill a new box with seeds (like you wanna know what I was doing ^.^) and was able to open 2 doors when I believe it then "lagged", was not able to open a 3rd door, tried to go further left on the map to see if my dad could open the door when it looked as if the world just dropped off. I've seen this kind of drop off with graphics before on other games and usually pushing past that unknown barrier (sorry don't know what to call this) usually I would lag completely and the game force closes or I would appear to walk on thin air. However, my avatar just stopped at that threshold. I don't know if this is relevant or not. My father, in the mean time of this happening, was trying to find where I was by following our group marker (again loss of words) and had "walked past me" and couldn't see me at all. We both went to a black screen together and both had to close the game. We were able to access the game again within minutes and the new structures we had made were gone and the world was reset.

    I know this is beta and there are still a lot of things that need fixing but I hope that this issue can be fixed asap if possible. We are all really excited about this game and are looking forward to see the completed product. If I can get anymore information for you or try to replicate the issue I'll take some screen shots next time, but as for logs or anything technical, that might be a bit difficult for me.

    Thanks for your hard work Devs. ~Eka

    Edit: The first world wipe was on Saturday 4/26/14
     
  6. Silverforte

    Silverforte Spaceman Spiff

    Of course it probably is affected by players modding their assets. Have you not seen what happens on servers when players use hacked items? It's insanity and usually crashes servers. If a crash can cause the corruption, and we have players hacking the game files and abusing the game and the servers they play on, then it's obviously possible that those crashes can cause it. A crash is a crash. Almost all files can become corrupted by not being able to write properly, by being interrupted during a write. Crashing is most likely the cause and anything that causes said crashing is to blame.
     
    Dunto likes this.
  7. This is still very much an issue and I'm upset at the lack of progress that has been made towards finding it's cause and resolve.
     
  8. quatch

    quatch Aquatic Astronaut


    Hi all, I am having this same corruption error ( Cause: DBException: Error, incorrect index block signature.), but I think I've caught starbound in the act of crashing to produce it.

    Code:
    Info: UniverseServer: Client 'Threep-IO' <1> (10.20.21.109:52046) connected
    Info: UniverseServer: Loading world db for world alpha:45:42:1727111:12:2
    Info: UniverseServer: Loading world db for world alpha:45:42:1727111:12:2
    Warn: Perf: UniverseServer::createWorld millis: 32420
    Error: WorldServerThread exception caught: IOException: Internal exception! block 5411 missing free index block marker!
    ...
    Error: UniverseServer: error during worker pooled job: IOException: Internal exception! block 5411 missing free index block marker!
    ...
    Error: UniverseServer: error during worker pooled job: StarException: No such client '0' found in UniverseServer
    
    ...
    
    Now when I try to restart starbound, and go there I get the generic incorrect block signature error. I've uploaded my log files here (attached them). The initial error is in .log.1, and a generic error happens in .log


    Platform: Mac 10.9 (client and server)

    I don't suppose there's any hope in recovering this file? Our latest backup is quite stale.
     

    Attached Files:

  9. I just want to make sure that the developers are still aware that this is a semi-common issue, it seems like.
     
  10. White935

    White935 Big Damn Hero

    Sure hope the developers still hold this as a high priority. A game which a large portion of the player base enjoys multiplayer, and whatever they build randomly wipes isn't exactly pretty.
    And thus a rather big problem no ?
     
  11. Most of this errors are due how newbie the coding is to make server able to deal correctly with errors, things end up on the most absurd points. To known is when they will fix it, nightly had some fixes but when it comes to the server software they don't go into detail of what they did on it. hm
     
  12. I ended up writing a script to automatically detect and fix these corruptions. It's not ready for public release yet, but I am planning on it. If you want to set it up privately, you can contact me and I'll be willing to help.
     
    White935 likes this.
  13. White935

    White935 Big Damn Hero

    Well, hopefully they'll patch "stable" soon if they've repaired the major server issues. There is a upcoming patch lucky :) or so I've read somewhere here.
     
  14. Far i see their development idea for next stable update, i would say 2015 till it happens unless they change their mind.

    Still that will break all our proxies and wrappers so much, eheheh, and likely introduce new issues, one thing that frustrates me is that server software needs some recodes on certain aspects and not only "fast-fixes", something devs refuse to do, so the hopes they will improve such things are not that high. It's very noticeable they are never happy to work on the server software, acting like it isn't one of the biggest parts of this game.
     

Share This Page