Server Discussion Another Server Memory Leak? Why such Linux Hate?

One major thing they CANNOT forget to do is allow, mirror linking for the mod's download. My mod pack varies between 30-60 MBs and I have upto 500/100Mbps at home on fiber, but thats still a bandwith killer. Being able to host it on drop box and X site, and be able to put the very direct download link, as in click it and it downloads or even external ftps and list the addresses in the config would be amazing.

 

That would be perfect. After they remove the version string usage from the .modinfo, they could work on adding a "getfrom" string or whatever, and just list the FTP there, so that even if, say, I download your modpack, and run a server my players will be able to auto DL the pack from whatever FTP is linked in the modinfo



EDIT: Just as well, we will kind of need a 'version' string anyways at that point.

Player A joins your server. Server runs check to see if ModPack A (current version 6) is installed.
Player A currently has ModPack A (@ version 5, which might cause a crash.)
Server runs version check, sees v.5, flushes mod assets, downloads modpack A v.6, initializes, connect.


And even then, if it didn't work 100% what's to stop player A from adding something malicious into your modpack, and then connecting

 

They got rid of strings already, which is awesome! Other games have this implemented very well. It can be cheated but does requre a lot of effort, more then most would put fourth. I know bacon keeps saying people will enter with other mods. But mod pack syncing means that it only uses X mod pack and X assets for X server only and players are locked into that while on your server.

 

Code:

<Underbalanced> What is the universe.dat used for? Well deleting it have adverse effects on the planet or items, same with universe.chunks?
<bartwe> both are part of the save data wrt the universe map
<Underbalanced> I ask because players are deleting it to fix mod issue. and Server owners too. So we wanted to make sure we are not causing harm by this, while using it to fix crashing or issues when making a mod change or removal
<bartwe> well it is deleting data that might be needed to return to specific worlds
<Underbalanced> so far we have not seen planets get changed or have issues, but just wanted to see if there was any unseen issues
<Underbalanced> so in the .chunks, what in a gist might be stored that would make planet builds or planet items lost?
<Underbalanced> I guess is most peoples concerns is the buildings and placed items.
<bartwe> universe.chunks doesn't contain the world itself
<bartwe> but it does contain the starmap
<bartwe> and if an upgrade happens the map might change if its not saved with the universe.chunk files
<Underbalanced> ok, so shutting a server off, and deleting it to clear up other issues, will effect the starmap and players getting to the planets containted within the universe folder?
<bartwe> possibly
<Underbalanced> I know your busy, I do thank you for your time on explaining this.

 

Hey Bacon.
Since we are running one of the biggest german Stabound Servers, I can confirm this bug.
Memory is getting full after some time and than server lags and crashes.

Thats already happening with just a few gamers...
So yes this bug is or was never really fixed for Linux.
Hope bartwe is reading this thread so he can take a look into it.

Also using Ubuntu 12.04 64 Bit.

Greetings!

 

I can verify the existence of memleaks very clearly aswell.

Having around 128GB ddr3ram, I found it quite entertaining to see the server running at 40GB Ram-usage after a couple of days with 10-30 ppl around.
I've shut it down at that point, cause I'd like to avoid stuff going into swap.

Running on CentOS6.5 64bit.

 

One other issue we experience only in Linux, is the server HANGING, this is very frustrating because the server from sudden disconnects everybody, it timeouts every try from people to connect, but it doesn't crash, so it's always running but non-working, what causes the scripts to restart it unable to detect this.
Anybody else experienced or have any idea why this happens?

You can run a couple of days without a crash? What dark magic is that?
We still crash like madness since the heavy crash issue was Introduced on Furious Koala and fixed on Enraged (for windows only).

 

I do get the hang.

 

Well now that you're mentioning it, yes it crashs a lot aswell.

At these dates for example:
10. Mar 20:119 CET 2014
11. Mar 024:27 CET 2014
11. Mar 19:464 CET 2014
12. Mar 155:14 CET 2014
12. Mar 22:29:10 CET 2014
13. Mar 203:56 CET 2014
14. Mar 18:24:08 CET 2014
15. Mar 005:10 CET 2014
15. Mar 05:47:06 CET 2014
15. Mar 21:01:00 CET 2014
16. Mar 03:48:10 CET 2014
16. Mar 15:26:12 CET 2014
16. Mar 17:494 CET 2014
16. Mar 18:42:14 CET 2014
16. Mar 223:45 CET 2014

 

I do get this behaviour too but my script is able to detect this because it scans the port availability and not just the process pid so it restarts any time the conn is refused twice

 

Windows or Linux?

 

We tried that same thing, the fix causes another issue, that is, once it kills the server after it did hang, it's likely it can take up to 45 Minutes down till it starts because it's "recovering files" for some mad reason and sometimes it simply fails to detect the server hanging.
The solutions to this issues are on the Devs hands, i'm yet to see who is responsible for the Linux version, as i only saw @bartwe fixing Windows and never linux, to find the right person to get in touch to fix this issues.

 

Linux with monit http://mmonit.com/monit/

Not getting this issue here, I guess this is because you used a normal kill -INT to "safely close the process" while in case of a crash you have to kill -9 to insta kill the process, indeed a kill -INT when the server crashes can takes forever, anyway there is no real benefit in gracefully shutting down the server when this one is already crashed and won't save anything

I don't get any trouble with such script for example (my only problem is when an exception is spamming the log files and causing huge lags I did not upgraded my script to check the log file size but should be easy to do with monit)

Code:

# Settings
set daemon  30              # check services at 1-minute intervals
   with start delay 30      # optional: delay the first check by 30s.
set logfile /var/log/monit.log

# Checks    
check host local with address 127.0.0.1
    start = "/usr/sbin/chroot /chroot-starbound /etc/init.d/starbound start"
    stop = "/usr/sbin/chroot /chroot-starbound /etc/init.d/starbound force-stop"
    # Save starbound_server.log
    if failed port 55102 type udp with timeout 5 seconds then exec "/bin/bash -c 'cp -f /chroot-starbound/starbound/starbound_server.log /chroot-starbound/starbound/starbound_server_crashed_$$$(date +%s%N).log'"
    # Restart
    if failed port 55102 type udp with timeout 5 seconds retry 2 then restart

for force-stop I have slightly modified my script to do "force-stop"

Code:

...
case "$1" in
    start)
        start
    ;;
    stop)
        stop 0
    ;;
    force-stop)
        stop 1
    ;;
...

stop() {
    echo -n $"Stopping $NAME: "
    RETVAL=1
    if ! checkScreen || ! checkPID; then
        echo -n "(not running, or stale pidfile)"
    else
        if [ $1 = 0 ]; then
            kill -INT $(cat ${PIDFILE} 2>/dev/null) > /dev/null 2>&1;
        else
            kill -9 $(cat ${PIDFILE} 2>/dev/null) > /dev/null 2>&1;
        fi;
        n=${EXIT_TIMEOUT}
        for (( i=0; i<n; i++ )); do
            if ! checkScreen; then RETVAL=0; break; fi;
            sleep 1;
        done
    fi
    if [ $RETVAL = 0 ]; then
        success
    else
        failure
    fi;
    echo
    return $RETVAL
}

 

It kills normally the process, this happens after it starts, but i already notice this on normal server crashes, not sure what i can do to avoid it, if i delete the universe folder i will create universe-wide rage because it's all deleted lol

 

still no linux server update.... go head starbound eat all the cpu you can... lol I seen it go up to 500%

 

Starbound is really hating on the whole dedicated server thing. Windows dedi is also not stable and there are a lot of problems.

 

wonder why they hating , I dont get it... multiplayer its whats keeping this game fun

 

I'm not sure what is happening but Server Linux Version is being completely neglected, no matter if we poke @Tiy @mollygos @OmnipotentEntity @kyren @Legris ... Nobody says nothing, it's like us, Linux Users do not exist (so does the Linux version of the game).
We just want the server to perform with the minimum stability with bigger numbers.

 

It's not being neglected. It's a mess of a compiler issue that I think I've worked out, but it didn't make it into the 4/25 update.

GCC has a bad issue with lambda closures and ensuring that destructors run. We've had a number of very subtle leaks as a result of this. For backwards compatibility on Linux we're using Debian stable which only carries up to gcc 4.7.2 in dpkg. So we can't easily upgrade past this bug, (which is also present in 4.8.1)

Anyway, as I mentioned, I think I hunted down this bug. I spent an entire day on it last week valgrinding it out. The patch is pushed to our progression branch and it will be in the next update. The source was a return value not being properly destructed within a lambda. The patch was essentially changing a "return x;" to "auto res = x; return res;"

I am a Linux user and I'm the Linux maintainer. I get it. I just haven't had the time until recently to spend an entire day hunting down this slow leak until after I got the patch stuff done.

 

It's good to hear you're still trying to hunt this problems down. We have duzens of pages of code with scripts trying to put SB Server in order, we deal with crashes, high ram usage, etc... But when there is things like incredible high CPU usages (up to 500%) that lags the entire server, or when the server "timeouts" everybody and the process simply hangs there, not accepting any connection but not crashing... We ran out of scripts that can detect / act on this more complex issues.
We have been a top server since the very launch day with amazing 98% Uptime till now, but the cost was frustration to a level i became suicidal!

I hope you can get server running nicely as soon possible, are you alone doing all this? Hire one helper