Info Server offline, long save duration, Quest Book resets (partially resolved on 1.7.10)

Discussion in 'Crash Landing' started by Slind, Oct 8, 2014.

Page 1 of 2
  1. Slind

    Slind Founder

    Warlock
    10 Points
    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    9:46 AM
    Hey there,

    I'm gonna use this thread to talk about a very annoying issue with Agrarian Skies and Crash Landing. This issue comes up all the time and drives all of us crazy.

    The save data of the Quest Books does corrupt it self from time to time and causes major lag spikes on world save, this leads to the following:
    • creating new parties won't work OR
    • server does freeze on startup (from the player perspective it is offline)
    • quest books data gets restored from backups

    Why we can't do anything about it:
    • all the data is being saved in one file as pure bytecode, bytecode is not human readable
    • the mod is closed source (the code is not public)
    • the mod authors are inactive
    • the mod authors made clear that they don't support any server related issues (if this issue only occurs on servers is unclear, reports from the average player about a quest book corruption are very very unlikely due to the nature of know how and probability of occurrence with 1 instead of thousands of players)
    • on server start-up-freeze we need to manually kill the server process and perform the quest book data restoration (requires admin)

    Things we discovered so far:
    • Players (every join counts, even if it is just for a second) are limited to 1024 -> this limit got increased by the authors on high pressure a few months ago, [1] but we remove data from inactive players and teams in order to help with the save duration (decreased it by 5 times)
    • Teams (every created team counts, also if there is only one player inside) are limited to 1024 -> implemented auto cleanup feature, see above [1]
    • It seems to be team related or at least happens while reading the first things (team data is the first that gets read)
      • servers with corrupted quest book save data mainly freeze on team data load
      • corruption happens more frequent on crash landing -> through the hardcore system with only one live there are more player entries and teams getting created per player over the same duration
    • Saving data in bytecode is a very unstable save format, it is enough if one single action fails or doesn't work as wanted to break the whole data. Overflowing integers, strings or similar might be able to cause corruption.

    To sum it up, we have invested days of time into this issue and didn't come up with any solution. One solution would be to rewrite the data storage process to something more stable like SQL but the mod is closed source and doing such a big change with only patching is time consuming and requires experts on this sort of topic. The system with the net handler doesn't allow easy exchange.

    If you are a developer with tons of experience (we had master students look into this without success) and the motivation to spend days on a resolution for this topic feel free to contact as, we will gladly give you all the information we can provide.
     
    Last edited: Aug 7, 2015
    Slind, Oct 8, 2014
    #1
    Werwolfi_de, cbrozak and Krole69 like this.
  2. Jeppe0701

    Jeppe0701 Active Member

    Messages:
    50
    Likes Received:
    7
    Local Time:
    9:46 AM
    They are not inactive..
    Crash Landing
    extally they are really busy with the modpack:D
     
    Jeppe0701, Oct 8, 2014
    #2
  3. Slind

    Slind Founder

    Warlock
    10 Points
    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    9:46 AM
    The HardcoreQuestingMod not the modpack.
     
    Slind, Oct 8, 2014
    #3
  4. Jeppe0701

    Jeppe0701 Active Member

    Messages:
    50
    Likes Received:
    7
    Local Time:
    9:46 AM
    Oh sorry! My bad!
    :D
     
    Jeppe0701, Oct 8, 2014
    #4
  5. SH4D0WN1NJ4

    SH4D0WN1NJ4 Well-Known Member

    Empress
    0 Points
    Messages:
    101
    Likes Received:
    7
    Local Time:
    8:46 AM
    also from what i understand is that the questbooks automatically reset at a 1000 unique players
     
    SH4D0WN1NJ4, Oct 24, 2014
    #5
  6. paulnator2112

    paulnator2112 Member

    Messages:
    898
    Likes Received:
    102
    Local Time:
    9:46 AM
    1024 uniue players ;)
     
    paulnator2112, Oct 24, 2014
    #6
  7. SH4D0WN1NJ4

    SH4D0WN1NJ4 Well-Known Member

    Empress
    0 Points
    Messages:
    101
    Likes Received:
    7
    Local Time:
    8:46 AM
    well there we go then not far off XD

    ^hehe a Gigabyte of players XD
     
    SH4D0WN1NJ4, Oct 25, 2014
    #7
  8. Jaded_Magic

    Jaded_Magic New Member

    Messages:
    21
    Likes Received:
    1
    Local Time:
    4:46 AM
    If I'm not mistaken should the book have restarted by now(picture), if this is all true. Has this number been raised higher than 1024? (3 months later) imgur.com/RYQrh3s
     

    Attached Files:

    Jaded_Magic, Feb 3, 2015
    #8
  9. Slind

    Slind Founder

    Warlock
    10 Points
    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    9:46 AM
     
    Slind, Feb 3, 2015
    #9
  10. Jaded_Magic

    Jaded_Magic New Member

    Messages:
    21
    Likes Received:
    1
    Local Time:
    4:46 AM
    oo haha, I read that as, it got raised to 1024, sorry
     
    Jaded_Magic, Feb 3, 2015
    #10
  11. Slind

    Slind Founder

    Warlock
    10 Points
    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    9:46 AM
    Found a way to fix the issue with corrupted teams, this should half the need of quest progressed resets. Currently deployed on 1.6 for testing. It saved the crash landing book from a reset a few minutes ago.
     
    Slind, Aug 2, 2015
    #11
    MrWisski, Bennyboy1695 and LucidTheStick like this.
  12. danielpmc

    danielpmc Well-Known Member

    Messages:
    249
    Likes Received:
    6
    Local Time:
    8:46 AM
    love ya me system in the background
    xD
     
    danielpmc, Aug 8, 2015
    #12
  13. egons_ghost

    egons_ghost Build Team

    Messages:
    50
    Likes Received:
    10
    Local Time:
    3:46 AM
    How freaking annoying to have the Quest book reset on us all the effin time. We understand the "FREE" stuff that we make and the doubles,triples and quadrupultes of machines and resources that we have to make over and over but what is the sense of playing a game that gets reset all the time? We find the server that offers the quest book because it brings a new light to the game but at what cost? There is no goal but to survive because there is NO END because you are being reset all the time. I suggest instead of getting pissed off players and bad reviews because of this nonsense just delete,remove or abolish the quest book altogether. Really what is the sense of putting hard work to "No Payout" at the end?

    None because there is NO END at all if you keep resetting the beginning.
     
    egons_ghost, Aug 23, 2015
    #13
  14. danielpmc

    danielpmc Well-Known Member

    Messages:
    249
    Likes Received:
    6
    Local Time:
    8:46 AM
    Yeh I agree in crash landing iv finished the quest book 3 tines from the quest reset I even left a bad report on the review
     
    danielpmc, Aug 23, 2015
    #14
  15. MrWisski

    MrWisski Well-Known Member

    Messages:
    1,110
    Likes Received:
    522
    Local Time:
    3:46 AM
    We run servers that run Minecraft Mod Packs.

    We do not develop (most) of these Mod Packs. We do not develop the Mods that these Mod Packs use. We have gone to the developer of HQM. He will not fix his code. He will not allow other people to help fix his code. He will not allow us to decompile his Mod, and fix the code for our own use.

    We have gone to the Mod Pack Authors - They have told us that there is no viable alternative to HQM, and therefore, they will not be removing HQM from their Mod Packs.

    We do what we can, to fix what we can, but it is a difficult, time consuming, and frustrating process.

    Therefore, you really have only 3 choices here :

    1) Continue playing, in the hopes the HQM mod author sees the light and either fixes, or allows others to fix, his broken code, or we (MyM) can patch it to the point where it's no longer corrupting data (we may never get it working completely, but I feel like we've taken HUGE steps, and you're not going to find that anywhere else).
    2) Stop playing HQM Mod Packs - We have many servers that have never seen HQM on them, such as Infinity, Crundeecraft, BTeam, etc.
    3) Learn Java, and write a viable alternative to HQM. Then get mod pack authors to use your new awesome mod.

    Screaming at us does nothing, as this is more or less completely out of our hands. We understand your frustration, and we share them, but there is only so much we can do about this.
     
    MrWisski, Aug 23, 2015
    #15
    Mouro and joices like this.
  16. egons_ghost

    egons_ghost Build Team

    Messages:
    50
    Likes Received:
    10
    Local Time:
    3:46 AM
    Then stop promoting the mod that cannot be repaired. Please understand that i am not yelling at the staff or the owner of the server. I understand 100% where they stand, but its the users you need to promote the server, to give the great reviews and refer new users. Maybe now is the time to take our advice and change it up to something new or something old. If the developers are not going to change their ways change yours and take a poll/vote to rid ourselves and yourselves of a book that in retrospect kill your votes and reputation as an awesome server.
     
    egons_ghost, Aug 23, 2015
    #16
  17. MrWisski

    MrWisski Well-Known Member

    Messages:
    1,110
    Likes Received:
    522
    Local Time:
    3:46 AM
    I'm sorry you feel that way, but rather than telling us to remove quite a few servers (Crash Landing, Ag1-3, Ag+, Agrarian Skies 2, SkyFactory2), that almost all have a substantial user base, I would recommend that you simply not play on the HQM packs. They are popular enough, even in spite of any occasional HQM glitches, that it would be silly and self-destructive for us to close them out.

    By not playing HQM packs, you avoid the frustration of having to play an HQM pack, but you allow other players, who are not you, to enjoy the things that they enjoy playing.
     
    MrWisski, Aug 23, 2015
    #17
  18. egons_ghost

    egons_ghost Build Team

    Messages:
    50
    Likes Received:
    10
    Local Time:
    3:46 AM
    Well i just found out from _ItzSquid_ your Ex Forum moderator that he knows the solution to issue and can off some stern advice. He stated that the reason why the Quest Books were reset was because there was a repair done on them yesterday. I would ask him to talk to you personally but my theasaurus brain and typing arent up to par to chat with him. sorry
     
    egons_ghost, Aug 23, 2015
    #18
  19. Slind

    Slind Founder

    Warlock
    10 Points
    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    9:46 AM
    It can't be fixed from our side. We have spent +80 hours on this issue but could never get far as we would need to rewrite the mode/data system of the mod. Unless your friend would write a new mod and get mod pack devs to use it instead there is nothing we could do.
     
    Slind, Aug 23, 2015
    #19
  20. danielpmc

    danielpmc Well-Known Member

    Messages:
    249
    Likes Received:
    6
    Local Time:
    8:46 AM
    I could take the old hqm mod and edit it to see if that's better[DOUBLEPOST=1440321249][/DOUBLEPOST]
    Don't think bteam has hqn
     
    danielpmc, Aug 23, 2015
    #20
Page 1 of 2

Share This Page