Info Recent network wide outages

Discussion in 'Announcements' started by Slind, Dec 23, 2016.

  1. Slind

    Slind Founder

    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    4:37 AM
    Hello MineYourMind Community,

    as you probably noticed did we have quite some difficulties with network up time recently. During the last few weeks there had been 4 network wide outages.
    Unfortunately we are not able to give you good news that it is resolved. To give you an idea of the issue here is the setup explained.

    In order for optimal up time we have our database spread across multiple servers. This means if one of the servers goes down the others can continue without downtime. The recent issues here are that for some reason the entire cluster (all connected database nodes) are freezing. This freeze is happening without any error reports or information we could use in the search of a solution.
    Therefor we are only left with trying to find out what change caused this. As this cluster had been running fine for months.

    But it is now time to set an ultimatum. Meaning if we can't resolve this issue this year with the next two freezes, we are giving up and revert to a simple single instance setup for the time being.
    I hope this gives you a good understanding of what is going on.

    I wish you all a great merry Christmas and a solution to this issue - as present- for us all :)
     
  2. Cubelex_YT

    Cubelex_YT The steampunker Lord

    Messages:
    104
    Likes Received:
    209
    Local Time:
    4:37 AM
    use some christmas magic
     
    Mouro likes this.
  3. mrmeep321

    mrmeep321 Well-Known Member

    Messages:
    57
    Likes Received:
    6
    Local Time:
    10:37 PM
    I was just able to get on regrowth. current on right now. 3:50 EST.
     
  4. Mouro

    Mouro Well-Known Member

    Messages:
    1,583
    Likes Received:
    325
    Local Time:
    3:37 AM
    You could switch the AC to warm up the room where the servers are, so they don't freeze :)
     
    Atum, cstalker, Slind and 1 other person like this.
  5. PhantomRTW

    PhantomRTW Retired Staff

    Messages:
    338
    Likes Received:
    141
    Local Time:
    10:37 PM
    Tell Lucid to keep his chickens out of the Server Rooms. :(
     
  6. NotChiken

    NotChiken Well-Known Member

    Messages:
    55
    Likes Received:
    17
    Local Time:
    4:37 AM
    @Slind Did you try turning it off and on again?
     
    grav0r, Mouro, cstalker and 2 others like this.
  7. Fracknoid

    Fracknoid Well-Known Member

    Messages:
    74
    Likes Received:
    11
    Local Time:
    10:37 PM
    Thank you for the update, and I wish you the best of luck resolving the issue.
     
  8. PhantomRTW

    PhantomRTW Retired Staff

    Messages:
    338
    Likes Received:
    141
    Local Time:
    10:37 PM
    [​IMG]

    I'm watching you, good sir!
     
    kindlich and littlejdude like this.
  9. NotChiken

    NotChiken Well-Known Member

    Messages:
    55
    Likes Received:
    17
    Local Time:
    4:37 AM
    @PhantomRTW No no Señor, no chickens here, Señor
     
    matiepie2 and Cubelex_YT like this.
  10. tractoid

    tractoid Ex-Staff

    Messages:
    284
    Likes Received:
    84
    Local Time:
    6:37 PM
    I've heard that duct tape fixes everything
     
    HelloKittyPhone likes this.
  11. chugga_fan

    chugga_fan ME 4M storage cell of knowledge, all the time

    Messages:
    5,861
    Likes Received:
    730
    Local Time:
    10:37 PM
    that's only in engineering
     
  12. EpicSkwrlBattle

    EpicSkwrlBattle Well-Known Member

    Messages:
    24
    Likes Received:
    8
    Local Time:
    10:37 PM
    maybe a little WD40 applied to it, then some strong bourban and a fine cigar while you play smooth jazz to relax the cluster's nerves.
     
    htko89 likes this.
  13. Datsaltysnek

    Datsaltysnek lover of NCIS

    Messages:
    406
    Likes Received:
    51
    Local Time:
    9:37 PM
    based off what your saying and after watching 50+ hours of tech vids on youtube what most likely is happening is a raid is failing or something or your hardware is takeing a hit from 1.10.2 forge servers
     
  14. Timkoo

    Timkoo Well-Known Member

    Messages:
    529
    Likes Received:
    39
    Local Time:
    4:37 AM
    we had Xmas 3hours ago
     
  15. wyndman

    wyndman Well-Known Member

    Messages:
    4,045
    Likes Received:
    2,014
    Local Time:
    10:37 PM
    Should've stuck with the gerbil story. So much more believable than this talk about some sort of database issue. I'm asking Santa for some new gerbils for Christmas.
     
  16. Azurith93

    Azurith93 Well-Known Member

    Messages:
    76
    Likes Received:
    13
    Local Time:
    11:37 PM
    @Slind dumb techy techy question for you. what version of Mysql are you talking about?
     
    Mouro likes this.
  17. Slind

    Slind Founder

    Messages:
    8,332
    Likes Received:
    3,018
    Local Time:
    4:37 AM
    mariadb 10.1
     
  18. Azurith93

    Azurith93 Well-Known Member

    Messages:
    76
    Likes Received:
    13
    Local Time:
    11:37 PM
    have you thought about migrating to MS SQL or oracle?
     
  19. chugga_fan

    chugga_fan ME 4M storage cell of knowledge, all the time

    Messages:
    5,861
    Likes Received:
    730
    Local Time:
    10:37 PM
    Why would this matter? MySQL works fine
     
  20. wyndman

    wyndman Well-Known Member

    Messages:
    4,045
    Likes Received:
    2,014
    Local Time:
    10:37 PM
    No one wants to know what version gerbils we're running?
     

Share This Page