STATUS: New server up and running

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • admin2
    Member
    • Jul 2013
    • 0

    #1

    STATUS: New server up and running

    Hi everyone -

    As you may have noticed, we had a major server malfunction this week which resulted in some fairly severe data corruption. Our only option to move forward was to set up a brand new server and do a complete system restore from our most recent set of complete backups. Unfortunately those backups were 4 weeks old. This means we've lost all data between roughly January 30th and February 27th.

    If you're reading this message, it means you're on our new, upgraded server. You should be able to play normally again from here on out, though you will notice that any points/trophies accrued over the past four weeks are now gone. The good news: the new server is a major step up from our last one, and should provide much, much faster response times and fewer needed maintenance outages.

    I sincerely apologize for the interruption and the headaches involved. Its been a rough week, but I'm hoping we're now through the worst of it.

    Thanks as always for playing!

    Stephen
  • LLapp
    Premium Member
    • Aug 2014
    • 462

    #2
    Thank you Stephen! And thanks for working straight into the weekend on this.

    Comment

    • abra
      Premium Member
      • Jun 2009
      • 158

      #3
      Wow, it must have been a rough few days. Thanks for fixing it.

      Comment

      • wvwoman
        Member
        • Feb 2019
        • 6

        #4
        bless his heart--we must drive him crazy!

        Comment

        • marnita
          Premium Member
          • Apr 2014
          • 42

          #5
          It's great to have things back to normal. There really was not that much disruption, all thing considered, though I am sorry for those who lost records. I for one would be lost without this site!

          Comment

          • damsel22
            Member
            • Jan 2011
            • 63

            #6
            STATUS: New server up and running

            I have tremendously enjoyed my time spent on this site over the years. Thank you, Stephen, for being genuinely caring about this site and its members. You are the best admin ever, and we are all so fortunate to be in such a great place here.

            Comment

            • bwbaer

              #7
              Thanks so much for gutting it out!

              “By "guts" I mean, grace under pressure” Earnest Hemingway

              Comment

              • jimdgar

                #8
                Stephen, thanks for the great support in keeping this and so many other puzzle sites running.

                Comment

                • admin2
                  Member
                  • Jul 2013
                  • 0

                  #9
                  Hi everyone -

                  Well, obviously, we're not quite past our server issues just yet - sorry about that! I'm working behind the scenes to iron out issues as they become apparent but we're running into a couple of issues I've frankly never even seen before, mostly having to do with newer versions of certain software packages and stricter requirements therein.

                  I'm optimizing as quickly as I can, and for what its worth I feel like we're making good progress. But it might still be a bumpy couple of days before we're completely out of the woods.

                  I want to assure everyone that this is my #1 priority, and I'm working non-stop to get things in order.

                  Thank you for your patience and understanding!

                  Stephen

                  Comment

                  • wvwoman
                    Member
                    • Feb 2019
                    • 6

                    #10
                    thanks, stephen, for everything!

                    Comment

                    • damsel22
                      Member
                      • Jan 2011
                      • 63

                      #11
                      STATUS: New server up and running

                      You are the best, Stephen. Thanks for all your hard work.

                      Comment

                      • Lurker
                        Member
                        • May 2009
                        • 379

                        #12
                        Your hard work is definitely appreciated. I'm not going back to paper-and-pencil puzzles yet!

                        Comment

                        • admin2
                          Member
                          • Jul 2013
                          • 0

                          #13
                          Quick update...

                          We found a major issue yesterday with a system process on the new server that was creating an unbelievably huge log file (700 GB by the time we caught it). That was fixed about 24 hours ago, and according to our monitoring services we haven't had any downtime issues since then. I'm keeping my fingers crossed that this was the culprit but we'll give it another few days just to be sure.

                          Thanks everyone!

                          Comment

                          • wvwoman
                            Member
                            • Feb 2019
                            • 6

                            #14
                            when i click on recent games, i'm still getting a blank screen that says "file not found."

                            Comment

                            • admin2
                              Member
                              • Jul 2013
                              • 0

                              #15
                              Hmm.... that page is working for me. Its possible you have a cached version of it stuck in your browser from when the site was buggy - can you try clearing your browser's cache and see if that helps?

                              Comment

                              Working...