Posted on 16 November 2015 - 12:24 PM

So I have been doing research all of yesterday and this morning and a lot of tests…like a lot.

After a lot of thinking we have decided that at the moment we are going to try a new OVH solution that includes the better parts of our old system when we were in Virginia/Amsterdam and the good DDoS protection that OVH offers.

The different network setups

Our current setup

Player <-> Outside Internet <-> OVH Firewalls <-> OVH VAC (Anti-DDoS Layer 1) <-> OVH Tilera (Game-Anti-DDoS) <-> OVH Arbor (Anti-DDoS Layer 2) <-> Game Server (Bungee) <-> OVH VAC (Anti-DDoS Layer 1) <-> OVH Tilera (Game-Anti-DDoS) <-> OVH Arbor (Anti-DDoS Layer 2) <-> Game Server (Arena)

This is a fucking mess and has proven to be a disaster. When originally purchasing these game servers I did not know that traffic between game servers is re-routed through their Anti-DDoS systems. This is why there is extra rod/bow/potion delay and why there have been so many mini lag spikes in EU and NA. The new solution is the one I would have originally gone with if I had known this extra information earlier.

The new setup

Player <-> Outside Internet <-> OVH Firewalls <-> OVH VAC (Anti-DDoS Layer 1) <-> OVH Arbor (Anti-DDoS Layer 2) <-> Infrastructure Server (Bungee) <-> Private Network <-> Infrastructure Server (Arena)

At OVH the Infrastructure servers allow for a private network (which they call VRack) to be setup. This allows us to have our servers communicate directly with each other without any extra filtering of any sort. This is what we need for optimal performance.

I will also show what we had with our last host to make players understand why this old model (with Virgina/Amsterdam) had issues too.

The setup we had at our last host

Player <-> Outside Internet <-> Staminus Firewalls <-> Staminus Anti-DDoS <-> Outside Internet <-> SecuredServers Anti-DDoS <-> SecuredServers (Bungee) <-> Private Network <-> SecuredServers (Arena)

In this model we are going through the outside internet twice. We are also going through two different layers of Anti-DDoS services. Not to mention that the routing points for traffic were only in Los Angeles, NYC, and Amsterdam (Staminus). People that live in the Midwest of the USA got absolutely fucked with high pings having to route their traffic to LA and then to our datacenter in Virginia.

Q & A

Why are we not moving out of OVH yet?

The reason is DDoS protection. The fact is that their new Game AntiDDoS system is shit. Their regular protection (VAC and Arbor) are still really good. I really really really wish we had never moved to the game servers from the original servers we had when we first moved to OVH. A lot of these problems could have been avoided. The data centers are also so massive that they are able to withstand huge attacks that most other data centers could not.

The reason we moved to the game servers is that they were supposed to offer special DDoS protection for Minecraft. Their protection for Minecraft is shit. Honestly figuring out if I want to put a lawsuit against them for false advertisement. It does not protect Minecraft at all.

What happens if we are still having major issues?

After this new solution is in place I will be doing further research at other data centers to have a backup plan in place in case this does not go well. Based on all of the things I have been monitoring/testing with I feel confident in saying that we will probably be ok with OVH, but I will have a backup just in case.

What the hell happened to NA on Saturday night?

The OVH VAC system in BHS (NA) had some hardware parts fail when a huge DDoS attack hit. These types of things (like the fiber cut a few weeks ago) are completely unpredictable. The entire DC was having issues because some of the line cards in their system burned out. They replaced them within a few hours and have not been having issues with it since.

What is the biggest issue with the game servers?

1. That we have to route through all of their protection a second time for no reason
2. That their Tilera system can't handle the traffic going through them (EU was the first to notice these issues. The system was being capped out during peak times. Now NA is having these same problems)
3. When a huge attack hits the Tilera (even if we are not the ones being attacked) we are still affected 2 times instead of 1

When will the transfers be done?

I am hoping to have all transfers of the servers done within the next 12 hours. I have already pulled off most of the NA servers to back up and prepare for transfer once I have the new servers setup. Once NA is done I will turn it back on and then move EU while they are asleep (later part of my day).

Closing Statement

I hope that this clarifies what we are doing and the reasoning behind it. Please note that just because I am not making posts everyday does not mean I am not working on things. It just means I don't have conclusive/precise information to share yet about what is going on. I only make a post when I am 100% sure on something and I try not to half ass things.

Posted on 16 November 2015 - 12:25 PM

thx for the hard word

Posted on 16 November 2015 - 12:32 PM

Thank you for finally informing us at what is happening in regards to the lag, nice to know that there is a plan going forward.

Posted on 16 November 2015 - 12:33 PM

Thank you so much

Posted on 16 November 2015 - 12:37 PM

You guys r amazing cheers homies

Posted on 16 November 2015 - 12:37 PM

Thx u for the information and the hard work

Posted on 16 November 2015 - 12:40 PM

yeeeeeeeeeees

Posted on 16 November 2015 - 12:53 PM

amazing how something so small can screw something up big time

def not worth the extra money for those servers then?

Posted on 16 November 2015 - 01:11 PM

I'm sorry? Can you elaborate?

Posted on 16 November 2015 - 01:20 PM

Thanks for all the hard work admins. I hope all these problems are resolved soon!

Posted on 16 November 2015 - 01:43 PM

Cool :)

Posted on 16 November 2015 - 02:24 PM

Thanks so much it's great to know there's a plan and what it is :D

Posted on 16 November 2015 - 02:54 PM

Thats great Gberry Im so glad you worked this hard on this Thank you for all your hard work.

Posted on 16 November 2015 - 03:03 PM

https://wemovetogether.files.wordpress.com/2012/11/snoopy_happy_dance.jpg

Posted on 16 November 2015 - 03:51 PM

Can i get a deal on a shirt

Posted on 16 November 2015 - 04:03 PM

Thank you so much ;D

Posted on 16 November 2015 - 04:04 PM

I bet 80% dont know what ur saying lmfao


I dont fully either

Posted on 16 November 2015 - 04:05 PM

So the host in moving back to Virginia?

Posted on 16 November 2015 - 04:17 PM

Yay! Thanks for your hard work :D

Posted on 16 November 2015 - 04:38 PM

I know this doesn't matter at all in terms of importance, but does this mean that the "ghosting" effect and the "laggy knockback" should be stabilized or will we just have to see? Like I said not crazy important seeing as your priority is to make the server playable, but just a curiosity question and sorry if this has already been asked in advance.