Lag lag lag

[Bheale]

This very long delay then catch up symptom smells more like packet loss leading to TCP timeouts/retries/window collapse. Would guess that the server enforces the GCD pretty strictly and therefore responding to many events at once means the server kept up while everything was going on, but the traffic from server back to client was delayed.

To totally speculate: let's say RIFT is served by blade servers in a hosting center, and that TRION has blades, talking to network switches in blade chassis, talking to a higher level switch for the whole cage/cluster/installation, talking to the one line they have to the router to the internet.

That means if there is a burst of traffic from the servers, those switches have to have enough space to hold onto all the packets until they can drain into the router at the rate of the one line in. If there isn't enough space, packets get dropped. Every time a packet gets dropped, the client waits a 2 second TCP timeout, then reports back to the server something got lost. The server then retransmits (and slows down its max transmission rate for that particular connection). A storm of such retransmissions can of course further overload the network, which of course can get rather ugly. Ugly, like, say, 15 seconds of traffic from server to client waiting and then all being delivered at the same time.

If there's enough network bandwidth for what the servers are trying to do, the fix is putting a deep buffered switch in between the blade enclosure servers and the uplink to the router to the internet.

If there isn't enough network bandwidth, the fix is either tuning the application to send less data, or writing a check for a bigger internet connection.

But again, this is all just speculation...

[Asidia]

Those latencies look like waiting for TCP timeouts and retries to me, not like the server actually got that far behind.

Then you have no idea what the lag was like. This was all taken during conquest and other stuff when conquest was on.

[bamabenny]

Still waiting for some sort of response or acknowledgement on this. Hard to meet enrage requirements on mobs when your entire raid has 2-3s ability lag on everything.

[Bheale]

Then you have no idea what the lag was like. This was all taken during conquest and other stuff when conquest was on.

Agreed a few TCP drops on the internet backbone wouldn't come close to this.

My point was that when round trip times go out to 8 seconds and then recover, that isn't the server getting behind and then catching up, that's repeated recovery from network packet loss.

Given that it occurs for "everyone" in the raid but not exactly consistent, and that it's sustained over a period of time (ie the average DPS is missing 1 GCD out of 4 or 5 over a 15 minute period) I would speculate that it's a packet loss - retry performance collapse occurring adjacent to Trion's servers, not out on the internet.

To oversimplify, TCP packet loss recovery occurs in increments of 2 seconds (the TCP timeout). If a retry storm is going on at 2 seconds, your traffic is likely dropped again, trying again at 4 seconds, 6 seconds, etc. The internet will drop packets more than 2 seconds old. So if you're seeing latencies above 2 seconds, they involve packet drops and retries. The only question left is whether it's your packets to the server which are being lost, or the traffic from the server back to you. Observing the results of 4 or 5 GCDs worth of actions suddenly played back means it's the traffic from the server to you. (If it were your traffic to the server, the server would have rejected all but one of the actions because they came before the GCD was up.)

So TCP drops are worse than the server getting behind, and that's what I think the picture showed...

speculating, of course.

[Sunfire]

Still waiting for some sort of response or acknowledgement on this. Hard to meet enrage requirements on mobs when your entire raid has 2-3s ability lag on everything.

Yea trying to do Rusilla with 3-5s ability lag was beyond stupid - we just cancelled our raids for the rest of the week and are in "wait and see mode" as far as this game overall is concerned.

[bamabenny]

Yea trying to do Rusilla with 3-5s ability lag was beyond stupid - we just cancelled our raids for the rest of the week and are in "wait and see mode" as far as this game overall is concerned.

We're in the same boat for now. Hopefully there are some fixes coming.

[Arir]

I was just standing in Ember Isle and my performance was in the red. I looked and my latency was over 2800. Its been like this since the streaming client was installed. This may be the final straw with me. Conquest bombed, the summerfest is a dud. I don't think I am going to pay for an expansion pack if this is the best they can do.

[Kulta]

I played for 9 months (paid time) actually started in beta, I have never seen lag like it is now. Not even in Beta. I use to run on ultra settings but now even at very minimum I lag and lag and lag. If in a town or even out pveing by myself I get loading screens and lag spikes

In Conquest it is INSANE lag, you lag or crash, come back and dead. I have more loading screens here now than there was in GW1 and that is saying ALOT.


Can we get a simply acknowledgement and reassurance this is being worked on? I mean getting it fixed

[McWaffles]

THIS IS ABSURD!

Can't raid with the lag, it's too terrible.

[Kutiel]

http://imgur.com/i6rOE

[Zyzyx]

Shut down conquest during prime time so we can raid plz tia.

[mistacrowley420]

Even lagging in my warfronts. If you do not have a situation in the forseeable futue please let me know so I can go elsewhere until this problem is fixed. This is preventing all types of gameplay for people, every single thing I have done has been plagued with this intolerable lag.

[Aelan]

Way to go Trion. You've lost me as a customer. How many more do you need to lose before you fix the problem?

[bamabenny]

Having to pause any raid you're doing for the duration of a Conquest instance before continuing = not cool, at all.

[Zyzyx]

Having to pause any raid you're doing for the duration of a Conquest instance before continuing = not cool, at all.

I see you are about to kill laethys with all of the bugs...

HAVE SOME LAG!!!!