ID Player Testing Failed

[Gyle]

I know i am going to get a fair amount of blow back from fellow testers on this, but i think it is clearly the truth. All player testing in ID managed to do was cause unnecessary drama and give a perceived (and in some cases real) advantage to the testing guilds. The argument for having so many players test ID content was that it was supposed to reduce bugs and make for more polished content upon release. Did we actually see that?


Drak - Pets getting explody bubbles if parked in the back of the room.


Fire Mini - Despawned and never came back, then the wall that never went down, crazy agro range, adds despawning immediately.


Ituzial - useless enrage timer, as well the whole point was to force you to kite him in a circle, which no one does because the encounter is not actually designed to make you do so. Also his AoE damage was too low at release. (This was probably the most polished fight at release)


Conclave - Hard mode is easier than normal mode, and Packmaster P3 was actually impossible. Also, again, useless enrage timer. Also the combat start is really buggy and still has not been addressed.


Scryer - Do i need to go there? Not a single thing on this encounter has worked right yet, insane agro range, also it still has no enrage timer.


Rusila - Reflects on Dread Shot, useless enrage timer, half the mechanics not actually working reliably, no snipers.


And i am sure there are some bugs that i left out. When HK was released it was buggy, for sure, but the bugs made it more difficult. In ID the bugs have made things much easier, leading IMO to most of the hardcore raiding scene's ire. Had ID been released as bug free and polished as it was supposed to be, there would have been MUCH less QQ from the top guilds, and there would have actually been some sort of a progression race.


In the future Trion needs to figure out a different way of dealing with their raid testing. All ID's testing did was create drama, give people insight into encounters before they were released, and give us an instance with just as many bugs, or at least IMO way too many bugs to justify the problems it caused.

[mavfin87]

In the future Trion needs to figure out a different way of dealing with their raid testing. All ID's testing did was create drama, give people insight into encounters before they were released, and give us an instance with just as many bugs, or at least IMO way too many bugs to justify the problems it caused.

So do it on PTS and post the schedules. Let anyone who thinks they can handle it see it. Sure, the World First guilds will get advantages, but they ALL will get the same, and, Trion can get much more varied testing. I think the mistake may have been the focused testing that didn't test outliers and such that people WiLL try.

[Muspel]

So do it on PTS and post the schedules. Let anyone who thinks they can handle it see it. Sure, the World First guilds will get advantages, but they ALL will get the same, and, Trion can get much more varied testing. I think the mistake may have been the focused testing that didn't test outliers and such that people WiLL try.

I'm inclined to agree.

Simply put, if you want to find bugs, the simplest way is to let as many people bang away at it as possible. There would absolutely still be bugs that slip through (relatively few people test things on the PTR, after all), but I'd bet good money that there would be a fair chunk less, and major issues/exploits would become far more apparent.

That being said, if they're looking for balance feedback, then it's still a good idea to do it in a more targeted way, because otherwise, you get dramatically varying input from different kinds of players, and it's not always simple to distinguish between them. (If anyone here ever read the feedback for raid testing in WoW, they'll know what I'm talking about, since the posts were an equal split between "too easy", "too hard", and "just right", because you had such a wide range of people trying out the encounters).

[Vioarr]

Give it a rest already, your constant complaining on the forums was old 1981 posts ago.

[TheGrinnz]

I'm inclined to agree.

Simply put, if you want to find bugs, the simplest way is to let as many people bang away at it as possible. There would absolutely still be bugs that slip through (relatively few people test things on the PTR, after all), but I'd bet good money that there would be a fair chunk less, and major issues/exploits would become far more apparent.

That being said, if they're looking for balance feedback, then it's still a good idea to do it in a more targeted way, because otherwise, you get dramatically varying input from different kinds of players, and it's not always simple to distinguish between them. (If anyone here ever read the feedback for raid testing in WoW, they'll know what I'm talking about, since the posts were an equal split between "too easy", "too hard", and "just right", because you had such a wide range of people trying out the encounters).

One issue is, they cannot simply have people test whatever they please. This leads to what everyone was accusing the testing guilds of for some reason, developing strategies for the bosses well before release, but at a much larger scale. So how do you restrict different groups of people to test different parts of different content, when it is open to the public for anyone to stumble in, with a limited development staff to oversee the operation?

[Vioarr]

Quantity testing is not as effective as quality testing and a test without a dev observing is not quality. No method of testing will bring to light all bugs so the argument that bugs existed so testing was a failure is short sighted and should not be taken seriously.

[TheGrinnz]

Quantity testing is not as effective as quality testing and a test without a dev observing is not quality. No method of testing will bring to light all bugs so the argument that bugs existed so testing was a failure is short sighted and should not be taken seriously.

I wouldn't go that far, but it is a large unsupported leap of incorrect logic.

[Vioarr]

I wouldn't go that far, but it is a large unsupported leap of incorrect logic.

Explain the implication that my statement is extreme.

[Leif]

It looks more and more apparent that player testers were not needed (with bugs being about as prevalent as in other non player tested dungeons). What is needed is better testing on the developer's end.

I would hope that the raid developers have a specific checklist to go over for every encounter that covers very common issues (can pets cheese a mechanic, can abilities be ignored which shouldn't be, is enrage timer accurate, etc) but it seems like that is less and less of a possibility.

Some of these bugs probably wouldn't have been found by a developer doing their own play testing, and that's pretty forgivable, but most of these things a group of 2 or 3 developers could have discovered by turning on GM god mode and playing through the encounter.

[Vioarr]

It looks more and more apparent that player testers were not needed (with bugs being about as prevalent as in other non player tested dungeons). What is needed is better testing on the developer's end.

I would hope that the raid developers have a specific checklist to go over for every encounter that covers very common issues (can pets cheese a mechanic, can abilities be ignored which shouldn't be, is enrage timer accurate, etc) but it seems like that is less and less of a possibility.

Some of these bugs probably wouldn't have been found by a developer doing their own play testing, and that's pretty forgivable, but most of these things a group of 2 or 3 developers could have discovered by turning on GM god mode and playing through the encounter.

You have no idea what the state of encounters were prior to testing. What dungeons were not player tested, please enlighten me. I don't know why I am even wasting time responding to you because you obviously have absolutely no clue what you are talking about based on your assumptions and idiotic suggestions like "devs should have a checklist" as if these guys havent been doing this for years across multiple major titles. If you think a "checklist" is some revolutionary idea then you need to refrain from making further suggestions.

[Muffin911]

Didn't your guild test ID as well?

Also, excluding the bugs on the earth wing bosses when they were released, no bugs in the fire wings are *Game Breaking*. Yeah there are minor bugs in the fire wing but it's nothing that made the fights trivial such as the bugs for the earth wing bosses.

I don't know what you mean by the start of the combat on Conclave being buggy, they agro one at a time starting with Ereetu and have a few second delay between each other but to my knowledge that was intended. I'm also not aware of Ituziel's AoE damage being increased in one of the hotfix so unless I'm missing anything he hasn't changed. Whether or not your guild uses the circle is irrelevant, there are more then 1 way to kill bosses.

I don't know how you even blame this on the players either, they test encounters and often not knowing what's intended vs what's not intended. It's not like we know every single mechanic in a fight like the devs do. They have the code for the bosses and tweak the data not players. We test and give our opinion / feedback based on the boss and his mechanics. If data gets altered and we happen to test the same boss in the future then we could give another round of feedback for this.

I could be wrong here but I don't think guilds tested the final version of bosses before they were released on Live. (Or if they did, it certainly wasn't the majority of bosses). I believe the reason for this is that Trion doesn't want to reveal all the cards for a boss which is understandable. Regardless of this, I think we can agree that earth wing bosses could of used more testing closer to their final version (regardless if it were guilds testing or internal testing). I think Drak and the fire wing bosses were properly tuned when they were released. I also don't believe they were intended to be very hard from the beginning (I think HM Conclave should have been harder for sure but that's about it).

From my perspective, I wish every bosses in ID were harder but there's just so much you can do with 20 people. Also, I believe Trion's intention for bosses in ID was not to take too long to kill (no more 15-20 minute fights). Taking this part out, that leaves encounters intended to be between 5 to 10 minutes in duration (Maybe a little higher for the dragons not sure). Enraged timers or Soft enrage mechanics are a good way to set time limits for boss but adjusting them to the top end guilds are only going to have a negative impact on all the guilds that are currently progressing in ID.

[Muffin911]

It looks more and more apparent that player testers were not needed (with bugs being about as prevalent as in other non player tested dungeons). What is needed is better testing on the developer's end.

I would hope that the raid developers have a specific checklist to go over for every encounter that covers very common issues (can pets cheese a mechanic, can abilities be ignored which shouldn't be, is enrage timer accurate, etc) but it seems like that is less and less of a possibility.

Some of these bugs probably wouldn't have been found by a developer doing their own play testing, and that's pretty forgivable, but most of these things a group of 2 or 3 developers could have discovered by turning on GM god mode and playing through the encounter.

You play HK on release? If you want to speak about bugs then ID is peanuts compared to HK.

[OwnaGeJuiCe]

leading IMO to most of the hardcore raiding scene's ire. Had ID been released as bug free and polished as it was supposed to be, there would have been MUCH less QQ from the top guilds, and there would have actually been some sort of a progression race.

Weren't the "top guilds" responsible for reporting these things? I don't know how the testing works since my guild wasn't selected but a lot of these things were fairly obvious from the start. Again this is just speculations and cause the drama because we can never know the truth about what actually happens. If there is not truth to that then I apologize to the testers.

[Kutiel]

To be honest ID has been pretty good vs HK. Things get missed, it happens, it's annoying but it gets fixed. I've had a lot of fun in ID so far.

[Leif]

You have no idea what the state of encounters were prior to testing. What dungeons were not player tested, please enlighten me. I don't know why I am even wasting time responding to you because you obviously have absolutely no clue what you are talking about based on your assumptions and idiotic suggestions like "devs should have a checklist" as if these guys havent been doing this for years across multiple major titles. If you think a "checklist" is some revolutionary idea then you need to refrain from making further suggestions.

Learn to read. I assume they already have a checklist and a thorough QA process for content in place. That process just doesn't catch some of those things that should be obvious during such QA testing. Ergo, it seems less and less likely that some of these things are even being checked for.

Also, unless you want to work under the assumption that every dungeon and raid has had the same level of, or any, player testing as ID, I can draw my comparisons across the population of instances since launch. Unless you want to take a stab at enlightening me with what must be your expansive knowledge of what players have and haven't tested prior to live.