Make Sulfuras great again. [Incorrect proc-rate]

[Storfan 24]

This is a copy of the bug-report I have made on github. I will also post this thread on Shaman and Paladin forums. What I would like from anyone else using this weapon is for you to test it on your own and report back in the thread, or as a bugreport on github your findings. Should anyone also happen to have it on their retail-characters please test there aswell. I would also like to point out that the Addons called ProcWatch and ProcWatcher is NOT reliable (I repeat, NOT reliable!) because they dont count all hits correctly. I have however noticed that DPSmate does infact work great for this purpose. Now, onto the report.

 

--------------------------------------------------------------------------------------------------------------------------------------------

BUGGED/INCORRECT BEHAVIOR:
Through extensive testing I have determined that the current proc chance on Elysium (Anathema) is 4.87% (Probably set to 5% by Devs). However, it is actually 6% on retail live server.

--------------------------------------------------------------------------------------------------------------------------------------------

Anathema testing method:
Recorded with the addon "DPSmate".

3179 hits only with sulfuras and only with abilities that are able to trigger the weapons proc.

- 155 procs
- Proc chance: 155/3179 = 0,0487574709027996 = 4.87%
- Screenshot: http://imgur.com/a/BSqG9

--------------------------------------------------------------------------------------------------------------------------------------------

Retail-live Testing method:
Recorded with the addon "Recount".

10k hits on a Target dummy in Ironforge.

- 619 procs
- Proc chance: 619/10001 = 6.19%
- Screenshot: http://imgur.com/a/p8iL2

--------------------------------------------------------------------------------------------------------------------------------------------

CONSLUSION: Devs have assigned Sulfuras a 5% proc-rate because of lack of in-depht testing. 5% is a "nice and even" number and could very well have been the actual rate. However, after 10k hits it has now been proven beyond a resonable doubt that the proc-rate is infact 6%. A 1% deviation is simply too large when it comes to a pool of 10k hits to chalk down to RNG.

[Guest]

1 час назад, Storfan сказал:

This is a copy of bug-report

[Justme 9]

such small data. You could've gotten 0 hits on either client

[Storfan 24]

7 minutes ago, Justme said:

such small data. You could've gotten 0 hits on either client

I'm not sure what you mean, or if it's sarcasm. Are you saying that 3k and 10k hits is a too small sample size? Normally even 1k hits will give you a pretty good idea of an items proc-rate. Even if 3k hits is too small, in the end the devs know what proc-rate they have assigned to the weapon so technically the only interesting sample is the retail test and 10k hits is definitely enough to give you a very solid proc-rate model.

TLDR: Devs know if they assigned Sulfuras with 4, 5 or even 6% proc-rate, all they have to do is look at the 10k hits that proves it to be 6% and adjust accordingly.

Edited March 27, 2017 by Storfan

[Justme 9]

With 10k hits and 1000 procs it could still be a 5% proc chance

[Lorilay 12]

8 hours ago, Justme said:

With 10k hits and 1000 procs it could still be a 5% proc chance

That's quite easy to test.

Sample 1: 6.19%, n=10,000

Sample 2 (expected): 5.00%, n=10,000

X^2 = 13.404 (1 degree of freedom)

p-value = 0.0003

 

TL;DR: It's extremely unlikely (0.03%) that with a sample size of 10,000 that you would see a difference of 1.19% based on random chance.

Edited March 27, 2017 by Lorilay

[Zarant 2]

I went through the elysium DB myself, and I found out that its proc rate is supposed to be 1 ppm (6.17%). What is your testing methodology? Maybe this is just a symptom of a larger issue.

From my own testings and from looking through the source code, I found out that in order to trigger weapon based procs, the ability must require a melee weapon and deal damage, which excludes bloodthirst, pummel and sunder armor. There is a small exception for shield bash (not shield slam) and I'm not totally sure about execute.

Execute does fall into that category but my own testing indicates that it can't trigger any kind of proc. It's kinda difficult to know for sure because i can't usually cast execute in a controlled environment. Pretty much all of my executes are overkill, i know that other abilities still triggers main hand procs from overkill and maybe its different with execute, I'm not 100% sure. I need to find out a clever way to test it, because currently I need more people to help me execute some large hp mob for long periods of time and I just can't be arsed with that.

Edited March 27, 2017 by Zarant

[Undertanker 88]

2 hours ago, Zarant said:

I need to find out a clever way to test it

Blasted Lands Elites near the Dark Portal.

[Storfan 24]

6 hours ago, Zarant said:

I went through the elysium DB myself, and I found out that its proc rate is supposed to be 1 ppm (6.17%). What is your testing methodology? Maybe this is just a symptom of a larger issue.

From my own testings and from looking through the source code, I found out that in order to trigger weapon based procs, the ability must require a melee weapon and deal damage, which excludes bloodthirst, pummel and sunder armor. There is a small exception for shield bash (not shield slam) and I'm not totally sure about execute.

Execute does fall into that category but my own testing indicates that it can't trigger any kind of proc. It's kinda difficult to know for sure because i can't usually cast execute in a controlled environment. Pretty much all of my executes are overkill, i know that other abilities still triggers main hand procs from overkill and maybe its different with execute, I'm not 100% sure. I need to find out a clever way to test it, because currently I need more people to help me execute some large hp mob for long periods of time and I just can't be arsed with that.

I actually considered this as I know some other procs wont trigger from e.g Sunder armor (I think TF doesnt?) but I too havnt been able to figured out a good to test and confirm this theory. If this is the case, it will mean that it can only prroc from from Autohit, cleave, WW and OP.

3 hours ago, Undertanker said:

Blasted Lands Elites near the Dark Portal.

I actually went there first to test the proc but soon found out that the mobs hit me too hard so I have to retreat after a minute or so. I guess a paladin would be much better suited for such a test given their ability to heal themselves.

[Undertanker 88]

A paladin can't test execute which is what he was commenting on a good place for testing execute.

 

Bring a friend.

[Storfan 24]

13 minutes ago, Undertanker said:

A paladin can't test execute which is what he was commenting on a good place for testing execute.

 

Bring a friend.

But they can test just autohitting and see if they get a higher proc-rate as a result, thus confirming that some abilities doesnt trigger proc.

[Zarant 2]

5 hours ago, Undertanker said:

Blasted Lands Elites near the Dark Portal.

That is not clever enough for my taste. You need way too many people in order to get a decent size sample.

[Undertanker 88]

1 Pally healer.

[Storfan 24]

13 minutes ago, Zarant said:

That is not clever enough for my taste. You need way too many people in order to get a decent size sample.

What do you mean? 10k autohits done by (say, a paladin) should be more than enough of a sample size.

10.000 hits

3.7 speed

37000 seconds = 10.27 hours to complete the test.

12 minutes ago, Undertanker said:

1 Pally healer.

I'm horde though, and I dont think I'm friendly enough with any healer to stay with me for 10h autohitting a mob.

[Fideus 0]

I have collected some data regarding this.

On 4k hits, just auto hit killing mobs i got 192 Fireball procs, which is 4.77%. https://gyazo.com/f085211498e539c5a2ff1ddfecf6d6bb

On the same test, above 4k hits, i got 264 procs of Darkmoon Card Maelstorm, which is 6.6% https://gyazo.com/fca69dbacab3b42f26b3716b5c9a30e0

Notice the difference in number of procs tho, to my knowledge, both items should have almost the same proc chance.

I have some data, sample around 1k hits, which includes paladin abilities, Seal of Command and Judgement of Command, which are considered melee attacks and can trigger weapon procs(this is 100% blizzlike). Here you can see that both Fireball and DCM proc almost same amount of time, around 6%. Tho sample is smaller, i think its sufficient and indicative of certain issue with proc mechanic and how it works. https://gyazo.com/581923e5ae4f796a66d8f468a3ebc77f https://gyazo.com/a09609e890044d6fb6ce7c322aed8735
 

14 hours ago, Zarant said:

I went through the elysium DB myself, and I found out that its proc rate is supposed to be 1 ppm (6.17%). What is your testing methodology? Maybe this is just a symptom of a larger issue.

I woud agree with this. Inconsistency in proc chance, diference between Sulfuras and DCM in given sample, tells me something regarding PPM and % on hit mechanic is not working right.

I can get bigger sample of lets say 10k hits, with/or without special abilities, if needed, but for now and sake of argument i think this is sufficient to show something is not working right(imho). Maybe these numbers can help someone more knowledgeable about mechanics  to see if this is working as intended or not.

I have no knowledge on warrior mechanics to know what should or should not trigger weapon procs.

[killerduki 54]

On 28.3.2017 at 9:48 AM, Storfan said:

What do you mean? 10k autohits done by (say, a paladin) should be more than enough of a sample size.

10.000 hits

3.7 speed

37000 seconds = 10.27 hours to complete the test.

I'm horde though, and I dont think I'm friendly enough with any healer to stay with me for 10h autohitting a mob.

Speaking of Paladin , make sure they don't use Seal/Judge of Command/Righteousness.

Otherwise it will end up higher proc rate.

/Kind regards Killerduki