Simulation: DPS Volatility

by Tachyon on Tue. 08. June 2010, 23:20

Filed under: simulation, theorycrafting, dps

Hey folks, it's been a while since my last post, sorry for that. I've been raiding with my new fire specs during the last weeks, and we managed to kill the Lich King in ICC 25, and are now working on the ICC 25 HC bosses (9/12 so far).



I've also been doing some research on stats scaling for the fire spec, but haven't drawn a conclusion yet. Some sources (like Rawr) state that crit rating is the best stat by far, but given the nature of the Ignite mechanics (ignite munching, losing a huge ignite bank when your target dies) I have some sincere doubts. Thaarsis (one of the other mages in my guild) pointed me to Greatschock <For the Horde>'s Armory profile (you remember, the first mage which participated in a 25 HC LK kill) who had replaced all his yellow gems with pure +20 crit rating gems. We both were puzzeled, but by now Greatschock went back to SP/crit gems, seems like too much crit was not that great after all.

Limits of the old simulation

For the Arcane posting series, I used a simulation program I wrote to observe and analyze the most interesting aspects of the arcane spec (spell rotations, the impact of the T10 boni, stats scaling).
Simulating arcane rotations was fairly easy, every spell does direct damage, and stuff procs on sucessful spell casts rather than on inpact, which was easy to model.

But now that I want to observe the Fire spec in detail, I have to rewrite the whole simulation from scratch. Fire has effects that proc on spell impact, so you have to take the flight time into consideration. Some effects even have a life of their own (hello Ignite!), so a radical new approach was necessary.

New simulation

The new casting simulation I'm writing is event driven, which eases dealing with concurrent effects (such as the Ignite proc still ticking, the player casting a new spell when the last spell is still on its flight to the target). Everything in the simulation is an Event, and can be scheduled for execution in the Simulation on a given time. Events can interact with the simulation, spawn and schedule new events and even reschedule themselves. Event execution can even be observed using Listeners, making it easy to modularize aspects of the simulation.

The simulation API is not yet complete, but it's growing more and more mature.
It may be a little late for the fire spec, regarding that the Cataclysm expansion is already in the beta, but when the simulation's done I think it will be quite easy to adapt it to the Cataclysm spell mechanics, so it's well worth developing it.

The simulation also doesn't use a fixed player stat model, but allows to take a character out of the armory, and take all individual stats and talent point distribution as well as glyph choices, raid buffs and temporary powerups into account.

I've only modelled arcane so far (so I can compare the outputs of the old and new simulation), but at least I can share some effects I'm dealing with at the moment, or rather one effect in particular: Volatility.

Volatility

As a great part of the logic and results depends on the RNG (random number generator), say whether your spell hits, crits or misses, and the procs involved, the outcome will vary each time the simulation is run. This variance is significant and tells a lot on how to interpret simulated results.

For testing purpose, I did 10 runs with my mage's stats, using an arcane rotation on a 5min fight (no raid buffs, no AP/IV etc). Here's the DPS numbers that came out:

10 Runs:


Once I had less than 6800 DPS, and another time over 7200 DPS, using the same rotation, same stats and same duration. When the outcome is so volatile, it's nigh impossible to draw any conclusions when you vary input parameters.
Any DPS tests you do ingame (be it on the target dummies, or in a raid) are rendered obsolete, as you just cannot obtain enough samples to observe what happens when you do something different. You could test on a target dummy, swap some gems to a useless stat (e.g. Agility), test again and have an increased DPS result.

I did some more runs with the simulation:

1'000 Runs:


Those of you with a background in statistics recognize a gaussian distribution in the samples, and if we do event more runs the bell shape of the distribution curve is obvious:

100'000 Runs:


Look at the x-scale to see how far the minimum and maximum value are apart!
What this means is that to be able to draw conclusions on the average DPS for a given situation, the simulation has to do serious number crunching and do lots and lots of runs to have a stable outcome.

Fascinating, isnt't it? I hope you just learned to not to trust any observations that are drawn out of a handful of samples

Stay tuned for more articles on the progress of the simulation!

EDIT 2010-06-12: Sorry, forgot to enable comments (was disabled by default), commenting is now allowed

• 8 Comments