Ok, simple thing. Optimizing rotations can be easier if you can see when cooldowns are up/durations expire and plot it on a timeline. So I made a simple inkscape file for my (26) rogue to see how hard it would be. I've not accounted for energy costs, and the numbers are all from the tooltips on my character when I made this, so just use it as an example. I should probably colour and overlap the abilities to make rotation generation take less space.
http://www.multiupload.com/269RM54GHD

It seems to help a bit. Mostly obvious (use dots before abilities with channeled cast times, refresh things when they run out and are off cooldown, etc), but it can be nice to have a premade rotation. (Until you get hit with an interrupt/knockback, of course.)

TL;DR: Play with blocks to optimize an ability rotation. Expandable to any class and spec.

Automating rotation optimization is an interesting problem. I think brute-force is around O(n!) time and thus rather impractical. Need to come up with a good algorithm.