Firstly note that I am speaking from an EU perspective and not US.
Special Olympics are currently on Blightweald shard a.k.a 'Deadweald'. We knew this when we transferred and its perfectly lag free for out main raiding however it presented an interesting predicament for my alts.
Do I leave them on Icewatch, by far and away the most active EU shard, or bring them with my main for crafting/support purposes. After experimenting with splitting my four characters across the 3 (excluding the RP shard) remaining EU shards I found that I was struggling money/crafting wise to sustain raid consumables for any and all of my characters. Thus I consolidated my warrior and rogue to Blightweald so I had all trade skills together. The downside to this is they struggle to find weekly groups or better yet HK pugs.
A year on from entering HK and I still love the raid, more so than the new and finished ID, and would happily do HK 3 or 4 times a week on all my characters. But with the way 'serious' raiding guilds (defiant) in EU are spread:
Bloodiron: Vengeance, eXile
Blightweald: Special Olympics, Heresy, MOIRAE
The skilled playerbase is too thinly spread making 'competent' HK pugs next to impossibe. Within the above guilds I know there is 20-30 HK and ID ready alts and a good solid alt raiding core could exist.
Now some say half the problem is me being elitist in requiring 11/11 when I put "LFM HK" but when you know your alt along with many many others are capable of clearing HK then you don't really want sicaron or grug wipes. Yet even on Icewatch at peak population requiring 11/11 resulted in a raid not filling, however with cross shard invites and raid groups this problem wouldn't exist.
A further benefit for cross shard invites and raid groups would be for trialing members, if you recruit someone from another shard. This prevents issues of if they're not good enough and transfer off a locked shard just to trial they are then stuck on your server.
TLDR - Cross shard invites would make pugs a lot more accessible to EU (and I'm sure US) players, especially the more serious guilds who make the difficult decision of lower population shards for better quality raiding.