+ Reply to Thread
Results 1 to 4 of 4

Thread: Data messaging logic discussion

  1. #1
    RIFT Guide Writer Redcruxs's Avatar
    Join Date
    Jan 2011
    Posts
    3,935

    Default Data messaging logic discussion

    Hey guys, was wondering if anyone want interested in sharing any knowledge they have about the most efficient way to transmit certain data across addon users. Sort of a messaging API "best practices" discussion

    I'll open with an example from RaidFinder:

    Currently I have each person who is LFG or LFM sending out a channel broadcast once every 15 seconds. They can only send either a LFG message or LFM message, not both.

    On the Recieving end the player adds all the broadcasts that they recieve into a table which updates the grid once every 15 seconds (so as not to spam update the list every time a message is recieved)

    Then, when players want to communicate directly with eachother for a raid invite the messages are sent directly, but if the direct send methods fail it broadcasts a tell to them, if that fails it broadcasts across the channel but gets ignored by anyone who isn't the intended recipient. I had to use this tripple redundant sending because for some reason the Send and Broadcast tell are not 100% reliable. I was unable to find what was causing them to fail in some cases but not in others.

    I should also note that all these messages are turned into strings, then compressed with zlib to make the sent data as compact as possible.

    Now, I'm starting to think about how I want to add a "calendar of future events". I don't even know where to begin but I think I should start with the Yague library which is used by the guild calendar addon (forgot the name).

    The main thing I have to figure out, is how to propogate the future event, even when the raid leader is not online. I also need to figure out if signups should be allowed, if so, then how Do I propagate those all while staying under the message limit.

    my initial idea:
    entire calendar is broadcast in chunks, every 2 minutes, by every person. However I think this could be made more efficient if a "pre-broadcast" message was sent so that people would only broadcast if there is someone who needs an update. It starts to get confusing though on how to get everyones calendar broadcasting "in sync".

    and then there's the problem of sign ups, if one person signs up for a raid, how can their new version of the calendar get sent out without overwriting the someone else's calendar which might have changed in the meantime?

    An analogy would be a crowded train station, someone new enters the station and says "What time is my train leaving?" and then one person shouts the schedule out. As opposed to every single person in the room shouting the entire schedule every 2 min.

    Any thoughts?
    Last edited by Redcruxs; 11-12-2013 at 09:10 AM.

  2. #2
    Plane Touched
    Join Date
    Feb 2012
    Posts
    228

    Default

    IIRC, you're currently using a purpose specific channel to send addon traffic. If that's not the case, some ideas below could not apply. I also assume broadcasts are reliable (while this is not guaranteed by the API, I believe it is most of the time).

    Having everybody broadcast a message every 15 seconds would generate a lot of redundant traffic. Instead, state should be made consistent between all participants (shared state), and only changes should be broadcasted.

    When a new participant joins the network, you could use a sequence like this:

    * Send a state transfer request on the group channel and wait for answers. If nobody responds after a given lapse of time, try on the guild channel, and if it fails too, use the main channel.

    Only one participant of the channel should respond to the request, by sending all events known by the network to the new one. You can use some kind of coordinator election algorithm per channel to prenegotiate who will be responsible of sending state transfers to new participants, or just choose the one with the highest ID.

    * As soon as you start receiving the shared state, start queueing all incoming change broadcasts. Once the shared state has been received completely, apply the queued changes and, from this point on, apply incoming change broadcasts immediately.

    * After having synced with the shared network state, you should check if you have any event on your own previous state (saved variables) that isn't known by the network, or that has been updated more recently than the timestamp received during the state transfer. These new and updated events should be broadcasted to the network so all other participants can add them to their state.

    * At this point, all participants of the network should have the same shared state, and the change broadcasts would be enough to keep everyone in sync.

    Extra tips:

    * If possible, use direct messages to transfer state. If not, use the less populated channel available (that's the idea behind checking group, then guild, then main channel).
    * Throttle state transfer broadcasts, to prevent flooding the channels those times that a lot of players join at nearly the same time (server maintenance, massive disconnects...). For example, if a state transfer has been responded to during the last minute, wait till the next minute to respond again.
    * Discard old/expired events so the state doesn't grow over time.

    For signup while the leader is offline, you could use two states: "player requested sign up" and "leader acknowledged request".

    To prevent message tampering by malicious users, you could use some digital signature scheme.

  3. #3
    RIFT Guide Writer Redcruxs's Avatar
    Join Date
    Jan 2011
    Posts
    3,935

    Default

    hmmm I've thought a little about that but the problem with sending state changes is that there is no way to tell when someone leaves the network other then when they voluntarily choose to.

    Right now, my grid discards players it has not received an update from for at least 20 seconds (to account for lag in the 15 second sends)

    I had to do it this way because if a player gets disconnected or force logs out there is no way to send a message saying "i'm no longer looking for a group"

    But in any case, 1 message every 15 seconds from every player LFG or LFM isn't too big of a deal because the messages are small. The real problem is the calendar which could be pretty big (but I don't know for sure)

    However you did give me some ideas

    How about this:

    There are 3 types of broadcasts, "Raid Leader: Add me to the calendar", "Player: Add me to an event", and "Propagate current calendar".

    The "Propagation" broadcast will be on a set timer, say every minute. BUT that timer will reset back to 1 minute every time a player receives a "Propagation" broadcast. So essentially, whoever sends it first will become the propagator and everyone else will be silenced until the first person stops broadcasting and then the next first broadcaster will silence everyone. The propagated calendar will update everyone elses.

    the raid leaders broadcast to add an event will be relatively small, just saying: "add this event to your calendar" it will broadcast every 15 seconds until the leader receives a "propagation" event that contains their event (aka it has spread) then it will stop. It will work the same with the players "Add me to this event" broadcast.

    Hopefully doing it like this will give the network some robustness/redundancy without spamming the channel much. just 1 propagator at a time and the occasional "add this" broadcasts.

    On a side note:

    I've noticed that broadcast via channel is by far the most reliable method of getting messages from one person to everyone. Group, guild, and yell won't work for me because the people who are LFG are across all shards and locations and usually not in a group at the time.

    Sending a message directly fails between some people randomly, I have no clue why. When i was first making the messaging system i was able to communicate easily between my two alts on separate shards, then as soon as i wanted to talk to someone else it wouldn't work. The only way to guarantee all messages are received is to fall back on "less desirable" methods

    1) Command.Message.Send(target, ...) = most desirable, oddly not reliable
    2) Command.Message.Broadcast("tell", target, ...) = functionally same as #1 but it's some sort of weird hybrid
    3) Command.Message.Broadcast("channel", channelname, ...) = least desirable for obvious reasons, but can be made to be ignored by everyone except the intended recipient so it is 100% reliable as far as I've seen. in my debug messages i call this "FAIL-SAFE SEND! SOMETHING FKED UP!! BAD!" method lol

  4. #4
    RIFT Guide Writer Redcruxs's Avatar
    Join Date
    Jan 2011
    Posts
    3,935

    Default

    ah this stuff makes my brain hurt... but in a good way... so much easier to organize my thoughts when i type it all down on the forums

+ Reply to Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts