From kbulgrien at att.net Thu Aug 6 23:31:45 2009 From: kbulgrien at att.net (Kevin Bulgrien) Date: Thu, 6 Aug 2009 23:31:45 -0500 Subject: [crossfire] Bug 2789515: output-count broken since 1.11 server Message-ID: <20090806233145.6986251d@a850srvr.kbulgrien.att.net> For those interested, see the SF tracker: https://sourceforge.net/tracker/index.php?func=detail&aid=2789515&group_id=13833&atid=113833 This is fair warning that the server support for output-count and output-sync functions is about to be removed rather than fixed. Ragnor states that jxclient already implements message folding in the client. After looking over the server code, it seems quite clear to me that mwedel's suggestion to move it to the client is a good idea. My plan at the moment is to have the server continue to save the output_count and output_sync fields to the player file, but to otherwise completely remove references to any of the code and declarations that supported the facility. The player save function would simply write the old defaults in to the player file so that an older server would run sanely on the player file even if it had been saved by the modified server. A comment will be placed there saying that this should/could be removed when the server goes 2.x. Comments welcome. The changes are sitting in my SVN checkout, but I'll hold back a bit in case there is any life out there that cares. Kevin From juhaj at iki.fi Fri Aug 7 05:24:56 2009 From: juhaj at iki.fi (Juha =?iso-8859-1?q?J=E4ykk=E4?=) Date: Fri, 07 Aug 2009 13:24:56 +0300 Subject: [crossfire] Bug 2789515: output-count broken since 1.11 server In-Reply-To: <20090806233145.6986251d@a850srvr.kbulgrien.att.net> References: <20090806233145.6986251d@a850srvr.kbulgrien.att.net> Message-ID: <200908071325.07407.juhaj@iki.fi> > After looking over the server code, it seems quite clear to me that > mwedel's suggestion to move it to the client is a good idea. Do you have any idea how this affects the latency on longer network links and when there are huge numbers of messages flowing? The output-count helped me a lot sometimes. > Comments welcome. The changes are sitting in my SVN checkout, but I'll > hold back a bit in case there is any life out there that cares. Contrary to you expectations, someone answered? =) -Juha -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. Url : http://mailman.metalforge.org/pipermail/crossfire/attachments/20090807/e9bfcc4e/attachment.pgp From kbulgrien at att.net Fri Aug 7 17:44:43 2009 From: kbulgrien at att.net (Kevin Bulgrien) Date: Fri, 7 Aug 2009 17:44:43 -0500 Subject: [crossfire] Bug 2789515: output-count broken since 1.11 server In-Reply-To: <200908071325.07407.juhaj@iki.fi> References: <20090806233145.6986251d@a850srvr.kbulgrien.att.net> <200908071325.07407.juhaj@iki.fi> Message-ID: <20090807174443.2dfb4f05@a850srvr.kbulgrien.att.net> > > After looking over the server code, it seems quite clear to me that > > mwedel's suggestion to move it to the client is a good idea. > > Do you have any idea how this affects the latency on longer network links and > when there are huge numbers of messages flowing? The output-count helped me a > lot sometimes. I cannot say, but the SF tracker contains mwedel's comments to the effect that the network bandwidth used by the messages is pitifully small compared to other components of the traffic content. While I am not sure that the comments really take everything into consideration, I didn't really stop to think that much about it either after I saw the number of conditions that were designed to avoid use of the output buffers. Still, there is that side that must admit that when I did not have a DSL connection, though I was able to play crossfire, it seems to me that I used to experience lag more often than I do now. Some of it was definitely not caused by output-count/sync issues, but I cannot say I could be so definite about some of it actually being related to it. Also, the output-sync/count facility appeared to only be expected to work under some conditions. I cannot say I fully followed the logic while looking at the code because it was very convoluted and appeared to have been so quite some time - notwithstanding it was broken the first time I looked at it. A part of me would like to fix it... but probably mostly along the lines that it is annoying when working code is broken because someone rewrites something, and, because crossfire always given the impression of being network link intensive. I may have given it a go if I felt more adept with svn. Also, now that I have DSL and it lets me be less concerned about speed, and as so few people seem to play via internet (by the metaserver status), it did not occur to me to argue long and loud against the suggestion to remove the code from the server. It is a feature that loads a server, whereas with a client operation, memory and processing is able to be off-loaded to clients very naturally - leaving the system more time and resources to support game play vs. network traffic management. What's more, I don't know if I ever knew what it did to know to turn it on until just recently. I suspect not many people use it either. Not knowing the server code vs. having done a lot of client work, I feel more likely to be able to effectively re-instantiate a working feature in the client than in the server without taking a risk of breaking other clients. Since there seem to be only a few people even interesting in working on CF these days... I'm also feeling that I have more freedom than if the case were different. It used to bug me in the past when people would argue something to the point where no code ever got done. Since its not a cut-and-dried thing, and since there are no actual metrics, and because I would like to have the feature working due to the horrible spew that happens because of melee changes (i want less spew during gameplay), it seems that the balance is tipped on the side of making the feature work in the client vs. the server. > > Comments welcome. The changes are sitting in my SVN checkout, but I'll > > hold back a bit in case there is any life out there that cares. > > Contrary to you expectations, someone answered? =) Amazing! Perhaps you'd like to start work today... ;-) Hmm... let see... to answer the question... one might start by collecting actual data about its impact on gameplay... ;-) That'd be a whole lot more helpful than all this conjecture and theory - most of which is probably worth less than a kobold's treasure drop. > -Juha > From mwedel at sonic.net Fri Aug 7 20:02:24 2009 From: mwedel at sonic.net (Mark Wedel) Date: Fri, 07 Aug 2009 18:02:24 -0700 Subject: [crossfire] Bug 2789515: output-count broken since 1.11 server In-Reply-To: <20090807174443.2dfb4f05@a850srvr.kbulgrien.att.net> References: <20090806233145.6986251d@a850srvr.kbulgrien.att.net> <200908071325.07407.juhaj@iki.fi> <20090807174443.2dfb4f05@a850srvr.kbulgrien.att.net> Message-ID: <4A7CCEA0.7040103@sonic.net> As the original write of the output sync, here is some background and my thoughts. That code was originally put back in before the client/server split. At that time, any control of messages had to be done is the server - there was no client to do that task. This really makes it a bit of legacy code. I have a feeling if that code never existed and someone today suggested adding code to the server to collapse messages, most all the devs would say no way - do it in the client. The fact that the code already exists in legacy form really shouldn't change the decision making process much. No one has ever done an analysis on how much bandwidth different aspects of the protocol consume. My gut feeling is that in general play, it is the map updates that take the most, but inventory/floor updates could take a considerable amount of bandwidth in certain circumstances (depending on how much stuff is on the floor or in inventory). Analysis of the messages is a bit easier than most - one could take an expected number of messages per second times an average size, and get pretty close to actual bandwidth uses (there is some overhead in the protocol, so maybe add 10 bytes for that). So if you say that you might get 50 messages/second, and each is 50 bytes (including that overhead), that is 2500 bytes/second (or 25 Kb/sec). If you're on a 56Kb modem, that is a big hunk. If you're on any form of broadband, that is trivial. Now a harder question would be how much would the output sync reduce that to, and that would involve a lot more assumptions or setting up what the circumstance is. I don't know what number of folks who play still have 56K modems - I doubt too many, and I'm not sure how hard we want to design towards that group. That doesn't mean we should waste bandwidth, but if the choice was between adding a feature that makes the game better but uses more bandwidth or trying to make it playable for people with 56K modems, I'd rather add the feature. We've never realyl come up with what a minimum standard for play is. All that said, output sync is much less useful than it once was. It can only combine messages that are exactly the same. Lots of messages basically set the flag that they can not be combined (this could be because ordering is important, like in lists, and output sync could mess that up, or in some cases where we know that the message is not likely to be something that we can combine, like in the case of examining items). So the case it was most often useful was the attack messages. When it was written, there were perhaps half a dozen different attack messages (you hit %s, you hit %s hard, you smash %s, you miss %s). So if you're fighting the same type of creatures, or case a spell into a mob of them, it does a good job of combining messages. However, when the attack messages were added, now there are a lot of different attack messages. As such, it is much less frequent for it to be able to combine messages - while they may convey the same information, they are different messages so can not be combined. I don't really have a good fix for that. So just as a general case, the output sync has less use than it did when first written. If one cared a lot about bandwidth of messages, then one could basically move localization for server to client and just send the i18n message id (as an example, look at lib/i18n/messages.* - just send '209 ' type of thing. Now at some of connecting, the client would have to get all the message strings, etc. I personally don't think that it would be worthwhile to do that. But if one was really concerned about amount of data used for transmitting messages, sending integers (a 4 byte int should be more than enough for all messages) instead of 30 byte strings would be much more efficient. From kbulgrien at att.net Tue Aug 18 20:59:30 2009 From: kbulgrien at att.net (Kevin Bulgrien) Date: Tue, 18 Aug 2009 20:59:30 -0500 Subject: [crossfire] MSG_TYPEs vs in-Client configuration of routing and message folding. Message-ID: <20090818205930.4cb40343@a850srvr.kbulgrien.att.net> The message types in the system are consecutive numbers from 1 to 20. They are: #define MSG_TYPE_BOOK 1 #define MSG_TYPE_CARD 2 #define MSG_TYPE_PAPER 3 #define MSG_TYPE_SIGN 4 #define MSG_TYPE_MONUMENT 5 #define MSG_TYPE_DIALOG 6 /* NPCs, magic mouths, and altars */ #define MSG_TYPE_MOTD 7 #define MSG_TYPE_ADMIN 8 #define MSG_TYPE_SHOP 9 #define MSG_TYPE_COMMAND 10 /* Responses to commands, eg, who */ #define MSG_TYPE_ATTRIBUTE 11 /* Changes to attributes (stats, */ /* resistances, etc) */ #define MSG_TYPE_SKILL 12 /* Messages related to using skills */ #define MSG_TYPE_APPLY 13 /* Applying objects */ #define MSG_TYPE_ATTACK 14 /* Attack related messages */ #define MSG_TYPE_COMMUNICATION 15 /* Communication between players */ #define MSG_TYPE_SPELL 16 /* Spell related info */ #define MSG_TYPE_ITEM 17 /* Item related information */ #define MSG_TYPE_MISC 18 /* Messages that don't go anyplace else */ #define MSG_TYPE_VICTIM 19 /* Something bad is happening to player */ #define MSG_TYPE_CLIENT 20 /* Messages originated by the client */ I would like to consider changing the message types to bit values so that it is easier to use masks to detect message types instead of having to do sequential tests for individual values. There are two immediate reasons for this: 1) I want to the gtk-v2 client to easily allow in-client message routing by type. The short-term goal is to allow players to pick which messages go to the so-called "critical messages" window. In the longer term, an idea was expressed that it might be nice to allow players to set up their own number of text windows and route messages as they pleased. In any case, with the current message type numbering, one has to have statements like: if (type == MSG_TYPE_ATTRIBUTE || type == MSG_TYPE_COMMUNICATION || type == MSG_TYPE_DIALOG || type == MSG_TYPE_VICTIM) For in-client configuration, this type of thing might be done by searching a list, which seems silly, or perhaps a table lookup. The table lookup is not really too bad, but it seems like it would be really nice to just use a bit mask instead. 2) The client now has an output-count/sync function. It would be nice to be able to select which message types are processed for duplicate suppression and which are not. Again, a bit mask selector for this would be great. A table lookup scheme isn't that bad, and could be fairly natural to set up a GUI control for, but the idea of using bit masks is more appealing at the moment, and with only 20 values in use, it seems quite reasonable to assume that 32 bits allows for adequate for future expansion. I see no real reason to modify the subtypes at this time, except perhaps for consistency. They also could be converted to bit values quite easily as the highest number of subtypes in one type is 18. I suppose the message routing or buffering could even offer control down to the subtype, but at first blush that seems a bit of overkill as a GUI control could get quite cluttered with so many options. If anyone has any feedback to offer on conversion of the message type values to powers of two, and/or considerations for message routing and configuration of output-count and sync, it might be worth airing them for consideration while I'm interested in doing the work. Kevin From mwedel at sonic.net Tue Aug 18 23:52:36 2009 From: mwedel at sonic.net (Mark Wedel) Date: Tue, 18 Aug 2009 21:52:36 -0700 Subject: [crossfire] MSG_TYPEs vs in-Client configuration of routing and message folding. In-Reply-To: <20090818205930.4cb40343@a850srvr.kbulgrien.att.net> References: <20090818205930.4cb40343@a850srvr.kbulgrien.att.net> Message-ID: <4A8B8514.4010200@sonic.net> Kevin Bulgrien wrote: > The message types in the system are consecutive numbers from 1 to 20. They are: > I would like to consider changing the message types to bit values so that it > is easier to use masks to detect message types instead of having to do > sequential tests for individual values. There are two immediate reasons for > this: First question I have would be can a message be of multiple types? By having distinct (non bitmask values), a message can only be of one type. There probably isn't a wrong or right answer to that question, but that should be answered because it can affect future game decisions, and as you note below, by allow customizable routing one needs to now how they will work. What you probably want to avoid is the case of assuming a message will only have a single bit set, and writing support for that, and then someone setting multiple bits on a message and then filing bugs saying support for that is broken. > > 1) I want to the gtk-v2 client to easily allow in-client message routing by > type. The short-term goal is to allow players to pick which messages go > to the so-called "critical messages" window. In the longer term, an idea > was expressed that it might be nice to allow players to set up their own > number of text windows and route messages as they pleased. In any case, > with the current message type numbering, one has to have statements like: > > if (type == MSG_TYPE_ATTRIBUTE > || type == MSG_TYPE_COMMUNICATION > || type == MSG_TYPE_DIALOG > || type == MSG_TYPE_VICTIM) I personally don't see any issue with that statement in itself - it's fairly clear in what it does and efficiency doesn't really matter. > > For in-client configuration, this type of thing might be done by searching > a list, which seems silly, or perhaps a table lookup. The table lookup is > not really too bad, but it seems like it would be really nice to just use > a bit mask instead. It would seem to me that the easiest thing to do would be convert the message type to a bit value upon receipt. Eg, something like: client_message_mask = 1 >> server_message_type That gets you the bitmask for easy compares, but does not break compatibility of existing message types, and does still enforce a message can only be a single type. It also means that if message types do increase beyond a 32 bit value, it is just a client change that is needed (to store in a larger size) vs potential server and protocol changes. > > 2) The client now has an output-count/sync function. It would be nice to be > able to select which message types are processed for duplicate suppression > and which are not. Again, a bit mask selector for this would be great. > > A table lookup scheme isn't that bad, and could be fairly natural to set up > a GUI control for, but the idea of using bit masks is more appealing at the > moment, and with only 20 values in use, it seems quite reasonable to assume > that 32 bits allows for adequate for future expansion. > > I see no real reason to modify the subtypes at this time, except perhaps for > consistency. They also could be converted to bit values quite easily as the > highest number of subtypes in one type is 18. I suppose the message routing > or buffering could even offer control down to the subtype, but at first blush > that seems a bit of overkill as a GUI control could get quite cluttered with > so many options Long term, I think a table with actions is the most flexible solutions. To me, these are attributes you may want to apply to the message: mergeable - should we try to merge message which pane(s) the message is drawn to. Drawing to multiple panes would need to be an option IMO - one of the biggest uses I would see for this is that I might have pane where everything goes to, but in course of combat, etc, that could get filled up pretty easily, so I would want chat type messages to also go into another pane so that I can review that pane after the battle. But I'd still like those to go to the main pane, so if things are quiet, I can still see the messages and I don't need to switch among several panes to find all the messages. color & font information - this is currently done via themes, but those are not especially friendly for end user editing - having an in-game way to set these values would be nice - it also lets you more easily see why a message may have shown up in some way (oh, that message was red because it was an attributed related message). The bitmask is useful if your just looking at global value (mergable message). But once you start dealing with message panes, it starts looking less good, because now you are having something like: if (message_bit & pane1) { draw_in_pane(1) } if (message_bit & pane2) { draw_in_pane(2) } and that still doesn't cover font and/or color information (I'm not really sure how you would do those with bitmasks). But those are just my thoughts - the gui to deal with all this is some work, but I always just thought of displaying it basically as a table, something like: pane1 pane2 pane3 style info book X X drawn in color/font card X paper X sign X X .... Where you just use checkboxes to decide where messages go. That said, there is a huge number of options, but I'm not sure how long it would take to make those choices. Also, just thinking on that, might even be a column to denote the message is mergeable or not - don't know. From kbulgrien at att.net Sat Aug 29 17:40:42 2009 From: kbulgrien at att.net (Kevin Bulgrien) Date: Sat, 29 Aug 2009 17:40:42 -0500 Subject: [crossfire] GTK-V2 Message Control Message-ID: <20090829174043.197129be@a850srvr.kbulgrien.att.net> SVN revision 12170 debuts the GTK-V2 Message Control system. Earlier clients had some of this functionality, but it could only be configured by changing defines or code. This revision implements a Client | Message Control menu option and corresponding dialog. The player can now select which message types pass through the client-side output-count feature (duplicate message suppression) by simply marking a checkbox alongside the message type. Additionally, the player can configure which types go to which message panes in the client - also by checking boxes. The choice is not either/or, but does allow messages to go to both (or none) if so desired. The changes are made as soon as the Apply button is pressed. There is no need to restart the client. This revision is not the final one though. Presently the output-count is fixed at 16 messages, and output-syn at 16 ticks (about 2 seconds). These options are slated for addition to the dialog. Further, the dialog has a Save button, but it is disabled. At some point it is hoped that the player configured settings will be saveable and that they will restore when the client is restarted. This functionality is not yet implemented. The code is not quite finished, but this announcement is made in case anyone wants to take a look. The message duplicate suppression apparently has a bit of a flaw. Sometimes it actually causes duplicates... though it seems it doesn't happen a lot so it is still very usable. :-( Anyway, that just means it will not be long before there are updates. Kevin From kbulgrien at att.net Sat Aug 29 23:55:19 2009 From: kbulgrien at att.net (Kevin Bulgrien) Date: Sat, 29 Aug 2009 23:55:19 -0500 Subject: [crossfire] GTK-V2 Message Control In-Reply-To: <20090829174043.197129be@a850srvr.kbulgrien.att.net> References: <20090829174043.197129be@a850srvr.kbulgrien.att.net> Message-ID: <20090829235519.144d6533@a850srvr.kbulgrien.att.net> > The message duplicate suppression apparently has a > bit of a flaw. Sometimes it actually causes duplicates... though it seems > it doesn't happen a lot so it is still very usable. :-( Anyway, that just > means it will not be long before there are updates. r12172 fixes that bug.