Hi All,
As my botnet increased in size, I experienced slowing down of the bots, particularly the main
and secondary hubs. I tracked down the problem to, apparently, a large traffic of (explicit)
"chattr handle flags" as observed on the partyline. Let me explain better: the bots tipically
share globally; the sharing works fine (no problem with +ps flags, I mean). Suppose the
bot A is the hub, and is on chan #c1 and #c2, there is a bot B linked to A and sitting on #c1 only: the bots have shared the userfile at linking.When a user added as h1 enters #c2, although B is not on #c2, I can see on the partyline of B that A is broadcasting a "chattr h1 globalflagsofh1". Now, given that some of my bots are sitting on channels with 300+
users, and I have some 800 ppl added, the traffic sometimes becomes horrible, bogging
down the hubs.
Is there a simply and neat way to solve the problem, apart from having to go over the all bots, removing the +g and start putting some |+s channel, instead? One thing which is
really bothering me is that I cannot see why the bots should be sending this information
(the flag of a user which enters a channel) all over the botnet.
Many thanks,
dun_dacil
This is the new home of the egghelp.org community forum.
All data has been migrated (including user logins/passwords) to a new phpBB version.
For more information, see this announcement post. Click the X in the top right-corner of this box to dismiss this message.
All data has been migrated (including user logins/passwords) to a new phpBB version.
For more information, see this announcement post. Click the X in the top right-corner of this box to dismiss this message.
Bots slowing down
No - there is no simple and neat way.
+g is dicribed well - changes are made globaly. The hub has not way of knowing that botb isn't in that channel. If it was to make checks for this every time, then it would be more likely, that the bot would be more bogged down with these checks.
Apart form this, why would share traffic be bogging down the bots?
800 users is a lot, but if you have the flag system working corectly, there should be no need to be making hundreds of chattr changes.
1 chattr a min, would allow for every user record, to be changed in under 14 hours. 5 chattr's a min, would be completed in under 3 hours.
These are not large figures, and as such, would sugest a change in the way the botnet is managed is changed.
botnet traffic would be expected to increase over times of expansion.
When I was running 100 bot net, we where using quality bots, on poor networks. They where not up to the job of being hubs, but we had to make do.
One of the things we did, was to change the tapolgy of the botnet.
You should start by aranging bots into there servers geographical location.
Using something like 4 groups, European bots, East coast bots, West coast bots and Eastern bots.
Take each group, and split it into 3 catagories. hgih capacity, medium capacity and low capactity. Where high are on stable shells, with good bandwidth links.
There should some form of ration system, for this. For every 5 low capacity bots, you need 1 medium. for each 5 medium, you need 1 high.
Out of all the bots, you should pick one, if you like, super capacity bot, and use that as the main hub, linking no more than 5 bots to this. If need be, you will need to assign extra bots, to prevent over loading of the master hub.
ANother sugestion, is to use "limbo" bots. Where you do not load the server module, and do not connect it to IRC. This will provide a faster bot, that doesn't have to handle IRC traffic.
+g is dicribed well - changes are made globaly. The hub has not way of knowing that botb isn't in that channel. If it was to make checks for this every time, then it would be more likely, that the bot would be more bogged down with these checks.
Apart form this, why would share traffic be bogging down the bots?
800 users is a lot, but if you have the flag system working corectly, there should be no need to be making hundreds of chattr changes.
1 chattr a min, would allow for every user record, to be changed in under 14 hours. 5 chattr's a min, would be completed in under 3 hours.
These are not large figures, and as such, would sugest a change in the way the botnet is managed is changed.
botnet traffic would be expected to increase over times of expansion.
When I was running 100 bot net, we where using quality bots, on poor networks. They where not up to the job of being hubs, but we had to make do.
One of the things we did, was to change the tapolgy of the botnet.
You should start by aranging bots into there servers geographical location.
Using something like 4 groups, European bots, East coast bots, West coast bots and Eastern bots.
Take each group, and split it into 3 catagories. hgih capacity, medium capacity and low capactity. Where high are on stable shells, with good bandwidth links.
There should some form of ration system, for this. For every 5 low capacity bots, you need 1 medium. for each 5 medium, you need 1 high.
Out of all the bots, you should pick one, if you like, super capacity bot, and use that as the main hub, linking no more than 5 bots to this. If need be, you will need to assign extra bots, to prevent over loading of the master hub.
ANother sugestion, is to use "limbo" bots. Where you do not load the server module, and do not connect it to IRC. This will provide a faster bot, that doesn't have to handle IRC traffic.
Re: bots slowing down
Dear ppslim,
Many thanks for your prompt reply. Well, I do have the bots arranged to reflect the topology of the servers locations. But I observe slowing down even between two bots located on the same LAN, with 100mb connection
: the hub may be totally unresponsive to anything typed on the partyline for 3 or 4 minutes while it is sending loads of "chattr" to another bot sitting next to it, on a fastethernetted netwtork. In other words, the limitation does not seem to come from physical bandwidth, but from some inner mechanism of the eggdrop.
Perhaps I misunderstood you: let me rephrase what is happening: suppose bot B is down, and comes up: it downloads the userfile and switch control over to bot A (the hub) in a matter of a couple of seconds, when coming up. Then bot B enters a channel where A is not, but which is nevertheless defined on A (and which is +share +inactive on A): there is a huge amount of "chattr" transfer (explicitly on the partyline, these, I mean not compressed like the userfile at the linking) between A and B, relative to the channel just entered: this is somehow bothering me, it seems that the information is sent twice, first when the userfile is downloaded, and again when the single channels are entered. Is there a way to prevent this "flags sharing", which is taking place even when no flag is actually changed on the botnet? You see, I see "chattr" flying around even when there is *no* flag sharing(. And no, no fancy tcl, the eggies are 1.6.10 or 1.6.12, and basically have only things like notes2 or netbots loaded, and little more, tcl wise.
Many thanks,
dun_dacil
Many thanks for your prompt reply. Well, I do have the bots arranged to reflect the topology of the servers locations. But I observe slowing down even between two bots located on the same LAN, with 100mb connection
: the hub may be totally unresponsive to anything typed on the partyline for 3 or 4 minutes while it is sending loads of "chattr" to another bot sitting next to it, on a fastethernetted netwtork. In other words, the limitation does not seem to come from physical bandwidth, but from some inner mechanism of the eggdrop.Perhaps I misunderstood you: let me rephrase what is happening: suppose bot B is down, and comes up: it downloads the userfile and switch control over to bot A (the hub) in a matter of a couple of seconds, when coming up. Then bot B enters a channel where A is not, but which is nevertheless defined on A (and which is +share +inactive on A): there is a huge amount of "chattr" transfer (explicitly on the partyline, these, I mean not compressed like the userfile at the linking) between A and B, relative to the channel just entered: this is somehow bothering me, it seems that the information is sent twice, first when the userfile is downloaded, and again when the single channels are entered. Is there a way to prevent this "flags sharing", which is taking place even when no flag is actually changed on the botnet? You see, I see "chattr" flying around even when there is *no* flag sharing
(. And no, no fancy tcl, the eggies are 1.6.10 or 1.6.12, and basically have only things like notes2 or netbots loaded, and little more, tcl wise.Many thanks,
dun_dacil
There should be no reason, why the bot shares the flags, if you like, twice.
It should send the userfile at link time, and only send shattr shares, wehn a change is made using either the chattr dcc command, or the chattr Tcl command.
I sugest killing the bots, unloading all Tcl scripts, and testing the bots again.
If that fails to help, then there is somthing more going on than meets the eye.
It should send the userfile at link time, and only send shattr shares, wehn a change is made using either the chattr dcc command, or the chattr Tcl command.
I sugest killing the bots, unloading all Tcl scripts, and testing the bots again.
If that fails to help, then there is somthing more going on than meets the eye.
Dear ppslim,
Well spotted! I needed some reassurance that the behaviour was not correct.
I did as suggested, taking down one of the bots which was giving trouble, and started it adding one tcl at the time: it turned out that the culprit is indeed a tcl: lol.tcl
Having found the culprit, it was straightforward to isolate the problem: when a user joins a chan, or leaves a chan, lol.tcl forces a "chattr handle -D" even if "handle" is not +D flagged.
I have now patched lol.tcl, and the bots run as smooth as silk. Many thanks, I owe you a luvely English warm beer, to be payed next time I pop around there if you like (I normally travel to Lancaster and the Lakes at least twice a year
)
Take care,
dun_dacil
Well spotted! I needed some reassurance that the behaviour was not correct.
I did as suggested, taking down one of the bots which was giving trouble, and started it adding one tcl at the time: it turned out that the culprit is indeed a tcl: lol.tcl
Having found the culprit, it was straightforward to isolate the problem: when a user joins a chan, or leaves a chan, lol.tcl forces a "chattr handle -D" even if "handle" is not +D flagged.
I have now patched lol.tcl, and the bots run as smooth as silk. Many thanks, I owe you a luvely English warm beer, to be payed next time I pop around there if you like (I normally travel to Lancaster and the Lakes at least twice a year
)Take care,
dun_dacil
-
TThor