how to have multi-thread processing..

[dj-zath]

This ones going to be kinda hard to explain, but I will try to explain through an example..

The situation:

an eggdrop sitting in a busy chatroom.. many users sending it commands all at the same time.

the commands are sent in a msg window, which require the bot to reply with a list- varable length/size

The problem:

user one sends a query for "tainted love" ,
user two sends a query for "beatles"
user three sends a query for "all throghh the night"

user one will get his list right away, user two will also get his list realitively quickly, but user 3 will not get ANYTHING untill user 2's list finishes displaying.. although the logic, internally, is quick.. the "having to write it all out to the server" really SLOWS DOWN the entire process.. and since "beatles" will render over 100 items, user three will most likely GIVE UP before he waits out 2 to 5 MINUTES for his list!

I have set a limit to 20 items- but theres still a realitively long wait time which woud double for each user past the previous... and, the limit also cuts off a possale querry for "item 21" and beyond... (an undesired side-effect)

the solution:

As I mentioned above, the logic itself is realitively fast- but the writing out to the server is what takes the time.. so if there was a way to "seperate", isolate or "multi-thread" the server write-out process.. so that each user gets his list at the same time as each other user.. this could work very nicely!

unless someone has a better idea how I can do this,

some things to note:

The database is *NOT* mysql or any form of established database- it is managed by a playout system and prepriortary to it.. it can ONLY be accessed through a web-channel (done internally) where its cached in a varable and parsed..

query:
!search <song/artsit> -> "arg"
socket -> push command $arg -> localhost -> webserver -> playout

return:
Playout -> webserver -> localhost-> eggdrop -> socket -> "GetInfo"
GetInfo -> parsers -> "Song", "Show", "DJ", (etc) -> putserv $nick $song1, Song 2, etc"

each result (song 1, song 2, etc) is a new write to putserv

I could display the code, but its really "involved" I don't want to post it "freely" however since it contains "sensitive elements" and if I remove/substitute said elements, the code wouldn't make sense.. also, the code resides on an entirely-different machine which does NOT contain a browser or any way to access the web.. I'd have to use "sneakernet" to gain the file I won't do the work if it won't be seriously-considered

-DjZ

[Trixar_za]

Well, TCL is technically not designed to be threaded at all which is probably where the trouble comes in. I have noticed the same problem with some of my scripts, especially those that have to send more than 4 lines over notice that it really slows down the whole bot. My fix for it was to use putquick PRIVMSG instead of NOTICE to help speed things up. Not exactly sure why this works, but it does. Maybe you could make it output to private if the results are more than 5 lines long?

[willyw]

...
user one will get his list right away, user two will also get his list realitively quickly, but user 3 will not get ANYTHING untill user 2's list finishes displaying.. although the logic, internally, is quick.. the "having to write it all out to the server" really SLOWS DOWN the entire process.. and since "beatles" will render over 100 items, user three will most likely GIVE UP before he waits out 2 to 5 MINUTES for his list!
...

I've read that Eggdrop is designed to NOT flood itself off a server. That is, to not suddenly dump a lot of stuff, such that the server detects it as a flood, and kicks the bot off.

From what you've described, to me it sounds like you've 'discovered' this.

If you really want to get around this, I have read about it, right here in this forum. I believe I read about methods to get around it.... but one must realize what one is asking for... the potential to get one's bot kicked off.

I'm replying without first searching the forum, so I can't give you a link to a specific thread.

I hope this helps somehow though.

[willyw]

Check out:
http://eggwiki.org/Eggdrop.conf
and scroll down to:

# Number of seconds to wait between transmitting queued lines to the server.
# Lower this value at your own risk. ircd is known to start flood control
# at 512 bytes/2 seconds.
set msg-rate 2

Next, check out:
http://www.eggheads.org/support/egghtml ... mands.html
and scroll down to:
putnow

Again, I hope this is somehow helpful

[nml375]

The trick here, is to use the event-engine in tcl, along with your own queues...
As you thought, the rough idea is: since each of your searches are fast in nature, store the results as lists, one for each user, which each song title as a separate list item.
Then have an event-driven function/proc poll these lists for new data to send to each user, only picking one list item for each user during each iteration. This way, all users will get their results at the same rate.

Code: Select all

proc doSearch {....} {
#some code to get the list of results here...
#assuming $result is a list of results, and $nick is the nickname of the requester...
  if {![info exists ::responseQueue($nick)]} {
    set ::responseQueue($nick) $result
  } {
    set ::responseQueue($nick) [concat $::responseQueue($nick) $result]
  }
}

proc poll {} {
  foreach nick [array names ::responseQueue] {
    if {[llength $::responseQueue($nick)] > 0} {
      #pop the first value off the list..
      set item [lindex $::responseQueue($nick) 0]
      set ::responseQueue($nick) [lrange $::responseQueue($nick) 1 end]
      #and send to the user
      puthelp "PRIVMSG $nick :$item"
    }
  }
  #Trigger a new event after 1000 ms
  after 1000 poll
}

#start the polling...
poll

Keep in mind, that this code currently does no checks for nickname changes or such, though that could be fairly easily solved by renaming the array index accordingly.