possible to make eggdrop auto-rehash every x minutes?

syfx · Post by **syfx** » Wed Jun 15, 2005 12:37 pm

Hello all, first post here, i was wondering if it is possible to make an eggdrop auto rehash itself every x minutes, my reason for this is i would like to use perpleXa's RSSNews.tcl but after so many news shows it seems to lock up, i contacted perpleXa about this and he said the XML parser is a bit unstable and uses a lot of CPU power, so i was wondering if rehashing the bot every 30 minutes or something would help keep the cpu levels down. is this possible?

demond · Post by **demond** » Wed Jun 15, 2005 1:08 pm

use this:

Code: Select all

#
# RSS news announcer by demond@demond.net
#
# works with either TclRSS by Tomoyasu Kobayashi (http://nul.jp/2003/tclrss),
# or rss package included with RSS Monitor by BDK, Blair Kitchen (http://wiki.tcl.tk/11184)
#
# configure it like this:  set feeds(#yourchan) {10 http://slashdot.org/slashdot.rss}
#

package require rss
package require http

namespace eval news {

variable version "rssnews 1.0"

bind time - * ::news::timer
bind pub - !read ::news::pub

putlog "$version by demond loaded"

proc timer {min hour day month year} {
	global feeds
	variable version
	if [info exists feeds] {
		foreach {chan feed} [array get feeds] {
			set chan [string tolower $chan]
			if {[lsearch -exact [string tolower [channels]] $chan] == -1} continue
			scan [join $feed] "%d %s" int feed
			if {$min != "00" && [string trimleft $min 0] % $int == 0} {
				process $chan [retrieve $feed]
			}
		}
	} else {
		putlog "$version: no configured feeds, exiting..."
		unbind time - * ::news::timer
	}
}

proc retrieve {feed} {
	set token [http::geturl $feed]
	set channel [rss::parse [http::data $token]]
	http::cleanup $token
	foreach item [$channel items] {
		set elem {}
		lappend elem [unixtime]
		lappend elem [$item title]
		lappend elem [$item link]
		lappend elem [$item description]
		lappend elem [md5 [$item description]]
		lappend res $elem
	}	
	return $res
}

proc process {chan data} {
	variable feedsdata
	set new 0
	set fetched [info exists feedsdata($chan)]
	foreach elem $data {
		if $fetched {
			set found 0
			foreach item $feedsdata($chan) {
				if {[lindex $elem 4] == [lindex $item 4]} {incr found}
			}
			if !$found {
				incr new; lappend feedsdata($chan) $elem
				if [botonchan $chan] {
					puthelp "privmsg $chan :($new) [lindex $elem 1]"
				}
			}
		} else {
			lappend feedsdata($chan) $elem
		}
	}
	if {$fetched && $new} {
		set feedsdata($chan) [lsort -decreasing -index 0 $feedsdata($chan)]
		set idx [expr [llength $feedsdata($chan)] - $new]
		set feedsdata($chan) [lreplace $feedsdata($chan) $idx end]
	}
}

proc pub {nick uhost hand chan text} {
	variable feedsdata
	if {$text == ""} {
		puthelp "notice $nick :usage: $::lastbind <#|all>"
		return
	}
	set chan [string tolower $chan]
	if [info exists feedsdata($chan)] {
		if [string equal -nocase $text "all"] {
			set num 0
			foreach item $feedsdata($chan) {
				incr num; show $nick $num $item 
			}
		} else {
			set num $text
			set len [llength $feedsdata($chan)]
			if {[string is integer $num] && ($num >= 1 && $num <= $len)} {
				show $nick $num [lindex $feedsdata($chan) [expr $num-1]]
			} else {
				puthelp "notice $nick :invalid index (must be >=1 and <=$len)"
			}
		}
	} else {
		puthelp "notice $nick :no feeds data for this channel"
	}
}

proc show {nick num item} {
	puthelp "notice $nick :title($num).........: [lindex $item 1]"
	puthelp "notice $nick :link($num)..........: [lindex $item 2]"
	puthelp "notice $nick :description($num)...: [lindex $item 3]"
}

}

it requires a bit of work to install (actually to install RSS and TclLib/TclXML packages), but once installed, it works far better than any other RSS news script I know of

syfx · Post by **syfx** » Wed Jun 15, 2005 2:54 pm

thanks i'll look into this now. no doubt i'll find a way to screw it up, but hey im trying

Thanks Demond

syfx · Post by **syfx** » Wed Jun 15, 2005 5:32 pm

ok ive installed the TclRSS by Tomoyasu Kobayashi but when i try to run eggdrop i get an error message requesting package 'dom' and i cant find this package anywhere on google. perhaps im searching for the wrong thing. anyone help out?

demond · Post by **demond** » Wed Jun 15, 2005 6:00 pm

http://tclxml.sourceforge.net/tcldom.html

however, you may discover that TclRSS from Tomoyasu Kobayashi causes errors... at least it didn't work for me, although it should have worked; so I resorted to BDK's rss package, which worked nicely (and it doesn't require TclDOM, just TclLib and TclXML)

TC^^ · Post by **TC^^** » Fri Jul 08, 2005 1:04 pm

*gees* I wish I had the patience to install the required packages, but gave up before I got it working.

Does your script only post items from the RSS feed, if they haven't been posted before? Or does it simply post the newest every 10th minute? I haven't got the knowledge to simply look at your script and get the answer myself, I'm afraid. As far as I can see, the script doesn't save any information about which items from the feed it has announced?

Please bear with me and my poor knowledge of TCL

demond · Post by **demond** » Fri Jul 08, 2005 2:23 pm

it posts only new (or changed) headlines, every X minutes where X is configurable per channel (note that you can configure multiple feeds on multiple channels, each one with different refresh rate); this is done by keeping md5 hash of description field and comparing it to newly fetched ones

TC^^ · Post by **TC^^** » Fri Jul 08, 2005 3:11 pm

I see.. Is the hash saved in memory then? What I'm concerned of, is that if the machine the bot is running on is restarted, or the bot is restarted, will it then remeber the hash? Or will it on startup announce the latest news again?

(I could try a scenario like it, but I'm trying to avoid going through alot of trouble installing the required packages, if I can't use the script anyways)

Thanks for your patience btw

demond · Post by **demond** » Fri Jul 08, 2005 3:37 pm

no, it doesn't save anything to a file, so on every restart it will initially announce all available headlines

chasingsol · Post by **chasingsol** » Sun Jul 17, 2005 5:19 pm

Hi there,

I've been trying (!) to get your script to work. I have installed ActiveTCL 8.4, since another script I tried required it (PerpleXa's). I have downloaded and placed BDK's rss.tcl in /usr/local/ActiveTcl/lib/tcl8.4/rss.tcl, verified that ld.so.conf is correct, and added the set feeds entries to the eggdrop config.

I keep getting this error message:

Tcl error [::news::timer]: wrong # args: should be "::struct::tree::_unset name node key"

I'm certain I'm doing something wrong here. Any thoughts?

Thanks in advance.

Mark.

demond · Post by **demond** » Sun Jul 17, 2005 6:00 pm

hmm I'm not sure what's causing your problem; I've installed TclLib and TclXML from source and never used ActiveTcl (I gather it has TclLib/TclXML bundled in the package); they might have changed something which introduces some incompatibility (my installation is not too current, at least one year old)

in any case, type .set errorInfo when you see that error, to obtain full error traceback

chasingsol · Post by **chasingsol** » Sun Jul 17, 2005 9:06 pm

Here's the error output.

Tcl error [::news::timer]: wrong # args: should be "::struct::tree::_unset name node key"

Code: Select all

Currently: while executing
Currently: "::struct::tree::_unset ::rss::tree1 channel.1"
Currently: ("_unset" body line 1)
Currently: invoked from within
Currently: "$tree unset channel.$channelcount"
Currently: (procedure "rss::parse" line 49)
Currently: invoked from within
Currently: "rss::parse [http::data $token]"
Currently: (procedure "retrieve" line 3)
Currently: invoked from within
Currently: "retrieve $feed"
Currently: ("foreach" body line 6)
Currently: invoked from within
Currently: "foreach {chan feed} [array get feeds] {
Currently: set chan [string tolower $chan]
Currently: if {[lsearch -exact [string tolower [channels]] $chan] == -1..."
Currently: invoked from within
Currently: "if [info exists feeds] {
Currently: foreach {chan feed} [array get feeds] {
Currently: set chan [string tolower $chan]
Currently: if {[lsearch -exact [string t..."
Currently: (procedure "::news::timer" line 4)
Currently: invoked from within
Currently: "::news::timer $_time1 $_time2 $_time3 $_time4 $_time5"

Incidentally, this is now using TclRSS, since I removed ActiveTcl, tried installing from source, and then couldn't figure out how to get auto_path and auto_mkindex (complained about a missing bracket in RSS Monitor) to work correctly, despite the location being correct. TclRSS simply worked, so my apologies for moving the goal posts, but the error is the same as previously.

Thanks again.

demond · Post by **demond** » Sun Jul 17, 2005 9:16 pm

hmm there is no line 49 in ::rss::parse proc of BDK's rss monitor, are you sure you are using this?

edit: well, simply place BDK's stuff in rss.tcl and do a pkg_mkIndex on that directory

chasingsol · Post by **chasingsol** » Sun Jul 17, 2005 9:19 pm

My apologies again, I switched to TclRSS since it worked, and I could no longer get BDK's to do so.

demond · Post by **demond** » Sun Jul 17, 2005 9:25 pm

so, is my script now working for you or not?

I mean with TclRSS