Getting values from a url (and count lines) - XML!

[JKM]

Let's say I've got a text file where it's added new rows - maybe 20 each day. How can I then count the rows for http://mydomain.com/list.txt and msg it like this msg $chan "$lines -1" (the last line at list.txt shouldn't count) - when someone writes !lines?

Also, I would like to make a function where the trigger is !check. If someone writes f.ex. "!check coffee", the script should get the response from http://mydomain.com/check.php?value=coffee and then msg $chan $result.

It would be awsome if some could help me out.

[tomekk]

both a simple

1st i mean, smth like:
row1
row2
row3

and just display the count of rows - 1, right?

2nd:

Also, I would like to make a function where the trigger is !check. If someone writes f.ex. "!check coffee", the script should get the response from http://mydomain.com/check.php?value=coffee and then msg $chan $result.

yeah, but what will be the output of this query? smth like:
xxxxxxxxxxxxxxxxxxxxx - one line output

or smth:
xxxxxxxxxx
xxxxxxx
xxxxxxxxxxxx - many lines as output

?

[JKM]

both a simple

1st i mean, smth like:
row1
row2
row3

and just display the count of rows - 1, right?

Like this:
list.txt --->
1. roooow1
2. row2
3. rooow3
4. this is the last row!

So if you writes !list, it should msg "3" (4-1).

Also, I would like to make a function where the trigger is !check. If someone writes f.ex. "!check coffee", the script should get the response from http://mydomain.com/check.php?value=coffee and then msg $chan $result.

yeah, but what will be the output of this query? smth like:
xxxxxxxxxxxxxxxxxxxxx - one line output

or smth:
xxxxxxxxxx
xxxxxxx
xxxxxxxxxxxx - many lines as output

?

On output could be like "xxxxxxx" and one could be "xxxxxxxxxxxxxx" - everything is at one row.

[tomekk]

Code: Select all

# Author: tomekk
# e-mail:  tomekk/@/oswiecim/./eu/./org
# home page: http://tomekk.oswiecim.eu.org/
#
# Version 0.1
#
# This file is Copyrighted under the GNU Public License.
# http://www.gnu.org/copyleft/gpl.html

# url to list file, row by row
set list_url "http://localhost/list.txt"

# url to data file, one row
set check_url "http://localhost/test.php"

# check_url script argument, 
# will be: $check_url?$check_url_arg=$query, http://localhost/test.php?value=$query
set check_url_arg "value"

#######################################################################
bind pub - !lines lines_proc
bind pub - !check check_proc

package require http

proc lines_proc { nick uhost hand chan arg } {
	global list_url

	set data_lines [grab_http_data 1 $list_url ""]

	if {$data_lines != "error"} {
		set lines_counter 0

		foreach data_line $data_lines {
			if {$data_line != ""} {
				incr lines_counter 1
			}
		}

		putquick "PRIVMSG $chan :[expr $lines_counter - 1]"
	}
}

proc check_proc { nick uhost hand chan arg } {
	global check_url check_url_arg

	set check_arg [lindex [split $arg] 0]

	if {$check_arg != ""} {
		set one_data_line [grab_http_data 2 $check_url $check_arg]

		if {$one_data_line != "error"} {
			putquick "PRIVMSG $chan :$one_data_line"
		}
	}
}

proc grab_http_data { type url query } {
	global check_url_arg

	set user_agent "http-query.tcl <eggdrop script>"
			
	set tiny_token [http::config -useragent $user_agent]

	if {$type == 1} {	
		set tiny_token [http::geturl $url -timeout 10000]
	} else {
		# for bigger queries should be ::http::formatQuery
		set proper_url "$url?$check_url_arg=$query"
		set tiny_token [http::geturl $proper_url -timeout 10000]
	}
						
	set html_data [http::data $tiny_token]
							
	if {$html_data != ""} {
		return $html_data
	} {
		return "error"
	}
}

putlog "http-query.tcl ver 0.1 by tomekk loaded"

should be easy to understand, try it

[JKM]

Thanks a lot!

But how can I modify it so that !lines work for only #chan1 and #chan2, and that !check only works for #chan3?

[tomekk]

Thanks a lot!

But how can I modify it so that !lines work for only #chan1 and #chan2, and that !check only works for #chan3?

fixed:

Code: Select all

# Author: tomekk
# e-mail:  tomekk/@/oswiecim/./eu/./org
# home page: http://tomekk.oswiecim.eu.org/
#
# Version 0.1
#
# This file is Copyrighted under the GNU Public License.
# http://www.gnu.org/copyleft/gpl.html

# lines chans
set lines_chans {#chan1 #chan2}

# check chans
set check_chans {#chan1 #chan2}

# url to list file, row by row
set list_url "http://localhost/list.txt"

# url to data file, one row
set check_url "http://localhost/test.php"

# check_url script argument, 
# will be: $check_url?$check_url_arg=$query, http://localhost/test.php?value=$query
set check_url_arg "value"

#######################################################################
bind pub - !lines lines_proc
bind pub - !check check_proc

package require http

proc lines_proc { nick uhost hand chan arg } {
	global list_url lines_chans

	if {[lsearch $lines_chans $chan] != -1} {
		set data_lines [grab_http_data 1 $list_url ""]

		if {$data_lines != "error"} {
			set lines_counter 0
	
			foreach data_line $data_lines {
				if {$data_line != ""} {
					incr lines_counter 1
				}
			}
			putquick "PRIVMSG $chan :[expr $lines_counter - 1]"
		}
	}
}

proc check_proc { nick uhost hand chan arg } {
	global check_url check_url_arg check_chans

	if {[lsearch $check_chans $chan] != -1} {
		set check_arg [lindex [split $arg] 0]

		if {$check_arg != ""} {
			set one_data_line [grab_http_data 2 $check_url $check_arg]

			if {$one_data_line != "error"} {
				putquick "PRIVMSG $chan :$one_data_line"
			}
		}
	}
}

proc grab_http_data { type url query } {
	global check_url_arg

	set user_agent "http-query.tcl <eggdrop script>"
			
	set tiny_token [http::config -useragent $user_agent]

	if {$type == 1} {	
		set tiny_token [http::geturl $url -timeout 10000]
	} else {
		# for bigger queries should be ::http::formatQuery
		set proper_url "$url?$check_url_arg=$query"
		set tiny_token [http::geturl $proper_url -timeout 10000]
	}
						
	set html_data [http::data $tiny_token]
							
	if {$html_data != ""} {
		return $html_data
	} {
		return "error"
	}
}

putlog "http-query.tcl ver 0.1 by tomekk loaded"

try :>

[JKM]

Thanks a lot! I can't - unfortunately - test it before friday (when I'm ordering my shell).

By the way, could you please make another thing for me?
!search xx -> paste the result from mypage.com?search.php?f=xx. IF the page shows more than three rows (ex. 20 lines), it should past it like this:

row1
row2
row3
... And further 17 (20-3) lines: mypage.com?search.php?f=xx

[tomekk]

Thanks a lot! I can't - unfortunately - test it before friday (when I'm ordering my shell).

By the way, could you please make another thing for me?
!search xx -> paste the result from mypage.com?search.php?f=xx. IF the page shows more than three rows (ex. 20 lines), it should past it like this:

row1
row2
row3
... And further 17 (20-3) lines: mypage.com?search.php?f=xx

after 3 lines script should write link to this or this script should print all 20 lines?

[JKM]

This script should only print the first three lines at the page (if there is more than three lines). At the end it should echo "... And further XX (Total lines - 3) lines - http://url".

[tomekk]

This script should only print the first three lines at the page (if there is more than three lines). At the end it should echo "... And further XX (Total lines - 3) lines - http://url".

Code: Select all

# Author: tomekk
# e-mail:  tomekk/@/oswiecim/./eu/./org
# home page: http://tomekk.oswiecim.eu.org/
#
# Version 0.1
#
# This file is Copyrighted under the GNU Public License.
# http://www.gnu.org/copyleft/gpl.html

# lines chans
set lines_chans {#chan1 #chan2}

# check chans
set check_chans {#chan1 #chan2}

# search chans
set search_chans {#chan1 #chan2}

# url to list file, row by row
set list_url "http://localhost/list.txt"

# url to check script, one row
set check_url "http://localhost/check.php"

# url to search script, row by row
set search_url "http://localhost/search.php"

# check_url script argument, 
# will be: $check_url?$check_url_arg=$query, http://localhost/check.php?value=$query
set check_url_arg "value"

# search_url script argument,
# will be: $search_url?$search_url_arg=$query, http://localhost/search.php?value=$query
set search_url_arg "value"

#######################################################################
bind pub - !lines lines_proc
bind pub - !check check_proc
bind pub - !search search_proc

package require http

proc lines_proc { nick uhost hand chan arg } {
	global list_url lines_chans

	if {[lsearch $lines_chans $chan] != -1} {
		set data_lines [grab_http_data 1 $list_url "" ""]

		if {$data_lines != "error"} {
			set lines_counter 0
	
			# we can count list with [llength] but we dont want to count empty lines { }
			foreach data_line [split $data_lines "\n"] {
				if {$data_line != ""} {
					incr lines_counter 1
				}
			}
			putquick "PRIVMSG $chan :[expr $lines_counter - 1]"
		}
	}
}

proc check_proc { nick uhost hand chan arg } {
	global check_url check_url_arg check_chans

	if {[lsearch $check_chans $chan] != -1} {
		set check_arg [lindex [split $arg] 0]

		if {$check_arg != ""} {
			set one_data_line [grab_http_data 2 $check_url $check_url_arg $check_arg]

			if {$one_data_line != "error"} {
				putquick "PRIVMSG $chan :$one_data_line"
			}
		}
	}
}

proc search_proc { nick uhost hand chan arg } {
	global search_url search_url_arg search_chans

	if {[lsearch $search_chans $chan] != -1} {
		set search_arg [lindex [split $arg] 0]

		if {$search_arg != ""} {
			set data_lines [grab_http_data 2 $search_url $search_url_arg $search_arg]

			if {$data_lines != "error"} {
				set lines_counter 0

				set data_lines [split $data_lines "\n"]

				foreach data_line $data_lines {
					if {$data_line != ""} {
						incr lines_counter 1
					}
				}

				if {$lines_counter > 3} {
					set three_lines [lrange $data_lines 0 2]

					foreach one_line $three_lines {
						putquick "PRIVMSG $chan :$one_line"
					}

					set the_rest [expr $lines_counter - 3]

					putquick "PRIVMSG $chan :... And further $the_rest lines - $search_url?$search_url_arg=$search_arg"
				} else {
					foreach one_line $data_lines {
						putquick "PRIVMSG $chan :$one_line"
					}
				}
			}
		}
	}
}

proc grab_http_data { type url query arg } {
	set user_agent "http-query.tcl <eggdrop script>"
			
	set tiny_token [http::config -useragent $user_agent]

	if {$type == 1} {	
		set tiny_token [http::geturl $url -timeout 10000]
	} else {
		# for longer queries should be ::http::formatQuery
		set proper_url "$url?$query=$arg"
		set tiny_token [http::geturl $proper_url -timeout 10000]
	}
						
	set html_data [http::data $tiny_token]
							
	if {$html_data != ""} {
		return $html_data
	} {
		return "error"
	}
}

putlog "http-query.tcl ver 0.1 by tomekk loaded"

try :>

[JKM]

Code: Select all

proc check_proc { nick uhost hand chan arg } {
	global check_url check_url_arg check_chans

	if {[lsearch $check_chans $chan] != -1} {
		set check_arg [lindex [split $arg] 0]

		if {$check_arg != ""} {
			set one_data_line [grab_http_data 2 $check_url $check_url_arg $check_arg]

			if {$one_data_line != "error"} {
				putquick "PRIVMSG $chan :$one_data_line"
			}
		}
	}
}

Again, thanks a lot! But I've got some questions.
1: I wan't to reply "Nothing found" if the line is " N/A N/A N/A N/A N/A N/A N/A 1 " - so would this work?

Code: Select all

proc check_proc { nick uhost hand chan arg } {
	global check_url check_url_arg check_chans

	if {[lsearch $check_chans $chan] != -1} {
		set check_arg [lindex [split $arg] 0]

		if {$check_arg != ""} {
			set one_data_line [grab_http_data 2 $check_url $check_url_arg $check_arg]

			if {$one_data_line != "  N/A  N/A  N/A  N/A  N/A  N/A  N/A  1  "} {
				putquick "PRIVMSG $chan :$one_data_line"
			
				else {
					putquick "PRIVMSG $chan :Nothing found"
				}
			}
		}
	}
}

2. If the code got DOCTYPE, and if the lines at the html page is listed with <p></p>, will the html code also be visible?

[tomekk]

Code: Select all

proc check_proc { nick uhost hand chan arg } {
        global check_url check_url_arg check_chans

        if {[lsearch $check_chans $chan] != -1} {
                set check_arg [lindex [split $arg] 0]

                if {$check_arg != ""} {
                        set one_data_line [grab_http_data 2 $check_url $check_url_arg $check_arg]

                        if {$one_data_line != "error"} {
                                if {$one_data_line != "  N/A  N/A  N/A  N/A  N/A  N/A  N/A  1  "} {
                                        putquick "PRIVMSG $chan :$one_data_line"
                                } else {
                                        putquick "PRIVMSG $chan :Nothing found"
                                }
                        }
                }
        }
}

"error" should stay because this inform script when page get timeout or some connection problem

" N/A N/A N/A N/A N/A N/A N/A 1 " - when line will be exactly like this one, without "enters" etc then should work

When line includes some html chars this should work too. (i mean whole line will be visible, with <p> etc tags.)

[JKM]

spanks!

[JKM]

Hi there! The scripts is working great, but I was wondering if you could extend the !search script. I wan't to get xml data (<field1></field1> and <field2></field2> but not the rest - including such as "<?xml version='1.0' encoding='us-ascii' ?>" and so on.).

Thanks for the help this far!

[tomekk]

Hi there! The scripts is working great, but I was wondering if you could extend the !search script. I wan't to get xml data (<field1></field1> and <field2></field2> but not the rest - including such as "<?xml version='1.0' encoding='us-ascii' ?>" and so on.).

Thanks for the help this far!

hey,

Paste here full example of this XML structure, I need to see it
Insert some fake data etc.