tweakers.net / pricewatch.tcl

[DiEPLZ]

Hiya people,

I'm a total n00b when it comes to scripting or coding; but I have searched for a tcl which will give me a result for the pricewatch @ tweakers.net (it's a dutch site wich checks the cheapest prices for internet stores)

The URL is: http://tweakers.net/pricewatch/

And I would like some tcl script that if I do: !pw samsung / !pricewatch samsung

it will announce a variable amount of results (wich could be set in the code)

The layout would be something like this:
!pw samsung
!pricewatch samsung

< pricewatch> Samsung Syncmaster XL2370 - Monitoren - € 298,- (31 winkels) - http://tweakers.net/pricewatch/245280/s ... l2370.html
< pricewatch> Samsung Syncmaster P2370 - Monitoren - € 179,15 (39 winkels) - http://tweakers.net/pricewatch/236702/s ... p2370.html
< pricewatch> Samsung Syncmaster 2443BW - Monitoren - € 227,- (19 winkels) - http://tweakers.net/pricewatch/245685/s ... 443bw.html



Would something like this be possible?
I hope someone is bored and would like to help a n00b out with a nice tcl script for this

It would be very much appreciated!

Thanks in advance


PS: I hope I've been clear enough about the request; if not; please ask for anything so I could try to make it more clear for those who are interested in making the script.

[arfer]

Limited testing done. Seems Ok. Please carefully read what it says.

Code: Select all

# pricewatch.tcl
# arfer <arfer.minute@gmail.com>
# DALnet #Atlantis

### ------------------------------------------------------------------------------------------- ###
### -------------------- OPERATION ------------------------------------------------------------ ###

# scrapes product price information from http://tweakers.net/pricewatch/

### ------------------------------------------------------------------------------------------- ###
### -------------------- CHANGELOG ------------------------------------------------------------ ###

# 1.0 beta release 28/12/09

### ------------------------------------------------------------------------------------------- ###
### -------------------- DEPENDENCIES --------------------------------------------------------- ###

# eggdrop 1.6.17+
# tcl 8.4.x+
# http 2.5.x+ (http package is normally installed by default with the shell's tcl installation)

# IRC client for results output must be UTF-8 compliant, such as mIRC or XChat
# IRC output channel must allow colour/bold text styles

### ------------------------------------------------------------------------------------------- ###
### -------------------- INSTALLATION --------------------------------------------------------- ###

# 1. configure pricewatch.tcl in a suitable text editor
# 2. put the configured pricewatch.tcl in the bot's scripts subdirectory
# 3. add a line to the bots .conf file 'source scripts/pricewatch.tcl'
# 4. restart the bot
# 5. requires partyline '.chanset #channelname +pricewatch' to function in #channelname

### ------------------------------------------------------------------------------------------- ###
### -------------------- SYNTAX --------------------------------------------------------------- ###

# assuming the default trigger character ! (see configuration section below)

# !pw <text>
# !pricewatch <text>

### ------------------------------------------------------------------------------------------- ###
### -------------------- CONFIGURATION -------------------------------------------------------- ###

# set here the single character command trigger
set vPricewatchTrigger !

# set here the bot user flag(s) required to permit use of the public commands
set vPricewatchFlag -

# set here the maximum number of results to return
# note that the actual results may be less than this value, but will never be more
set vPricewatchResults 4

### ------------------------------------------------------------------------------------------- ###
### -------------------- CODE ----------------------------------------------------------------- ###

### ********************************* ###
### ********** DO NOT EDIT ********** ###
### ********************************* ###

### ----- initialise ------------------------ ###

set vPricewatchVersion 1.0

package require http

setudef flag pricewatch

proc pPricewatchTrigger {} {
  global vPricewatchTrigger
  return $vPricewatchTrigger
}

### ----- binds ----------------------------- ###

bind PUB $vPricewatchFlag [pPricewatchTrigger]pw pPricewatchScrape
bind PUB $vPricewatchFlag [pPricewatchTrigger]pricewatch pPricewatchScrape

### ----- procs ----------------------------- ###

proc pPricewatchParse {chan text token} {
  global vPricewatchResults
  regexp -- {<tbody>(.+)</tbody>} [::http::data $token] -> data
  if {[info exists data]} {
    for {set count 1} {$count <= $vPricewatchResults} {incr count} {
      if {[regexp -indices -- {(</tr>)} $data -> idx]} {
        set product($count) [regsub -all -- {[\n\t]} [string range $data 0 [expr {[lindex $idx 0] - 1}]] {}]
        set data [string range $data [expr {[lindex $idx 1] + 1}] end]
      }
    }
    if {[array size product] > 0} {
      set results 0
      foreach name [lsort -increasing [array names product]] {
        if {[regexp -- {<td class=\"title ellipsis\"><p><a href[^>]+>([^<]+)</a>} $product($name) -> model]} {
          if {[regexp -- {title=\"Specificaties[^>]+>([^<]+)</a>} $product($name) -> type]} {
            if {[regexp -- {<td class=\"price\"><p><a href[^>]+>([^<]+)</a>} $product($name) -> price]} {
              set price [regsub -- {€ } $price [encoding convertto utf-8 \u20AC]]
              if {[regexp -- {<p class=\"subtitle\">([^<]+)</p>} $product($name) -> shops]} {
                if {[regexp -- {<td class=\"first thumb\"><a href=\"([^"]+)\"} $product($name) -> link]} {
                  putserv "PRIVMSG $chan :<pricewatch> \002$model\002 -> $type -> \002$price\002 ($shops) -> \00303\037$link\037\003"
                  incr results
                }
              }
            }
          }
        }
      }
      if {$results == 0} {putserv "PRIVMSG $chan :Nothing found for $text"}
    } else {putserv "PRIVMSG $chan :Nothing found for $text"}
  } else {putserv "PRIVMSG $chan :Nothing found for $text"}
  return 0
}

proc pPricewatchScrape {nick uhost hand chan text} {
  if {[channel get $chan pricewatch]} {
    set keyword [regsub -all -- {[\s]} [regsub -all -- {[\s]{2,}} [string trim $text] { }] \+]
    set url http://tweakers.net/pricewatch/zoeken/?keyword=$keyword
    set agent [::http::config -useragent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"]
    if {![catch {set token [::http::geturl $url -timeout 20000]}]} {
      switch -exact -- [::http::status $token] {
        "timeout" {putserv "PRIVMSG $chan :Operation timed out"}
        "error" {putserv "PRIVMSG $chan :Error encountered [::http::error $token]"}
        "ok" {
          switch -- [http::ncode $token] {
            200 {pPricewatchParse $chan $text $token}
            default {putserv "PRIVMSG $chan :Unexpected Ncode [http::ncode $token]"}
          }
        }
        default {}
      }
      ::http::cleanup $token
    } else {putserv "PRIVMSG $chan :Failed to locate URL"}
  }
  return 0
}

### ----- finalise -------------------------- ###

putlog "pricewatch.tcl version $vPricewatchVersion by arfer loaded"

# eof

[DiEPLZ]

Awesome; thank you very much; it seems working perfectly!

I'm really thankful, perhaps in the future i could help you out with something; as I'm learning the language

[DiEPLZ]

Could someone please update this script so it works with the new layout of the website?

Thanks in advance.

I really love this script!

[caesar]

Every time they will update HTML code of the pages the script will stop working until it's changed again in the TCL code.

The best option would be to look if there's an API for searching tough the site. I've googled about it and found some links but don't understand dutch so..

If there isn't any, try contacting the owner and ask if there's one, or maybe he can create one or whatever.

[DiEPLZ]

Thanks for the reply!

unfortunatly there isn't a public API available; although one of the devvers of the website hinted this should always keep working:

http://tweakers.net/xmlhttp/xmlHttp.php ... utput=json

For example:
http://tweakers.net/xmlhttp/xmlHttp.php ... utput=json

I don't know if that outcome is workable to have the same outcome as from when the script was still working....