Http Script

[semisonic]

Hey Guys,

I own a radio scanner site and would like to pull data from a html page that is generated by a pager decode program into irc when new lines are added/updated in the html file..

http://www.centralcoastonlinescanner.co ... ilter.html is where the data is stored..

As you can see there is alot of double which hopefully the tcl script can get rid of.. if not that's cool..

So i need the data in between each ------ (<HR>) displayed in the channel
say the last 5 lines when !trig and just the last update line automaticly (If possible) if not the last 5 updated lines will be fine..

How do i parse the crap and just leaving the data between <tr> and </tr> or even<td> and </td> in a format like on the Html page minus color?

Thanks in advance
Sonic

[semisonic]

*Bump*

*Bump*

Can someone at least help me with Regsub to get the info out

or how to get the code to display?

Thanks
Sonic

[arfer]

You may be able to make something of this code, It takes the site data and lappends it to a list variable. For testing a PUB bind using the command !scan is used and the data written to a text file (scanner.txt) in the bots root directory.

Note that it should be considered very much a work in progress for you to adapt if it is any use. No error messages are coded in case the bot cannot connect, or for http timeout. error or unexpected ncode. Likewise html codes such as > or & etc have not been translated.

Code: Select all

# scanner.tcl

package require http

bind PUB - !scan pScannerScrape

proc pScannerScrape {minute hour day weekday year} {
    set agent [::http::config -useragent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"]
    set url http://www.centralcoastonlinescanner.com/pager/filter.html
    if {![catch {set token [::http::geturl $url -timeout 10000]}]} {
        switch -- [::http::status $token] {
            "timeout" {}
            "error" {}
            "ok" {
                switch -- [::http::ncode $token] {
                    200 {
                        regexp -nocase -- {<table>(.*?)</table>} [::http::data $token] -> data
                        pScannerParse $data
                    }
                    default {}
                }
            }
            default {}
        }
        ::http::cleanup $token
    } else {}
    return 0
}

proc pScannerParse {data} {
    global vScannerDatalist
    foreach line [split $data \n] {
        if {![regexp -nocase -- {<tr><td colspan=\"3\"><hr></td></tr>} $line]} {
            set temp [string trim [regsub -all -- {\s{2,}} [regsub -all -- {(<[^>]+>)|(\t)} $line { }] { }]]
            if {[string length $temp] != 0} {
                lappend vScannerDatalist $temp
            }
        } else {lappend vScannerDatalist "--------------------------------------------------"}
    }
    pScannerWrite
    return 0
}

proc pScannerWrite {} {
    global vScannerDatalist
    set fp [open scanner.txt w]
    foreach line $vScannerDatalist {
        puts $fp $line
    }
    close $fp
    unset -nocomplain -- vScannerDatalist
    return 0
}

putlog "scanner.tcl loaded"

# eof

Here is a portion of the beginning of the text file :-

18:00:32 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS HIGH PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Wyong RFS Message -
18:00:18 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS HIGH PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Wyong RFS Message -
18:00:11 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS HIGH PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Wyong RFS Message -
--------------------------------------------------
17:58:28 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS LOW- MODERATE PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Lake Macquarie RFS Message -
17:58:21 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS LOW- MODERATE PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Lake Macquarie RFS Message -
17:58:14 04-03-11 FIRE DANGER RATINGS FOR SATURDAY IS LOW- MODERATE PLEASE ADJUST ROADSIDE FIRE DANGER METERS. THANK YOU - Lake Macquarie RFS Message -
--------------------------------------------------
17:49:04 04-03-11 CAT 2 BACK IN SERVICE - Wyee Point RFS -
17:48:59 04-03-11 CAT 2 BACK IN SERVICE - Wyee Point RFS -
17:48:54 04-03-11 CAT 2 BACK IN SERVICE - Wyee Point RFS -
--------------------------------------------------
17:39:54 04-03-11 STOP MESSAGE - Awaba RFS -
17:39:50 04-03-11 STOP MESSAGE - Awaba RFS -
17:39:40 04-03-11 STOP MESSAGE - Awaba RFS -
--------------------------------------------------
17:25:05 04-03-11 thanks. - Arcadia RFS -
--------------------------------------------------

[semisonic]

Thanks Very Much Arfer....

That is a start

I will try and build on from there

Thanks for your time..

Sonic