This is the new home of the egghelp.org community forum.
All data has been migrated (including user logins/passwords) to a new phpBB version.

For more information, see this announcement post. Click the X in the top right-corner of this box to dismiss this message.

Scraping pages while logged in

Help for those learning Tcl or writing their own scripts.
Post Reply
p
pen
Voice
Posts: 4
Joined: Thu Apr 23, 2009 2:04 pm

Scraping pages while logged in

Post by pen »

Hey, I'm trying to figure out how to scrape some text and output it to my irc channel. But the problem is that the text I want is on a page that's only accessible while I'm logged in. Is there a way to do this with tcl?
Top
n
nml375
Revered One
Posts: 2860
Joined: Fri Aug 04, 2006 2:09 pm

Post by nml375 »

Most likely, yes.

I'm assuming you're talking 'bout scraping text off a webpage. "All" you'll have to do, is replicate the login procedure, and you should be all done. Since logons can be done in a multitude of different fashions, that's as good as the answer gets.
NML_375
Top
p
pen
Voice
Posts: 4
Joined: Thu Apr 23, 2009 2:04 pm

Post by pen »

Yes, it's just a regular web page, but unfortunately, you have to be logged in to see it. I can log in and read it just fine, but I wasn't sure if tcl could be used to do the same.
Top
User avatar
tomekk
Master
Posts: 255
Joined: Fri Nov 28, 2008 11:35 am
Location: Oswiecim / Poland
Contact:
Contact tomekk

Post by tomekk »

remember about cookies, most of login "systems" uses session IDs on the server side and session cookies on the client side (web browser)

some hints,
http://wiki.tcl.tk/3828
http://wiki.tcl.tk/17611
Top
Post Reply

Return to “Scripting Help”