UNOFFICIAL incith-google 2.1x (Nov30,2o12)

[speechles]

Only USA results?

At first, I thought I was just getting USA results because google was using IP based country lookup irregardless of what country server your actually using. After I coded and released this new feature I actually read the about.html...

About Hot Trends

With Hot Trends, you can see a snapshot of what's on the public's collective mind by viewing the fastest-rising searches for different points of time. You can see a list of today’s top 100 fastest-rising search queries in the U.S. You can also select a recent date in history to see what the top rising searches were and what the search activity looked like over the course of that day. We update Hot Trends hourly.

I expect soon they will enlarge this to encompass other countries instead of being so biased towards the USA. This was just added by google on 2007-05-15 as you can tell by searching any results prior to this, there are none. So when they work the bugs out of their collection/ranking scheme and feel confident it works right within the USA, they will expand it to all countries (maybe?! who knows. let's hope ). When it does, the script should already work and correctly display your countries results.

[speechles]

1st question) Would anyone be interested in the wikipedia portion of this script becoming a "standalone" script?
2nd question) Is this against some wikipedia rule??? Illegal even?? Does this script violate some policy they have?

I never thought to ask before, but I see no real wikipedia scripts in the TCL archive and could easily 'rip' out the !wikipedia and all its features into a standalone script for submitting. Just need to know if first people even care. Thanks.

--- for those interested, read on ----
Keep in mind, there are more features to the wikipedia portion than you can possibly imagine. I haven't divulged all the features.. yet.. it's complete though, you can try it (here as part of the google script where it's limited on how much it can display, in a "standalone" it would be capable of more) and maybe discover the features by using the !wiki trigger and seeing what it supports that wikipedia does (hint: the list of what it can't do is far far shorter )
Try for instance: !w talk:internet relay chat#toc
also a fun read is: !w talk:internet relay chat#piracy

Also, there are two forms of the Table-of-Contents: ToC or [ToC]. The bracketed ToC designates that the wiki page found lacks one, and bot is creating one for you. The subtagging of bracketed ToC's will use some strange encodings perhaps, these cannot be presently matched completely. Bracketed ToC's will use underscores, you can use spaces to reference them as well. The Table-of-Contents tag (#toc) is special when used, it will always take priority and show a full content listing of what is available on the page. There is a priority to matching subtags; First is a direct match of the subtag, second is if it begins the subtag, third and final is anywhere within the subtag. This should address most issues involving bracketed ToC's giving results with encoded subtags and being able to still easily match them.

Sidenote: To foreigners; use this instead:
!w .de diskussion:Internet Relay Chat#toc
!w .fr discuter:internet relay chat#toc
etc, etc, etc for other languages.

Some of the foreign wiki search engines aren't as advanced as the english one, so when using terms with colon's : you sometimes need to be precise and case does matter.

[Alchera]

2nd question) Is this against some wikipedia rule??? Illegal even?? Does this script violate some policy they have?

Wikimedia Foundation

You're going to have do some extensive research.

[Zircon]

1st question) Would anyone be interested in the wikipedia portion of this script becoming a "standalone" script?

Yup me, very interested in such thing, wikipedia gives very good results. I use it extensively.

2nd question) Is this against some wikipedia rule??? Illegal even?? Does this script violate some policy they have?

NO policy should be against spreading knowledge

[speechles]

2nd question) Is this against some wikipedia rule??? Illegal even?? Does this script violate some policy they have?

Wikimedia Foundation

You're going to have do some extensive research.

Okay, here's what I have to report after my research.

Reusers' rights and obligations

If you want to use Wikipedia materials in your own books/articles/web sites or other publications, you can do so, but you have to follow the GFDL. If you are simply duplicating the Wikipedia article, you must follow section two of the GFDL on verbatim copying, as discussed at Wikipedia:Verbatim copying.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

Since the bot gives verbatim the exact same information visible to users on the page, and links relevantly back to the exact article which it is parsing.. I think this falls within fair use of their site. Just reminding everyone that I haven't forgotten about Wikipedia.. In the meantime, I've updated the incith-google link which includes some new fixes to the wikipedia engine (this is how the final design will look, albeit final design will expand on the amount of the article displayed). So if you want to see what it will look like, just click here.

Also about this snippet of text: However, you may accept compensation in exchange for copies
Keep in mind that this is what normally motivates people to create content, in this instance this is not the case. Eggdrop users are especially lucky in that regard as most scripts developed are completely free and written by talented invdividuals. You are getting alot of quality bang for zero dollar, this does not happen often in the real world.

This is why I like this forum and especially those offering tcl advice and scripts.

EDiT: just for fun, and to give an example, here is how the bot would defend itself as being fair use:

<speechles> !w Wikipedia:Copyrights#Reusers
<sp33chy> Wikipedia:Copyrights | Reusers' rights and obligations If you want to use Wikipedia materials in your own books/articles/web sites or other publications, you can do so, but you have to follow the GFDL. If you are simply duplicating the Wikipedia article, you must follow section two of the GFDL on verbatim copying, as discussed at Wikipedia:Verbatim copying. If you create a derivative ve @
<sp33chy> http://en.wikipedia.org/wiki/Wikipedia: ... bligations [1 Redirect(s)]
<speechles> !w Wikipedia:Text of the GNU#2. verbatim
<sp33chy> Wikipedia:Text of the GNU Free Documentation License | 2. VERBATIM COPYING You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical
<sp33chy> me @ http://en.wikipedia.org/wiki/Wikipedia: ... IM_COPYING [2 Redirect(s)]

[Domin]

wouldt it be possible to include or exchange the !locate function whit info from http://www.ip-adress.com since that is much more exact on the ips that i have checked so fare ?

Great script btw

[speechles]

wouldt it be possible to include or exchange the !locate function whit info from http://www.ip-adress.com since that is much more exact on the ips that i have checked so fare ?

--- IP-ADDRESS.COM --->
You reached the limit of 3 lookup queries per day.
Due to some automated bots spidering our
database we had to set a limit.
<----------------------------

They are pretty useless I would say, heh. Geo-Bytes allows FAARRR more than this before they count it as abuse, and their ban is lifted relatively quickly. ip-address.com keeps you banned for whatever a day means to them (24hrs?). So if you have other suggestions for a site to add that isn't this restrictive I will add it. In the meantime IP Locator Script by Rosc2112 is good, she codes expertly and is active here as well.

Great script btw

All credit for the script belongs to incith, the original coder. Madwoota you should thank second, the current maintainer. Once you get to me, all I did was add versatility (adding lots of popular sites) and extensibility (the added ability to regionalize every aspect of searches into any language using a dynamic .country switch) to their already existing code. Making an already great script, even greater. The ironic thing here, is that this unofficial script now fulfills almost the entire wishlist from users in the original incith-google thread (which you can read here). I was unware of this until I read through the entire thread front to back the other day. I got a giggle out of it.

[speechles]

As I promised earlier, the wikipedia script was going to be seperated. But.. I've put that on hold for the moment while I beta-test a new feature. One that doesn't specifically involve wikipedia, but something much more interesting. It involves parsing any Mediawiki enabled site, and works exactly in the same way wikipedia does, but of course instead of the country switch, you now have a site switch.

<speechles> !wm .wiki.pocketheaven.com Silver moon team
<sp33chy> NDS Rom Dumper | The first NDS Rom Dumper, made by DarkFader but released anonymously as being a Silver Moon Team, can dump DS roms using the GBA Movie Player Download @ http://wiki.pocketheaven.com/NDS_Rom_Dumper

<speechles> !wm .www.sabayonlinux.org/wiki FireBug
<sp33chy> FireBug | One of the most useful web developer tools is the FireFox extension FireBug. It allows direct access to the DOM and many other features. However, by default it fails to start in the FireFox that's build with sabayon's default USE flags. In order to make it work add the USE flag "mozdevelop" I added it to the /etc/make.conf and then ran "emerge mozilla-firefox" @
<sp33chy> http://www.sabayonlinux.org/wiki/index. ... le=FireBug

<speechles> !wm .wiki.contextgarden.net Vim#toc
<sp33chy> Vim | ToC: Filetype detection; Spell-checker; ftplugin and syntax file; TODO; compiler plugin; Other useful vim plugins; using latex-suite; Powerful key mappings; key mappings borrowed from scite; Makefile; MetaPost extension @ http://wiki.contextgarden.net/Vim#toc

<speech|es> !wm .wiki.pocketheaven.com the.error.messaging.needs.work.still
<sp33chy> Search results | For more information about searching PHWiki, see Searching PHWiki. Showing below 0 results starting with #1. @ http://wiki.pocketheaven.com/index.php? ... ext=Search

You can probably tell by the last entry above, there isn't exactly a proper error message displayed yet. Now this isn't entirely my fault as mediawiki pages don't exactly give you one. So this is what you get for the moment when nothing is found, not pretty, but at least it gives you something.

You will also now find some new variables to define in the config:

Code: Select all

variable wikimedia_binds "wm wikim wikimedia"
...
variable wikimedia_results 1
...
# set this to the default website you would like wikimedia to use when
# no website is specified.
variable wikimedia_site "wiki.gbatemp.net"

The format for wikimedia_site can be: wiki.yoursite.com or www.yoursite.com/wiki
I'm sure more than this format exist, and you can freely use those as well. This is very much beta, but you can help me by testing it with your favorite mediawiki site and letting me know how it handles them. There will be a stand-alone wikipedia/mediawiki script once the bugs (are there any?) get worked out.

If you ready to try it out you can find it here: UNOFFICIAL-incith-google-v1.96.tcl (July, 27th 2007)

[Domin]

wouldt it be possible to include or exchange the !locate function whit info from http://www.ip-adress.com since that is much more exact on the ips that i have checked so fare ?

--- IP-ADDRESS.COM --->
You reached the limit of 3 lookup queries per day.
Due to some automated bots spidering our
database we had to set a limit.
<----------------------------

They are pretty useless I would say, heh. Geo-Bytes allows FAARRR more than this before they count it as abuse, and their ban is lifted relatively quickly. ip-address.com keeps you banned for whatever a day means to them (24hrs?). So if you have other suggestions for a site to add that isn't this restrictive I will add it. In the meantime IP Locator Script by Rosc2112 is good, she codes expertly and is active here as well.

Great script btw

All credit for the script belongs to incith, the original coder. Madwoota you should thank second, the current maintainer. Once you get to me, all I did was add versatility (adding lots of popular sites) and extensibility (the added ability to regionalize every aspect of searches into any language using a dynamic .country switch) to their already existing code. Making an already great script, even greater. The ironic thing here, is that this unofficial script now fulfills almost the entire wishlist from users in the original incith-google thread (which you can read here). I was unware of this until I read through the entire thread front to back the other day. I got a giggle out of it.

Perhaps you shouldt take a look once again at the adress i wrote, its not the same as the one your checking, the one i wrote have no limit and is by fare bette that geobytes

And yes lets not forget the other that helped here, a big thanks to all who contributed to this script

[speechles]

Perhaps you shouldt take a look once again at the adress i wrote, its not the same as the one your checking, the one i wrote have no limit and is by fare bette that geobytes

Actually, I was checking the correct site. I just gave an extra D when referencing the site and giving the message it eventually gives the bot. It doesn't care to block my normal http queries using firefox no matter how many I make. That is up until the bot has used up 3 queries of it's own, then it does block firefox. This is because the bot uses the same IP as I do on firefox (since I test using windrop on the same machine), so maybe I just need a more clever user agent string to make the bot not look like a bot to them. I'll check into this shortly and see if it's possible. Until then, test out the mediawiki trigger with your favorite pages and let me know how it traverses them. I'm curious if it is truly universal or not. Thanks

[Alchera]

so maybe I just need a more clever user agent string to make the bot not look like a bot to them.

I use a script that uses the following User-Agent string without any dramas:

Code: Select all

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

Not sure if helpful to you or not.

[speechles]

@Alchera, I'll give that a try and see if that keeps it playing nice, thanks *crosses fingers*

<speechles> !wm .www.dslinux.org/wiki sadhfsjadfjhasdf
<sp33chy> Sorry, there were no exact matches to your query.
<speechles> !wm .uncyclopedia.org/wiki qwtye
<sp33chy> You searched for qwtye For more information about searching Uncyclopedia, see Help. No page title matches No page text matches

With the MediaWiki, I've now worked in a proper error messaging system, and done extensive work on the parser detection (mediawiki markups differ from wikipedia ones) and found a small inconsistency with pages not being found correctly. This should make it more compatible for every mediawiki site and return proper results. Get the update at the same link in my post above.

<speech|es> !wm .www.dslinux.org/wiki .User:Stsp
<sp33chy> User:Stsp | I am a DSLinux developer. So far, I have contributed the following to the project: A Mini 6x6 font which is more readable than the Mini 4x6 font. It sacrifices a bit of display width though.. Ipcsync (inter-processor synchronization), a small API allowing device drivers to use the IPCSYNC register of the NDS, which is used for synchronization between the NDS's two @ http://www.dslinux.org/wiki/User:Stsp

<speech|es> !wm .www.dslinux.org/wiki User:Stsp
<sp33chy> Contacting Developers | DSLinux is actively being developed by: Malcolm Parsons (User:Pepsiman), creator of DSLinux. Stefan Sperling (User:Stsp) (see user page for contributions). Amadeus, who designed the DSMEM, made mp3 playback work, did the massive amount of work involved in solving the 8bit write problem, enabling DSLinux to use the RAM in the SuperCard and the M3, and wrote most of @
<sp33chy> http://www.dslinux.org/wiki/Contacting_Developers

The example above is only to demonstrate FORCED destinations. If you prefix your search term with a period (ie .User:Stsp) it will assume you are being direct in your query and will take you directly to that page. This is skipping it's normal search engine routine it normally performs. It's needed, because some (read this as quite a few) mediawiki pages do not allow you to directly search for special pages like you normally can, as you can see above. So without a way to directly access them, you could not otherwise view them. This only works at the moment for Mediawiki pages using /index.php/ design, some wiki designs don't presently use this. I will fix this to work for those sites soon, this was just to give you something now to play with.

One day I will write a readme.txt and include it with this script to somewhat explain in full detail how to use all the powerful features hidden within. I really don't think the config comment section is really explaining enough to be honest. It barely scratches the surface.

[speechles]

One of my buddies on EFnet asked why doesn't the bot dynamically offer help so users can understand what the bot does? Well, finally, the script can do just this.. and it uses the triggers you define, and will let user know how many results you have set and if trigger is disabled.

<speechles> !help hashjadshjadsjhads
<sp33chy> Help is only available for the following: all,google,images,groups,news,local,book,video,fight,youtube,translate,gamespot,
gamefaqs,blog,ebay,ebayfight,wikipedia,wikimedia,locate,torrent,top,popular,dailymotion,ign,myspace,trends

<speechles> !help all
-sp33chy- --> Bot triggers available:
-sp33chy- !g,!google [.google.country.code] [define:|spell:|movie:] <search terms> <1+1> <1 cm in ft> <patent ##> <weather city|zip> <??? airport> with 4 results.
-sp33chy- !i,!gi,!image,!images [.google.country.code] <search terms> with 4 results.
-sp33chy- !gg,!group,!groups [.google.country.code] <search terms> with 3 results.
-sp33chy- !n,!gn,!news [.google.country.code] <search terms> with 3 results.
-sp33chy- !l,!gl,!local [.google.country.code] <what> near <where> with 4 results.
-sp33chy- !gb,!book,!books [.google.country.code] <search terms> with 3 results.
-sp33chy- !v,!gv,!video [.google.country.code] <search terms> with 4 results.
-sp33chy- !f,!fight,!googlefight <word(s) one> vs <word(s) two> with 1 results.
-sp33chy- !y,!yt,!youtube [.youtube.country.code] <search terms> with 5 results.
-sp33chy- !tr,!trans,!translate region@region <text> with 1 results.
-sp33chy- !gs,!game,!gamespot <search terms> with 3 results.
-sp33chy- !gf,!gamefaq,!gamefaqs <system> in <region> with 20 results.
-sp33chy- !b,!blog,!blogsearch [.google.country.code] <search terms> with 3 results.
-sp33chy- !e,!ebay [.ebay.country.code] <search terms> with 3 results.
-sp33chy- !ef,!ebayfight <word(s) one> vs <word(s) two> with 1 results.
-sp33chy- !w,!wiki,!wikipedia [.wiki-country-code] <search terms>[#subtag] with 1 results.
-sp33chy- !wm,!wikim,!wikimedia [.www.wikisite.org[/wiki]] <search terms>[#subtag] with 1 results.
-sp33chy- !geo,!loc,!locate <ip or hostmask> with 1 results.
-sp33chy- !r,!review <gamename> [@ <system>] with 1 results.
-sp33chy- !t,!torrent,!mininova <search terms> with 3 results.
-sp33chy- !top,!best <system> with 10 results.
-sp33chy- !popular,!pop <system> with 10 results.
-sp33chy- !dm,!daily,!dailymotion <search terms> with 4 results.
-sp33chy- !ign,!igame <search terms> with 3 results.
-sp33chy- !m,!myspace,!myvids <search terms> with 3 results.
-sp33chy- !gt,!trends [.google.country.code] <YYYY-MM-DD> with 20 results.

Keep in mind this uses NOTICE and PUTHELP queueing and cannot be changed in settings. This is done to avoid clashing with its normal activities, as well as avoiding the server flood kill (Quit: excess flood). You can grab it here or at any of the v1.96 links above, all are updated the same with this version. Now with the extra power of !help.

[Domin]

Cool, i will test it when i get home from work

[djevrek]

I have this error when im' trying to get wikipedia result from Serbian (Cyrillic) wikipedia.

Code: Select all

<djevrek> !w .sr Srbija
<Grgo> !@18X0 |  5?C1;8:0 !@18X0 X5 :>=B8=5=B0;=0 4@6020 :>X0 A5 =0;078 C XC3>8AB>G=>X 2@>?8 (=0 0;:0=A:>;C>AB@2C) 8 C A@54Z>X 2@>?8 (0=>=A:>X =878X8). # A0AB02C 5?C1;8:5 !@18X5 AC 8 425 0CB>=>:@0X8=5 >X2>48=0 8 >A>2> 8 5B>E8X0. 5?C1;8:0 !@18X0 X5 453 =0@>40 8 A28E 4@C38E 3@0R0=0 :>X8 C Z>X 6825, 70A=>20=0 =0 45X ?@82@548, ?>HB>20ZC YC4A:8E ?@020 8 2;04028=8 ?@020. !@18X0 A @ http://sr.wikipedia.org/wiki/Srbija

I think so that output must be in UTF-8, or it will not work. Well, Serbian language have two scripts, maybe you can make some converter to output those Cyrillic letters in Latin alphabet. You can find dozens of converters online.

P.S. Thanks for this great script !