SpamScan

[metroid]

I made this script awhile back and have been testing it quite alot.
You can download the script here
http://www.development.woosah.org/download.php?view.8
Tell me what you think of it

I'm sending it to slennox soon so you won't find it in the tcl archive right now

[demond]

care to explain how does it do its job, maybe paste some relevant code?

I'm curious, but can't be bothered to download and examine it hehe

[metroid]

Here are parts of the script, that should make you understand i guess

Code: Select all

# /* Configuration
# * In seconds, If the last time they spammed was 20 seconds they will get 15 points added (default settings)
variable time      "10"
variable time2     "15"
variable time3     "20"

# * Points we give for spamming/advertising
variable advert    "40"
variable advert2   "50"
variable advert3   "60"

# * Points we give out for flood/spam
variable flood     "15"
variable flood2    "10"
variable flood3    "5"

# * Points they need to get punished.
variable warn      "80"
variable kick      "100"
variable ban       "120"

Code: Select all

  if {$spamscan::spamscan} {
   if {[advertisement $arguments]} {
    if {$checktime >= $spamscan::time3} {
     incr db($channel,$ident) $spamscan::advert3
    } elseif {$checktime >= $spamscan::time2} {
     incr db($channel,$ident) $spamscan::advert2
    } else {
     incr db($channel,$ident) $spamscan::advert
    }
   }
  }

Code: Select all

  if {$db($channel,$ident) >= $ban} {
    putlog "$nickname was \0034banned\003 in $channel. \(score: $db($channel,$ident)\)"
    set banmask [banmask $hostname]
    putquick "MODE $channel +b $banmask"
    putquick "KICK $channel $nickname :[string map "%nickname $nickname %channel $channel %id [expr [channel get $channel spamkicked] + 1]" [join $kickmsg]]"
    channel set $channel spamkicked "[expr [channel get $channel spamkicked] + 1]"
    if {$bantime != "0"} {
     timer $bantime [list pushmode $channel -b $banmask]
    }
  } elseif {$db($channel,$ident) >= $kick} {
    putlog "$nickname was \0038kicked\003 in $channel. \(score: $db($channel,$ident)\)"
    putquick "KICK $channel $nickname :[string map "%nickname $nickname %channel $channel %id [expr [channel get $channel spamkicked] + 1]" [join $kickmsg]]"
    channel set $channel spamkicked "[expr [channel get $channel spamkicked] + 1]"
  }

It will just add a score for each line said and decrease the score over time, If the score gets to a certain point the script will do something, like warn or kick that person.

It's purely for text and stuff, not other things

(This is how the spamscan service on Quakenet works)

[demond]

hmm I was actually more interested in the spam detection method - what your definition for spam is and how it gets applied to channel traffic

I presume it's in the [advertisement] proc

[Dizzle]

I see its reacting on these advertise lines

Code: Select all

proc spamscan::advertisement {line} {
 if {[regexp -- {\x23\S+|[a-zA-Z0-9]+://\S+\.[a-zA-Z0-9]+|www[0-9]*\.\S+\.[a-zA-Z]+} $line]} {
  return 1
 }
 return 0
}

Does this detect most off the advertisment??

Im going too test it outonmy channel if i find something ill post it here

[metroid]

That proc will detect if an channel name or website is being advertised and that's all it does. Hence the procname

[Dizzle]

i tested the script untill now i like it, no bugs or weird things happend, butt one question metroid.

Is there a way i can putt in aline where the doesnt react on ??

in my channel ppl can request relay's (based on your forum script)

butt ppl get warned or even kicked when they use !request 01 #chan

is there a way i can make the script ingore ONLY this ??

greetz Dizzle

[awyeah]

Here is a good match pattern for regexp to detect all types of urls:

Code: Select all

(https?|ftp|file)://[-A-Z0-9+&@#/%?=~_|!:,.;]*[-A-Z0-9+&@#/%=~_|]

Also I was wondering how can I detect repeated words in a string through regexp or even string match? Then count how many are present? For regexp I found this:

Code: Select all

(\w+)\s+\1

I can do that by using 2 foreach loops on the same string, checking one word, matching against others and going for the next one and doing all again and going further. But is there a simpler and easier way?

[greenbear]

or you can use this if you want to find a url inside a string and dont want to use a regexp longer than your arm..

Code: Select all

regexp -nocase {((http|ftp)://[^\s]+)} $text url

[Sir_Fz]

I suggest adding a check for server advertising and those lame decode messages.

[awyeah]

So about that:

Also I was wondering how can I detect repeated words in a string through regexp? Then count how many are present like with the -all switch I can do that. The main thing is to detect repeated words not characters.

Anyone?

[demond]

if by "word" you mean sequence of non-whitespace characters, you need to split, trim and then count repeating list elements (be careful to avoid redundant string comparisons); hint: use Tcl's associative array

[awyeah]

First of all:
set string [string trim [split $text]]

Then:
Umm, tcl assoicated arrays. Actually I am unable to get the logic on how to count for repeated words using the array logic in my head currently. Have any rough example or so?

[demond]

Code: Select all

foreach word $words {
   if {[info exists count($word)]} {
      incr count($word)
   } {
      set count($word) 1
   }
}

on a second thought, you can do that without splitting into a list, just by using [scan] - but I'm going to bed already

[awyeah]

Hehe, okay that wasn't difficult enough, just using normal arrays. Incr count if a similar word is found, else there is a new word.

Sometimes the easiest things can take alot of time to receive from the brain.
No I am not good with scan, I use lindex, split to get away from it.

NOTE: I was trying to develop a way to deteriorate flood bots who use messages with repeating words and text. Based on varying score for words in a string this can be handy.