Accepting DCC Files, without filesys.mod

[Empus]

Hi,

I am wanting to write some code whereby an eggdrop can accept DCC SENDs to it.

However, the Filesys module is not approriate, for a few reasons:

a). I do not want to use the eggdrop userfile for matching, but rather SQL
b). I need to screen what files will be accepted (sizes, file types, source)
c). I need the files to not be all saved to a generic dir

I really dont know how possible this is with TCL, it has me stumped.

Direction would be great - all the other things are fine, it's the actually transferring of files I am at a loss about. I have no idea how to write something to allow this to happen, without the filesys module.

Cheers,

-Mike

[user]

First, read this to understand how dcc send works.
Then make a script that does something like this (i've left out parsing and the tcp/file parts )

Code: Select all

bind ctcp - DCC mydcc
proc mydcc {n u h t a} {
	if {![isbotnick $t]} {return 0}
	switch -- [scan $a %s] {
		"SEND" {
			# determine if you want to allow it and resume or create a new file
			# then ignore/reply(+remember)/connect based on that
		}
		"ACCEPT" {
			# if expected, connect
		}
		default {
			# allow further processing of other keywords (to allow ACTION, CHAT etc.)
			return 0
		}
	}
	return 1;# this will halt further processing of SEND and ACCEPT requests
}

To fetch the data, you make an outgoing tcp connection using the tcl command 'socket' (make sure you don't mess up the data by making the 'translation' for both the socket and the local file 'binary' (using fconfigure)).

[Empus]

I've seen some of your posts on this forum, and to be honest I was hoping you may be the one who replied to this :>

It's the TCP/File bits of this that I'm not up with; As far as I know, DCC requires acknowledgements (every 2K) where the file is written in blocks, as opposed to a streamed connection. I really have no idea how to handle this, or how to specify where the file is saved to.

Any ideas?

[user]

You have to decide where to put the file, as there is (or should be) no directory included in the file name recieved in the ctcp.
The "blocks" is just packet size (I think), so it should be handled by the OS. All you need to do is create the socket, wait for the connection to complete, open the file and fcopy... something like this:

Code: Select all

set sock [socket -async $ipFromCtcp $portFromCtcp]
fconfigure $sock -blocking 0 -translation binary
fileevent $sock writable [list sockConnected $sock $fileName etc...]
proc sockConnected {sock filename etc...} {
	fileevent $sock writable {}
	if {![eof $sock]&&[fconfigure $sock -error]==""} {
		# connected... open the file and start fcopy (that's a tcl command - look it up in the manual)
	} else {
		# complain
	}
}

Edit: I was wrong about the packet size stuff, but I think you can ignore that part of the protocol (it doesn't seem to be required by clients)
If you HAVE to implement that part, you will have to postpone the fcopy untill you've read the first packet to determine the "block size", then use it with the -size option when you set up the fcopy (then transmit the file size each time your callback command is invoked)

[Empus]

user you are a champion, your help is much appreciated.

Whilst I've scripted for quite a few years now, I've never really had to play with sockets so no doubt I will need a bit more help along the way, but you have definitely given me enough to start with.

[Empus]

Ok, I used the code example you gave me, and also from the FCOPY man page. The file is transferring, but I'm not getting an acknowledgement of a successful transfer.

I initially wasn't declaring the variable "done" as global, so when I added that (in sockConnected), the CopyMore proc was running. However, I'm not seeing the completion, and now the bot is locking up, as if it is entering an infinite loop:


--snip--

Code: Select all

proc sockConnected {sock file} {
   global total done
   fileevent $sock writable {}
   if {![eof $sock]&&[fconfigure $sock -error]==""} {

        set out [open /home/bot/upltmp/[unixtime]-$file w]
        set in $sock
        set chunk 2048
        set total 0

        fcopy $in $out -command [list CopyMore $in $out $chunk] -size $chunk
        vwait done

   } else {
      # complain
   }
}

proc CopyMore {in out chunk bytes {error {}}} {
    global total done
    incr total $bytes

    if {([string length $error] != 0) || [eof $in]} {
        putlog "file saved. total $total"
        set done $total
        close $in
        close $out

    } else {
        fcopy $in $out -command [list CopyMore $in $out $chunk] \
            -size $chunk
    }
}

So, like I say: the file is transferring, and CopyMore is being ran, but it seems to lock the bot, and I don't get an acknowledgement that the transfer completed.

Also, you mentioned about the packet sizes - how should I determine what these should be?

Thanks again.

[Empus]

Ok, so I have an update.

I'm using fcopy, but the callback proc doesn't appear to be running, or I'm having mixed results (ie. the last block isn't copied).

Also, I just read http://forum.egghelp.org/viewtopic.php?t=11518 and am now worried about this implementation. If I use vwait during the transfer, then nothing else within the bot (or even the bot as an irc client) will operate.

This will cause major problems.

Any ideas?

Code snip (with a bunch of debug putlogs):

Code: Select all

# notes
# socket, file and size are set elsewhere


proc sockConnected {sock file} {
   global total done size chunk
   fileevent $sock writable {}
   if {![eof $sock]&&[fconfigure $sock -error]==""} {
      # connected... open the file and start fcopy

	# FCOPY

	set out [open /tmp/[clock-seconds]-$file w]
	set in $sock
        putlog "in: $in | out: $out"
        set chunk [expr $size / 2]

        set total 0
        putlog "chunk is $chunk, size is $size, starting fcopy"

        fcopy $in $out -command [list CopyMore $in $out] -size $chunk

        vwait done

        putlog "done?"        

   } else {
      # complain
   }
}


proc CopyMore {in out bytes {error {}}} {

    set timestamp [clock format [clock seconds]]
    set fl [open CopyMore {WRONLY CREAT APPEND}]
    puts $fl "$timestamp: CopyMore started (chunk: $chunk)"
    close $fl

    global total done size chunk

    putlog "CopyMore chunk is $chunk and bytes is $bytes"
    incr total $bytes

    if {([string length $error] != 0)} {
        putlog "no error"
        if {[eof $in]} { 
          putlog "in ($in) EOF"
        }
        if {$total >= $size} {
         putlog "file saved. total is $total and size is $size"
	 set done $total
	 close $in
	 close $out
        }

    } else {
        putlog "total is now $total (of $size) bytes - chunk is $chunk"
	fcopy $in $out -command [list CopyMore $in $out $chunk] -size $chunk
    }
}

proc bgerror {message} {
    set timestamp [clock format [clock seconds]]
    set fl [open bgerror.txt {WRONLY CREAT APPEND}]
    puts $fl "$timestamp: bgerror in $::argv '$message'"
    close $fl
}

[nml375]

You really don't need vwait, as eggdrop calls Tcl_DoOneEvent() in the main loop, and thus works in a similar fashion as "wish" (Tk-enabled tcl-shell).

Using bgerror is a very good idea, as eggdrop otherwize lack any means of logging tcl-event errors. Keep in mind that ::errorInfo will be properly defined, and may hold some additional - and helpful - information.

[Empus]

But I need something to call the event loop, this is the reason I'm using vwait...

Can you see what is wrong with the code I've pasted, or suggest another method to try?

I really seem to be getting nowhere fast.

[Empus]

It shall also be noted, that my bgerror code shown here, has not produced anything in bgerror.txt thus far.

[nml375]

Roughly, all vwait does is: 1. call Tcl_DoOneEvent(), and 2. Check wether the variable specified on the commandline has been altered; if not, back to 1, else return.

Since eggdrop itself calls Tcl_DoOneEvent(), there's no need for vwait unless you want your script to block until done has been set.

I do see a possible error in CopyMore, you try to log the contents of chunk, but it's not set there. You also try to increase the value of total, which also isn't set there. Would seem you're abit confused with namespaces. Further, you don't set the globalspace variable done monitored by vwait, but rather a local one in CopyMore - possibly causing your script to wait forever.

To simplify logging, you might simply use putlog in your bgerror function.

Finally, I don't see you sending any Ack's back to the sending party.

[Empus]

You're right, the script does seem to wait forever.

I've removed vwait, and it no longer does this, but CopyMore seems to still not run.

Can you clarify your point of namespaces, and offer advice as to what to do to get this working?

I used file writes in bgerror to ensure that if the eggdrop itself hangs, the tcl will write to the file. Plus, when asking TCL folk about this, I've tried to eliminate eggdrop references, otherwise no-one will want to help me.

This is driving me nuts and I very much so want to get the code working.

How should the acknowledgements be sent?

[nml375]

Namespaces; any variable is defined within a certain namespace, and cannot be accessed from another namespace without full addressing.

The ones to keep track of in this case are globalspace (::), and and the anonymous spaces within a given invocation of a proc (also known as local variables). That is, $myvar in proc1 is not the same as $myvar in proc2; nor is it the same the first and the second time you call proc1 (ie when doing recursive calls).

To access the globalspace variables, you have two options, either use the globalspace address ($::myvar) or use the global command to link a local variable to the globalspace one.

Most likely, you intended to use the global command in CopyMore just as you did in sockConnected.


DCC Ack's;
The ack should be a 4 byte, network order, unsigned integer of the number of bytes recieved sofar.
A further explanation may be found here: http://www.irchelp.org/irchelp/rfc/dccspec.html

[Empus]

Ah yes, I've changed little bits and pieces in this code for days now trying to work out why it isn't working, but to no avail.

I did have them declared with 'global', perhaps just not in that example.

CopyMore doesnt seem to be looping properly.

In CopyMore, i've added a: puts $in $total, as acknowledgement.

I've read that page, but the issue seems to be TCL related. It's not looping so not all blocks are transferred, meaning there is no EOF, and $size never equals $total (because $total is not incrementing.

Are you able to be any more specific?

[nml375]

I'm not familiar with fcopy in particular, I'm afraid.

A possible issue might be that the client is waiting for the ack before sending the next packet

One approach I'd considder is using fileevents to trigger some proc whenever there is data to be read at the input socket. I'll see if I can't craft a rough example in the next day or two.