mysql.mod

tajson · Post by **tajson** » Tue Jan 01, 2002 2:24 pm

Hi.

I have created a snapshot release of my mysql.mod for eggdrop because some of you here were interested in it.

Some quick notes:
It only logs channel activities right now, not updateing the users database. That is comming in the initial stable release.
The logging should work perfectly thou I hope, and that is the main purpose with the module.

I want some feedback since I'm a newbee C programmer.
Please also contact me if you make any third party scripts/modules/standalone software that uses mysql.mod so I can link you from my hompage. I am going to do some PHP scripts to demonstrate to use mysql.mod collected data.

You can download the latest snapshot from my homepage wich is http://joakim.gissberg.nu/

With best regards, Joakim Gissberg.

_________________
http://joakim.gissberg.nu/ | joakim@gissberg.nu

[ This Message was edited by: tajson on 2002-01-01 11:25 ]

Yourname · Post by **Yourname** » Thu Jan 03, 2002 4:13 am

I always wondered why people want MySQL do do their seen requests.. mind explaining me please?

tajson · Post by **tajson** » Thu Jan 03, 2002 6:24 am

On 2002-01-03 01:13, Yourname wrote:
I always wondered why people want MySQL do do their seen requests.. mind explaining me please?

("select nick, channel from irclog where nick='%s'order by timestamp desc", seennick)

Now just print out the result. Hard? Nope.

And you can also use thesame information to do !online, !top10, webstats and much much more. It's not just seen, its the information. What you do with it is up to you.
_________________
http://joakim.gissberg.nu/ | joakim@gissberg.nu

[ This Message was edited by: tajson on 2002-01-03 03:25 ]

stdragon · Post by **stdragon** » Thu Jan 03, 2002 10:01 am

Hi, I was looking in mysql.c and I noticed your question about mysql_expmem. You did the correct thing. The only thing the expmem function is supposed to report is the dynamic memory your module uses via nmalloc. Since you don't use nmalloc, the expected memory usage from eggdrop's point of view is 0.

Here's a problem:

Code: Select all

char escape(char *string)
  {
    char *tmpString[2048];
    Context;

    mysql_real_escape_string(&mysql, tmpString, string, (unsigned int)strlen(string));
    snprintf(string, sizeof(tmpString), tmpString);

    nfree(tmpString);
    return *string;
  }

If tmpString is supposed to be a string, it should be declared
char tmpString[2048];
The * makes it an array of 2048 pointers, not characters. Also, it should not be nfree()'d, because it was not nmalloc()'d.

And, escape() should not modify its argument. When you call escape(nick); escape(host); escape(handle); escape(channel); etc. you are running the risk of a buffer overflow (if the escaped string is larger, it will overflow the nick/host/handle buffer) and you may corrupt future binds, because you are modifying the bind arguments.

Your overall module structure is good. I'm curious, did you figure it all out yourself (like importing the H_* tables) or did you use a template or some tutorial? Several people on this forum have asked for a module tutorial. If you used one, that could be helpful to share

Finally, as sort of a feature suggestion.. once you have all the mysql code there, why not create some simple tcl commands to access them? Like mysql_connect, mysql_execute, etc. Then your module would be useful to everybody who wants to use mysql in their tcl scripts (but not necessarily for channel logging).

tajson · Post by **tajson** » Thu Jan 03, 2002 12:16 pm

First before answering all the parts I must say - Thank you! This is what a community is all about and in specific OpenSource, helping each other.
I have never claimed to be an programmer, but since I didn't find some programs I wanted I just learned C. I started a couple of months ago with some basic C and here we are...

Hi, I was looking in mysql.c and I noticed your question about mysql_expmem. You did the correct thing. The only thing the expmem function is supposed to report is the dynamic memory your module uses via nmalloc. Since you don't use nmalloc, the expected memory usage from eggdrop's point of view is 0.

As I will say below (jumping in my answers when I'm writing), that is something I must study harder on.

Here's a problem:
Code: Select all
char escape(char *string)
  {
    char *tmpString[2048];
    Context;

    mysql_real_escape_string(&mysql, tmpString, string, (unsigned int)strlen(string));
    snprintf(string, sizeof(tmpString), tmpString);

    nfree(tmpString);
    return *string;
  }
If tmpString is supposed to be a string, it should be declared
char tmpString[2048];
The * makes it an array of 2048 pointers, not characters. Also, it should not be nfree()'d, because it was not nmalloc()'d.

Make sense to me...
And this with dynamic memory allocation is something I must study more on, I'm still on basic C programming.

And, escape() should not modify its argument. When you call escape(nick); escape(host); escape(handle); escape(channel); etc. you are running the risk of a buffer overflow (if the escaped string is larger, it will overflow the nick/host/handle buffer) and you may corrupt future binds, because you are modifying the bind arguments.

The escaped string is always larger when returned if it contains anything that must be escaped (like '%_) etc. A "'" will be escaped like "'". How to deal with that?
Make a copy?

Your overall module structure is good. I'm curious, did you figure it all out yourself (like importing the H_* tables) or did you use a template or some tutorial? Several people on this forum have asked for a module tutorial. If you used one, that could be helpful to share

I looked at the woobie source and figured out with time. Its all there in the source somewhere, you just have to find it.
I am willing to learn, that is my secret. I just never give up.

Finally, as sort of a feature suggestion.. once you have all the mysql code there, why not create some simple tcl commands to access them? Like mysql_connect, mysql_execute, etc. Then your module would be useful to everybody who wants to use mysql in their tcl scripts (but not necessarily for channel logging).

Infact, that is something that will come in the feature. It's in my plans for the module. But I first want to do STABLE logging, that's the main purpose for me since I did this module for myself.

Edit: The script didn't like my escape example, it should be a backslash before the escaped character.

_________________
http://joakim.gissberg.nu/ | joakim@gissberg.nu

[ This Message was edited by: tajson on 2002-01-03 09:26 ]

blackwidow · Post by **blackwidow** » Thu Jan 10, 2002 9:24 pm

two errors, one of them critical:

gcc -pipe -fPIC -g -O2 -Wall -I. -I../../.. -I../../.. -I../../../src/mod -DHAVE_CONFIG_H -I/usr/local/mysql/include -DMAKING_MODS -c .././mysql.mod/mysql.c .././mysql.mod/mysql.c:27:25: mysql/mysql.h: No such file or directory

I fixed this by creating a mysql directory under /usr/local/mysql/include and copying /usr/local/mysql/include/mysql.h into it.

error 2, I can't get past:

gcc -pipe -shared -nostartfiles -L/usr/local/mysql/lib/mysql -lmysqlclient -o ../../../mysql.so ../mysql.o
/usr/bin/ld: cannot find -lmysqlclient

I'm on a redhat 7.2 linux install, using mysql 3.23.43 and eggdrop 1.6.8. thanks for any help you might have

blackwidow · Post by **blackwidow** » Thu Jan 10, 2002 9:36 pm

ok, I figured out my second error. I changed the makefile to look in /usr/local/mysql/lib instead of /usr/local/mysql/lib/mysql. odd.

tajson · Post by **tajson** » Sun Jan 13, 2002 12:57 am

On 2002-01-10 18:24, blackwidow wrote:
two errors, one of them critical:

gcc -pipe -fPIC -g -O2 -Wall -I. -I../../.. -I../../.. -I../../../src/mod -DHAVE_CONFIG_H -I/usr/local/mysql/include -DMAKING_MODS -c .././mysql.mod/mysql.c .././mysql.mod/mysql.c:27:25: mysql/mysql.h: No such file or directory

Point --with-mysql to the directory that contains include/mysql.h.
Like /usr/local/mysql, /usr, /usr/local etc...

error 2, I can't get past:

gcc -pipe -shared -nostartfiles -L/usr/local/mysql/lib/mysql -lmysqlclient -o ../../../mysql.so ../mysql.o
/usr/bin/ld: cannot find -lmysqlclient

If not in --with-mysql path, you can export it into $LD_LIBRARY_PATH and/or by adding it into /etc/ld.so.conf

I (with the help of ppslim) are working on a better configure script that will do this work better.

Yourname · Post by **Yourname** » Tue Jan 15, 2002 2:53 am

And you can also use thesame information to do !online, !top10, webstats and much much more. It's not just seen, its the information. What you do with it is up to you.

Well, but then gseen.mod by G`Quann also works the same way, except for that !online thingie or so... but then sometimes gseen sez Returned too many matches... thats one thing i dislike about it.

ppslim · Post by **ppslim** » Tue Jan 15, 2002 11:23 am

if gseen returns too many matches, then a Tcl using seen data from the MySQL.mod would certainly return too many matches.

Instead of searching a list (bseen) or a struct (faster gseen), it uses it's own mini programing language to locate and return records.

The bseen ans gseen issue is easily solved. It only needs to return the top 6 results (EG the latest 6 events recorded).

The advantage of mysql oin seen scripts.

For a start, it only requires one snapshot
of data, instead of each bot doing it's own logging.

Only one bot required to write to file, avoiding conflicts.

If data logging script is coded correctly, it is possible to log channel accross networks, or even channels that some bots where never intended for. Providing a much more complete seen database.

No memory requirments. Data is logged to file, by a server designed to do the task, rather than a thought up data format. There is no need to store seen info in local memory.

Ability to colect and store info not even intented for seen purposes, yet retrievable by other bots too, without the requiremnet of sending files. With MySQL selects, you limit the results returned, instead of having to search a file yourself, or transfer the whole file to a local file, just to search. This is usful if you are using very large datasources (20MB +).

Ability to interact with other systems, and web programs, without the need to create your own parsers. EG, webseen, the module addon for gseen, allows gseen results to be returned as a searched webpage. This is more memory consuming for eggdrop, as it has to search, enconde and handle http transactions. PHP, perl, Tcl and many other MySQL able (addons maybe needed) languages can simply query the database, and the information is then rediliy available, gurenteeing 100% pure HTTP headers, and 0 extra load on eggdrop.

There are hundreds of advantages. SOme of them may not be suited, or just seem like there is no reason, but the general idea is, that this information is sharable.

Yourname · Post by **Yourname** » Sat Jan 19, 2002 3:19 pm

Hmmmm, so i see that mysqlseen has more advantages than the other ones, i.e bseen, gseen.

Ok, but then do the other shells allow usage of MySQL?
And if the mod is made, what about the scripts that will be using them..Im sure even that needs to be very well coded.

ppslim · Post by **ppslim** » Sat Jan 19, 2002 10:23 pm

Good question indeed, do shell providers allow MySQL?

Well, for ones that provide you with a MySQL DB with the account, I supose they do.

As for others, unknown, you would have to ask.

But then again, what about the uptime.mod - Did any1 bother to ask the shell provider about using this?

tajson · Post by **tajson** » Sat Jan 19, 2002 11:54 pm

I just want to say that the next mysql.mod release seems stable in the logging part wich is the main purpose of the module. I will start adding the cool features after that wich also will do the sometimes hard job for other scripts, like keep track of wich users that are currently in the channel in a mysql table.

I also want to thank the betatesters (you know who you are) wich helped me with feedback and ideas. Thanks!

Yourname · Post by **Yourname** » Sun Jan 20, 2002 5:35 am

Why? whats about uptime.mod? Does it also use MySQL?

tajson, pal, i'd request you to make a good one (the mod) i hope you get a stable release soon.

And why dont u try to incorporate webseen into the mod too?.. that'd give an edge. As well as the goodies gseen offers (bseen doesnt have so many) This will make ppl use MySQL more than these scripts.

Wish you luck guyz.. you cud add thegoodies after stabilizing the mod first

tajson · Post by **tajson** » Sun Jan 20, 2002 7:20 am

On 2002-01-20 02:35, Yourname wrote:
tajson, pal, i'd request you to make a good one (the mod) i hope you get a stable release soon.

And why dont u try to incorporate webseen into the mod too?.. that'd give an edge. As well as the goodies gseen offers (bseen doesnt have so many) This will make ppl use MySQL more than these scripts.

The module is stable now. I just have to make a new tarball and upload it.
This module is a data collector, it doesn't include any userfunctions and will newer do. If you want a webseen, use the data collected by this module. It's up to third party modules/scripts to take the credit. I did an example for ppslim who asked the same question, it's a simple webstatistics in real-time. Check my homepage (source is online).