A truely "random" random number generator

[dj-zath]

Hi Nml, Mindles, Caesar, everyone!

Heres one to knaw on!

First: the task..

I need to create a number genny that can output three (or possably, in the future, more) different cascaded outputs, but not exceed a specific set value.. yet not include the previous value(s) consectively.


Now, the (current) varables...

note: these values were chosen during testing.. and are expected to change as more of the system comes online

PromoCnt is the number of promos included in the playset; its from 0 to a number.. currently "32"

$PromoCnt = 32

VoiceCnt is the number of "vocal promos" in the playset; it excludes the first set of promos ($PromoCnt) and includes to a given number... there are currently no voice promos- so its set to 0

$VoiceCnt = 0 (not yet used)

PlayCnt is the number of music tracks in total, excluding $PromoCnt and $VoiceCnt; it has a defined limit that cannot be exceeded.

$PlayCnt = 38502

oh, and one more thing.. any output cannot match the previous output.. that results in the SAME song (or promo) playing again (sometimes more than once) in a row!

How this code is ran, is, its called in another proc and ran "on the spot"

example:

Code: Select all

proc   GetRand {} {
         putlog "Rand 1: [Rand1]"
         putlog "Rand 2: [Rand2]"
         putlog "Rand 3: [Rand3]"
         return "0"
}

note: This is just an example on how I 'call' the rand's.. I do not use this piece if code itself.

and now, heres the code I came up with:

Code: Select all

proc    Rand1 {} {
        global Rnd1 PromoCnt;
        if {([catch {set Rnd1 [rand $PromoCnt];}])} {
                set Rnd1 "0";
        };
        set Rnd1 $Rnd1;
};

proc    Rand2 {} {
        global Rnd2 VoiceCnt PromoCnt;
        if {([catch {set Rnd2 [expr [rand $VoiceCnt] + $PromoCnt];}])} {
                set Rnd2 "0";
        };
        set Rnd2 $Rnd2;
};

proc    Rand3 {} {
        global Rnd3 PlayCnt VoiceCnt PromoCnt;
        if {([catch {set VarA [expr [rand [expr $PlayCnt - [expr $VoiceCnt + $PromoCnt]]] + [expr $VoiceCnt + $PromoCnt]];}])} {
                set VarA "0";
        };

        if {($VarA == $Rnd3)} {
                set VarA [expr $VarA + [rand [expr $Rnd3 + "1"]]];
        };
        if {($VarA > [$PlayCnt])} {
                set VarA [expr $VarA / [expr [rand "3"] + "1"]];
        };
        set Rnd3 $VarA;
};

As you can see, it has "problems".. it can crash on "devide by zero" and "limit cannot be zero" errors..

its also not truely "random".. it has a tendency to "hover around" a specific set of numerals for some reason, (which is why I tried adding some more "math" to the output on #3) but its not that good .. the initial "if statements" are Okay, though since they fix one specific error.. but, as mentioned eariler.. the "rand" command seems "not too random" to begin with.

I was told that its is because "rand" is only a 16 bit output

so I need a "better" random gennerator to address this requirement.

-DjZ-

[nml375]

Well...
True random numbers don't exist... Atleast not unless we dig deep into Quantum Theory.. With Computer Science, we can achieve a Psuedo Random Number Generator, which for most cases works well enough.

Then what..
The restrictions you impose actually makes the number less random, as we have to disqualify some generated numbers based on a well-known set of rules..

I'm not sure what you are thinking of when you say cascading numbers..
A PRNG generally uses a state-machine where the generated number depends on the previous state, and the generation of the number alters the state so that the next generated number is generated under different states than the current one. More advanced PRNG will also use external sources to continually update the states to make it harder to predict the state of the PRNG. As such, all generated numbers do depend to some extent (depending on the PRNG) on the previously generated values - or in some extend cascading...

Regarding the rand command:
This is pretty much an interface to the stdlib.c random() function. The exact implementation of this function may wary between different system and libraries, but generally it can be considered a " non-linear additive feedback random number generator". This function will generate a number between 0 and RAND_MAX, and needs to be "seeded" prior use (setting an initial state - using a static seed will always create the same sequence of numbers. Eggdrop combines the current timestamp with the process id to create a hard-to-predict seed).

Regarding the "quality" of random numbers:
True random numbers make no assumptions of "spread" for a finite number of data. For an infinite or "near-infinite" (lim x -> inf) number of data, one could expect an even distribution. Noticing a tendency to "hover" around a subset of values is not unusual for a limited subset of data, and is not a measurement of "poor" randomization. For a PRNG, the definition of quality is the likelyness that an external observer may predict the outcome of the next value - not how close to the value the prediction was.

To exclude subsets within the range of the generated output, there's two possible solutions;
First one;
Create a mapping function that will project a smaller set of PRNG data onto a larger set of outputs (that excludes the undesired subsets):
Simple example, we have PRNG data ranging 0-9, but we don't accept 5

Code: Select all

set data [rand 9] #generate a number between 0 and 8, this has the same size as the output set
#If data is 5, project it at 6, if data is 6, project it at 7, etc...
if {$data >= 5} {
  incr data
}
return $data

For a fixed number of unwanted sets, this is easily implemented. Supporting a dynamic number of sets or dynamic sets of unwanted data, the code quickly gets alot more complicated...

Second one;
Compare the generated number against the unwanted subsets. If we match one of the subsets, reject the data, and generate a new one. Repeat until we find a data that does not match.
This code is easy to implement for any number of unwanted subsets, but has the drawback that it might never find a non-matching data with a poor PRNG.

[dj-zath]

HI Nml!

I see what you are saying...

I [kinda] came up with the same results and solutions as you stated- mostly to "throw away" any result that doesn't meet the given critera of course, this puts "holes" in the playout format.. its not TOO bad as long as it doesn't exceed three times.. otherwise, the playlist format shows extreme inbalence- like playing 5 promos in a row then one tune.. or 30 tunes with no promos (because the "nulls" were simply skipped and the list continues on)

The other issue is songs in the 300 to 1800 range seem to get picked- and re-picked.. a LOT.. (can't tell you how many times I heard Fleetwood mac's "rumors" over and over and over! hehehe) example: 300, 1253,300,300,5342,543,56,764,300, etc etc etc

sure, it picks "a random seclection" but it seems to "favor" specific numbers over others hehe

I also came across another post someone made some time ago asking about the same problem- though his application was different, the results were the same "the code tends to pick the same few [random] answers over other ones".. I studied the code you posted in that post.. appears to be some "8-bit shift regester" of some sort.. I tried it out, and it created HUUUGE numbers (but they really were random!) hehehe

I guess I'll hash it out some more, down the line.. for now, I removed the last "math" line in my code to repair Rnd3 (by preventing the "divide by zero" error) and I'll leave it for now.

again, as always.. you are VERY insightful and knowledgable!

awesome way to go!

-DjZ-

[nml375]

Btw, you do know you don't need one expr command for each mathematical operation?

Taking the horrible Rand3 proc, I'd probably re-write it like this...

Code: Select all

proc    Rand3 {} {
  global Rnd3 PlayCnt VoiceCnt PromoCnt;
  set VarA [rand [expr $PlayCnt - $VoiceCnt - $PromoCnt]]
  incr VarA $VoiceCnt
  incr VarA $PromoCnt

...

I could use the rand() function provided through expr to tidy up the code even further, though this function uses a weaker PRNG to my best knowledge..

Btw, I assume you do not intend to execute the value of $PlayCnt as a separate tcl command? aka don't wrap it with []..


That said, I would probably look at a quite different set of algorithms alltogether for an implementation like this..
First off, I'd put Promos in a separate bucket alltogether, then use a time-weighted randomizer to decide whether to pick which bucket to play (promo or music). Simplified, the longer time music is played, the more likely it is the next selection comes from the promo-bucket. At that point I'd probably reset the time-weighting to return preference to music.

For the music-bucket, I'd probably build a large set of tiny sets ("playlists", roughly 10 tracks each). When one set is selected, the tracks within this set is played at a random order until they all have been played, and a new set is selected. To increase variation, you could limit this to perhaps 75% (7-8 ) of the tracks within the set before picking a new set. This will make repeated tracks almost non-existant, and also gives you control of "music matching" while you still avoid the impression of running a playlist.

[dj-zath]

Hi Nml:

As usual, your intelect amazes me!

I had to look at your example a few times.. then realized.. I better explain a few things. First off, your suggestions are very good.. but I'm not quite following along in some parts.. but I think thats because you might not understand the "problem" I have in the first place.. so let me try to explain the situation and "logic" I was following:

I'm using a program called "OtsAV" for the music playout.. its a rather nice piece of software, however, its still quite "under-developed" in its "remote controls" facility (one of the few playout managers. though, that actually has some sort of a remote control facility in the first place!)

Although OtsAV has an extensive "playlist template manager", that part of the program isn't accessable via the RAC (Remote Access Controller) facility.. at all! And, to add salt to the wound.. the RAC is a unsecured javascript web interface with frames and subsets! OUCH! (cringe)

The RAC can stop, start, play, pause (a stupid feature.. "pause") and search.. and thats about it; using java buttons and http forms! its "nasty" for a poor eggie to interface!

so, heres what *I* did (and I'm the first, to date, to have pulled this off) to make it work; after setting up an IP-based access tunnel (though the server firewall)...

The buttons were easy.. I merely "minick the return code" and send that back "blindly" to the RAC.. and put each set in a proc, attached some logic controllers (like if $DetOT and $DetPY is true/false etc etc) and bind it to a command string (like !play, for example) and boom... works great!

The search facility, however, was a different story alltogether!

Unlike the simple java buttons, The search, however, now requires returned RESULTS and they come in "index numbers" which had to be converted from song names and other querys.. no easy task.. you'd think they would allow a result to be "handed over" from one operation to another.. NOT THE CASE AT ALL! - thats why the code has 2200 lines in it.. whew! I mention the search because its involved of the next part.. which contains the number gennys...

now I have to give ya a small example of what all this is..

so, lets's say you want to hear "tainted love" by "Soft Cell"..
so you type it in the chatroom:

!search tainted love

The bot now takes that and forwards it via $arg to the command line cgi script (I had fun getting that line out of the javascript!) and then logs into the WEB page interface "on call" and pushes the command through..

Next, the bot has to wait for a reply from the interface.. it then has to parse the entire webpage(s) to extract just the names of the search results AND then to get their index numbers.. more logic determines which kind of page was returned and how it terminates.. (example.. if theres more than 10 results, then a second page is loaded.. up to 5 turns in total to be allowed) then these results are pushed back to $nick to whom requested it (you, in this case) so now the bot will open a PM to you with a list of tracks containing your search query(ies):

DJ 5000 - Search Results for "Tainted Love"

929 - Tainted love - Soft Cell
1623 - Tainted Love - Manson

at this point, you would take the index number and then type in the chatroom: !request 929 at which the bot returns via notice: "your request #929 has been successfully submitted."

the bot passed the index number via another "blind command string" back to the web interface as a "pick #929 and cue as next" while another command sends a "play" directive afterwards...

its really quite CLEVER if you ask me!

now, about the number gennys.. and those "index numbers"..

unfortunately, (as I mentioned above) the RAC doesn't have access to any of the playlist template tools or generators.. it merely sees a "raw list of index numbers".. and thats about it.. -and- these index numbers represent NOTHING more than "index numbers" to the RAC and eggie... and this is a PROBLEM! a nice challenge I had to come up with in order to sort out "what was what" and "how was where"... Now, heres where your suggestions makes sense, in fact, I have allready implemented JUST that.. about the "buckets" and what is where...

The first thing I had to do was (re)sort the "media library" (what they call the index database) so that specific groups of numbers represents specific items that are played- and what items these are.. AND that these numbers CAN (and WILL) change as the library grows (or shrinks) and that these "boundries" are non-fixed.. but must still represent a FIXED value at the time of the parsing. (is this making any sense?) or, in other words: "item 0 to 32 is a promo", "item 33 to 100 is a vocal" and items 101 to the end of the index database is a track"... I'm sure you've gotten the idea!

Now, the NEXT part..

I figured "wow since I have all these nice formatted index numbers.. why not let the bot "auto DJ"?!.. and so, after some thought, I came up with a nice way to do just that.. AUTO DJ! The bot can now seclectively generate a playlist based on a given (or fixed) playout and/or ruleset..

i.e.
1 promo (rand 1), 5 songs (rand 3), 1 ID (rand 2), 1 promo (rand 1), 4 songs (rand 3), 1 promo (rand 1), 5 songs (rand 3).. and you get the idea.. actually works VERY well in fact.

So, thats what governs the "conditions" of the rands.. they are used to generate "subsets" of playlists based on what the track is- a promo, a vocal or a song entry itself (later I may add more as I figure out ways to resort the media library into GENRES.. but that might be TOO much at this point) at least I got the promos and vocals figured out!

I hope this helps to explain some of my "twisted scripting" you have seen here.. it is a dauting task.. and I'm not a "good programmer".. I only squeeze by with what I need to do to get the job done (at least I TRY to do it right though I'm sure I don't quite succeed ) I have taken a lot of your invalable help and put it to good use in the parsers, and sockets, the callback code.. and now the number gennys just to name a few..

if you ever feel like you'd like to see the whole thing (or even give it a shot and rewrite it ) please let me know! I'll send ya a copy and even give ya access to the RAC.. I think you would be impressed (or, maybe not hehe) at what I was able to accomplish without former programming skills,training, or experience I have you, mindless, Caesar, Fez.. and a few others who have posted various stuff.. and in reading the posts, I have learned and adopted a lot of great minds in writing this thing.. believe it or not.. it hasn't crashed in a long time

I still have a few things to figure out.. like that oauth thing for twitter and a way to multi-thread.. (the search has a problem as when its "busy" with the writeout, the bot does nothing else) oh, and is there a way to expand the output buffer size? (my server can handle more than 3 lines at a time <smile> ) but first, today I need to build a new machine for the playout...

again, thanks Nml and everyone else for al the great help and suggestions- both directly and indirectly!

-DjZ-


UPDATE:

after I wrote this post, I went back and re-read your post and looked at the code example again and DING it CLICKED.. so I DO get what you're saying now!

makes LOTS of sense - I LIKE IT!

as for the inital code, yeah the code I posted is "broken" and was allready fixed; the "rand 3 in []'s for example was an error from when I took out what was inside there and forgot the []'s.. I dumped that entire line of code by this point..

about your code: what I didn't know was that you can incr a NUMBER besides 1.. I have read (by now) that you can.. which is why initally, I didn't understand what you were saying... but, NOW I get it.. (see? my code is FULL of mistakes like that; stuff being done "the long way" because I don't know or understand the commands designed for that task) -DjZ-