Liquidsoap

randomize for playlists is not so random

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None

Description

I use "playlist(directory)" for testing, and by default the mode is randomize. The starting song is "random", but I got many times the same song as first song, too many times for me to believe in randomness (the directory is my mp3 library which is fairly large...).

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
David Baelde added a comment -
Please provide evidence. I ran some quick tests here and I don't believe the shuffling function does what you say, unless Random.self_init is really bad. Note that shuffling isn't the only aspect: after that, a valid request has to be found, which may take several attempts.


# let randomize a =
    let permute i j =
      let tmp = a.(i) in
        a.(i) <- a.(j);
        a.(j) <- tmp
    in
    let l = Array.length a in
      if l>=2 then
        for i=0 to l-2 do
          permute i (i + Random.int (l-i))
        done
  ;;

# Random.self_init () ; let a = Array.init 100 (fun i -> i) in randomize a ; a ;;
- : int array =
[|9; 84; 50; 96; 23; 27; 43; 36; 70; 4; 5; 6; 74; 69; 95; 60; 37; 80; 98; 0;
  54; 83; 53; 39; 89; 81; 15; 79; 32; 55; 66; 40; 91; 99; 62; 30; 67; 48; 75;
  85; 46; 76; 33; 35; 11; 61; 26; 71; 44; 34; 2; 57; 92; 29; 93; 94; 86; 56;
  49; 78; 17; 88; 1; 41; 87; 82; 68; 58; 13; 59; 73; 42; 3; 25; 7; 8; 65; 19;
  21; 12; 38; 20; 64; 47; 10; 72; 77; 45; 24; 31; 51; 18; 90; 22; 52; 28; 97;
  14; 63; 16|]

Repeat it as many times as you want...
Show
David Baelde added a comment - Please provide evidence. I ran some quick tests here and I don't believe the shuffling function does what you say, unless Random.self_init is really bad. Note that shuffling isn't the only aspect: after that, a valid request has to be found, which may take several attempts. # let randomize a =     let permute i j =       let tmp = a.(i) in         a.(i) <- a.(j);         a.(j) <- tmp     in     let l = Array.length a in       if l>=2 then         for i=0 to l-2 do           permute i (i + Random.int (l-i))         done   ;; # Random.self_init () ; let a = Array.init 100 (fun i -> i) in randomize a ; a ;; - : int array = [|9; 84; 50; 96; 23; 27; 43; 36; 70; 4; 5; 6; 74; 69; 95; 60; 37; 80; 98; 0;   54; 83; 53; 39; 89; 81; 15; 79; 32; 55; 66; 40; 91; 99; 62; 30; 67; 48; 75;   85; 46; 76; 33; 35; 11; 61; 26; 71; 44; 34; 2; 57; 92; 29; 93; 94; 86; 56;   49; 78; 17; 88; 1; 41; 87; 82; 68; 58; 13; 59; 73; 42; 3; 25; 7; 8; 65; 19;   21; 12; 38; 20; 64; 47; 10; 72; 77; 45; 24; 31; 51; 18; 90; 22; 52; 28; 97;   14; 63; 16|] Repeat it as many times as you want...
Hide
Permalink
Samuel Mimram added a comment -
Ok, so here you go. My test script pl2.liq is

#!../../liquidsoap/src/liquidsoap
%include "../../liquidsoap/scripts/pervasives.liq"
set("log.level",4)
set("log.file",false)
set("log.stdout",true)
source = playlist(id="src","~/music")
output.oss(fallible=true,source)

Notice that I have a fairly large number of files in this dir:

2011/06/29 15:39:14 [src:3] Playlist is a directory.
2011/06/29 15:39:14 [src:3] Successfully loaded a playlist of 3182 tracks.

I ran the target test2 in the following Makefile:

test2:
while true; do $(MAKE) prep2; done

prep2:
./pl2.liq | grep Prepared >> pl2 &
sleep 3
killall pl2.liq

I get the following result:

$ cat pl2 | wc -l
298
$ cat pl2 | grep Shirley | wc -l
36

(I checked that I have only one file with "Shirley" in the title).

So, 36 / 298 = 12% is much much higher than random which should be around 1 / 3182 = 0.03%. I think that this confirms that I often get the same song, no?

An interesting thing, this Shirley song is the first I get if I run the playlist in "normal" mode...
Show
Samuel Mimram added a comment - Ok, so here you go. My test script pl2.liq is #!../../liquidsoap/src/liquidsoap %include "../../liquidsoap/scripts/pervasives.liq" set("log.level",4) set("log.file",false) set("log.stdout",true) source = playlist(id="src","~/music") output.oss(fallible=true,source) Notice that I have a fairly large number of files in this dir: 2011/06/29 15:39:14 [src:3] Playlist is a directory. 2011/06/29 15:39:14 [src:3] Successfully loaded a playlist of 3182 tracks. I ran the target test2 in the following Makefile: test2: while true; do $(MAKE) prep2; done prep2: ./pl2.liq | grep Prepared >> pl2 & sleep 3 killall pl2.liq I get the following result: $ cat pl2 | wc -l 298 $ cat pl2 | grep Shirley | wc -l 36 (I checked that I have only one file with "Shirley" in the title). So, 36 / 298 = 12% is much much higher than random which should be around 1 / 3182 = 0.03%. I think that this confirms that I often get the same song, no? An interesting thing, this Shirley song is the first I get if I run the playlist in "normal" mode...
Hide
Permalink
David Baelde added a comment -
Your experiment is quite convincing. I simplified it it a little bit (only works with a directory containing files with different track numbers, such as a cleanly ripped album):

$ cat random.liq
set("log.file",false)
source = playlist(id="src","~/media/audio/albums/The Bad Plus - Prog")
output.dummy(fallible=true,
  on_track(fun(m)-> begin
             print("#{m['tracknumber']}")
             shutdown()
           end,source))

$ cat random.pl
#!/usr/bin/perl
for (0..100) {
  $count{int(`./src/liquidsoap random.liq`)}++ ;
}
for (sort { $a <=> $b } (keys %count)) {
  print "$_ $count{$_}\n" ;
}

The liq script prints the first tracknumber and exits, the perl script runs it a hundred times and reports how many times each track number was obtained. A typical output is:
1 24
2 3
3 12
4 11
5 11
6 6
7 11
8 2
9 11
10 10

Interesting problem...
Show
David Baelde added a comment - Your experiment is quite convincing. I simplified it it a little bit (only works with a directory containing files with different track numbers, such as a cleanly ripped album): $ cat random.liq set("log.file",false) source = playlist(id="src","~/media/audio/albums/The Bad Plus - Prog") output.dummy(fallible=true,   on_track(fun(m)-> begin              print("#{m['tracknumber']}")              shutdown()            end,source)) $ cat random.pl #!/usr/bin/perl for (0..100) {   $count{int(`./src/liquidsoap random.liq`)}++ ; } for (sort { $a <=> $b } (keys %count)) {   print "$_ $count{$_}\n" ; } The liq script prints the first tracknumber and exits, the perl script runs it a hundred times and reports how many times each track number was obtained. A typical output is: 1 24 2 3 3 12 4 11 5 11 6 6 7 11 8 2 9 11 10 10 Interesting problem...
Hide
Permalink
David Baelde added a comment -
Fixed in [83addd3ddbae]: a race condition when reloading playlist made it possible to play a file from a new playlist before shuffling, thus breaking randomness.
Show
David Baelde added a comment - Fixed in [83addd3ddbae]: a race condition when reloading playlist made it possible to play a file from a new playlist before shuffling, thus breaking randomness.
Hide
Permalink
David Baelde added a comment -
You may check that the issue seems fixed for you, and close the issue. For my part, I get much more reasonable results now.

$ ./random.pl
1 13
2 9
3 5
4 11
5 10
6 9
7 11
8 5
9 19
10 9

Some runs have more deviation, but it doesn't seem abnormal, in any case the bias is not specific to the first item anymore.
Show
David Baelde added a comment - You may check that the issue seems fixed for you, and close the issue. For my part, I get much more reasonable results now. $ ./random.pl 1 13 2 9 3 5 4 11 5 10 6 9 7 11 8 5 9 19 10 9 Some runs have more deviation, but it doesn't seem abnormal, in any case the bias is not specific to the first item anymore.

People

  • Assignee:
    David Baelde
    Reporter:
    Samuel Mimram
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: