References
A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications,
accompanying the NIST Statistical Test Suite.
https://csrc.nist.gov/Projects/Random-Bit-Generation/Documentation-and-Software
See also http://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/cusumtes.htm (more recent)
which has a slightly different, and seemingly more consistent, equation for the cumulative sum p-value. However, the values given in the examples actually use this equation ( = the published equation - 1 ):
Random Number Generators: An Evaluation and Comparison of Random.org and Some Commonly Used Generators.
https://www.random.org/analysis/
(Analysis2005.pdf by Charmaine Kenny, which corrects errors in the NIST publication.)
https://en.wikipedia.org/wiki/Error_function
where p = 0.3275911, a1 = 0.254829592, a2 = -0.284496736, a3 = 1.421413741, a4 = -1.453152027, a5 = 1.061405429.
The complementary error function erfc(z) = 1 - erf(z).
The standard normal (cumulative distribution) function:
The NIST Statistical Test Suite
You may save binary number output of the random number generator to a plain text file to use as input for the NIST Statistical Test Suite.
The suite is very easy to install and run on Linux, but the documentation in section 5 of the NIST document referenced above needs some clarification. The current download file is sts-2_1_2.zip. Compile it with gcc as follows, which should already be installed on Fedora-based systems, and is easily installed via the Ubuntu Software Center. First, edit the makefile text file per the documentation, with the path to gcc, and the folder (sts-2.1.2) that contains makefile. Then run the following commands:
cd /<path to containing folder>/sts-2.1.2/
make
The executable file assess should now be present in the folder.
The command to run the program is
./assess <bitstream length>
such as ./assess 1000000 (note the dot and slash!). Run at least 10 bitstreams to get complete results (you will be prompted for the number). The number of binary digits in the input text file must be at least the length of the bitstream × the number of bitstreams; for example, ./assess 3000000 × 10 bitstreams = 30,000,000 total digits. The Universal and ApproximateEntropy tests seem to need 1,000,000 bits per stream to work; RandomExcursions also needs sufficient bits and streams to generate full output. Per the documentation, the uniformity p-value is undefined if fewer than 10 sequences are processed, and at least 55 sequences must be processed to derive statistically meaningful results for the uniformity of the individual p-values. The overall test results are saved to sts-2.1.2/experiments/AlgorithmTesting/finalAnalysisReport.txt; be sure to rename it to save it. P-values for the individual tests are saved to sts-2.1.2/experiments/AlgorithmTesting/<test name>/results.txt; more detailed information is found in stats.txt.
I ran the suite using the default parameters on 100,000,000 digits, generated with the script above at ×1000 speed. 100,000,000 digits crashed Firefox, so I generated 20,000,000 at a time and appended them to a text file (101.2 MB). I ran them first as 10 streams of 10,000,000, then as 100 streams of 1,000,000. All the tests met the minimum p-values and pass rates (if they do not, they are marked with an asterisk).
------------------------------------------------------------------------------ RESULTS FOR THE UNIFORMITY OF P-VALUES AND THE PROPORTION OF PASSING SEQUENCES ------------------------------------------------------------------------------ generator is <100Million.txt> ------------------------------------------------------------------------------ C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 P-VALUE PROPORTION STATISTICAL TEST ------------------------------------------------------------------------------ 1 0 0 0 0 0 2 2 4 1 0.066882 10/10 Frequency 0 1 2 0 3 1 1 1 1 0 0.534146 10/10 BlockFrequency 1 0 1 2 0 1 1 1 1 2 0.911413 9/10 CumulativeSums 1 0 1 0 2 2 0 1 1 2 0.739918 10/10 CumulativeSums 1 0 0 0 0 2 2 1 3 1 0.350485 10/10 Runs 0 1 1 1 0 4 0 1 1 1 0.213309 10/10 LongestRun 0 2 2 0 1 2 0 0 2 1 0.534146 10/10 Rank 1 0 1 0 2 1 0 3 1 1 0.534146 10/10 FFT 1 2 1 0 0 2 2 0 0 2 0.534146 10/10 NonOverlappingTemplate 1 2 2 0 1 0 1 0 1 2 0.739918 10/10 NonOverlappingTemplate 1 1 1 0 3 1 0 0 2 1 0.534146 10/10 NonOverlappingTemplate 2 1 1 0 1 1 0 1 1 2 0.911413 9/10 NonOverlappingTemplate 1 0 2 2 2 0 0 0 2 1 0.534146 9/10 NonOverlappingTemplate 2 1 1 0 1 2 0 0 1 2 0.739918 10/10 NonOverlappingTemplate 1 1 1 1 2 0 2 0 1 1 0.911413 10/10 NonOverlappingTemplate 1 0 1 0 0 3 1 4 0 0 0.035174 10/10 NonOverlappingTemplate 1 3 0 1 0 0 1 0 2 2 0.350485 10/10 NonOverlappingTemplate 0 1 1 0 1 1 3 0 3 0 0.213309 10/10 NonOverlappingTemplate 0 2 1 1 0 1 0 1 3 1 0.534146 10/10 NonOverlappingTemplate 0 0 2 1 3 0 0 1 2 1 0.350485 10/10 NonOverlappingTemplate 2 2 0 0 1 0 2 2 0 1 0.534146 10/10 NonOverlappingTemplate 1 0 0 2 1 3 1 0 1 1 0.534146 10/10 NonOverlappingTemplate 1 0 0 0 2 2 2 1 2 0 0.534146 10/10 NonOverlappingTemplate 0 0 2 1 0 1 1 1 3 1 0.534146 10/10 NonOverlappingTemplate 0 1 1 2 3 1 1 0 1 0 0.534146 10/10 NonOverlappingTemplate 1 4 1 1 1 1 0 0 1 0 0.213309 10/10 NonOverlappingTemplate 1 0 1 1 0 3 2 1 1 0 0.534146 10/10 NonOverlappingTemplate 1 2 1 1 0 1 2 1 0 1 0.911413 10/10 NonOverlappingTemplate 1 1 2 0 3 1 0 1 0 1 0.534146 10/10 NonOverlappingTemplate 1 1 2 0 2 2 0 0 1 1 0.739918 10/10 NonOverlappingTemplate 1 1 1 1 2 0 1 2 1 0 0.911413 9/10 NonOverlappingTemplate 2 1 0 0 1 1 1 1 1 2 0.911413 10/10 NonOverlappingTemplate 0 0 0 0 3 1 1 1 2 2 0.350485 10/10 NonOverlappingTemplate 1 0 0 2 1 1 3 1 1 0 0.534146 10/10 NonOverlappingTemplate 2 0 2 0 2 0 2 0 1 1 0.534146 10/10 NonOverlappingTemplate 1 2 0 3 1 1 1 0 0 1 0.534146 10/10 NonOverlappingTemplate 2 0 0 2 1 0 1 1 1 2 0.739918 10/10 NonOverlappingTemplate 0 0 2 1 0 0 2 2 0 3 0.213309 10/10 NonOverlappingTemplate 0 0 2 2 3 1 0 1 1 0 0.350485 10/10 NonOverlappingTemplate 0 2 0 0 2 1 1 1 2 1 0.739918 10/10 NonOverlappingTemplate 1 1 0 0 0 0 2 1 2 3 0.350485 10/10 NonOverlappingTemplate 0 3 0 2 2 1 0 1 1 0 0.350485 10/10 NonOverlappingTemplate 1 1 0 3 0 1 1 1 0 2 0.534146 10/10 NonOverlappingTemplate 2 1 2 1 2 2 0 0 0 0 0.534146 10/10 NonOverlappingTemplate 1 4 0 0 1 1 1 0 1 1 0.213309 10/10 NonOverlappingTemplate 1 0 1 0 5 0 0 1 1 1 0.017912 10/10 NonOverlappingTemplate 3 0 0 1 2 1 1 0 0 2 0.350485 10/10 NonOverlappingTemplate 3 1 0 0 3 1 0 1 1 0 0.213309 9/10 NonOverlappingTemplate 2 1 3 0 1 0 0 1 1 1 0.534146 10/10 NonOverlappingTemplate 3 0 1 2 1 0 1 0 1 1 0.534146 10/10 NonOverlappingTemplate 1 1 0 0 0 2 2 2 1 1 0.739918 10/10 NonOverlappingTemplate 0 0 0 2 2 1 2 1 0 2 0.534146 10/10 NonOverlappingTemplate 0 0 0 0 0 2 2 1 2 3 0.213309 10/10 NonOverlappingTemplate 1 0 2 0 0 1 3 0 1 2 0.350485 10/10 NonOverlappingTemplate 0 1 0 1 1 0 1 2 2 2 0.739918 10/10 NonOverlappingTemplate 1 0 0 0 1 3 3 0 2 0 0.122325 10/10 NonOverlappingTemplate 0 0 1 1 2 2 0 3 1 0 0.350485 10/10 NonOverlappingTemplate 1 4 1 0 2 0 1 0 1 0 0.122325 10/10 NonOverlappingTemplate 0 0 0 1 1 1 2 1 1 3 0.534146 10/10 NonOverlappingTemplate 0 0 0 0 2 2 0 3 2 1 0.213309 10/10 NonOverlappingTemplate 0 1 2 1 0 2 3 0 0 1 0.350485 10/10 NonOverlappingTemplate 1 0 3 1 0 1 0 1 3 0 0.213309 10/10 NonOverlappingTemplate 1 1 3 2 0 1 0 1 0 1 0.534146 10/10 NonOverlappingTemplate 1 1 0 2 0 1 0 1 2 2 0.739918 10/10 NonOverlappingTemplate 0 1 3 1 1 1 1 0 1 1 0.739918 10/10 NonOverlappingTemplate 1 1 1 1 0 1 1 2 1 1 0.991468 10/10 NonOverlappingTemplate 1 0 1 1 1 3 0 0 2 1 0.534146 10/10 NonOverlappingTemplate 2 1 2 1 0 2 1 0 0 1 0.739918 10/10 NonOverlappingTemplate 2 1 3 1 1 0 0 1 0 1 0.534146 10/10 NonOverlappingTemplate 1 2 0 1 1 0 0 2 1 2 0.739918 9/10 NonOverlappingTemplate 1 4 0 0 0 0 3 1 0 1 0.035174 10/10 NonOverlappingTemplate 2 0 2 1 1 1 1 1 1 0 0.911413 10/10 NonOverlappingTemplate 0 0 1 0 1 0 2 3 2 1 0.350485 10/10 NonOverlappingTemplate 0 2 1 1 1 2 2 0 0 1 0.739918 10/10 NonOverlappingTemplate 1 1 1 0 2 2 0 1 2 0 0.739918 10/10 NonOverlappingTemplate 1 2 0 2 2 0 1 1 1 0 0.739918 9/10 NonOverlappingTemplate 2 0 1 0 0 1 1 1 1 3 0.534146 10/10 NonOverlappingTemplate 1 1 1 1 1 1 0 0 2 2 0.911413 10/10 NonOverlappingTemplate 2 1 1 0 1 1 0 2 0 2 0.739918 9/10 NonOverlappingTemplate 0 1 1 0 0 0 3 2 3 0 0.122325 10/10 NonOverlappingTemplate 0 1 0 0 1 2 1 1 1 3 0.534146 10/10 NonOverlappingTemplate 1 0 0 1 1 2 2 1 1 1 0.911413 10/10 NonOverlappingTemplate 1 2 1 0 0 2 2 0 0 2 0.534146 10/10 NonOverlappingTemplate 1 1 0 0 0 2 2 2 1 1 0.739918 10/10 NonOverlappingTemplate 1 2 0 1 1 0 3 0 1 1 0.534146 10/10 NonOverlappingTemplate 2 2 1 1 0 1 2 0 1 0 0.739918 10/10 NonOverlappingTemplate 0 1 0 0 1 3 0 2 3 0 0.122325 10/10 NonOverlappingTemplate 1 0 1 3 1 1 0 1 1 1 0.739918 10/10 NonOverlappingTemplate 1 1 0 1 0 1 0 1 3 2 0.534146 10/10 NonOverlappingTemplate 0 0 0 3 2 1 0 0 1 3 0.122325 10/10 NonOverlappingTemplate 1 5 1 0 2 0 1 0 0 0 0.008879 10/10 NonOverlappingTemplate 3 2 0 0 0 0 0 2 3 0 0.066882 10/10 NonOverlappingTemplate 2 2 1 0 1 2 0 0 0 2 0.534146 10/10 NonOverlappingTemplate 4 0 3 0 1 1 0 0 0 1 0.035174 9/10 NonOverlappingTemplate 0 2 1 0 2 0 2 0 2 1 0.534146 10/10 NonOverlappingTemplate 2 1 1 1 1 0 3 0 1 0 0.534146 10/10 NonOverlappingTemplate 2 0 2 0 2 2 1 1 0 0 0.534146 10/10 NonOverlappingTemplate 2 1 2 1 1 0 2 0 1 0 0.739918 10/10 NonOverlappingTemplate 1 2 1 1 1 3 0 0 1 0 0.534146 10/10 NonOverlappingTemplate 0 3 1 2 1 3 0 0 0 0 0.122325 10/10 NonOverlappingTemplate 1 2 1 1 0 1 0 0 2 2 0.739918 10/10 NonOverlappingTemplate 2 0 2 0 1 1 2 0 1 1 0.739918 10/10 NonOverlappingTemplate 3 2 0 1 0 1 0 1 1 1 0.534146 10/10 NonOverlappingTemplate 0 1 1 3 0 1 1 1 1 1 0.739918 10/10 NonOverlappingTemplate 0 1 2 1 3 1 0 1 1 0 0.534146 10/10 NonOverlappingTemplate 3 1 0 1 0 0 0 1 1 3 0.213309 9/10 NonOverlappingTemplate 0 2 1 1 3 1 0 1 0 1 0.534146 10/10 NonOverlappingTemplate 1 0 0 2 0 1 0 1 3 2 0.350485 10/10 NonOverlappingTemplate 2 1 3 1 1 1 0 1 0 0 0.534146 10/10 NonOverlappingTemplate 0 0 0 2 3 1 1 1 0 2 0.350485 10/10 NonOverlappingTemplate 1 2 1 0 0 3 1 1 0 1 0.534146 10/10 NonOverlappingTemplate 2 1 1 2 2 0 1 0 0 1 0.739918 10/10 NonOverlappingTemplate 0 1 2 2 0 2 1 1 1 0 0.739918 10/10 NonOverlappingTemplate 1 0 2 0 1 2 0 1 3 0 0.350485 10/10 NonOverlappingTemplate 0 0 1 1 2 2 0 2 1 1 0.739918 10/10 NonOverlappingTemplate 2 1 2 1 1 0 0 0 2 1 0.739918 10/10 NonOverlappingTemplate 0 2 0 1 2 0 2 0 3 0 0.213309 10/10 NonOverlappingTemplate 0 0 0 2 3 0 0 2 1 2 0.213309 10/10 NonOverlappingTemplate 1 3 1 0 2 2 0 1 0 0 0.350485 10/10 NonOverlappingTemplate 0 0 1 1 3 1 1 1 2 0 0.534146 10/10 NonOverlappingTemplate 1 0 0 5 0 0 3 1 0 0 0.002043 10/10 NonOverlappingTemplate 2 0 1 0 0 1 1 1 1 3 0.534146 10/10 NonOverlappingTemplate 1 1 0 2 0 1 1 1 2 1 0.911413 10/10 NonOverlappingTemplate 0 2 2 0 0 0 2 0 0 4 0.035174 10/10 NonOverlappingTemplate 1 1 2 1 1 0 2 1 0 1 0.911413 10/10 NonOverlappingTemplate 0 0 3 1 0 1 2 1 1 1 0.534146 10/10 NonOverlappingTemplate 3 1 2 1 1 0 1 0 1 0 0.534146 10/10 NonOverlappingTemplate 0 1 0 1 3 0 1 1 2 1 0.534146 10/10 NonOverlappingTemplate 1 4 0 2 0 1 0 0 1 1 0.122325 10/10 NonOverlappingTemplate 1 0 0 1 1 0 1 3 1 2 0.534146 10/10 NonOverlappingTemplate 0 0 2 0 2 0 2 1 2 1 0.534146 10/10 NonOverlappingTemplate 0 2 1 0 1 1 2 0 1 2 0.739918 10/10 NonOverlappingTemplate 0 2 3 0 0 0 1 0 2 2 0.213309 10/10 NonOverlappingTemplate 2 3 0 1 2 0 1 0 0 1 0.350485 10/10 NonOverlappingTemplate 1 1 2 0 3 0 0 2 1 0 0.350485 10/10 NonOverlappingTemplate 1 1 1 1 0 0 1 2 0 3 0.534146 10/10 NonOverlappingTemplate 0 2 1 1 2 2 1 0 0 1 0.739918 10/10 NonOverlappingTemplate 1 0 2 0 1 1 1 2 2 0 0.739918 10/10 NonOverlappingTemplate 2 0 0 2 1 0 0 1 1 3 0.350485 10/10 NonOverlappingTemplate 2 0 3 0 0 2 2 0 0 1 0.213309 10/10 NonOverlappingTemplate 1 2 0 0 2 0 3 1 1 0 0.350485 10/10 NonOverlappingTemplate 1 1 3 1 2 0 0 0 1 1 0.534146 10/10 NonOverlappingTemplate 1 0 1 1 2 0 2 1 1 1 0.911413 10/10 NonOverlappingTemplate 1 1 1 2 2 0 1 1 0 1 0.911413 10/10 NonOverlappingTemplate 3 1 0 2 1 0 1 0 2 0 0.350485 10/10 NonOverlappingTemplate 1 1 1 2 1 0 1 1 1 1 0.991468 10/10 NonOverlappingTemplate 2 1 2 1 0 0 2 2 0 0 0.534146 10/10 NonOverlappingTemplate 1 2 1 3 1 1 1 0 0 0 0.534146 10/10 NonOverlappingTemplate 0 3 2 1 0 1 0 1 0 2 0.350485 10/10 NonOverlappingTemplate 0 2 1 2 2 0 2 0 1 0 0.534146 10/10 NonOverlappingTemplate 0 0 1 0 1 2 1 3 0 2 0.350485 10/10 NonOverlappingTemplate 0 1 1 3 0 2 1 1 1 0 0.534146 10/10 NonOverlappingTemplate 1 0 2 1 0 0 3 1 1 1 0.534146 10/10 NonOverlappingTemplate 0 0 2 1 2 0 1 1 1 2 0.739918 10/10 NonOverlappingTemplate 1 1 1 0 0 3 2 1 1 0 0.534146 10/10 NonOverlappingTemplate 1 0 0 1 1 2 2 1 1 1 0.911413 10/10 NonOverlappingTemplate 3 1 2 0 1 0 1 1 0 1 0.534146 10/10 OverlappingTemplate 3 0 0 2 0 1 1 1 1 1 0.534146 10/10 Universal 0 1 0 2 1 1 1 1 1 2 0.911413 10/10 ApproximateEntropy 1 1 2 0 0 1 1 1 1 0 ---- 8/8 RandomExcursions 0 2 1 1 0 1 0 1 2 0 ---- 8/8 RandomExcursions 2 0 1 0 1 2 1 1 0 0 ---- 7/8 RandomExcursions 0 1 0 3 0 1 1 1 0 1 ---- 8/8 RandomExcursions 1 1 1 2 1 0 2 0 0 0 ---- 8/8 RandomExcursions 0 3 1 0 0 0 2 1 0 1 ---- 8/8 RandomExcursions 1 1 0 0 1 1 1 0 2 1 ---- 8/8 RandomExcursions 1 1 0 1 1 0 1 0 1 2 ---- 8/8 RandomExcursions 2 1 0 0 0 1 2 1 0 1 ---- 8/8 RandomExcursionsVariant 3 0 0 0 0 2 1 0 1 1 ---- 8/8 RandomExcursionsVariant 2 1 0 0 0 1 1 1 0 2 ---- 7/8 RandomExcursionsVariant 2 0 1 0 0 2 1 1 1 0 ---- 7/8 RandomExcursionsVariant 1 2 1 0 1 1 0 0 1 1 ---- 7/8 RandomExcursionsVariant 1 2 2 1 0 1 0 0 0 1 ---- 8/8 RandomExcursionsVariant 2 1 0 2 0 1 0 1 1 0 ---- 8/8 RandomExcursionsVariant 0 1 0 1 0 3 1 0 0 2 ---- 8/8 RandomExcursionsVariant 0 0 2 1 0 1 1 0 2 1 ---- 8/8 RandomExcursionsVariant 0 2 0 1 0 3 0 1 1 0 ---- 8/8 RandomExcursionsVariant 1 0 0 3 0 1 1 1 0 1 ---- 8/8 RandomExcursionsVariant 1 2 0 0 0 2 1 1 1 0 ---- 8/8 RandomExcursionsVariant 1 2 0 1 0 1 1 0 1 1 ---- 7/8 RandomExcursionsVariant 1 3 1 0 0 0 1 0 0 2 ---- 7/8 RandomExcursionsVariant 2 2 1 0 0 1 0 0 1 1 ---- 7/8 RandomExcursionsVariant 2 1 2 0 0 1 0 0 0 2 ---- 8/8 RandomExcursionsVariant 3 0 0 2 1 0 0 1 0 1 ---- 8/8 RandomExcursionsVariant 3 0 1 2 0 1 0 0 0 1 ---- 8/8 RandomExcursionsVariant 0 0 4 1 1 1 2 1 0 0 0.122325 10/10 Serial 0 1 2 1 2 2 1 1 0 0 0.739918 10/10 Serial 0 0 2 2 1 1 0 1 1 2 0.739918 10/10 LinearComplexity - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The minimum pass rate for each statistical test with the exception of the random excursion (variant) test is approximately = 8 for a sample size = 10 binary sequences. The minimum pass rate for the random excursion (variant) test is approximately = 7 for a sample size = 8 binary sequences. For further guidelines construct a probability table using the MAPLE program provided in the addendum section of the documentation. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
------------------------------------------------------------------------------ RESULTS FOR THE UNIFORMITY OF P-VALUES AND THE PROPORTION OF PASSING SEQUENCES ------------------------------------------------------------------------------ generator is <100Million.txt> ------------------------------------------------------------------------------ C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 P-VALUE PROPORTION STATISTICAL TEST ------------------------------------------------------------------------------ 8 13 11 15 4 4 12 8 15 10 0.108791 99/100 Frequency 8 13 9 12 10 12 8 9 8 11 0.955835 99/100 BlockFrequency 9 11 15 6 13 7 10 8 11 10 0.678686 100/100 CumulativeSums 10 8 14 10 8 8 12 11 4 15 0.401199 100/100 CumulativeSums 17 7 8 10 9 11 12 7 11 8 0.514124 99/100 Runs 11 13 3 9 13 9 7 14 10 11 0.383827 100/100 LongestRun 9 7 10 9 10 8 9 12 16 10 0.779188 99/100 Rank 9 14 8 14 10 7 12 10 10 6 0.678686 100/100 FFT 12 5 7 13 13 6 12 9 9 14 0.401199 99/100 NonOverlappingTemplate 7 14 15 4 13 10 5 7 12 13 0.115387 98/100 NonOverlappingTemplate 10 12 9 10 11 11 8 12 10 7 0.983453 99/100 NonOverlappingTemplate 13 16 2 11 9 9 7 12 9 12 0.162606 99/100 NonOverlappingTemplate 12 12 7 10 5 10 12 11 7 14 0.616305 99/100 NonOverlappingTemplate 7 10 10 7 12 15 12 6 13 8 0.534146 99/100 NonOverlappingTemplate 14 10 6 7 8 9 8 15 10 13 0.494392 100/100 NonOverlappingTemplate 10 7 13 8 6 9 15 12 7 13 0.474986 100/100 NonOverlappingTemplate 7 11 9 15 14 11 7 9 10 7 0.616305 100/100 NonOverlappingTemplate 7 9 12 13 10 10 16 7 9 7 0.554420 100/100 NonOverlappingTemplate 9 7 6 11 10 6 13 8 14 16 0.289667 99/100 NonOverlappingTemplate 7 4 9 11 11 18 9 7 14 10 0.129620 100/100 NonOverlappingTemplate 12 4 15 10 7 7 8 10 11 16 0.191687 99/100 NonOverlappingTemplate 9 7 9 15 6 13 8 8 15 10 0.401199 100/100 NonOverlappingTemplate 14 9 8 8 12 6 5 11 12 15 0.350485 98/100 NonOverlappingTemplate 10 13 13 12 7 10 9 6 12 8 0.779188 99/100 NonOverlappingTemplate 10 11 8 10 10 12 11 6 9 13 0.935716 100/100 NonOverlappingTemplate 11 9 7 11 16 10 11 10 6 9 0.678686 100/100 NonOverlappingTemplate 9 12 5 13 12 8 10 5 17 9 0.202268 99/100 NonOverlappingTemplate 16 11 9 11 11 9 10 3 9 11 0.419021 99/100 NonOverlappingTemplate 9 7 14 7 12 10 10 16 5 10 0.350485 98/100 NonOverlappingTemplate 9 12 17 9 12 9 7 9 9 7 0.534146 98/100 NonOverlappingTemplate 13 9 13 9 11 6 9 10 10 10 0.924076 98/100 NonOverlappingTemplate 9 11 11 8 10 6 15 11 7 12 0.719747 98/100 NonOverlappingTemplate 6 11 9 11 10 10 14 9 8 12 0.883171 99/100 NonOverlappingTemplate 7 17 7 11 7 13 11 9 11 7 0.366918 100/100 NonOverlappingTemplate 13 5 7 9 10 13 10 12 8 13 0.637119 99/100 NonOverlappingTemplate 10 8 11 11 11 9 5 11 9 15 0.739918 100/100 NonOverlappingTemplate 11 10 10 9 8 12 10 8 10 12 0.994250 98/100 NonOverlappingTemplate 10 10 12 13 9 13 9 7 10 7 0.897763 99/100 NonOverlappingTemplate 9 14 7 6 6 12 13 10 6 17 0.137282 100/100 NonOverlappingTemplate 10 11 11 13 5 13 11 6 12 8 0.637119 99/100 NonOverlappingTemplate 9 15 6 12 9 8 8 9 14 10 0.616305 100/100 NonOverlappingTemplate 8 8 10 8 5 19 13 15 4 10 0.026948 99/100 NonOverlappingTemplate 8 10 12 10 14 11 8 10 10 7 0.924076 98/100 NonOverlappingTemplate 5 4 15 14 9 6 8 14 17 8 0.023545 99/100 NonOverlappingTemplate 13 7 11 6 8 10 14 10 9 12 0.739918 98/100 NonOverlappingTemplate 7 9 12 8 9 15 12 9 8 11 0.798139 100/100 NonOverlappingTemplate 9 8 6 13 13 13 13 3 12 10 0.275709 99/100 NonOverlappingTemplate 13 15 5 8 12 9 13 10 5 10 0.334538 98/100 NonOverlappingTemplate 10 9 11 17 16 5 4 8 13 7 0.048716 97/100 NonOverlappingTemplate 17 12 17 5 7 10 11 5 7 9 0.045675 98/100 NonOverlappingTemplate 3 10 9 12 15 15 12 9 6 9 0.181557 100/100 NonOverlappingTemplate 9 12 6 12 9 11 7 11 11 12 0.897763 100/100 NonOverlappingTemplate 8 8 17 9 7 10 17 14 8 2 0.017912 97/100 NonOverlappingTemplate 15 7 7 12 12 9 8 7 8 15 0.401199 97/100 NonOverlappingTemplate 11 4 8 6 10 15 10 13 12 11 0.383827 97/100 NonOverlappingTemplate 9 14 9 9 9 5 7 13 13 12 0.574903 100/100 NonOverlappingTemplate 7 8 8 10 5 12 14 11 12 13 0.574903 98/100 NonOverlappingTemplate 10 11 18 4 6 10 9 10 13 9 0.171867 99/100 NonOverlappingTemplate 5 6 13 8 17 9 8 12 8 14 0.153763 100/100 NonOverlappingTemplate 5 16 11 9 12 8 6 14 11 8 0.289667 99/100 NonOverlappingTemplate 12 11 6 9 10 15 9 6 11 11 0.678686 99/100 NonOverlappingTemplate 6 9 10 12 9 7 13 16 11 7 0.474986 99/100 NonOverlappingTemplate 5 12 11 12 10 10 12 11 6 11 0.779188 99/100 NonOverlappingTemplate 12 7 10 9 11 12 10 9 10 10 0.991468 99/100 NonOverlappingTemplate 8 8 10 15 10 5 11 11 14 8 0.534146 99/100 NonOverlappingTemplate 9 13 6 8 10 12 9 14 10 9 0.816537 100/100 NonOverlappingTemplate 14 10 7 4 9 9 11 10 10 16 0.350485 99/100 NonOverlappingTemplate 14 15 7 13 5 13 9 9 5 10 0.213309 98/100 NonOverlappingTemplate 15 9 6 13 10 5 13 11 3 15 0.066882 99/100 NonOverlappingTemplate 13 14 12 11 12 7 7 7 11 6 0.554420 98/100 NonOverlappingTemplate 9 6 10 11 9 11 10 8 12 14 0.883171 100/100 NonOverlappingTemplate 14 6 11 10 10 11 3 11 6 18 0.058984 99/100 NonOverlappingTemplate 7 8 5 17 6 11 9 16 14 7 0.055361 99/100 NonOverlappingTemplate 7 12 13 13 9 6 13 5 13 9 0.419021 100/100 NonOverlappingTemplate 12 11 11 8 9 13 8 8 7 13 0.867692 100/100 NonOverlappingTemplate 8 7 10 19 8 8 8 13 7 12 0.171867 100/100 NonOverlappingTemplate 8 10 11 8 13 11 10 13 8 8 0.935716 100/100 NonOverlappingTemplate 13 9 7 14 7 8 11 8 8 15 0.514124 100/100 NonOverlappingTemplate 19 7 11 8 10 5 7 18 5 10 0.009535 97/100 NonOverlappingTemplate 12 12 5 10 11 7 14 9 12 8 0.657933 99/100 NonOverlappingTemplate 9 7 8 6 12 8 8 14 10 18 0.202268 100/100 NonOverlappingTemplate 11 10 16 6 11 10 8 10 10 8 0.719747 99/100 NonOverlappingTemplate 12 5 7 13 13 6 12 9 9 14 0.401199 99/100 NonOverlappingTemplate 11 15 6 12 5 10 11 11 9 10 0.595549 100/100 NonOverlappingTemplate 12 5 8 12 13 11 12 9 8 10 0.779188 98/100 NonOverlappingTemplate 14 14 12 13 13 9 6 6 6 7 0.262249 97/100 NonOverlappingTemplate 16 9 11 8 7 9 14 8 9 9 0.595549 99/100 NonOverlappingTemplate 10 10 12 8 8 10 9 12 13 8 0.964295 100/100 NonOverlappingTemplate 14 9 12 6 15 12 7 7 11 7 0.401199 99/100 NonOverlappingTemplate 3 13 10 5 7 7 17 10 17 11 0.017912 100/100 NonOverlappingTemplate 9 9 12 8 9 11 13 15 8 6 0.678686 100/100 NonOverlappingTemplate 11 9 9 9 13 15 10 14 5 5 0.319084 100/100 NonOverlappingTemplate 10 9 15 14 9 10 8 7 10 8 0.739918 99/100 NonOverlappingTemplate 15 13 8 11 11 8 8 11 5 10 0.595549 98/100 NonOverlappingTemplate 6 10 8 11 9 14 10 12 8 12 0.834308 100/100 NonOverlappingTemplate 5 9 11 8 9 11 12 17 9 9 0.455937 98/100 NonOverlappingTemplate 10 6 8 11 15 3 9 12 12 14 0.213309 98/100 NonOverlappingTemplate 11 8 7 11 9 12 10 13 10 9 0.964295 99/100 NonOverlappingTemplate 6 11 13 7 13 6 9 10 12 13 0.595549 100/100 NonOverlappingTemplate 12 10 16 9 14 10 7 7 10 5 0.350485 99/100 NonOverlappingTemplate 14 11 7 13 9 10 10 5 11 10 0.719747 98/100 NonOverlappingTemplate 5 8 11 12 12 14 11 7 14 6 0.383827 100/100 NonOverlappingTemplate 11 9 13 11 7 5 10 13 10 11 0.779188 99/100 NonOverlappingTemplate 9 9 14 8 10 11 10 12 12 5 0.779188 99/100 NonOverlappingTemplate 6 6 12 13 13 12 14 4 8 12 0.224821 100/100 NonOverlappingTemplate 10 13 9 3 7 9 11 11 15 12 0.350485 99/100 NonOverlappingTemplate 7 12 13 6 7 10 5 17 8 15 0.090936 98/100 NonOverlappingTemplate 8 13 5 6 13 11 14 10 10 10 0.534146 100/100 NonOverlappingTemplate 8 15 9 8 6 10 10 14 12 8 0.595549 98/100 NonOverlappingTemplate 8 10 5 17 9 7 8 13 13 10 0.275709 99/100 NonOverlappingTemplate 13 6 9 11 12 9 13 9 8 10 0.867692 100/100 NonOverlappingTemplate 8 12 10 12 13 7 10 6 12 10 0.834308 99/100 NonOverlappingTemplate 10 7 10 7 8 11 15 13 10 9 0.759756 99/100 NonOverlappingTemplate 6 15 11 11 11 10 10 8 10 8 0.816537 100/100 NonOverlappingTemplate 10 6 9 15 8 11 8 14 9 10 0.657933 100/100 NonOverlappingTemplate 10 7 12 8 5 8 9 14 14 13 0.455937 99/100 NonOverlappingTemplate 12 7 13 9 12 12 9 9 6 11 0.834308 96/100 NonOverlappingTemplate 8 11 9 10 13 12 15 6 9 7 0.637119 100/100 NonOverlappingTemplate 8 12 10 11 13 12 10 7 9 8 0.935716 99/100 NonOverlappingTemplate 8 11 9 9 12 15 6 13 8 9 0.678686 99/100 NonOverlappingTemplate 13 8 3 10 10 13 13 10 11 9 0.514124 100/100 NonOverlappingTemplate 9 7 9 11 9 5 14 10 10 16 0.437274 98/100 NonOverlappingTemplate 9 8 11 5 7 12 13 16 8 11 0.401199 98/100 NonOverlappingTemplate 11 12 14 7 10 4 10 7 12 13 0.455937 99/100 NonOverlappingTemplate 10 5 15 11 6 12 13 12 5 11 0.275709 98/100 NonOverlappingTemplate 9 10 7 14 3 9 15 14 11 8 0.202268 100/100 NonOverlappingTemplate 10 11 7 16 12 10 7 8 10 9 0.699313 100/100 NonOverlappingTemplate 9 13 10 11 10 11 8 13 10 5 0.834308 98/100 NonOverlappingTemplate 19 7 15 7 10 5 16 6 7 8 0.010988 99/100 NonOverlappingTemplate 11 13 12 12 8 7 8 7 8 14 0.699313 100/100 NonOverlappingTemplate 8 12 9 13 10 11 6 6 13 12 0.699313 99/100 NonOverlappingTemplate 12 7 13 9 4 10 8 15 10 12 0.419021 97/100 NonOverlappingTemplate 9 10 7 17 10 7 14 10 7 9 0.401199 100/100 NonOverlappingTemplate 7 12 15 7 14 9 9 6 12 9 0.474986 98/100 NonOverlappingTemplate 6 8 13 12 13 11 11 9 8 9 0.834308 99/100 NonOverlappingTemplate 5 12 9 7 11 12 17 10 11 6 0.275709 100/100 NonOverlappingTemplate 8 13 8 9 12 10 8 10 13 9 0.935716 100/100 NonOverlappingTemplate 11 9 14 9 8 6 12 4 11 16 0.236810 99/100 NonOverlappingTemplate 13 8 9 9 9 13 7 10 11 11 0.935716 97/100 NonOverlappingTemplate 17 7 7 10 8 15 6 7 14 9 0.129620 100/100 NonOverlappingTemplate 9 9 8 14 8 13 9 12 6 12 0.739918 99/100 NonOverlappingTemplate 18 9 12 5 15 11 8 9 6 7 0.090936 98/100 NonOverlappingTemplate 12 7 7 7 10 12 14 10 15 6 0.419021 100/100 NonOverlappingTemplate 8 11 10 7 17 5 12 9 11 10 0.401199 99/100 NonOverlappingTemplate 12 12 12 11 9 4 11 10 9 10 0.816537 98/100 NonOverlappingTemplate 8 12 5 6 18 11 9 5 9 17 0.025193 99/100 NonOverlappingTemplate 7 9 13 12 15 4 5 12 11 12 0.224821 100/100 NonOverlappingTemplate 9 13 13 13 12 7 8 9 8 8 0.798139 99/100 NonOverlappingTemplate 11 8 10 8 6 17 15 13 7 5 0.115387 100/100 NonOverlappingTemplate 10 9 8 13 10 14 4 14 8 10 0.474986 99/100 NonOverlappingTemplate 3 5 11 11 9 17 14 7 11 12 0.075719 100/100 NonOverlappingTemplate 6 12 6 15 10 11 7 6 11 16 0.191687 100/100 NonOverlappingTemplate 8 8 7 9 14 10 7 15 12 10 0.616305 100/100 NonOverlappingTemplate 13 6 6 9 15 14 7 11 11 8 0.366918 100/100 NonOverlappingTemplate 12 11 12 8 12 6 11 9 11 8 0.911413 99/100 NonOverlappingTemplate 11 10 16 6 11 10 8 10 10 8 0.719747 99/100 NonOverlappingTemplate 12 7 9 10 15 11 8 6 10 12 0.699313 99/100 OverlappingTemplate 8 12 14 8 12 7 12 11 9 7 0.779188 100/100 Universal 10 7 8 10 7 12 13 12 12 9 0.883171 96/100 ApproximateEntropy 4 6 6 7 6 7 6 7 8 7 0.991468 63/64 RandomExcursions 8 12 3 6 4 9 2 8 6 6 0.134686 63/64 RandomExcursions 6 3 9 9 6 7 4 5 9 6 0.637119 63/64 RandomExcursions 5 5 8 2 9 4 7 4 8 12 0.148094 64/64 RandomExcursions 4 6 7 4 7 5 9 7 9 6 0.834308 64/64 RandomExcursions 3 11 5 9 6 5 8 6 6 5 0.500934 64/64 RandomExcursions 4 5 6 8 3 5 5 6 10 12 0.213309 63/64 RandomExcursions 8 5 7 3 9 6 5 7 5 9 0.739918 62/64 RandomExcursions 4 8 8 10 4 9 9 6 3 3 0.253551 64/64 RandomExcursionsVariant 4 5 9 8 7 8 8 5 7 3 0.706149 64/64 RandomExcursionsVariant 6 2 9 8 7 3 9 9 8 3 0.232760 63/64 RandomExcursionsVariant 4 5 4 8 8 6 9 5 10 5 0.602458 62/64 RandomExcursionsVariant 5 4 5 3 7 5 10 10 7 8 0.437274 63/64 RandomExcursionsVariant 5 4 5 4 9 5 13 3 7 9 0.100508 63/64 RandomExcursionsVariant 6 7 2 8 6 8 7 10 4 6 0.568055 63/64 RandomExcursionsVariant 7 6 5 5 7 6 9 8 5 6 0.964295 63/64 RandomExcursionsVariant 5 8 4 8 6 6 5 5 3 14 0.100508 63/64 RandomExcursionsVariant 3 8 6 7 0 5 10 9 11 5 0.048716 64/64 RandomExcursionsVariant 4 4 7 5 10 1 8 7 10 8 0.178278 64/64 RandomExcursionsVariant 3 7 5 8 7 4 4 8 9 9 0.568055 64/64 RandomExcursionsVariant 6 3 10 6 3 5 5 6 9 11 0.232760 64/64 RandomExcursionsVariant 4 10 9 7 4 5 1 8 9 7 0.195163 64/64 RandomExcursionsVariant 5 11 6 12 2 2 6 5 9 6 0.043745 64/64 RandomExcursionsVariant 4 7 12 6 2 8 6 6 9 4 0.195163 64/64 RandomExcursionsVariant 4 7 10 7 0 8 7 9 9 3 0.090936 63/64 RandomExcursionsVariant 3 8 5 10 5 5 8 6 6 8 0.671779 63/64 RandomExcursionsVariant 11 11 9 10 5 13 9 11 5 16 0.350485 100/100 Serial 10 11 10 8 8 9 14 9 10 11 0.971699 99/100 Serial 7 11 12 10 11 8 11 10 13 7 0.924076 100/100 LinearComplexity - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The minimum pass rate for each statistical test with the exception of the random excursion (variant) test is approximately = 96 for a sample size = 100 binary sequences. The minimum pass rate for the random excursion (variant) test is approximately = 60 for a sample size = 64 binary sequences. For further guidelines construct a probability table using the MAPLE program provided in the addendum section of the documentation. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -