-r--r--r-- 6851 djbsort-20260127/doc/html/test.html raw
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<style type="text/css">
html{overflow-y:scroll;background-color:#012345}
body{font-family:"Noto Sans","Droid Sans","DejaVu Sans","Arial",sans-serif;line-height:1.5}
tt,code{background-color:#f0f0f0;font-family:"Noto Sans Mono","Droid Sans Mono","DejaVu Sans Mono","Courier New",monospace,sans-serif;font-size:1em;}
pre{margin-left:3em}
p,ul,ol,blockquote,pre{font-size:1.0em;line-height:1.6}
li p{font-size:1.0em}
blockquote p{font-size:1.0em}
h1{font-size:1.5em}
h2{font-size:1.3em}
h3{font-size:1.0em}
h1 a{text-decoration:none}
table{border-collapse:collapse}
th,td{border:1px solid black}
table a{text-decoration:none}
table tr{font-size:1.0em;line-height:1.6em}
table tr{font-size:1.0em;line-height:1.5}
tbody tr:nth-child(2n+1){background-color:#f0ffff}
tbody tr:nth-child(2n+2){background-color:#fffff0}
#headline{display:block;margin:0;padding:0;color:#ffffff;background-color:#012345}
#headline .text{font-weight:bold;font-size:1.0em}
#headline input{display:none}
#nav ul{margin:0;padding:0}
#nav li{list-style-type:none;margin:0;padding:0}
.navtop{padding-bottom:0.5em;font-weight:bold;font-size:1.0em}
.navtop{background-color:#012345;color:#ffffff}
#nav .here{background-color:#012345;color:#ffffff}
#nav .away{background-color:#012345;color:#ffffff}
#nav .away a{text-decoration:none;display:block;color:#ffffff}
#nav .away a:hover,.away a:active{text-decoration:underline}
#hidemenu{visibility:hidden;display:none;overflow:hidden;position:fixed;top:0;left:0;height:100%;width:100%}
.main{padding:5px}
.main{background-color:#ffffff}
.pagetitle{font-size:1.4em;font-weight:bold}
@media only screen and (min-width:512px) {
.navtop{padding-top:5px}
#headline{top:0;margin:0;width:160px;height:100%;position:fixed;overflow:auto}
#headline .noselect{display:none}
#headline #nav{visibility:visible;display:block;width:auto;height:auto}
.main{margin-left:170px}
#headline #hidemenu{visibility:hidden}
}
@media not screen and (min-width:512px) {
#headline .noselect{-webkit-user-select:none;-ms-user-select:none;user-select:none;}
#headline #nav #navbot{visibility:hidden;position:fixed;top:0;left:-70%;z-index:2;transition:0.2s;margin:0;padding:0}
#headline input:checked ~ #nav #navbot{height:100%;position:fixed;top:0;left:0;visibility:visible;display:block;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;vertical-align:center;font-size:1.0em;width:70%;overflow:auto}
#headline input:checked ~ #hidemenu{visibility:visible;display:block;background:black;opacity:0.3;z-index:1}
}
</style>
<title>
djbsort: Test</title>
</head>
<body>
<label id=headline>
<input type=checkbox />
<nav id=nav>
<div class=navtop>
<span class=noselect>≡</span>
djbsort</div>
<ul id=navbot>
<li class=away><a href=index.html>Intro</a>
</li><li class=away><a href=download.html>Download</a>
</li><li class=away><a href=install.html>Install</a>
</li><li class=here>Test
</li><li class=away><a href=api.html>API</a>
</li><li class=away><a href=speed.html>Speed</a>
</li><li class=away><a href=security.html>Security</a>
</li><li class=away><a href=verif.html>Verification</a>
</li><li class=away><a href=internals.html>Internals</a>
</li><li class=away><a href=license.html>License</a>
</li><li class=away><a href=refs.html>References</a>
</li><li class=away><a href=comparison.html>Comparison</a>
</li></ul></nav>
<div id=hidemenu></div>
</label>
<div class=main>
<div class=pagetitle>djbsort: Test</div>
<p>To run the full test suite
after compiling and installing djbsort,
run <code>djbsort-fulltest</code>.
This indicates success in two ways:
it prints <code>full tests succeeded</code> as its last line of output;
it exits 0.</p>
<p>Any change in the compiled library
(compiling for a different architecture, compiling with a different compiler, etc.) must be subjected to a new round of tests.
A compiled version of djbsort that does not pass the full test suite is <strong>not supported</strong>.</p>
<p>One run of <code>djbsort-fulltest</code>
was observed to take 39 core-minutes on cezanne (AMD Ryzen 5 PRO 5650G; 6 cores, 2 threads/core)
with overclocking disabled (so running at 3.9GHz).
This test finished in under 4 minutes of real time;
<code>djbsort-fulltest</code> includes some automatic parallelization.
To limit the number of threads used to 1,
run <code>env THREADS=1 djbsort-fulltest</code>.</p>
<p>djbsort automatically selects
AVX2 implementations when it is running on an Intel/AMD CPU that supports AVX2,
while falling back to portable implementations otherwise.
Running <code>djbsort-fulltest</code> on an Intel/AMD CPU without AVX2
will say <code>CPU does not support implementation</code> for the AVX2 implementations
and will fail.
To test a compilation of djbsort for Intel/AMD CPUs,
you have to run <code>djbsort-fulltest</code> on an Intel/AMD CPU with AVX2.</p>
<p>The rest of this page says more about what is happening inside <code>djbsort-fulltest</code>.</p>
<h2 id="conventional">Conventional tests</h2>
<p>The workhorse inside <code>djbsort-fulltest</code>
is a separate <code>djbsort-test</code> program.</p>
<p>Simply calling <code>djbsort-test</code> without arguments
will run various tests on the subroutines in djbsort,
and will indicate success in two ways:
printing <code>all tests succeeded</code> as the last line of output,
and exiting 0.</p>
<p>For parallelism,
<code>djbsort-fulltest</code> calls <code>djbsort-test</code> many times,
using optional <code>djbsort-test</code> arguments to narrow what is being tested in each call.</p>
<h2 id="dataflow">Data-flow tests</h2>
<p>Another way that <code>djbsort-fulltest</code>
runs <code>djbsort-test</code> is as follows,
running TIMECOP-style tests that branch conditions and array indices
are independent of secrets:</p>
<pre><code>env valgrind_multiplier=1 \
valgrind -q \
--error-exitcode=99 \
djbsort-test
</code></pre>
<p>This requires <code>valgrind</code> to be installed at test time.
The output will include a line <code>valgrind 1 declassify 1</code>
if the library was compiled with <code>--valgrind</code> (which is the only supported option),
or a line <code>valgrind 1 declassify 0 (expect false positives)</code> otherwise.</p>
<p>These data-flow tests
do not supersede the conventional tests.
The conventional tests run code directly on the CPU
and might catch issues hidden by the emulation in <code>valgrind</code>.
The conventional tests also include some memory tests that are disabled to improve the <code>valgrind</code> memory tests
but that are not necessarily superseded by the <code>valgrind</code> memory tests.</p><hr><font size=1><b>Version:</b>
This is version 2026.01.26 of the "Test" web page.
</font>
</div>
</body>
</html>