opensource/podman
1
0
Fork 0
mirror of https://github.com/containers/podman.git synced 2025-07-02 08:47:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

parse-localbenchmarks: separate standard deviation

Browse Source 
Go benchmark results include an Average, represented as

    <mean> ± <standard deviation>

This is suboptimal for many reasons:

  * Some web server somewhere in our CI pipeline (Cirrus?
    Google? Gitlab? I have no idea) sends the wrong mime-type
    header, rendering the CSV weird-looking in a browser.
    Not that it's intended for a browser, but we have to
    debug/verify manually once in a while.

  * The spaces and +/- makes it less machine-readable.

Solution: split the "Average" field into two: Average, and
Standard Deviation. And, as a courtesy to human readers,
add a new column with SD as a percentage.

Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit is contained in:
Ed Santiago
2022-12-05 08:31:09 -07:00
parent 6f1bc98dce
commit 08741496d8
1 changed files with 36 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
hack/parse-localbenchmarks
View File

@ -71,13 +71,7 @@ while (my $line = <STDIN>) {
# e.g., 'Fastest Time: 0.265s'
elsif ($line =~ /^(\S.*?\S):\s+(.*)/) {
my $benchmark = "$type $1";
$results{$testname}{$benchmark} = $2;
# Keep an ordered list of benchmark names (as in, the order we
# encounter them)
push @benchmarks, $benchmark
unless grep { $_ eq $benchmark } @benchmarks;
log_result($testname, $type, $1, $2);
}
else {
@ -102,3 +96,38 @@ for my $t (sort keys %results) {
}
print "\n";
}
exit 0;
################
# log_result # Preserve one record
################
sub log_result {
my $testname = shift; # in: test name (eg "podman foo")
my $type = shift; # in: CPU or MEM
my $name = shift; # in: benchmark name (eg "Fastest")
my $result = shift; # in: benchmark value
my $benchmark = "$type $name";
$results{$testname}{$benchmark} = $result;
# Keep an ordered list of benchmark names (as in, the order we
# encounter them)
push @benchmarks, $benchmark
unless grep { $_ eq $benchmark } @benchmarks;
# Special case: "Average X" may be of the form "xxx ± yyy". Move the
# standard deviation to its own column.
if ($name =~ /Average/) {
if ($results{$testname}{$benchmark} =~ s/^(\S+)\s+.*\s+(\S+)$/$1/) {
my ($average, $sd) = ($1, $2);
log_result($testname, $type, 'StdDev', $sd);
# Strip off units, so we can determine it as a percentage
$average =~ s/[a-z]+$//i;
$sd =~ s/[a-z]+$//i;
my $pct = sprintf("%.1f%%", $sd * 100.0 / $average);
log_result($testname, $type, 'StdDev (Percent)', $pct);
}
}
}