There was an omission on 3e6dc39ed7a8 "sim/testsuite: Set
global_cc_os also when no compiler is found"; global_cc_os
wasn't set for other than the primary target, which means
that the "unguarded" use of global_cc_os in
testsuite/cris/c/c.exp caused the dreaded "ERROR: can't read
"global_cc_os": no such variable" when e.g. configuring for
pru-elf and doing "make check-sim". Better initializing
both variables at the top to default values, rather than
adding another single 'set global_cc_os ""', to reduce the
risk of not setting them properly if or when that
if-statement-chain is made longer.
sim/testsuite:
* lib/sim-defs.exp (sim_init_toolchain): Default
global_cc_os and global_cc_works properly, before if-chain.
Simulator features can be present or not, typically
depending on different-valued configure options, like
--enable-sim-hardware[=off|=on]. To avoid failures in
test-suite-runs when testing such configurations, a new
predicate is needed, as neither "target", "progos" nor
"mach" fits cleanly.
The immediate need was to check for presence of a simulator
option, but rather than a specialized "requires-simoption:"
predicate I thought I'd handle the general (parametrized)
need, so here's a generic predicate machinery and a (first)
predicate to use together with it; checking whether a
particular option is supported, by looking at "run --help"
output. This was inspired by the check_effective_target_
machinery in the gcc test-suite.
Multiple "requires: <requirement> <parameter>" form a list of
predicates (with parameters), to be used as a conjunction.
sim/testsuite:
* lib/sim-defs.exp (sim_check_requires_simoption): New function.
(run_sim_test): Support "requires: <requirement> <parameter>".
If we don't set this variable, it doesn't exist, and using "#progos:"
in an assembler-file will cause an error rather than just skipping the
test, viz:
Running /src/sim/testsuite/cris/hw/rv-n-cris/rvc.exp ...
ERROR: tcl error sourcing /src/sim/testsuite/cris/hw/rv-n-cris/rvc.exp.
ERROR: can't read "global_cc_os": no such variable
while executing
"if { $opts(progos) != "" && $opts(progos) != $global_cc_os } {
untested $subdir/$name
return
}"
(procedure "run_sim_test" line 102)
Neither the commit introducing progos, nor the top comment
in run_sim_test, mentions progos as intended only for C
tests, or that its use must be gated on $global_cc_works !=
0, so (not) setting it in the no-working-compiler path seems
just overlooked.
Allowing it to be used for assembler tests makes it usable
for e.g. an always-false predicate and in expressions in
.exp files without gating on $global_cc_works != 0.
With this patch, global_cc_os is set to "", just as for "unknown OS".
sim/testsuite:
* lib/sim-defs.exp (sim_init_toolchain): Set global_cc_os also when
no working target C compiler is found.
Leverage the new per-port toolchain settings to initialize the env
for eeach set of tests. This allows us to run all the tests in a
multitarget build if the user sets up the vars. If they don't, we
can still skip all the tests.
As we setup more reliable CC_FOR_TARGET variables for each target, the
bfin way of overriding it to stuff custom CFLAGS doesn't scale well.
Add a dedicated CFLAGS_FOR_TARGET_init setting that each set of tests
can setup if they want to add custom options.
We usually test against the newlib/libgloss environment, but for a
few ports that also support Linux apps, we want to test that logic
too. A lot of the C code is written such that it works with either
newlib/libgloss or glibc/linux toolchains, but we have some tests
that end up being Linux-specific. Cris has been using the target
tuple as a rough proxy for this (where cris*-*-elf is assumed to be
newlib/libgloss, and everything else is glibc/linux), but that is a
bit too rough, and it doesn't work in a multitarget build.
So lets create a few stub files that we can do compile tests with
to detect the different setups, and then let tests declare which
one they require (if they require any at all).
Both bfin & cris ports test the C compiler to see if it works, but in
their own way. Unify the checks in the common code so we can leverage
them in more ports in the future, and collapse the bfin & cris code.
The sim_init function was called by runtest for each test when --tool
was set to sim. When we changed to --tool '' to collapse the testsuite
dir, the init function was no longer called on every test. However, it
was still being called explicitly by config/default.exp. It's not clear
why that explicit call ever existed since, in the past, it meant it was
redundant.
Lets drop the single sim_init call in config/default.exp and move it out
to all our tests. This replicates the runtest behavior so we can setup
variables on a per-test basis which allows us to recollapse the sim_path
logic back. We'll also leverage this in the future for toolchain setup.
Also add a few comments clarifying the overall runtime behavior.
The tests assume that the cwd is the objdir directory and write its
intermediates to there all the time. When using runtest's --objdir
setting though, this puts the files in the wrong place. This isn't
a big problem currently as we never change --objdir, but in order to
support parallel test execution, we're going to start setting that
option, so clean up the code ahead of time.
We also have to tweak some of the cris tests which were making
assumptions about the argv[0] value.
Now that all the other toolchain settings have been renamed to match
the dejagnu settings of XXX_FOR_TARGET, rename global_sim_options to
SIMFLAGS_FOR_TARGET too.
Only a few tests actually use global_ld_options, but we can replace the
sim-specific settings with the dejagnu common LDFLAGS_FOR_TARGET and get
the same result.
Only a few tests actually use global_as_options, but we can replace the
sim-specific settings with the dejagnu common ASFLAGS_FOR_TARGET and get
the same result.
Nothing in the testsuite is using this setting, so let's drop it.
Any code that wants to set compiler flags can use CFLAGS_FOR_TARGET
instead to get the same effect.
These haven't been used in over 20 years. The sim testsuite used to
run these tools itself directly, but back in ~1999 it switched to the
dejagnu helpers (e.g. target_assemble & target_link), and the dejagnu
logic only utilizes XXX_FOR_TARGET variables. Punt them here to avoid
confusion with dead code.
The v850 testsuite code has been testing the $opt variable, but this
was never actually set anywhere globally or v850-specific. Instead,
this was a random variable leaking out of the sh testsuite code. As
far as I can tell, it has always been this way. That means the code
only ever tested the v850 cpu target (which is the default).
This failure can be easily seen in practice by running the v850 code
in isolation and seeing it crash:
$ runtest v850/allinsns.exp
...
Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
Using ../../../sim/testsuite/config/default.exp as tool-and-target-specific interface file.
WARNING: Assuming target board is the local machine (which is probably wrong).
You may need to set your DEJAGNU environment variable.
Running ../../../sim/testsuite/v850/allinsns.exp ...
ERROR: tcl error sourcing ../../../sim/testsuite/v850/allinsns.exp.
ERROR: tcl error code TCL LOOKUP VARNAME opt
ERROR: can't read "opt": no such variable
while executing
"switch -regexp -- $opt {
Backing up a bit, the reason for this logic in the first place is
because the common sim testsuite code makes an assumption about the
assembler options with cpu_option -- the option and its value are
always separated by an =. This is not the case with v850. So tweak
the core sim logic a bit to support omitting the = so that we can
switch v850 to the standard all_machs setting and avoid opt entirely.
Objcopy was used to create a binary file of just the executable code
since the environment requires code to based at address 0. We can
accomplish the same thing with the -Ttext=0 flag, so switch to that
to get rid of custom logic.
Exit status 77 is common (including the autotools world) to indicate
"skip this test". Add support for mapping that to "unsupported" as
that's the closest in the dejagnu world.
If the port hasn't been enabled, don't try to run its tests. Making
this dynamic simplifies the test harnesses and avoids duplicating a
bunch of target tuple checks.
Provide a simple example simulator for people porting to new targets
to use as a reference. This one has the advantage of being used by
people and having a fun program available for it.
It doesn't require a special target -- the example simulators can be
built for any existing port.
Some tests want to verify they can control the exit status, and
allowing any non-zero value would allow tests to silently fail:
if it crashed & exited 1, or forced all non-zero to 1, then we
wouldn't be able to differentiate with a test exiting with a
status like 47.
Extend the test harness to allow tests to declare their expected
exit status that would be defined as a "pass". This requires a
small tweak to the sim_run API to return the status directly, but
that shouldn't be a big deal as it's only used by sim code.
Sometimes in tests, we need supplemental files like linker scripts or
board helper files. There's no way to set those flags in the tests
currently and relative paths don't work (breaks out of tree builds).
Update the main option parser to replace some strings on the fly. Now
tests can do things like:
Long term we'll want to switch the framework to use the dejagnu helpers
like dg-xxx that gcc & gdb utilize. But that'll require more rework.
If a test doesn't write anything at all to stdout, the current test
framework can't support that. Even if you put a blank output line:
# output:
the setup happily clobbers that with a default pass/fail string.
Tweak the parsing logic so we only set the output to pass/fail when
the test has no output marker.
Rather than requiring all sim tests to be preprocessed .s files, add
support for .S and .c files so we can easily write code using a higher
level language like C.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
The testsuite subdir has a note about unifying the target->subdir logic,
so do just that. The end goal here is to have `make check` work out of
the box without having to delve into dejagnu internals.
The target-specific logic is split out of the top level configure.ac file
and into a dedicated configure.tgt similar to other subprojects (gdb and
ld and etc...) with the difference that this file has to be included at
the m4 level instead of the shell level. This is necessary only because
autoconf requires AC_CONFIG_SUBDIRS be given a string literal and not a
variable value.
Then the toplevel and the testsuite configure files pull this in, the sim
subdir gets expanded into testsuite/site.exp, and the default sim run code
uses this info to set the sim path to the local compiled run file if it
hasn't already been specified.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
* lib/sim-defs.exp (run_sim_test): Include a description such as
"assembling" or "linking" that identifies the phase a test fails
in, for easier analysis of failures.
* lib/sim-defs.exp (run_sm_test): Correct comment. "output" and
"xerror" options do not use a list of machines. Clear options from
previous test case. Use "$cpu_option" to identify the machine to the
assembler, if specified.
Mon Jun 1 18:54:22 1998 Frank Ch. Eigler <fche@cygnus.com>
* lib/sim-defs.exp (sim_run): Add possible environment variable
list to simulator run.
start-sanitize-sky
* sim/sky/sky-defs.tcl: Use it.
* sim/sky/t-pke2.vif1out: Update to match recent word-precise
tracking table change in sim/mips/sky-pke.c.
* sim/sky/t-pke3.trc: Ditto.
* sim/sky/t-pke4.vif0expect: Ditto.
end-sanitize-sky
Mon May 18 10:37:47 1998 Doug Evans <devans@canuck.cygnus.com>
