While working on another series, I noticed that c-lang.h does not need
to include parser-defs.h. This patch makes this change, and fixes up
the two .c files that needed this include. Tested by rebuilding.
gdb/ChangeLog
2020-11-25 Tom Tromey <tom@tromey.com>
* d-lang.c: Include parser-defs.h.
* rust-lang.c: Include parser-defs.h.
* c-lang.h: Do not include parser-defs.h.
Previously, ld merged duplicate output sections if such existed in
scripts, except for those with a constraint of SPECIAL. This makes
scripts with duplicate output section statements create duplicate
output sections in the linker output file.
* ldlang.c (lang_output_section_statement_lookup): Change "create"
parameter to a tristate, if 2 then always create a new output
section statement. Update all callers, with
lang_enter_output_section_statement using "2".
(map_input_to_output_sections): Don't ignore SPECIAL constraint
here.
* ldlang.h (lang_output_section_statement_type): Update prototype.
(lang_output_section_find): Update.
Seen on arm-elf, where ELFOSABI_ARM is set too late to get a warning
when processing ifunc related directives on their source line.
../gas/as-new ifunc.s -o tmpdir/ifunc.o
../gas/as-new: symbol type STT_GNU_IFUNC is supported only by GNU and FreeBSD targets
ifunc.s: Assembler messages:
ifunc.s: Fatal error: can't close tmpdir/ifunc.o: sorry, cannot handle this file
This patch doesn't fix the real underlying problem, just the late
error message where "can't close" is a misdirection in this case.
* output-file.c (output_file_close): Remove "can't close" from
error message.
* testsuite/gas/mips/reginfo-2.l: Update expected output.
GOTPCRELX relocations can be transformed only when addend == -4. Add
tests for GOTPCRELX relocations with addend != -4.
PR gold/26939
* testsuite/ld-x86-64/pr26939-x32.d: New file.
* testsuite/ld-x86-64/pr26939.d: Likewise.
* testsuite/ld-x86-64/pr26939.s: Likewise.
* testsuite/ld-x86-64/x86-64.exp: Run PR gold/26939 tests.
Since we are in C++, this typedef is no longer necessary, we can just
refer to the struct name directly.
gdb/ChangeLog:
* regcache.h (struct cached_reg): Remove typedef.
Change-Id: I0168b5a9cf88e9b962521760c7e2d0e6f0b52cdf
Breakpoint locations are sorted according to their addresses. The
addresses are determined by how the compiler emits the code.
Therefore, we may have a different order of locations depending on the
compiler we use. To make the gdb.base/condbreak-multi-context.exp
test flexible enough for different compilers' output, do not hard-code
location indices.
Tested with GCC and Clang.
gdb/testsuite/ChangeLog:
2020-11-24 Tankut Baris Aktemur <tankut.baris.aktemur@intel.com>
* gdb.base/condbreak-multi-context.exp: Do not hard-code location
indices.
This commit enhances print_type_scalar to include support for
TYPE_CODE_FIXED_POINT. This way, any language falling back to
this function for printing the description of some types
also gets basic ptype support for fixed point types as well.
This fixes a couple of XFAILs in gdb.dwarf2/dw2-fixed-point.exp.
gdb/ChangeLog:
* typeprint.c (print_type_scalar): Add handling of
TYPE_CODE_FIXED_POINT.
gdb/testsuite/ChangeLog:
* gdb.dwarf2/dw2-fixed-point.exp: Fix the expected output of
the "ptype pck__fp1_range_var" test for the module-2 and pascal
languages. Remove the associated setup_xfail.
gdb/ChangeLog (Simon Marchi <simark@simark.ca>):
* valarith.c (fixed_point_binop): Replace the
INIT_VAL_WITH_FIXED_POINT_VAL macro by a lambda. Update all
users accordingly.
This logically connects this function to the object it inspects.
gdb/ChangeLog:
* gdbtypes.h (struct type) <fixed_point_scaling_factor>: New method,
replacing fixed_point_scaling_factor. All callers updated
throughout this project.
(fixed_point_scaling_factor): Delete declaration.
* gdbtypes.c (type::fixed_point_scaling_factor): Replaces
fixed_point_scaling_factor. Adjust implementation accordingly.
As suggested by Simon, to logically connect this function to
the object it inspects.
Note that, logically, this method should be "const". Unfortunately,
the implementation iterates on struct type objects starting with "this",
and thus trying to declare the method "const" triggers a compilation
error.
gdb/ChangeLog:
* gdbtypes.h (struct type) <fixed_point_type_base_type> New method,
replacing the fixed_point_type_base_type function. All callers
updated throughout this project.
(fixed_point_type_base_type): Remove declaration.
* gdbtypes.c (type::fixed_point_type_base_type): Replaces
fixed_point_type_base_type. Adjust implementation accordingly.
This is one step further towards the removal of all these macros.
gdb/ChangeLog:
* gdbtypes.h (struct type) <fixed_point_info, set_fixed_point_info>:
New methods.
(INIT_FIXED_POINT_SPECIFIC): Adjust.
(TYPE_FIXED_POINT_INFO): Delete macro.
(allocate_fixed_point_type_info): Change return type to void.
* gdbtypes.c (copy_type_recursive): Replace the use of
TYPE_FIXED_POINT_INFO by a call to the fixed_point_info method.
(fixed_point_scaling_factor): Likewise.
(allocate_fixed_point_type_info): Change return type to void.
Adjust implementation accordingly.
* dwarf2/read.c (finish_fixed_point_type): Replace the use of
TYPE_FIXED_POINT_INFO by a call to the fixed_point_info method.
This commit changes the interfaces of some of the methods declared
in gmp-utils to take a gdb::array_view of gdb_byte instead of a
(gdb_byte *, size) couple.
This makes these methods' API probably more C++-idiomatic.
* gmp-utils.h (gdb_mpz::read): Change buf and len parameters
into one single gdb::array_view parameter.
(gdb_mpz::write): Likewise.
(gdb_mpq::read_fixed_point, gdb_mpq::write_fixed_point): Likewise.
* gmp-utils.c (gdb_mpz::read): Change buf and len parameters
into one single gdb::array_view parameter.
Adjust implementation accordingly.
(gdb_mpz::write): Likewise.
(gdb_mpq::read_fixed_point, gdb_mpq::write_fixed_point): Likewise.
* unittests/gmp-utils-selftests.c: Adapt following changes above.
* valarith.c, valops.c, valprint.c, value.c: Likewise.
This was suggested by Simon during a code review of this package upstream.
The upside is that this makes the function's API more natural and C++.
The downside is an extra malloc, which might be the reason why we went
for using a unique_xmalloc_ptr in the first place. Since this function
is not expected to be called frequently, the API improvement might be
worth the performance impact.
gdb/ChangeLog:
* gmp-utils.h (gmp_string_printf): Rename from gmp_string_asprintf.
Change return type to std::string. Update all callers.
* gmp-utils.c (gmp_string_printf): Likewise.
When building GDB using Ubuntu 20.04's system libgmp and compiler,
running the "maintenance selftest" command triggers the following error:
| Running selftest gdb_mpq_write_fixed_point.
| *** stack smashing detected ***: terminated
| [1] 1092790 abort (core dumped) ./gdb gdb
This happens while trying to construct an mpq_t object (a rational)
from two integers representing the numerator and denominator.
In our test, the numerator is -8, and the denominator is 1.
The problem was that the rational was constructed using the wrong
function. This is what we were doing prior to this patch:
mpq_set_ui (v.val, numerator, denominator);
The 'u' in "ui" stands for *unsigned*, which is wrong because
numerator and denominator's type is "int".
As a result of the above, instead of getting a rational value of -8,
we get a rational with a very large positive value (gmp_printf
says "18446744073709551608").
From there, the test performs an operation which is expected to
write this value into a buffer which was not dimensioned to fit
such a number, thus leading GMP into a buffer overflow.
This was verified by applying the formula that GMP's documentation
gives for the required memory buffer size needed during export:
| When an application is allocating space itself the required size can
| be determined with a calculation like the following. Since
| mpz_sizeinbase always returns at least 1, count here will be at
| least one, which avoids any portability problems with malloc(0),
| though if z is zero no space at all is actually needed (or written).
|
| numb = 8*size - nail;
| count = (mpz_sizeinbase (z, 2) + numb-1) / numb;
| p = malloc (count * size);
With the very large number, mpz_sizeinbase returns 66 and thus
the malloc size becomes 16 bytes instead of the 8 we allocated.
This patch fixes the issue by using the correct "set" function.
gdb/ChangeLog:
* unittests/gmp-utils-selftests.c (write_fp_test): Use mpq_set_si
instead of mpq_set_ui to initialize our GMP rational.
Let's say you put this gdb_assert in a test:
gdb_assert "some invalid tcl code"
You just get:
FAIL: gdb.base/template.exp: some invalid tcl code
That's not very easy to debug, since you don't know what's invalid in
your code.
Change gdb_assert to print the error message when catch's return code is
1 (TCL_ERROR). The "warning" is shown both on stdout and in the log
file. Mark the test as unresolved, because the evaluation error means
we couldn't reach a valid pass/fail conclusion.
gdb/testsuite/ChangeLog:
* lib/gdb.exp (gdb_assert): Show error message on error.
Change-Id: Ie6477859554e909ed8d07fb2769c6f2f55e7cce6
With current master I see a couple of KPASSes:
...
KPASS: gdb.ada/enum_idx_packed.exp: scenario=minimal: ptype small \
(PRMS minimal encodings)
...
KPASS: gdb.ada/mod_from_name.exp: scenario=minimal: print xp \
(PRMS minimal encodings)
KPASS: gdb.ada/pckd_arr_ren.exp: scenario=minimal: print var \
(PRMS minimal encodings)
...
The corresponding setup_kfail is called for everything before gnat 11.
However, the test-cases also PASS for me with gnat-4.8, gnat-7.5.0 and
gnat-8.4.0.
Fix the KPASSes by limiting the setup_kfail to gnat 9 and 10.
Tested on x86_64-linux.
gdb/testsuite/ChangeLog:
2020-11-23 Tom de Vries <tdevries@suse.de>
* gdb.ada/enum_idx_packed.exp: Limit setup_kfail to gnat 9 and 10.
* gdb.ada/mod_from_name.exp: Same.
* gdb.ada/pckd_arr_ren.exp: Same.
When setting env var DEBUGINFOD_URLS to " " and running the testsuite, we run
into these regressions:
...
FAIL: gdb.base/list-missing-source.exp: info source
FAIL: gdb.base/source-dir.exp: info source before setting directory search list
...
Setting var DEBUGINFOD_URLS to " " allows the debuginfod query function
debuginfod_source_query to get past its early exit.
The function debuginfod_source_query is documented as: "If the file is
successfully retrieved, its path on the local machine is stored in DESTNAME".
However, in case we get back -ENOENT from libdebuginfod, we still set
DESTNAME:
....
if (fd.get () < 0 && fd.get () != -ENOENT)
printf_filtered (_("Download failed: %s. Continuing without source file %ps.\n"),
safe_strerror (-fd.get ()),
styled_string (file_name_style.style (), srcpath));
else
*destname = make_unique_xstrdup (srcpath);
return fd;
...
Fix this by making debuginfod_source_query fit it's documentation and only
setting DESTNAME when successfully retrieving a file. Likewise in
debuginfod_debuginfo_query.
gdb/ChangeLog:
2020-11-23 Tom de Vries <tdevries@suse.de>
* debuginfod-support.c (debuginfod_source_query)
(debuginfod_debuginfo_query): Only set DESTNAME if successful.
PR 26931
* elf-bfd.h (struct elf_backend_data): Add bfd_boolean field to
slurp_secondary_relocs field.
(_bfd_elf_slurp_secondary_reloc_section): Update prototype.
* elf.c (_bfd_elf_slurp_secondary_reloc_section): Add new
parameter. Compute number of symbols based upon the new
parameter.
* elfcode.h (elf_slurp_reloc_table): Pass dynamic as new
parameter.
This is to address the regressions addressed by Nic [1].
The regular expression pattern for the tls_ie-01 test was
too strict and raising false alarms. The new pattern only
looks for matches that should be there AND ignores the boiler
plates from the object dump.
[1] New failures for ARC targets in linker testsuite
https://sourceware.org/pipermail/binutils/2020-November/114177.html
ld/
* testsuite/ld-arc/tls_ie-01.d: Use a more general pattern.
The wiki contains a template for new test cases:
https://sourceware.org/gdb/wiki/GDBTestcaseCookbook#Building_the_Example_Program
... which is helpful, because even after many years I can't write all
the boilerplate for writing a test case without doing some mistakes.
However, I think it would be nice to have it in the tree. It's much
faster to cp the files than going to the wiki and copy/pasting the
contents.
As a bonus, the copyright years will get updated in these files, unlike
those in the wiki. So they will always be good when we start a new
test.
If this patch is merged, I plan to change the wiki to just point to
these files.
gdb/testsuite/ChangeLog:
* gdb.base/template.exp: New.
* gdb.base/template.c: New.
Change-Id: I7dbf068a043b48f83cc325087d70e868eee998c6
Clang describes the upper bounds of variable length arrays using
a DW_AT_count attribute which references the DIE of a synthetic
variable whose value is specified using a DW_AT_location. GDB handles
these incorrectly if the corresponding DWARF expression finishes with a
DW_OP_stack_value (PR26905). This commit adds a new kfailed test to
gdb.dwarf2/count.exp with the same DWARF as that generated by Clang for
gdb.base/vla-optimized-out.exp, one of the failing tests.
Checked on Fedora 32 x86_64, with GCC and Clang.
gdb/testsuite/ChangeLog:
2020-11-22 Gary Benson <gbenson@redhat.com>
PR gdb/26905
* gdb.dwarf2/count.exp: Add test for an array whose upper bound
is defined using a DW_AT_count which references another DIE.
When running gdb.base/vla-ptr.exp with clang-10, we run into this FAIL:
...
(gdb) print td_vla^M
$6 = 0x7fffffffd2b0^M
(gdb) FAIL: gdb.base/vla-ptr.exp: print td_vla
...
Clang 10.0.1 generates the following DWARF for td_vla. A variable DIE:
...
<2><19f>: Abbrev Number: 6 (DW_TAG_variable)
<1a0> DW_AT_location : 0x39 (location list)
<1a4> DW_AT_name : td_vla
<1aa> DW_AT_type : <0x1ae>
....
with typedef type:
...
<2><1ae>: Abbrev Number: 7 (DW_TAG_typedef)
<1af> DW_AT_type : <0x1fc>
<1b3> DW_AT_name : typedef_vla
...
pointing to:
...
<1><1fc>: Abbrev Number: 11 (DW_TAG_array_type)
<1fd> DW_AT_type : <0x1d3>
<2><201>: Abbrev Number: 14 (DW_TAG_subrange_type)
<202> DW_AT_type : <0x1f5>
...
The subrange type is missing the count attribute. This was filed as
llvm PR48247 - "vla var with typedef'd type has incomplete debug info".
Mark this as xfail.
gdb/testsuite/ChangeLog:
2020-11-21 Tom de Vries <tdevries@suse.de>
* gdb.base/vla-ptr.exp: Add XFAIL.
I noticed that watchpoint_exp_is_const should return bool; this patch
implements this change.
gdb/ChangeLog
2020-11-21 Tom Tromey <tom@tromey.com>
* breakpoint.c (watchpoint_exp_is_const): Return bool.
I get the feedback recently that enable linker relaxations may fail to
build some program. Consider the following case,
.text
foo:
addi a0, a0, %pcrel_lo(.L2)
call foo
.L1: auipc a1, %pcrel_hi(data_g)
addi a1, a1, %pcrel_lo(.L1)
lui a2, %hi(data_g)
addi a2, a2, %lo(data_g)
lui a3, %tprel_hi(data_t)
add a3, a3, tp, %tprel_add(data_t)
addi a3, a3, %tprel_lo(data_t)
.L2: auipc a0, %pcrel_hi(data_g)
.data
.word 0x0
.global data_g
data_g: .word 0x1
.section .tbss
data_t: .word 0x0
The current ld reports `dangerous relocation error` when doing the
pcgp relaxation,
test.o: in function `foo':
(.text+0x0): dangerous relocation: %pcrel_lo missing matching %pcrel_hi
The .L2 auipc should not be removed since it is behind the corresponding
addi, so we record the information in the pcgp_relocs table to avoid
removing the auipc later. But current ld still remove it since we do not
update the pcgp_relocs table while doing other relaxations. I have two
solutions to fix the problem,
1. Update the pcgp_relocs table once we actually delete the code.
2. Add new relax pass to do the pcgp relaxations
At first I tried to do the first solution, and we need to update at
least three information - hi_sec_off of riscv_pcgp_lo_reloc, hi_sec_off
and hi_addr (symbol value) of riscv_pcgp_hi_reloc. Update the hi_sec_off
is simple, but it is more complicate to update the symbol value, since we
almost have to do parts the same works of _bfd_riscv_relax_call again in
the riscv_relax_delete_bytes to get the correct symbol value.
Compared with the first solution, the second one is more intuitive and
simple. We add a new relax pass to do the pcgp relaxations later, so
we will get all the information correctly in the _bfd_riscv_relax_call,
including the symbol value, without changing so much code. I do not see
any penalty by adding a new relax pass for now, so it should be fine
to delay the pcgp relaxations.
Besides, I have pass all riscv-gnu-toolchain regressions for this patch.
bfd/
* elfnn-riscv.c (_bfd_riscv_relax_section): Add a new relax pass
to do the pcgp relaxation later, after the lui and call relaxations,
but before the delete and alignment relaxations.
ld/
* emultempl/riscvelf.em (riscv_elf_before_allocation): Change
link_info.relax_pass from 3 to 4.
* testsuite/ld-riscv-elf/pcgp-relax.d: New testcase.
* testsuite/ld-riscv-elf/pcgp-relax.s: Likewise.
* testsuite/ld-riscv-elf/ld-riscv-elf.exp: Updated.
When building on solaris (gcc farm machine gcc211), I get:
CXX unittests/gmp-utils-selftests.o
/export/home/simark/src/binutils-gdb/gdb/unittests/gmp-utils-selftests.c: In function 'void selftests::gdb_mpz_read_all_from_small()' :
/export/home/simark/src/binutils-gdb/gdb/unittests/gmp-utils-selftests.c:128:43: error: call of overloaded 'pow(int, int)' is ambiguous
LONGEST l_min = -pow (2, buf_len * 8 - 1);
^
In file included from /opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/math.h:22:0,
from ../gnulib/import/math.h:27,
from /export/home/simark/src/binutils-gdb/gdb/unittests/gmp-utils-selftests.c:23:
/opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/iso/math_iso.h:210:21: note: candidate: long double std::pow(long double, long double)
inline long double pow(long double __X, long double __Y) { return
^
/opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/iso/math_iso.h:170:15: note: candidate: float std::pow(float, float)
inline float pow(float __X, float __Y) { return __powf(__X, __Y); }
^
/opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/iso/math_iso.h:71:15: note: candidate: double std::pow(double, double)
extern double pow __P((double, double));
^
The "pow" function overloads only exist for float-like types, and the
compiler doesn't know which one we want. Change "2" for "2.0", which
makes the compiler choose one alternative (the double one, I believe).
gdb/ChangeLog:
* unittests/gmp-utils-selftests.c (gdb_mpz_read_all_from_small):
Pass 2.0 to pow.
(gdb_mpz_write_all_from_small): Likewise.
Change-Id: Ied2ae0f01494430244a7c94f8a38b07d819f4213
When building on solaris (gcc farm machine gcc211), I get:
CXX dwarf2/read.o
/export/home/simark/src/binutils-gdb/gdb/dwarf2/read.c: In function 'void finish_fixed_point_type(type*, die_info*, dwarf2_cu*)':
/export/home/simark/src/binutils-gdb/gdb/dwarf2/read.c:18204:42: error: call of overloaded 'abs(LONGEST&)' is ambiguous
*num_or_denom = 1 << abs (scale_exp);
^
In file included from /usr/include/stdlib.h:11:0,
from ../gnulib/import/stdlib.h:36,
from /opt/csw/include/c++/5.5.0/cstdlib:72,
from /export/home/simark/src/binutils-gdb/gdb/../gdbsupport/common-defs.h:90,
from /export/home/simark/src/binutils-gdb/gdb/defs.h:28,
from /export/home/simark/src/binutils-gdb/gdb/dwarf2/read.c:31:
/opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/iso/stdlib_iso.h:163:16: note: candidate: long int std::abs(long int)
inline long abs(long _l) { return labs(_l); }
^
/opt/csw/lib/gcc/sparc-sun-solaris2.10/5.5.0/include-fixed/iso/stdlib_iso.h:117:12: note: candidate: int std::abs(int)
extern int abs(int);
^
I don't know why, but using std::abs instead of just abs fixes it.
gdb/ChangeLog:
* dwarf2/read.c (finish_fixed_point_type): Use std::abs instead
of abs.
Change-Id: I57b9098351f2a8b2d2f61e848b97f7b2dfe55908
libstdc++ might change so that it always implements std::thread, but
then have thread startup simply fail. This is being discussed here:
https://gcc.gnu.org/pipermail/gcc-patches/2020-November/558736.html
This patch pre-emptively changes gdb to handle this scenario. It
seemed fine to me to ignore all system errors at thread startup, so
that is what this does.
gdbsupport/ChangeLog
2020-11-20 Tom Tromey <tromey@adacore.com>
* thread-pool.cc (thread_pool::set_thread_count): Ignore system
errors.
When linking fails, we delete all the generated outputs, but we fail to
remove them from the ctf_link_outputs hash we stuck them in before doing
symbol and variable section linking (which we had to do because that's
where ctf_create_per_cu, used by both, looks for them). This leaves
stale pointers to freed memory behind, and crashes soon follow.
Fix obvious.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-link.c (ctf_link_deduplicating): Clean up the ctf_link_outputs
hash on error.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-create.c (ctf_dtd_insert): Set ENOMEM on the dict if out of memory.
(ctf_dvd_insert): Likewise.
(ctf_add_function): Report ECTF_RDONLY if this dict is not writable.
* ctf-subr.c (ctf_err_warn): Only debug-dump passed-in warnings if
the passed-in error code is nonzero: the error on the dict for
warnings may relate to a previous error.
libctf has long provided ctf_getdatasect, which hands back a pointer to
the CTF section a (read-only) dict came from. But it has no such
functions to return pointers to the ELF symbol table or string table
it's working from, which is unfortunate because several libctf functions
(ctf_open, ctf_fdopen, and ctf_bfdopen) figure out which string and
symbol table to use themselves, and don't tell the user what they
decided, so the caller can't agree on which symtab to use with libctf
even if it wanted to.
Add a pair of functions to return the symtab and strtab in use. Like
ctf_getdatasect, these return ctf_sect_t structures by value, filled
with all-NULL/0 content if a symtab or strtab is not being used.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_getsymsect): New.
(ctf_getstrsect): Likewise.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-open.c (ctf_getsymsect): New.
(ctf_getstrsect): Likewise.
* libctf.ver: Add them.
CTF archives may contain multiple dicts, each of which contain many
types and possibly a bunch of symtypetab entries relating to those
types: each symtypetab entry is going to appear in exactly one dict,
with the corresponding entries in the other dicts empty (either pads, or
indexed symtypetabs that do not mention that symbol). But users of
libctf usually want to get back the type associated with a symbol
without having to dig around to find out which dict that type might be
in.
This adds machinery to do that -- and since you probably want to do it
repeatedly, it adds internal caching to the ctf-archive machinery so
that iteration over archives via ctf_archive_next and repeated symbol
lookups do not have to repeatedly reopen the archive. (Iteration using
ctf_archive_iter will gain caching soon.)
Two new API functions:
ctf_dict_t *
ctf_arc_lookup_symbol (ctf_archive_t *arc, unsigned long symidx,
ctf_id_t *typep, int *errp);
This looks up the symbol with index SYMIDX in the archive ARC, returning
the dictionary in which it resides and optionally the type index as
well. Errors are returned in ERRP. The dict should be
ctf_dict_close()d when done, but is also cached inside the ctf_archive
so that the open cost is only paid once. The result of the symbol
lookup is also cached internally, so repeated lookups of the same symbol
are nearly free.
void ctf_arc_flush_caches (ctf_archive_t *arc);
Flush all the caches. Done at close time, but also available as an API
function if users want to do it by hand.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_arc_lookup_symbol): New.
(ctf_arc_flush_caches): Likewise.
* ctf.h: Document new auto-ctf_import behaviour.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (struct ctf_archive_internal) <ctfi_dicts>: New, dicts
the archive machinery has opened and cached.
<ctfi_symdicts>: New, cache of dicts containing symbols looked up.
<ctfi_syms>: New, cache of types of symbols looked up.
* ctf-archive.c (ctf_arc_close): Free them on close.
(enosym): New, flag entry for 'symbol not present'.
(ctf_arc_import_parent): New, automatically import the parent from
".ctf" if this is a child in an archive and ".ctf" is present.
(ctf_dict_open_sections): Use it.
(ctf_archive_iter_internal): Likewise.
(ctf_cached_dict_close): New, thunk around ctf_dict_close.
(ctf_dict_open_cached): New, open and cache a dict.
(ctf_arc_flush_caches): New, flush the caches.
(ctf_arc_lookup_symbol): New, look up a symbol in (all members of)
an archive, and cache the lookup.
(ctf_archive_iter): Note the new caching behaviour.
(ctf_archive_next): Use ctf_dict_open_cached.
* libctf.ver: Add ctf_arc_lookup_symbol and ctf_arc_flush_caches.
Some type kinds in CTF (functions, arrays, pointers, slices, and
cvr-quals) are intrinsically nameless: the ctt_name field in the CTF
is always zero, and the libctf API provides no way to set a name.
But the compiler can and does sometimes set names for some of these
kinds: in particular, the name it sets on CTF_K_FUNCTION types is the
means it uses to force the name of the function into the string table
so that it can point at it from the function info section.
So null out the name at hashing time so that the deduplicator can
correctly detect that e.g. function types identical but for name should
be considered truly identical, since they will not have a name when the
deduplicator re-emits them into the output.
ld/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* testsuite/ld-ctf/data-func-conflicted.d: Shrink the expected
size of the type section now that function types are being
deduplicated properly.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-dedup.c (ctf_dedup_rhash_type): Null out the names of nameless
type kinds, just in case the input has named them.
The flags word is nonzero now (so all the tests have been adjusted to
not depend on its content): some of them have data objects and functions
in the data object and function info sections now, rather than in the
variable section or recorded nowhere. There is a new test for
parent/child relationships and index section emission.
ld/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* testsuite/ld-ctf/array.d: Adjust for nonzero flags word and
public symbols in the data section rather than variables: use
sysv hash style to keep test results the same on non-GNU targets.
* testsuite/ld-ctf/diag-cttname-null.d: Likewise.
* testsuite/ld-ctf/diag-cuname.d: Likewise.
* testsuite/ld-ctf/diag-parlabel.d: Likewise.
* testsuite/ld-ctf/slice.d: Likewise.
* testsuite/ld-ctf/function.d: Likewise, but in the function section.
* testsuite/ld-ctf/conflicting-cycle-1.B-1.d: Adjust for nonzero
flags word.
* testsuite/ld-ctf/conflicting-cycle-1.B-2.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-1.parent.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-2.A-1.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-2.A-2.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-2.parent.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-3.C-1.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-3.C-2.d: Likewise.
* testsuite/ld-ctf/conflicting-cycle-3.parent.d: Likewise.
* testsuite/ld-ctf/cross-tu-noncyclic.d: Likewise.
* testsuite/ld-ctf/cycle-1.d: Likewise.
* testsuite/ld-ctf/cycle-2.A.d: Likewise.
* testsuite/ld-ctf/cycle-2.B.d: Likewise.
* testsuite/ld-ctf/cycle-2.C.d: Likewise.
* testsuite/ld-ctf/diag-wrong-magic-number-mixed.d: Likewise.
* testsuite/ld-ctf/super-sub-cycles.d: Likewise.
* testsuite/ld-ctf/data-func-1.c: New test.
* testsuite/ld-ctf/data-func-2.c: Likewise.
* testsuite/ld-ctf/data-func-conflicted.d: Likewise.
Now that we have a new format for the function info section, it's much
easier to dump it: we can use the same code we use for the object type
section, and that's got simpler too because we can use ctf_symbol_next.
Also dump the new stuff in the header: the new flags bits and the index
section lengths.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-dump.c (ctf_dump_header): Dump the new flags bits and the index
section lengths.
(ctf_dump_objts): Report indexed sections. Also dump functions. Use
ctf_symbol_next, not manual looping.
(ctf_dump_funcs): Delete.
(ctf_dump): Use ctf_dump_objts, not ctf_dump_funcs.
This adds facilities to write out the function info and data object
sections, which efficiently map from entries in the symbol table to
types. The write-side code is entirely new: the read-side code was
merely significantly changed and support for indexed tables added
(pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff
header fields).
With this in place, you can use ctf_lookup_by_symbol to look up the
types of symbols of function and object type (and, as before, you can
use ctf_lookup_variable to look up types of file-scope variables not
present in the symbol table, as long as you know their name: but
variables that are also data objects are now found in the data object
section instead.)
(Compatible) file format change:
The CTF spec has always said that the function info section looks much
like the CTF_K_FUNCTIONs in the type section: an info word (including an
argument count) followed by a return type and N argument types. This
format is suboptimal: it means function symbols cannot be deduplicated
and it causes a lot of ugly code duplication in libctf. But
conveniently the compiler has never emitted this! Because it has always
emitted a rather different format that libctf has never accepted, we can
be sure that there are no instances of this function info section in the
wild, and can freely change its format without compatibility concerns or
a file format version bump. (And since it has never been emitted in any
code that generated any older file format version, either, we need keep
no code to read the format as specified at all!)
So the function info section is now specified as an array of uint32_t,
exactly like the object data section: each entry is a type ID in the
type section which must be of kind CTF_K_FUNCTION, the prototype of
this function.
This allows function types to be deduplicated and also correctly encodes
the fact that all functions declared in C really are types available to
the program: so they should be stored in the type section like all other
types. (In format v4, we will be able to represent the types of static
functions as well, but that really does require a file format change.)
We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the
new function info format is in use. A sufficiently new compiler will
always set this flag. New libctf will always set this flag: old libctf
will refuse to open any CTF dicts that have this flag set. If the flag
is not set on a dict being read in, new libctf will disregard the
function info section. Format v4 will remove this flag (or, rather, the
flag has no meaning there and the bit position may be recycled for some
other purpose).
New API:
Symbol addition:
ctf_add_func_sym: Add a symbol with a given name and type. The
type must be of kind CTF_K_FUNCTION (a function
pointer). Internally this adds a name -> type
mapping to the ctf_funchash in the ctf_dict.
ctf_add_objt_sym: Add a symbol with a given name and type. The type
kind can be anything, including function pointers.
This adds to ctf_objthash.
These both treat symbols as name -> type mappings: the linker associates
symbol names with symbol indexes via the ctf_link_shuffle_syms callback,
which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the
ctf_dict. Repeated relinks can add more symbols.
Variables that are also exposed as symbols are removed from the variable
section at serialization time.
CTF symbol type sections which have enough pads, defined by
CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols
where most types are unknown, or in archive where most types are defined
in some child or parent dict, not in this specific dict) are sorted by
name rather than symidx and accompanied by an index which associates
each symbol type entry with a name: the existing ctf_lookup_by_symbol
will map symbol indexes to symbol names and look the names up in the
index automatically. (This is currently ELF-symbol-table-dependent, but
there is almost nothing specific to ELF in here and we can add support
for other symbol table formats easily).
The compiler also uses index sections to communicate the contents of
object file symbol tables without relying on any specific ordering of
symbols: it doesn't need to sort them, and libctf will detect an
unsorted index section via the absence of the new CTF_F_IDXSORTED header
flag, and sort it if needed.
Iteration:
ctf_symbol_next: Iterator which returns the types and names of symbols
one by one, either for function or data symbols.
This does not require any sorting: the ctf_link machinery uses it to
pull in all the compiler-provided symbols cheaply, but it is not
restricted to that use.
(Compatible) changes in API:
ctf_lookup_by_symbol: can now be called for object and function
symbols: never returns ECTF_NOTDATA (which is
now not thrown by anything, but is kept for
compatibility and because it is a plausible
error that we might start throwing again at some
later date).
Internally we also have changes to the ctf-string functionality so that
"external" strings (those where we track a string -> offset mapping, but
only write out an offset) can be consulted via the usual means
(ctf_strptr) before the strtab is written out. This is important
because ctf_link_add_linker_symbol can now be handed symbols named via
strtab offsets, and ctf_link_shuffle_syms must figure out their actual
names by looking in the external symtab we have just been fed by the
ctf_link_add_strtab callback, long before that strtab is written out.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_symbol_next): New.
(ctf_add_objt_sym): Likewise.
(ctf_add_func_sym): Likewise.
* ctf.h: Document new function info section format.
(CTF_F_NEWFUNCINFO): New.
(CTF_F_IDXSORTED): New.
(CTF_F_MAX): Adjust accordingly.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New.
(_libctf_nonnull_): Likewise.
(ctf_in_flight_dynsym_t): New.
(ctf_dict_t) <ctf_funcidx_names>: Likewise.
<ctf_objtidx_names>: Likewise.
<ctf_nfuncidx>: Likewise.
<ctf_nobjtidx>: Likewise.
<ctf_funcidx_sxlate>: Likewise.
<ctf_objtidx_sxlate>: Likewise.
<ctf_objthash>: Likewise.
<ctf_funchash>: Likewise.
<ctf_dynsyms>: Likewise.
<ctf_dynsymidx>: Likewise.
<ctf_dynsymmax>: Likewise.
<ctf_in_flight_dynsym>: Likewise.
(struct ctf_next) <u.ctn_next>: Likewise.
(ctf_symtab_skippable): New prototype.
(ctf_add_funcobjt_sym): Likewise.
(ctf_dynhash_sort_by_name): Likewise.
(ctf_sym_to_elf64): Rename to...
(ctf_elf32_to_link_sym): ... this, and...
(ctf_elf64_to_link_sym): ... this.
* ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO
flag, and presence of index sections. Refactor out
ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use
ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func
sxlate sections if corresponding index section is present. Adjust
for new func info section format.
(ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error
handling. Report incorrect-length index sections. Always do an
init_symtab, even if there is no symtab section (there may be index
sections still).
(flip_objts): Adjust comment: func and objt sections are actually
identical in structure now, no need to caveat.
(ctf_dict_close): Free newly-added data structures.
* ctf-create.c (ctf_create): Initialize them.
(ctf_symtab_skippable): New, refactored out of
init_symtab, with st_nameidx_set check added.
(ctf_add_funcobjt_sym): New, add a function or object symbol to the
ctf_objthash or ctf_funchash, by name.
(ctf_add_objt_sym): Call it.
(ctf_add_func_sym): Likewise.
(symtypetab_delete_nonstatic_vars): New, delete vars also present as
data objects.
(CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters:
this is a function emission, not a data object emission.
(CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit
pads for symbols with no type (only set for unindexed sections).
(CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters:
always emit indexed.
(symtypetab_density): New, figure out section sizes.
(emit_symtypetab): New, emit a symtypetab.
(emit_symtypetab_index): New, emit a symtypetab index.
(ctf_serialize): Call them, emitting suitably sorted symtypetab
sections and indexes. Set suitable header flags. Copy over new
fields.
* ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an
order on symtypetab index sections.
* ctf-link.c (ctf_add_type_mapping): Delete erroneous comment
relating to code that was never committed.
(ctf_link_one_variable): Improve variable name.
(check_sym): New, symtypetab analogue of check_variable.
(ctf_link_deduplicating_one_symtypetab): New.
(ctf_link_deduplicating_syms): Likewise.
(ctf_link_deduplicating): Call them.
(ctf_link_deduplicating_per_cu): Note that we don't call them in
this case (yet).
(ctf_link_add_strtab): Set the error on the fp correctly.
(ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add
a linker symbol to the in-flight list.
(ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the
in-flight list into a mapping we can use, now its names are
resolvable in the external strtab.
* ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with
external strtab offsets.
(ctf_str_rollback): Adjust comment.
(ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from
writeout time...
(ctf_str_add_external): ... to string addition time.
* ctf-lookup.c (ctf_lookup_var_key_t): Rename to...
(ctf_lookup_idx_key_t): ... this, now we use it for syms too.
<clik_names>: New member, a name table.
(ctf_lookup_var): Adjust accordingly.
(ctf_lookup_variable): Likewise.
(ctf_lookup_by_id): Shuffle further up in the file.
(ctf_symidx_sort_arg_cb): New, callback for...
(sort_symidx_by_name): ... this new function to sort a symidx
found to be unsorted (likely originating from the compiler).
(ctf_symidx_sort): New, sort a symidx.
(ctf_lookup_symbol_name): Support dynamic symbols with indexes
provided by the linker. Use ctf_link_sym_t, not Elf64_Sym.
Check the parent if a child lookup fails.
(ctf_lookup_by_symbol): Likewise. Work for function symbols too.
(ctf_symbol_next): New, iterate over symbols with types (without
sorting).
(ctf_lookup_idx_name): New, bsearch for symbol names in indexes.
(ctf_try_lookup_indexed): New, attempt an indexed lookup.
(ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol.
(ctf_func_args): Likewise.
(ctf_get_dict): Move...
* ctf-types.c (ctf_get_dict): ... here.
* ctf-util.c (ctf_sym_to_elf64): Re-express as...
(ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and
st_nameidx_set (always 0, so st_nameidx can be ignored). Look in
the ELF strtab for names.
(ctf_elf32_to_link_sym): Likewise, for Elf32_Sym.
(ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be.
* libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and
ctf_add_func_sym.
This is embarrassing.
The whole point of CTF is that it remains intact even after a binary is
stripped, providing a compact mapping from symbols to types for
everything in the externally-visible interface of an ELF object: it has
connections to the symbol table for that purpose, and to the string
table to avoid duplicating symbol names. So it's a shame that the hooks
I implemented last year served to hook it up to the .symtab and .strtab,
which obviously disappear on strip, leaving any accompanying the CTF
dict containing references to strings (and, soon, symbols) which don't
exist any more because their containing strtab has been vaporized. The
original Solaris design used .dynsym and .dynstr (well, actually,
.ldynsym, which has more symbols) which do not disappear. So should we.
Thankfully the work we did before serves as guide rails, and adjusting
things to use the .dynstr and .dynsym was fast and easy. The only
annoyance is that the dynsym is assembled inside elflink.c in a fairly
piecemeal fashion, so that the easiest way to get the symbols out was to
hook in before every call to swap_symbol_out (we also leave in a hook in
front of symbol additions to the .symtab because it seems plausible that
we might want to hook them in future too: for now that hook is unused).
We adjust things so that rather than being offered a whole hash table of
symbols at once, libctf is now given symbols one at a time, with st_name
indexes already resolved and pointing at their final .dynstr offsets:
it's now up to libctf to resolve these to names as needed using the
strtab info we pass it separately.
Some bits might be contentious. The ctf_new_dynstr callback takes an
elf_internal_sym, and this remains an elf_internal_sym right down
through the generic emulation layers into ldelfgen. This is no worse
than the elf_sym_strtab we used to pass down, but in the future when we
gain non-ELF CTF symtab support we might want to lower the
elf_internal_sym to some other representation (perhaps a
ctf_link_symbol) in bfd or in ldlang_ctf_new_dynsym. We rename the
'apply_strsym' hooks to 'acquire_strings' instead, becuse they no longer
have anything to do with symbols.
There are some API changes to pieces of API which are technically public
but actually totally unused by anything and/or unused by anything but ld
so they can change freely: the ctf_link_symbol gains new fields to allow
symbol names to be given as strtab offsets as well as strings, and a
symidx so that the symbol index can be passed in. ctf_link_shuffle_syms
loses its callback parameter: the idea now is that linkers call the new
ctf_link_add_linker_symbol for every symbol in .dynsym, feed in all the
strtab entries with ctf_link_add_strtab, and then a call to
ctf_link_shuffle_syms will apply both and arrange to use them to reorder
the CTF symtab at CTF serialization time (which is coming in the next
commit).
Inside libctf we have a new preamble flag CTF_F_DYNSTR which is always
set in v3-format CTF dicts from this commit forwards: CTF dicts without
this flag are associated with .strtab like they used to be, so that old
dicts' external strings don't turn to garbage when loaded by new libctf.
Dicts with this flag are associated with .dynstr and .dynsym instead.
(The flag is not the next in sequence because this commit was written
quite late: the missing flags will be filled in by the next commit.)
Tests forthcoming in a later commit in this series.
bfd/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* elflink.c (elf_finalize_dynstr): Call examine_strtab after
dynstr finalization.
(elf_link_swap_symbols_out): Don't call it here. Call
ctf_new_symbol before swap_symbol_out.
(elf_link_output_extsym): Call ctf_new_dynsym before
swap_symbol_out.
(bfd_elf_final_link): Likewise.
* elf.c (swap_out_syms): Pass in bfd_link_info. Call
ctf_new_symbol before swap_symbol_out.
(_bfd_elf_compute_section_file_positions): Adjust.
binutils/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* readelf.c (dump_section_as_ctf): Use .dynsym and .dynstr, not
.symtab and .strtab.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* bfdlink.h (struct elf_sym_strtab): Replace with...
(struct elf_internal_sym): ... this.
(struct bfd_link_callbacks) <examine_strtab>: Take only a
symstrtab argument.
<ctf_new_symbol>: New.
<ctf_new_dynsym>: Likewise.
* ctf-api.h (struct ctf_link_sym) <st_symidx>: New.
<st_nameidx>: Likewise.
<st_nameidx_set>: Likewise.
(ctf_link_iter_symbol_f): Removed.
(ctf_link_shuffle_syms): Remove most parameters, just takes a
ctf_dict_t now.
(ctf_link_add_linker_symbol): New, split from
ctf_link_shuffle_syms.
* ctf.h (CTF_F_DYNSTR): New.
(CTF_F_MAX): Adjust.
ld/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ldelfgen.c (struct ctf_strsym_iter_cb_arg): Rename to...
(struct ctf_strtab_iter_cb_arg): ... this, changing fields:
<syms>: Remove.
<symcount>: Remove.
<symstrtab>: Rename to...
<strtab>: ... this.
(ldelf_ctf_strtab_iter_cb): Adjust.
(ldelf_ctf_symbols_iter_cb): Remove.
(ldelf_new_dynsym_for_ctf): New, tell libctf about a single
symbol.
(ldelf_examine_strtab_for_ctf): Rename to...
(ldelf_acquire_strings_for_ctf): ... this, only doing the strtab
portion and not symbols.
* ldelfgen.h: Adjust declarations accordingly.
* ldemul.c (ldemul_examine_strtab_for_ctf): Rename to...
(ldemul_acquire_strings_for_ctf): ... this.
(ldemul_new_dynsym_for_ctf): New.
* ldemul.h: Adjust declarations accordingly.
* ldlang.c (ldlang_ctf_apply_strsym): Rename to...
(ldlang_ctf_acquire_strings): ... this.
(ldlang_ctf_new_dynsym): New.
(lang_write_ctf): Call ldemul_new_dynsym_for_ctf with NULL to do
the actual symbol shuffle.
* ldlang.h (struct elf_strtab_hash): Adjust accordingly.
* ldmain.c (bfd_link_callbacks): Wire up new/renamed callbacks.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-link.c (ctf_link_shuffle_syms): Adjust.
(ctf_link_add_linker_symbol): New, unimplemented stub.
* libctf.ver: Add it.
* ctf-create.c (ctf_serialize): Set CTF_F_DYNSTR on newly-serialized
dicts.
* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Check for the flag: open the
symtab/strtab if not present, dynsym/dynstr otherwise.
* ctf-archive.c (ctf_arc_bufpreamble): New, get the preamble from
some arbitrary member of a CTF archive.
* ctf-impl.h (ctf_arc_bufpreamble): Declare it.
We were failing to report errors from ctf_archive_iter, which results in
silent early termination if (for example) one CTF archive member in a
.ctf section is corrupted and cannot be opened. Report the error in the
usual fashion instead.
binutils/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* objdump.c (dump_ctf): Report errors from ctf_archive_iter.
* readelf.c (dump_section_as_ctf): Likewise.
