When plugin_object_p is called by elf_link_is_defined_archive_symbol to
check if a symbol in archive is a real definition, set archive member
plugin_format to bfd_plugin_yes_unused to avoid including the unused LTO
archive members in linker output. When plugin_object_p is called as
known used, call plugin claim_file if plugin_format is bfd_plugin_unknown
or bfd_plugin_yes_unused.
To get the proper support for archives with LTO common symbols with GCC,
the GCC fix for
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116361
is needed.
bfd/
PR ld/32083
* archures.c (bfd_arch_get_compatible): Treat bfd_plugin_yes_unused
the same as bfd_plugin_yes.
* elflink.c (elf_link_is_defined_archive_symbol): Likewise.
* bfd.c (bfd_plugin_format): Add bfd_plugin_yes_unused.
* plugin.c (try_claim): Try claim_file_v2 first.
* bfd-in2.h: Regenerated.
ld/
PR ld/32083
* plugin.c (plugin_call_claim_file): Add an argument to return
if LDPT_REGISTER_CLAIM_FILE_HOOK_V2 is used.
(plugin_object_p): When KNOWN_USED is false, we call plugin
claim_file if plugin_format is bfd_plugin_unknown and set
plugin_format to bfd_plugin_yes_unused on LTO object. When
KNOWN_USED is true, we call plugin claim_file if plugin_format
is bfd_plugin_unknown or bfd_plugin_yes_unused.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
(cherry picked from commit a6f8fe0a9e)
When VexVVVV handling was re-worked, .insn broke: When an opcode
extension is in use, VexVVVV_DST needs using now, as ModR/M.reg is
already occupied, matching what c8866e3ec5 ("x86: Drop using
extension_opcode to encode vvvv register") did.
While adding (bad) POP2 forms, also slightly adjust existing ones:
No need to use XMM registers, and no need to specify %r8 when really
%rax is meant twice (EVEX.vvvv not really being the culprit there, or
else EVEX.V' would also have needed mentioning).
Add PR ld/32067 tests with the compiler driver since the -plugin option
is needed to trigger this --oformat binary bug.
PR ld/32067
* testsuite/ld-i386/i386.exp: Run PR ld/32067 test.
* testsuite/ld-x86-64/x86-64.exp: Likewise.
* testsuite/ld-i386/start.s: Add .note.GNU-stack section.
* testsuite/ld-x86-64/pr32067.s: New file.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
(cherry picked from commit 602f5cf7e3)
The direct fix for this segfault is to test for a non-NULL bed in
_bfd_elf_link_keep_memory, but also there isn't much point in running
code for LTO if the output is binary.
PR 32067
* elflink.c (_bfd_elf_link_keep_memory): Test for non-NULL bed.
(elf_link_add_object_symbols): Don't run the loop setting
non_ir_ref_regular if the output hash table is not ELF.
(cherry picked from commit ec8f5671b4)
The first operand is a general register, not an fp register;
the third operand is encoded into RS2, not RS3;
the second operand must match the destination operand.
Call unwrap_hash_lookup to restore the wrapper symbol check for standard
function since reference to standard function may not show up in LTO
symbol table:
[hjl@gnu-tgl-3 pr31956-3]$ nm foo.o
00000000 T main
U __real_malloc
00000000 T __wrap_malloc
[hjl@gnu-tgl-3 pr31956-3]$ lto-dump -list foo.o
Type Visibility Size Name
function default 0 malloc
function default 0 __real_malloc
function default 3 main
function default 5 __wrap_malloc
[hjl@gnu-tgl-3 pr31956-3]$ make
gcc -O2 -flto -Wall -c -o foo.o foo.c
gcc -Wl,--wrap=malloc -O2 -flto -Wall -o x foo.o
/usr/local/bin/ld: /tmp/ccsPW0a9.ltrans0.ltrans.o: in function `main':
<artificial>:(.text.startup+0xa): undefined reference to `__wrap_malloc'
collect2: error: ld returned 1 exit status
make: *** [Makefile:22: x] Error 1
[hjl@gnu-tgl-3 pr31956-3]$
Also add a test to verify that the unused wrapper is removed.
PR ld/31956
* plugin.c (get_symbols): Restore the wrapper symbol check for
standard function.
* testsuite/ld-plugin/lto.exp: Run the malloc test and the
unused test.
* testsuite/ld-plugin/pr31956c.c: New file.
* testsuite/ld-plugin/pr31956d.c: New file.
* testsuite/ld-plugin/pr31956d.d: New file.
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
(cherry picked from commit 3221c470f0)
strchr is redefined as a macro in /usr/include/bits/string.h on CentOS 6/7.
In this case, we may not use our CALL_UTIL macro for strchr.
Use __collector_strchr instead of "CALL_UTIL (strchr)".
gprofng/ChangeLog
2024-07-28 Vladimir Mezentsev <vladimir.mezentsev@oracle.com>
PR 32018
* libcollector/hwprofile.c (open_experiment): Use __collector_strchr.
This failed to properly byteswap its return value.
The ctf_archive format predates the idea of "just write natively and
flip on open", and byteswaps all over the place. It's too easy to
forget one. The next revision of the archive format (not versioned,
so we just tweak the magic number instead) should be native-endianned
like the dicts inside it are.
libctf/
* ctf-archive.c (ctf_archive_count): Byteswap return value.
If you asprintf something and then use it only as input to another asprintf,
it helps to free it afterwards.
libctf/
* ctf-dump.c (ctf_dump_header): Free the flagstr after use.
(ctf_dump): Make a NULL return slightly clearer.
A bug in ctf_dtd_delete led to refs in the string table to the
names of non-root-visible types not being removed when the DTD
was. This seems harmless, but actually it would lead to a write
down a pointer into freed memory if such a type was ctf_rollback()ed
over and then the dict was serialized (updating all the refs as the
strtab was serialized in turn).
Bug introduced in commit fe4c2d5563
("libctf: create: non-root-visible types should not appear in name tables")
which is included in binutils 2.35.
libctf/
* ctf-create.c (ctf_dtd_delete): Remove refs for all types
with names, not just root-visible ones.
The dict and archive opening code in libctf is somewhat unusual, because
unlike everything else, it cannot report errors by setting an error on the
dict, because in case of error there isn't one. They get passed an error
integer pointer that is set on error instead.
Inside ctf_bufopen this is implemented by calling ctf_set_open_errno and
passing it a positive error value. In turn this means that most things it
calls (including init_static_types) return zero on success and a *positive*
ECTF_* or errno value on error.
This trickles down to ctf_dynhash_insert_type, which is used by
init_static_types to add newly-detected types to the name tables. This was
returning the error value it received from a variety of functions without
alteration. ctf_dynhash_insert conformed to this contract by returning a
positive value on error (usually OOM), which is unfortunate for multiple
reasons:
- ctf_dynset_insert returns a *negative* value
- ctf_dynhash_insert and ctf_dynset_insert don't take an fp, so the value
they return is turned into the errno, so it had better be right, callers
don't just check for != 0 here
- more or less every single caller of ctf_dyn*_insert in libctf other than
ctf_dynhash_insert_type (and there are a *lot*, mostly in the
deduplicator) assumes that ctf_dynhash_insert returns a negative value
on error, even though it doesn't. In practice the only possible error is
OOM, but if OOM does happen we end up with a nonsense error value.
The simplest fix for this seems to be to make ctf_dynhash_insert and
ctf_dynset_insert conform to the usual interface contract: negative
values are errors. This in turn means that ctf_dynhash_insert_type
needs to change: let's make it consistent too, returning a negative
value on error, putting the error on the fp in non-negated form.
init_static_types_internal adapts to this by negating the error return from
ctf_dynhash_insert_type, so the value handed back to ctf_bufopen is still
positive: the new call site in ctf_track_enumerator does not need to change.
(The existing tests for this reliably detect when I get it wrong.
I know, because they did.)
libctf/
* ctf-hash.c (ctf_dynhash_insert): Negate return value.
(ctf_dynhash_insert_type): Set de-negated error on the dict:
return negated error.
* ctf-open.c (init_static_types_internal): Adapt to this change.
The recent change to detect duplicate enum values and return ECTF_DUPLICATE
when found turns out to perturb a great many callers. In particular, the
pahole-created kernel BTF has the same problem we historically did, and
gleefully emits duplicated enum constants in profusion. Handling the
resulting duplicate errors from BTF -> CTF converters reasonably is
unreasonably difficult (it amounts to forcing them to skip some types or
reimplement the deduplicator).
So let's step back a bit. What we care about mostly is that the
deduplicator treat enums with conflicting enumeration constants as
conflicting types: programs that want to look up enumeration constant ->
value mappings using the new APIs to do so might well want the same checks
to apply to any ctf_add_* operations they carry out (and since they're
*using* the new APIs, added at the same time as this restriction was
imposed, there is likely to be no negative consequence of this).
So we want some way to allow processes that know about duplicate detection
to opt into it, while allowing everyone else to stay clear of it: but we
want ctf_link to get this behaviour even if its caller has opted out.
So add a new concept to the API: dict-wide CTF flags, set via
ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags
but simple arbitrary integers and an on/off value, stored in an unspecified
manner (the one current flag, we translate into an LCTF_* flag value in the
internal ctf_dict ctf_flags word). If you pass in an invalid flag or value
you get a new ECTF_BADFLAG error, so the caller can easily tell whether
flags added in future are valid with a particular libctf or not.
We check this flag in ctf_add_enumerator, and set it around the link
(including on child per-CU dicts). The newish enumerator-iteration test is
souped up to check the semantics of the flag as well.
The fact that the flag can be set and unset at any time has curious
consequences. You can unset the flag, insert a pile of duplicates, then set
it and expect the new duplicates to be detected, not only by
ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now
have to maintain the ctf_names and conflicting_enums enum-duplication
tracking as new enums are added, not purely as the dict is opened.
Move that code out of init_static_types_internal and into a new
ctf_track_enumerator function that addition can also call.
(None of this affects the file format or serialization machinery, which has
to be able to handle duplicate enumeration constants no matter what.)
include/
* ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New.
(ECTF_NERR): Update.
(CTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_dict_set_flag): New function.
(ctf_dict_get_flag): Likewise.
libctf/
* ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag.
(ctf_track_enumerator): Declare.
* ctf-dedup.c (ctf_dedup_emit_type): Set it.
* ctf-link.c (ctf_create_per_cu): Likewise.
(ctf_link_deduplicating_per_cu): Likewise.
(ctf_link): Likewise.
(ctf_link_write): Likewise.
* ctf-subr.c (ctf_dict_set_flag): New function.
(ctf_dict_get_flag): New function.
* ctf-open.c (init_static_types_internal): Move enum tracking to...
* ctf-create.c (ctf_track_enumerator): ... this new function.
(ctf_add_enumerator): Call it.
* libctf.ver: Add the new functions.
* testsuite/libctf-lookup/enumerator-iteration.c: Test them.
We set this flag at the top of ctf_link_write (to tell ctf_serialize, way
down under the archive file writing functions, to do the various link- time
serialization things like symbol filtering and the like), but we never
remember to clear it except on error. This is probably bad if you want to
serialize the dict yourself directly in the future after linking it (which
is... definitely a *possible* use of the API, if rather strange).
libctf/
* ctf-link.c (ctf_link_write): Clear LCTF_LINKING before exit.
libctf's dynsets are a straight wrapper around libiberty hashtab, storing
the key directly in the hashtab slot. However, we'd often like to be able
to store 0 and 1 (HTAB_EMPTY_ENTRY and HTAB_DELETED_ENTRY) in there, so we
move them out of the way and replace them with huge unlikely values
instead. Unfortunately we failed to do this replacement in one place, so
insertion of 0 or 1 ended up misinforming the hashtab machinery that an
entry was empty or deleted when it wasn't.
libctf/
* ctf-hash.c (ctf_dynset_insert): Call key_to_internal properly.
Drop an unnecessary variable, and fix a buggy comment.
No effect on generated code.
libctf/
* ctf-dedup.c (ctf_dedup_detect_name_ambiguity): Drop unnecessary
variable.
(ctf_dedup_rwalk_output_mapping): Fix comment.
This erorr doesn't just indicate that there is no parent dictionary
(that's routine, and true of all dicts that are parents themselves)
but that a parent is *needed* but wasn't found.
include/
* ctf-api.h (_CTF_ERRORS) [ECTF_NOPARENT]: Improve error message.
ld/
* testsuite/ld-ctf/diag-parname.d: Adjust.
Commit 483546ce4f ("libctf: make ctf_serialize() actually serialize")
accidentally broke dict compression. There were two bugs:
- ctf_arc_write_one_ctf was still making its own decision about
whether to compress the dict via direct ctf_size comparison, which is
unfortunate because now that it no longer calls ctf_serialize itself,
ctf_size is always zero when it does this: it should let the writing
functions decide on the threshold, which they contain code to do which is
simply not used for lack of one trivial wrapper to write to an fd and
also provide a compression threshold
- ctf_write_mem, the function underlying all writing as of the commit
above, was calling zlib's compressBound and avoiding compression if this
returned a value larger than the input. Unfortunately compressBound does
not do a trial compression and determine whether the result is
compressible: it just adds zlib header sizes to the value passed in, so
our test would *always* have concluded that the value was incompressible!
Avoid by simply always compressing if the raw size is larger than the
threshold: zlib is quite clever enough to avoid actually compressing
if the data is incompressible.
Add a testcase for this.
libctf/
* ctf-impl.h (ctf_write_thresholded): New...
* ctf-serialize.c (ctf_write_thresholded): ... defined here,
a wrapper around...
(ctf_write_mem): ... this. Don't check compressibility.
(ctf_compress_write): Reimplement as a ctf_write_thresholded
wrapper.
(ctf_write): Likewise.
* ctf-archive.c (arc_write_one_ctf): Just call
ctf_write_thresholded rather than trying to work out whether
to compress.
* testsuite/libctf-writable/ctf-compressed.*: New test.
If you deduplicate non-root-visible types, the resulting type should still
be non-root-visible! We were promoting all such types to root-visible, and
re-demoting them only if their names collided (which might happen on
cu-mapped links if multiple compilation units with conflicting types are
fused into one child dict).
This "worked" before now, in that linking at least didn't fail (if you don't
mind having your non-root flag value destroyed if you're adding
non-root-visible types), but now that conflicting enumerators cause their
containing enums to become conflicted (enums which might have *different
names*), this caused the linker to crash when it hit two enumerators with
conflicting values.
Not testable in ld because cu-mapped links are not exposed to ld, but can be
tested via direct creation of libraries and calls to ctf_link directly.
(This also tests the ctf_dump non-root type printout, which before now
was untested.)
libctf/
* ctf-dedup.c (ctf_dedup_emit_type): Non-root-visible input types
should be emitted as non-root-visible output types.
* testsuite/libctf-writable/ctf-nonroot-linking.c: New test.
* testsuite/libctf-writable/ctf-nonroot-linking.lk: New test.
The flag test when dumping non-root-visible tyeps was doubly wrong: the
flags word is a *bitfield* containing CTF_ADD_ROOT as one possible
value, so needs | and & testing, not just ==, and CTF_ADD_NONROOT is 0,
so cannot be tested for this way: one must check for the non-presence of
CTF_ADD_ROOT.
libctf/
* ctf-dump.c (ctf_dump_format_type): Fix non-root flag test.
In commit 149ce5c263 we introduced the concept of "movable" refs,
which are refs that can be moved in batches, to let us maintain valid ref
lists even when adding refs to blocks of memory that can be realloced (which
is any type containing a vlen which can expand, like names contained within
enum or struct members). Movable refs need a backpointer to the movable
refs dynhash for this dict; since non-movable refs are very common, we tried
to save memory by having a slightly bigger struct for moveable refs with a
backpointer in it, and casting appropriately, indicating which sort of ref
we were dealing with via a flag on the atom.
Unfortunately this doesn't work reliably, because you can perfectly well
have a string ("foo", say) which has both non-movable refs (say, an external
symbol and a variable name) and movable refs (say, a structure member name)
to the same atom. Indicate which struct we're dealing with with an atom
flag and suddenly you're casting a ctf_str_atom_ref to a
ctf_str_atom_ref_movable (which is bigger) and dereferencing random memory
off the end of it and interpreting it as a backpointer to the movable refs
dynhash. This is unlikely to work well.
So bite the bullet and split refs into two separate lists, one for movable
refs, one for immovable refs. It means some annoying code duplication, but
there's not very much of it, and it means we can keep the movable refs
hashtab (which in turn means we don't have to do linear searches to find all
relevant refs when moving refs, which in turn means that
structure/union/enum member additions remain amortized O(n) time, not
O(n^2).
Callers can now purge movable and non-movable refs independently of each
other. We don't use this yet, but a use is coming.
libctf/
* ctf-impl.h (CTF_STR_ATOM_MOVABLE): Delete.
(struct ctf_str_atom) [csa_movable_refs]: New.
(struct ctf_dict): Adjust comment.
(ctf_str_purge_refs): Add MOVABLE arg.
* ctf-string.c (ctf_str_purge_movable_atom_refs): Split out of...
(ctf_str_purge_atom_refs): ... this.
(ctf_str_free_atom): Call it.
(ctf_str_purge_one_atom_refs): Likewise.
(aref_create): Adjust accordingly.
(ctf_str_move_refs): Likewise.
(ctf_str_remove_ref): Remove movable refs too, including
deleting the ref from ctf_str_movable_refs.
(ctf_str_purge_refs): Add MOVABLE arg.
(ctf_str_update_refs): Update movable refs.
(ctf_str_write_strtab): Check, and purge, movable refs.
