Wiki - https://fedoraproject.org/wiki/Changes/StaticLibraryPreserveDebuginfo
Discussion thread -
https://discussion.fedoraproject.org/t/f43-change-proposal-lets-preserve-debuginfo-in-static-libraries-system-wide/157325
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
RPMs supplying static libraries should build with useful & preserved debuginfo.
Let's stop stripping them by default, and rely on new debugedit to process
static library debuginfo in a way that's useful for full debugging.
See also: [[Changes/StaticLibraryDebuginfo|an earlier attempt]] at this idea.
== Owner ==
* Name: [[User:Fche| Frank Ch. Eigler]]
* Adjutants: [[User:Mjw| Mark Wielaard]] [[User:Fweimer | Florian Weimer]]
* Email: fche@redhat.com
== Detailed Description ==
The current rpmbuild environment includes a post-build process that includes:
<pre>
%__os_install_post \
...
%{!?__debug_package:\
%{?__brp_strip} \
%{?__brp_strip_comment_note} \
} \
%{?__brp_strip_lto} \
%{?__brp_strip_static_archive} \
...
</pre>
The last part, __brp_strip_static_archive, removes all debuginfo from
any static archive (.a) files that may be packaged into foo-static
subrpms. These are not stripped into a separate -debuginfo file
(there's no tech for that); they are simply removed. This makes those
functions permanently undebuggable.
This was undesirable for several packages, so they experimented with
disabling the stripping step by redefining the
__brp_strip_static_archive macro to nothing. This would allow the
original object file / archive debuginfo to proceed into RPMs.
However, these files were not subject to source-file rewriting
normally performed by find-debuginfo.sh / debugedit, which let random
build-time directory names propagate into those debuginfo files.
(Normally, debugedit rewrites them to standard
/usr/src/debug/PACKGE/... paths and arranges to ship them into
debugsource RPMs.) This made their source files unpackaged and/or
inaccessible. But now debugedit master (coming to release 5.2
shortly) can process .a files, rewriting source path names and
collecting source files.
Fedora is finally in a position to '''preserve all static library
debuginfo''', and also arrange for distribution of their source code.
This Change proposes that this be done '''by default''':
* adopt the forthcoming version of debugedit 5.2
* change the %__os_install_post rpm macro to avoid the
__brp_strip_static_archive step
* offer an opt-in option for packages to retain stripping
We performed test mass-rebuilds of a number of packages, including all
fedora 42 packages that include "static" subrpms. There were no
build-breaking concerns.
We measured file size differences before/after to quantify the size
impact. Some interesting corner cases:
* xz is a typical case: small C library
* fftw is a worst-case scenario: template meta-programming C++ causes
huge debuginfo.
* gcc is good case: includes small C static-library, linked into many
fedora binaries. For this test, we removed the manual strip of libgcc
and other static libraries done in gcc.spec.
* llvm is also C++ and relatively debuginfo intensive
{| class="wikitable"
|-
! package or file !! old size !! new size !! delta !! notes
|-
| xz-static-5.8.1-2.fc42.x86_64.rpm || 119173 || 506301 || 4.2x bigger
|-
| liblzma.a || 392948 || 2172276 || 5.6x bigger
|-
| fftw-static-3.3.10-13.fc42.x86_64.rpm || 2904595 || 34036502 || 11x bigger
|-
| libfftw3f.a || 6976592 || 104539928 || 15x bigger
|-
| gcc-15.1.1-2.fc42.x86_64.rpm || 41440269 || 45581745 || +9% ||
includes libgcc.a
|-
| libgcc.a || 3349468 || 8271084 || 2.5x bigger || 64-bit
|-
| gcc debugsource rpm || 8243 || 10582 || +2339 || number of files
(mostly offload libraries)
|-
| llvm 20.1.7 lib64/libMLIRArithDialect.a || 6441876 || 45053964 || 7x bigger
|}
Note that figures like "10x bigger" should be taken in context. These
static libraries / RPMs form a tiny fraction of the entire Fedora
distro. Binary packages are unaffected, and their debuginfo would be
only trivially affected (e.g., by including snippets of libgcc.a).
Preserving debuginfo in the small number of static libraries should
not dramatically grow the overall distro. However, we don't have a
complete mass-build-based measurement on the overall impact.
== Feedback ==
In previous discussions over the last years, some concerns have come up:
==== Larger .a files and static RPMs ====
Static archive files become 2-12 times bigger after they carry
debuginfo content, and cannot be compressed by DWZ. (Hypothetically,
they could be processed with
objcopy --compress-debug-sections to win back some of this space.)
However, subrpms carrying those larger .a files are '''are compressed
at the rpm level''', so the storage / transport penalty of these
subrpms can be reasonable. Downstream packages that use static
libraries can be fully compressed by DWZ in the usual way. A small
number of very common libraries (e.g., libgcc.a) with newly preserved
debuginfo would enlarge many other Fedora -debuginfo packages, but
only by a trivial relative amount.
We believe the RPM size increases for the ~300ish
static-library-bearing packages are bearable to Fedora infrastructure.
For those who develop against static libraries, we expect the larger
disk footprint of those libraries is acceptable, in exchange for the
value of getting full debuggability of the finished binaries.
==== Alternative idea of moving debuginfo-carrying static .a files
into a -debuginfo subrpm ====
The gcc.spec has an option _enable_debug_packages, which if set, moves
original unstripped libFOO.a files under /usr/lib/debug/..., thus
getting them packaged into a gcc-debuginfo subrpm. This macro is not
enabled in Fedora builds, but the logic is consistent with ideas
expressed [https://bugzilla.redhat.com/show_bug.cgi?id=1395280|in
rhbz1395280].
We believe these measures are unnecessary. Users of -static libraries
should not need to use cumbersome custom -L/usr/lib/debug/.... paths
nor BuildRequire debuginfo RPMs just to make their programs
debuggable. Keeping debuginfo data in the original .a files & subrpms
is simple and effective.
==== Non-RPM developers getting some debuginfo even without gcc -g ====
Since basic runtime libraries like crt, libgcc would now carry
debuginfo, that data would be linked into binaries produced with "gcc
foo.c", even if "-g" was not specified. This is a relatively small
amount: x86_64 libgcc.a without -g is about 3MB, with -g is about 8MB,
and that's for all the individual runtime .o files. A developer
concerned about this could opt to strip his/her binaries after
linking. (Note that the toolchain has never guaranteed that there
would be zero debuginfo in a binary compiled/linked without -g.)
We believe this is not a serious impediment, when developers can work
around the distro change with classic tooling.
==== What about source code ====
In the status quo, a static library that is somehow packaged
unstripped ends up with broken DWARF source references, and thus no
files in the -debugsource. This change actually solves the "what
about the source" problem, because new debugedit (find-debuginfo.sh)
rewrites the source file names in the preserved DWARF, and arranges
for those source files to be included in the FOO-debugsource subrpm.
In the new scheme, debuginfod can automatically fetch the static
library sources!
== Benefit to Fedora ==
Binaries that link Fedora static libraries would become fully
debuggable. Crashed programs that include static libraries would
become fully backtraceable.
== Scope ==
* Proposal owners:
** Update debugedit rpm to bring in static-archive support
** Update redhat-rpm-config to disable __brp_strip_static_archive by
default in /usr/lib/rpm/redhat/macros, details tbd
** Identify packages that perform manual static-library stripping now
(e.g., gcc), file bugs to request discontinuation.
* Other developers:
** foo-static packagers who embrace the change: do nothing
** foo-static packagers who prefer to keep rpmbuild-stripped .a status
quo: activate __brp_strip_static_archive by tbd rpm macro
** foo-static packagers who want to keep manually stripping their own
.a: do nothing
** foo-static packagers who no longer want to manually strip their own
.a: strip that part from the .spec files
** Non-package developers linking against foo-static who embrace the
change: do nothing
** Non-package developers linking against foo-static who want stripped
binaries: strip binaries after linking
* Release engineering: [https://pagure.io/releng/issue/12799 #12799]
to note somewhat larger RPMs, which would be generated during a
mass-rebuild.
* Policies and guidelines: No formal change is necessary, as the
packaging guidelines do not specify whether static libraries do or do
not carry debuginfo.
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy: N/A
== Upgrade/compatibility impact ==
The only upgrade/compat impact appears to be larger disk footprint if
updating from previous -static subrpms.
== Early Testing (Optional) ==
Do you require 'QA Blueprint' support? N.
== How To Test ==
* Build RPMs or non-RPM binaries that link against -static libraries.
* Observe they function just like before.
* Run readelf -w on the static .a files. It should print lots of
.debug_info stuff (not just .eh_frame information)
* Run under debugger. Observe that the portions that came from
-static libraries now have full debuginfo and source code.
== User Experience ==
Ordinary Fedora user experience is unaffected, since this is a
developer rather than user change. Users of packages that incorporate
static library elements would, if the program crashed, be able to get
complete traceback information to assist with problem reporting.
Developers of static-library related packages would need to accept
somewhat larger disk footprint for the debuginfo-carrying files, and
would benefit from the thorough debuggability.
== Dependencies ==
This change is not blocked on others, nor on new upstream work. The
precise mechanism by which the __brp_strip_static_archive macro would
be disabled by default is tbd. We'd provide a way of forcing
stripping back on for packages who prefer the status quo.
The changes should land before the mass-rebuild for full effectiveness.
== Contingency Plan ==
* Contingency mechanism: If debugedit changes were to somehow break
builds for this or unrelated reasons, we can roll that back
immediately. If the rpm-side macro changes cannot be committed in
time, then status quo (stripping .a files) will happen. Packages will
be able to force static-library stripping for themselves.
* Contingency deadline: Beta freeze. Even if the rpmbuild-default
reconfiguration were to miss the mass-rebuild, subsequently rebuilt
packages could gradually take advantage.
* Blocks release? No.
== Documentation ==
End-user documentation should not be necessary.
== Release Notes ==
The release notes should mention that developers who do not want
debugging information in statically linked applications should link
with the -s flag or run the strip command after linking; analogous
instructions for RPM builders tbd.
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
--
_______________________________________________
devel-announce mailing list -- devel-announce@lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
No comments:
Post a Comment