Tuesday, September 24, 2024

F42 Change Proposal: Setuptools 74+ (System-Wide)

Wiki - https://fedoraproject.org/wiki/Changes/Setuptools_74%2B
Discussion Thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-setuptools-74-system-wide/132052

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.


== Summary ==

Update to a new upstream release of {{package|python-setuptools}} that
is not completely compatible with previous releases. Most notably,
version 72.0.0 removed support for the `setup.py test` command
(deprecated for 5 years). This is a breaking change and Fedora
packages that use the `setup.py test` command during the build need to
be adapted to use a different test runner, such as unittest, pytest,
etc.
There might be other breaking changes. We estimate 142 to 196 Fedora
Rawhide packages to fail to build due to this upgrade (as of Sep
2024).

== Owner ==

* Name: [[User:Churchyard|Miro Hrončok]]

* Email: mhroncok@redhat.com



== Detailed Description ==

Setuptools is a package development process library designed to
facilitate packaging Python projects by enhancing the former Python
standard library distutils (distribution utilities). It changes fast
and it often introduces breaking changes such as the removal of the
`setup.py test` command in version 72.0.0 which was deprecated since
2019.

If your Fedora package still uses the `setup.py test` command
(deprecated for 5 years), please update to `%pytest`, `%tox`,
`%{python3} -m unittest`, etc.

List of packages that will likely fail to build, because they use the
removed `setup.py test` command in the specfile:

$ rg '^[^-#].+(\b|_)setup(\.py|\})\s+test\b' -l | sed -E
's|^rpm-specs/(.+)\.spec$|\1|' | sort
adapt
ansible-inventory-grapher
binwalk
btrfs-sxbackup
calibre
cepces
classification-banner
ddiskit
fprettify
fros
grammalecte
gtts
gtts-token
hddfancontrol
hgsvn
ioc-writer
isrcsubmit
kitty
lazygal
libvirt-test-API
mopidy
mopidy-mpd
nototools
past-time
petsc
pycairo
PyGreSQL
pyjokes
pyowm
python3-lxc
python3-mallard-ducktype
python-aioresponses
python-apsw
python-auth-credential
python-autopage
python-binaryornot
python-booleanoperations
python-bracex
python-can
python-case
python-casttube
python-cccolutils
python-chai
python-cliff
python-collada
python-colorclass
python-CommonMark
python-cpuinfo
python-cwcwidth
python-cx-oracle
python-decorator
python-dfdatetime
python-dictdiffer
python-digitalocean
python-dirq
python-distroinfo
python-django-contrib-comments
python-django-prometheus
python-dkimpy
python-enlighten
python-epub
python-events
python-fedmsg-meta-fedora-infrastructure
python-flask-wtf-decorators
python-flask-xml-rpc
python-flufl-i18n
python-fontMath
python-freeipa
python-gammu
python-genshi
python-genty
python-geojson
python-gerritlib
python-glyphsLib
python-gmqtt
python-hgapi
python-html5-parser
python-httpsig-cffi
python-hypothesis-fspaths
python-imagehash
python-ipgetter2
python-iptools
python-jack-client
python-jcconv
python-jinja2_pluralize
python-jinja2-time
python-journal-brief
python-json-minify
python-jsonpath-ng
python-jsonpath-rw
python-klusta
python-launchpadlib
python-lazr-uri
python-libusb1
python-lupa
python-markups
python-messaging
python-moksha-hub
python-murano-pkg-check
python-mutatormath
python-mwclient
python-neurosynth
python-num2words
python-oletools
python-openidc-client
python-oslo-context
python-oslo-metrics
python-os-service-types
python-ouimeaux
python-pdfkit
python-phonenumbers
python-poyo
python-pyasn1
python-pyclipper
python-pycryptodomex
python-pydispatcher
python-pyfim
python-pymediainfo
python-pyramid_sawing
python-pyramid-tm
python-pysignals
python-pysocks
python-pystray
python-pyte
python-pytest-fixture-config
python-pytest-flakes
python-pytest-runner
python-pytimeparse
python-repoze-lru
python-rmtest
python-rsd-lib
python-scripttest
python-serpy
python-shodan
python-simplegeneric
python-simplevisor
python-singledispatch
python-slacker
python-smartypants
python-sphinxcontrib-autoprogram
python-sphinxcontrib-trio
python-sql
python-ssdeep
python-statsd
python-svg-path
python-testscenarios
python-textile
python-textparser
python-tgscheduler
python-tidy
python-timeout-decorator
python-tosca-parser
python-translationstring
python-tree-format
python-typing-inspect
python-volatile
python-wadllib
python-whoosh
python-wsgi_intercept
python-www-authenticate
python-xmp-toolkit
python-xunitparser
python-yapsy
python-yaql
python-zc-customdoctests
python-zc-lockfile
python-zope-deprecation
python-zope-testing
python-zstd
sepolicy_analysis
tpm2-pkcs11
transifex-client

List of packages that actually failed in our
[https://copr.fedorainfracloud.org/coprs/churchyard/setuptools-74/
testing copr] but did not fail in a control copr:

ansible-inventory-grapher
btrfs-sxbackup
cepces
classification-banner
ddiskit
flent
fprettify
fros
gpgme
grammalecte
grpc
hddfancontrol
ioc-writer
lazygal
mopidy
mopidy-mpd
mozjs102
past-time
PyGreSQL
pyproject-rpm-macros
python3-lxc
python3-mallard-ducktype
python-aioresponses
python-binaryornot
python-bracex
python-case
python-cccolutils
python-chai
python-cliff
python-collada
python-colorclass
python-cpuinfo
python-cwcwidth
python-daemon
python-decorator
python-dfdatetime
python-digitalocean
python-distroinfo
python-django-contrib-comments
python-django-prometheus
python-epub
python-events
python-flake8-class-newline
python-flask-wtf-decorators
python-flask-xml-rpc
python-flufl-i18n
python-fontMath
python-freeipa
python-genshi
python-genty
python-geojson
python-hypothesis-fspaths
python-imagehash
python-iptools
python-jack-client
python-jcconv
python-jinja2_pluralize
python-jinja2-time
python-json-minify
python-jsonpath-ng
python-jsonpath-rw
python-jupyter-packaging
python-klusta
python-launchpadlib
python-lazr-uri
python-libusb1
python-lupa
python-markups
python-moksha-hub
python-mutatormath
python-mwclient
python-nose
python-oletools
python-openidc-client
python-oslo-context
python-oslo-metrics
python-ouimeaux
python-parsel
python-pdfkit
python-phonenumbers
python-poyo
python-pyasn1
python-pycdio
python-pyclipper
python-pydispatcher
python-pyemd
python-pygmtools
python-pymediainfo
python-pyproject-api
python-pyproject-hooks
python-pyramid_sawing
python-pyramid-tm
python-pysignals
python-pysrt
python-pytaglib
python-pyte
python-pytest-fixture-config
python-pytest-flakes
python-pytest-runner
python-pytimeparse
python-pyvmomi
python-repoze-lru
python-rmtest
python-robot-detection
python-scikit-build
python-scripttest
python-serpy
python-simplegeneric
python-singledispatch
python-slacker
python-sphinxcontrib-autoprogram
python-sphinxcontrib-trio
python-sql
python-ssdeep
python-statsd
python-svg-path
python-testscenarios
python-textparser
python-tgscheduler
python-timeout-decorator
python-translationstring
python-tree-format
python-trml2pdf
python-typing-inspect
python-virtualenv
python-volatile
python-wadllib
python-whoosh
python-wsgi_intercept
python-www-authenticate
python-xmp-toolkit
python-xunitparser
python-yaql
python-zc-customdoctests
python-zc-lockfile
python-zope-deprecation
python-zope-testing
python-zope-testrunner
python-zstd
sentencepiece
tpm2-pkcs11
transifex-client

Both lists together, by maintainers:


abbra python-whoosh
abompard python-flufl-i18n python-jinja2_pluralize
python-pyramid_sawing python-pysocks python-translationstring
python-zope-deprecation python-zope-testing python-zope-testrunner
adamwill python-mwclient
adrian flent
aekoroglu python-django-prometheus python-pytaglib python-repoze-lru
python-serpy python-xunitparser
ajax binwalk
alexl pycairo
amigadave isrcsubmit python3-mallard-ducktype
amoralej python-rsd-lib python-yaql
ankursinha python-klusta python-num2words python-pyemd python-pyfim
apevec python-oslo-context python-testscenarios python-wsgi_intercept
asn cepces
athoscr python-booleanoperations python-epub python-fontMath
python-glyphsLib python-mutatormath python-pyclipper
atim kitty
aviso python-enlighten
besser82 python-jcconv
caolanm pycairo
carlwgeorge python-pyproject-hooks
chandankumar python-oslo-context python-tosca-parser python-wsgi_intercept
chedi python-jinja2-time python-poyo python-pytest-flakes
python-tree-format
chkr calibre
churchyard pyproject-rpm-macros python-genty python-launchpadlib
python-lazr-uri python-nose python-scripttest python-simplegeneric
python-virtualenv python-wadllib
cicku python-apsw
ckalina ddiskit
cottsay python-flake8-class-newline
cqi python-django-contrib-comments
cstratak python-virtualenv
dcallagh python-phonenumbers python-www-authenticate
dcavalca tpm2-pkcs11
dchen cepces
defolos grpc
dfateyev python-apsw
dherrera python-pytest-runner
dminer python-zstd
dshea python-jinja2_pluralize
echevemaster python-pydispatcher transifex-client
eeickmeyer python-jack-client
esyr ddiskit
fab ioc-writer past-time python-cx-oracle python-dfdatetime
python-geojson python-gmqtt python-imagehash python-ipgetter2
python-jsonpath-ng python-jsonpath-rw python-shodan python-slacker
python-textparser python-zc-customdoctests
fale python-decorator python-pyvmomi
fed500 python-pygmtools
fedepell python-jinja2-time
filiperosset hddfancontrol
fjanus PyGreSQL
frantisekz mozjs102
frostyx python-flask-wtf-decorators
fschwarz python-digitalocean python-genshi
girst mopidy mopidy-mpd
gsauthof python-aioresponses
gui1ty python-moksha-hub
hhorak PyGreSQL
hiwkby python-oslo-metrics
hobbes1069 python-collada
hvad python-events
ignatenkobrain python-pytest-flakes python-simplegeneric
jamatos python-markups python-smartypants
jcapitao python-cliff python-distroinfo python-timeout-decorator
jcaratzas python-repoze-lru python-simplegeneric
python-singledispatch python-sphinxcontrib-trio python-zc-lockfile
jjelen tpm2-pkcs11
jkucera gpgme
jmontleon python-dictdiffer
jonathanspw kitty python-freeipa python-parsel python-pystray
python-pytest-runner python-zstd
jonny python-libusb1
jorti python-pysrt
jpena python-murano-pkg-check python-tosca-parser
jruzicka python-tosca-parser
jujens python-CommonMark python-hgapi
jussilehtola fprettify
kalev pycairo
kevin calibre python-chai python-decorator python-html5-parser
python-pycdio python-pyramid-tm python-pysocks
python-pytest-fixture-config python-svg-path python-tgscheduler
kni python-ouimeaux python-pysignals
ktdreyer python-testscenarios
kushal python-daemon
laxathom python-gammu
lbalhar python-jupyter-packaging python-pyproject-api
python-pyproject-hooks python-virtualenv
lbazan python-neurosynth transifex-client
lberk python-rmtest
lbrabec python-xunitparser
lcons python-auth-credential python-dirq python-messaging
python-simplevisor
lecris python-scikit-build
limb python-typing-inspect
lnie libvirt-test-API
lyessaadi python-yapsy
maha python-sphinxcontrib-autoprogram
melmorabity grammalecte python-pycryptodomex
mfabian nototools
mhayden python-virtualenv
mhlavink gpgme
mohanboddu python-openidc-client
mrc0mmand gpgme
mrunge python-pytimeparse python-singledispatch
python-testscenarios python-virtualenv
nathans python-rmtest
nb python-digitalocean
neil grpc python-gerritlib
ngompa python-case python-dkimpy python-freeipa python-pystray
python-robot-detection python-zstd
opohorel python-launchpadlib python-lazr-uri python-wadllib
orion python-virtualenv python-zope-testing
orphan python-case python-fedmsg-meta-fedora-infrastructure
python-hypothesis-fspaths python-iptools
osloup PyGreSQL
pbrobinson adapt gtts gtts-token pyjokes python-can python-casttube tpm2-pkcs11
peter python-libusb1
pingou python-binaryornot python-case python-chai python-flask-xml-rpc
pkubat PyGreSQL
pnemade ansible-inventory-grapher python-bracex python-cpuinfo
poros ddiskit
praiskup PyGreSQL
puiterwijk python-cccolutils
pwhalen pyjokes pyowm
pwu nototools
qulogic python-json-minify python-xmp-toolkit
radez python-zc-lockfile python-zope-testing
raphgro python-pdfkit python-pyvmomi
rathann lazygal
rcallicotte python-pyvmomi
rcritten cepces python-pyasn1
rdieter gpgme python-mwclient
rebus python-colorclass python-oletools
rhughes pycairo
robert python-oletools
rstrode pycairo
sagitter petsc
salimma kitty python-lupa python-pytest-runner
python-robot-detection python-volatile python-zope-testrunner
python-zstd
sergiomb python-gammu
sgallagh python-whoosh
sharkcz python-sql
shoracek tpm2-pkcs11
slinabery python-jsonpath-rw
solopasha kitty
spichugi python-pyasn1
suve python-ssdeep
swt2c binwalk
tdecacqu python-statsd
terjeros hgsvn python-cwcwidth python-httpsig-cffi python-pyte python-tidy
tflink python-xunitparser
thebeanogamer classification-banner
thm python-sphinxcontrib-trio python-textile python3-lxc
thofmann btrfs-sxbackup
thrnciar python-scikit-build
tieugene python-trml2pdf
tohojo flent
tomspur python-daemon python-simplegeneric
topazus sentencepiece
trawets fros
ttorling python-pysocks
twaugh python-journal-brief
vascom python-pymediainfo
vmojzis sepolicy_analysis
vtrefny python-pytimeparse
xavierb python-daemon
ykarel python-os-service-types
zaneb python-autopage
zawertun kitty
zbyszek calibre
zsun ddiskit

We will be slowly crushing the failures and opening bugzillas before
we actually update setuptools.

== Feedback ==

python-devel thread from August 2024:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/YOC5527SRKNY2XG6CMS3KNGJ54O7SMXI/

Listed ~200 packages, ~30 were fixed in a month. Thank you, maintainers!

== Benefit to Fedora ==

Fedora offers cutting-edge technologies for Python and setuptools is
one of the most important Python packages. By doing this update we
bring users the latest features that setuptools offers and also we
enable Fedora 42+ to receive future updates as well.

== Scope ==
* Proposal owners: update python-setuptools to 74.x.x or newer, open
bugzillas, provide help

* Other developers: report problems to the upstream and backport
patches to the affected packages. The impact can be tested using
[https://copr.fedorainfracloud.org/coprs/churchyard/setuptools-74/
copr repository where setuptools 74+ has been built].
* Release engineering: [https://pagure.io/releng/issue/12323 #12323]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy: First

== Upgrade/compatibility impact ==

There is a clean upgrade path from current version to
python-setuptools 69.x.x. Fedora users using RPM-packaged setuptools
will use setuptools 74+ by default.

== Early Testing (Optional) ==

Do you require 'QA Blueprint' support? N


== How To Test ==

* Find the package you want to update in the
[https://copr.fedorainfracloud.org/coprs/churchyard/setuptools-74/
copr repository] and check the build logs to determine the fail cause.
* Work with the upstream to resolve the issue or drop the need for
`setup.py test` during the build of the package.
* When updating the package, you can test it using the same copr
repository where the latest version of setuptools has been built.
* Let us know if something doesn't work as it should.

== User Experience ==

Regular distro users shouldn't notice any change in setuptools
behavior, except for upstream-intended changes.


== Dependencies ==


== Contingency Plan ==

* Contingency mechanism: (What to do? Who will do it?) Provide a
compat package with setuptools 69
* Contingency deadline: mass rebuild
* Blocks release? No

== Documentation ==

[https://setuptools.pypa.io/en/stable/history.html Setuptools changelog]

This page is the documentation.

== Release Notes ==


--
Aoife Moloney

Fedora Operations Architect

Fedora Project

Matrix: @amoloney:fedora.im

IRC: amoloney

--
_______________________________________________
devel-announce mailing list -- devel-announce@lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

No comments:

Post a Comment