F45 Change Proposal: ODBC Stack Modernization (self-contained)

Wiki - https://fedoraproject.org/wiki/Changes/ODBC_stack_modernization Discussion thread - https://discussion.fedoraproject.org/t/f45-change-proposal-odbc-stack-modernization-self-contained/196627 This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Modernize the Fedora ODBC stack across 7 packages (<code>unixODBC</code> and 6 driver packages: <code>mariadb-connector-odbc</code>, <code>mysql-connector-odbc</code>, <code>postgresql-odbc</code>, <code>freetds</code>, <code>sqliteodbc</code>, <code>mdbtools</code>). The central change is replacing the static, centrally-maintained <code>/etc/odbcinst.ini</code> driver registration file with an auto-generated configuration assembled from per-driver drop-in snippets, using RPM file triggers — the same pattern used by <code>ldconfig</code>, <code>ca-certificates</code>, and <code>crypto-policies</code>. Supporting changes include switching to bare library names resolved via a compiled-in search path, moving driver plugins to a dedicated directory, and general spec cleanups. == Owner == * Name: [[User:mschorm|Michal Schorm]] * Email: mschorm@redhat.com === Proposed code changes === All changes are in the <code>dropin-registration</code> topic branches in the Chnage owner's forks: * [https://src.fedoraproject.org/fork/mschorm/rpms/unixODBC/c/fddf280ed2a7afdc6dda18996bad8b20425f00b6 unixODBC] — drop-in infrastructure, file triggers, <code>%ghost</code> config, migration scriptlets, [https://src.fedoraproject.org/fork/mschorm/rpms/unixODBC/c/70abcfdeeb41bacacc7da75b0a9d9f58a225e11b?branch=dropin-registration manual page] for the new script * [https://src.fedoraproject.org/fork/mschorm/rpms/mariadb-connector-odbc/c/d5cecc1055d4077b8f4a1c9b80d3cd49240f027c mariadb-connector-odbc] — ship <code>10-mariadb.ini</code> drop-in snippet * [https://src.fedoraproject.org/fork/mschorm/rpms/mysql-connector-odbc/c/0b03c28a7485d7ed446c63f70e9b537e5611e5c0 mysql-connector-odbc] — ship <code>10-mysql.ini</code> drop-in snippet * [https://src.fedoraproject.org/fork/mschorm/rpms/postgresql-odbc/c/e136923324f3956e599c15851d524d7bbb3c0576 postgresql-odbc] — ship <code>10-postgresql.ini</code> drop-in snippet * [https://src.fedoraproject.org/fork/mschorm/rpms/freetds/c/d36341635aa8a230abb072ac4c56539f9609ac48 freetds] — ship <code>10-freetds.ini</code> drop-in snippet * [https://src.fedoraproject.org/fork/mschorm/rpms/sqliteodbc/c/316aa92e9326d15ce43ba26675554ddfcb0d2a71 sqliteodbc] — ship <code>10-sqlite.ini</code> drop-in snippet, remove legacy <code>odbcinst</code> scriptlets * [https://src.fedoraproject.org/fork/mschorm/rpms/mdbtools/c/b6f54b24fb8de6f00eb883c37f20f0762d257dcf mdbtools] — ship <code>10-mdbtools.ini</code> drop-in snippet == Detailed Description == === Background === The ODBC driver manager (<code>unixODBC</code>) uses <code>/etc/odbcinst.ini</code> to map driver names to shared libraries. In Fedora, this file has historically been shipped as <code>%config(noreplace)</code> inside the <code>unixODBC</code> package, pre-populated with entries for all known Fedora ODBC drivers. The driver entries in Fedora were in poor shape. <code>FileUsage</code> values were wrong for multiple drivers, library paths were hardcoded and multilib-unfriendly, and the FreeTDS entry contained a dead <code>Port</code> key that the driver never reads from <code>odbcinst.ini</code>. These problems persisted because driver entries lived in the wrong package — fixing a driver's registration required a change to <code>unixODBC</code>, not to the driver package itself. This coupling created several structural problems: * '''Wrong ownership:''' Driver packages had no control over their own registration. A driver maintainer who wanted to fix or update their entry had to coordinate a change in <code>unixODBC</code>. * '''Stale entries:''' All known drivers were pre-populated in the config regardless of whether they were installed, leading to broken references. * '''Invisible drivers:''' New driver packages (e.g. <code>mdbtools-odbc</code>) were not registered at all until someone manually added them to the <code>unixODBC</code> spec. <code>mdbtools-odbc</code> was invisible to ODBC applications after install. * '''Fragile upgrades:''' The <code>%config(noreplace)</code> semantics mean RPM silently keeps the old file on upgrade, so corrections (like fixed <code>FileUsage</code> values) never reach existing installations. Of all 6 driver packages in Fedora, only <code>sqliteodbc</code> attempted self-registration via <code>%post</code>/<code>%preun</code> scriptlets calling <code>odbcinst -i</code>/<code>odbcinst -u</code>. The other 5 relied entirely on <code>unixODBC</code> pre-populating their entries. === What is changing === The modernization consists of two groups of changes: '''Already in Fedora Rawhide (foundation work):''' * ODBC driver plugins moved from <code>%{_libdir}</code> to a dedicated <code>%{_libdir}/odbc/</code> directory across all driver packages. * <code>unixODBC</code> compiled with <code>--with-odbc-driver-path=%{_libdir}/odbc</code>, enabling bare library names (e.g. <code>Driver = libmaodbc.so</code> instead of full paths). * <code>FileUsage</code> values corrected across all drivers ([https://bugzilla.redhat.com/show_bug.cgi?id=2453060 BZ#2453060]). * <code>Suggests:</code> added for all 6 ODBC driver packages. * Removal of a 17-year-old dead patch (<code>keep-typedefs.patch</code>). '''Proposed in this change (driver registration):''' * Each driver package ships a static <code>.ini</code> snippet file to <code>/usr/lib/odbc/odbcinst.d/</code> (vendor defaults). * <code>unixODBC</code> owns <code>%transfiletriggerin</code> and <code>%transfiletriggerpostun</code> scriptlets that run a regeneration script (<code>odbcinst-generate</code>) whenever snippets are installed or removed. * The regeneration script merges all snippets (vendor defaults overlaid by admin overrides) and atomically replaces <code>/etc/odbcinst.ini</code>. * <code>/etc/odbcinst.ini</code> changes from <code>%config(noreplace)</code> to <code>%ghost</code> — RPM tracks ownership but the file is never shipped; it is always generated. * Administrators can override or disable vendor drivers via <code>/etc/odbc/odbcinst.d/</code> (see below). === Drop-in directory layout === /usr/lib/odbc/odbcinst.d/ ← vendor snippets (shipped by driver RPMs) 10-mariadb.ini 10-mysql.ini 10-postgresql.ini 10-freetds.ini 10-sqlite.ini 10-mdbtools.ini /etc/odbc/odbcinst.d/ ← admin overrides and additions /etc/odbcinst.ini ← generated output (%ghost) '''Override semantics:''' * Files are named <code>NN-name.ini</code>. The numeric prefix controls merge order. * Vendor range: 10–49. Admin range: 50–99. * When two files share the same name after stripping the prefix (e.g. <code>10-mariadb.ini</code> and <code>60-mariadb.ini</code>), the higher-numbered one wins. * To disable a vendor driver, symlink its snippet to <code>/dev/null</code>: ln -sf /dev/null /etc/odbc/odbcinst.d/10-freetds.ini * To apply changes after editing drop-in files manually, run: <code>odbcinst-generate</code> These semantics follow the established systemd convention used throughout Fedora. === Precedent === This is the standard Fedora pattern for generated configuration. At least 22 packages use the same <code>%transfiletriggerin</code> + regeneration model, including: * <code>glibc-common</code> — <code>/etc/ld.so.conf.d/</code> → <code>ldconfig</code> → <code>/etc/ld.so.cache</code> * <code>ca-certificates</code> — <code>/etc/pki/ca-trust/source/</code> → <code>update-ca-trust</code> → <code>ca-bundle.crt</code> * <code>crypto-policies</code> — <code>/usr/share/crypto-policies/</code> → <code>update-crypto-policies</code> * <code>fontconfig</code> — <code>/usr/share/fonts/</code> → <code>fc-cache</code> * <code>shared-mime-info</code> — <code>/usr/share/mime/</code> → <code>update-mime-database</code> * <code>glib2</code> — <code>/usr/share/glib-2.0/schemas/</code> → <code>glib-compile-schemas</code> === Migration strategy (for FESCo consideration) === This is the area where community input is most welcome. When upgrading from a system where <code>/etc/odbcinst.ini</code> was <code>%config(noreplace)</code> to the new <code>%ghost</code> model, RPM does '''not''' automatically create an <code>.rpmsave</code> backup. Without intervention, any user modifications to <code>odbcinst.ini</code> would be silently lost. The proposed migration strategy: # A <code>%pretrans</code> Lua scriptlet in <code>unixODBC</code> detects first-time upgrades (the drop-in directory does not yet exist) and renames <code>/etc/odbcinst.ini</code> to <code>/etc/odbcinst.ini.rpmsave</code>. On subsequent upgrades (drop-in directory already exists), the <code>%pretrans</code> is a no-op. # <code>%post</code> checks whether <code>/etc/odbcinst.ini.rpmsave</code> exists. If so, it prints a notice to stderr explaining that driver registration has moved to drop-in snippets, pointing users to the <code>.rpmsave</code> file and explaining how to migrate custom entries. This notice is repeated on every upgrade as long as the <code>.rpmsave</code> file remains, serving as a persistent reminder until the user completes the migration and removes it. '''What users need to do:''' * Users who never modified <code>odbcinst.ini</code>: nothing. All standard drivers are registered automatically via snippets. * Users who added custom driver entries: copy their custom <code>[sections]</code> from <code>odbcinst.ini.rpmsave</code> into a new file under <code>/etc/odbc/odbcinst.d/</code> and run <code>odbcinst-generate</code>. '''Alternatives considered:''' * ''Content-based detection'' (parse the file to detect user modifications): rejected as unreliable in RPM's limited Lua environment. * ''Always preserve'' (never create <code>.rpmsave</code>): rejected because it leaves the old static file alongside the new generated one, causing confusion. * ''In-place migration'' (automatically convert user entries to drop-in files): rejected as too complex and opaque. Users should be aware of the new mechanism. === Rejected ideas: scriptlet-based registration (Debian/Ubuntu, openSUSE) === Other distributions use <code>odbcinst -i</code>/<code>odbcinst -u</code> commands in driver package scriptlets (<code>postinst</code>/<code>postrm</code> on Debian, <code>%post</code>/<code>%preun</code> on openSUSE). I considered and rejected this for Fedora due to various bugs present in this approach. '''How it works in Debian/Ubuntu:''' Each driver package calls <code>odbcinst -i -d -f /usr/share/<driver>/odbcinst.ini.template</code> in its <code>postinst</code> and <code>odbcinst -u -d</code> in its <code>postrm</code>. This approach has generated real bugs over more than a decade: * [https://bugs.launchpad.net/ubuntu/+source/freetds/+bug/1173083 Ubuntu Bug #1173083]: <code>tdsodbc</code> used a debconf prompt to ask whether to register the driver. In automated installs (containers, CI, preseed), the question was never answered, leaving FreeTDS silently unregistered. Open for 12 years before being fixed. * [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001141 Debian Bug #1001141]: <code>odbc-mariadb</code> called <code>odbcinst</code> in its maintainer scripts but did not declare a dependency on the <code>odbcinst</code> binary, causing silent registration failures. '''How it works in openSUSE:''' Similar, using RPM <code>%post</code>/<code>%preun</code> scriptlets. All errors are suppressed with <code>> /dev/null 2>&1 || true</code>, so failures are completely silent. Coverage is inconsistent: only 3 of 5 driver packages self-register; <code>mariadb-connector-odbc</code> ships a sample <code>.ini</code> but no scriptlets; <code>mdbtools</code> has its ODBC driver incorrectly placed in the <code>-devel</code> subpackage. '''Why the scriptlet approach is fundamentally fragile:''' * '''Race conditions:''' <code>odbcinst</code> performs a read-modify-write cycle on <code>/etc/odbcinst.ini</code> without file locking. If two driver packages run scriptlets concurrently, the file can be corrupted or one registration silently lost. * '''Reference counting:''' <code>odbcinst</code> maintains a reference count per driver entry. A failed or interrupted scriptlet can leave the count out of sync, resulting in stale entries that never get removed or premature removal of valid entries. * '''Cross-package file modification:''' Multiple independent packages all modify a file owned by <code>unixODBC</code>, violating single file ownership. * '''Per-driver boilerplate:''' Every driver package must independently implement registration logic, template files, and correct dependency declarations. Each one is a potential source of bugs. '''Why the drop-in approach is better:''' RPM file triggers (<code>%transfiletriggerin</code>) are transactional — they fire once after the entire transaction completes, not once per package. The regeneration script rebuilds <code>/etc/odbcinst.ini</code> from scratch using only the snippet files currently on disk. There is no read-modify-write race, no reference counting, no possibility of stale entries, and driver packages need zero scriptlet logic — they ship a static file and nothing else. With this change, Fedora would be the first distribution with a systemic, race-free, declarative ODBC driver registration mechanism with first-class admin override semantics. == Feedback == &nbsp; == Benefit to Fedora == * '''Decoupled driver registration:''' Each driver package owns its own registration. Adding a new ODBC driver to Fedora no longer requires changes to <code>unixODBC</code>. * '''Correct out-of-the-box experience:''' Only installed drivers appear in the config. Installing a driver automatically makes it visible to applications; removing it automatically cleans up. This fixes the long-standing problem where <code>mdbtools-odbc</code> was installable but invisible to ODBC applications. * '''Admin override mechanism:''' System administrators can override vendor defaults or disable drivers without editing generated files, using the same drop-in pattern familiar from systemd, ldconfig, and crypto-policies. * '''Bare library names:''' Driver entries use plain library names (e.g. <code>Driver = libmaodbc.so</code>) resolved via a compiled-in search path, eliminating hardcoded <code>%{_libdir}</code> paths and multilib ambiguity. * '''Correct metadata:''' <code>FileUsage</code> values are fixed across all drivers. Fedora becomes the first distribution to ship correct values for all its ODBC drivers. * '''Reduced spec complexity:''' Driver packages no longer need <code>%post</code>/<code>%preun</code> scriptlets for registration. A single static <code>.ini</code> file per driver replaces all of it. == Scope == * Proposal owners: ** Implement drop-in infrastructure in <code>unixODBC</code> (directories, regeneration script, file triggers, <code>%ghost</code> config, migration scriptlets) ** Add drop-in snippet files to all 6 driver packages: <code>mariadb-connector-odbc</code>, <code>mysql-connector-odbc</code>, <code>postgresql-odbc</code>, <code>freetds</code>, <code>sqliteodbc</code>, <code>mdbtools</code> ** Remove legacy <code>%post</code>/<code>%preun</code> <code>odbcinst</code> scriptlets from <code>sqliteodbc</code> ** Test upgrade scenarios in containers via COPR * Other developers: ** No action required from other package maintainers. The change is self-contained within the 7 packages listed above. * Release engineering: N/A (not a System Wide Change) * Policies and guidelines: N/A * Trademark approval: N/A * Alignment with the Fedora Strategy: Fedora leads in the Linux distribution development. This change adopts a modern, well-established Fedora pattern for an area of the stack that has lagged behind. == Upgrade/compatibility impact == On upgrade from a previous Fedora release: * <code>/etc/odbcinst.ini</code> is saved as <code>/etc/odbcinst.ini.rpmsave</code>. * A new <code>/etc/odbcinst.ini</code> is generated from drop-in snippets. * All standard drivers (MariaDB, MySQL, PostgreSQL, FreeTDS, SQLite, MDBTools) are registered automatically if their packages are installed. * Users with custom driver entries in the old <code>odbcinst.ini</code> need to migrate them to drop-in snippet files under <code>/etc/odbc/odbcinst.d/</code>. A notice is printed during upgrade with instructions. Applications using ODBC continue to work without changes. The generated <code>odbcinst.ini</code> uses the same INI format as before; only the mechanism that produces it has changed. == How To Test == Test builds are available in COPR: [https://copr.fedorainfracloud.org/coprs/mschorm/ODBC/ mschorm/ODBC] '''Fresh install (no previous ODBC config):''' # Install <code>unixODBC</code> and one or more driver packages # Verify <code>/etc/odbcinst.ini</code> is generated and contains entries for the installed drivers # Verify that uninstalling a driver package removes its entry from <code>odbcinst.ini</code> '''Upgrade from current Rawhide (unmodified config):''' # Start with a system running the current <code>unixODBC</code> (pre-drop-in) # Upgrade to the new version # Verify <code>/etc/odbcinst.ini.rpmsave</code> is created # Verify the new <code>/etc/odbcinst.ini</code> contains correct entries for installed drivers '''Upgrade with custom driver entries:''' # Start with a system where <code>/etc/odbcinst.ini</code> has been manually edited (e.g. a custom <code>[MyDriver]</code> section added) # Upgrade to the new version # Verify <code>/etc/odbcinst.ini.rpmsave</code> preserves the custom entries # Copy the custom section into <code>/etc/odbc/odbcinst.d/60-mydriver.ini</code> # Run <code>odbcinst-generate</code> # Verify the custom driver appears in the regenerated <code>/etc/odbcinst.ini</code> '''Admin override:''' # Install <code>mariadb-connector-odbc</code> (ships <code>10-mariadb.ini</code>) # Create <code>/etc/odbc/odbcinst.d/60-mariadb.ini</code> with modified settings # Run <code>odbcinst-generate</code> # Verify the admin override takes precedence '''Driver disable:''' # <code>ln -sf /dev/null /etc/odbc/odbcinst.d/10-freetds.ini</code> # Run <code>odbcinst-generate</code> # Verify FreeTDS no longer appears in <code>/etc/odbcinst.ini</code> == User Experience == For most users, the change is invisible. Installing an ODBC driver package automatically registers it; removing it automatically unregisters it. No manual <code>odbcinst</code> commands are needed. Power users and administrators gain a familiar drop-in override mechanism. Customizing ODBC driver registration now works the same way as customizing library paths (<code>ld.so.conf.d</code>) or CA certificates (<code>ca-trust</code>). Users upgrading from a previous Fedora release who had custom entries in <code>odbcinst.ini</code> will see a migration notice on each upgrade until they complete the migration and remove the <code>.rpmsave</code> file. == Dependencies == All 7 affected packages are maintained by the change owner: * <code>unixODBC</code> * <code>mariadb-connector-odbc</code> * <code>mysql-connector-odbc</code> * <code>postgresql-odbc</code> * <code>freetds</code> * <code>sqliteodbc</code> * <code>mdbtools</code> No other packages are affected. The generated <code>/etc/odbcinst.ini</code> is format-compatible with the previous static file. == Contingency Plan == * Contingency mechanism: Revert the <code>unixODBC</code> spec to ship a static <code>%config(noreplace)</code> <code>odbcinst.ini</code> and remove drop-in snippet files from driver packages. This is a straightforward revert of the topic branch in each package. * Contingency deadline: N/A (not a System Wide Change) * Blocks release? '''No''' == Documentation == * [https://bugzilla.redhat.com/show_bug.cgi?id=2453060 BZ#2453060] — Standardize unixODBC connector installation directory * [https://copr.fedorainfracloud.org/coprs/mschorm/ODBC/ COPR test repository: mschorm/ODBC] * A new <code>odbcinst-generate(1)</code> man page will be shipped with the <code>unixODBC</code> package, documenting the regeneration tool, drop-in directory layout, and override semantics * The existing upstream <code>odbcinst.ini(5)</code> man page will be updated or supplemented with a note about the drop-in mechanism == Release Notes == The ODBC driver stack has been modernized. ODBC driver registration now uses a drop-in snippet mechanism: each driver package ships a small <code>.ini</code> file that is automatically merged into <code>/etc/odbcinst.ini</code> when the package is installed or removed. Administrators can override vendor defaults or add custom drivers by placing files in <code>/etc/odbc/odbcinst.d/</code>. Users who had custom entries in <code>/etc/odbcinst.ini</code> will find their previous configuration saved as <code>/etc/odbcinst.ini.rpmsave</code> after upgrading. Custom driver sections should be migrated to individual files under <code>/etc/odbc/odbcinst.d/</code>. -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney -- _______________________________________________ devel-announce mailing list -- devel-announce@lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org Do not reply to spam, report it: https://forge.fedoraproject.org/infra/tickets/issues/new