autotools-dev for Debian ======================== (Before packaging something in Debian that uses GNU autoconf, you must read the "Introduction" and the "Calling GNU configure properly" sections) Note: libtool is part of the autotools toolchain, but we don't cover libtool well in this document. However, there are a few big fat warnings that I will repeat here: Your package must use an up-to-date libtool. See the libtool package for more information. Always retool (rebuild all autotools-generated files) on package build time when using libtool. Introduction: ------------- The autotools are a set of tools that help one (most of the time, anyway) write architecture-independent code, and write a build system capable of supporting a large number of architectures (plus cross-compiling and a few extras). The main components are autoconf (processes .in and .ac files), and automake (processes Makefile.am into Makefile.in). Other common components are aclocal (helper for autoconf), libtool (complex tool for library linking support), gettext/autopoint (i18n support). Autoconf is architecture-aware, as this is required for cross-compiling. This means it should always be called in a certain way (in Debian package builds) so that it picks up the proper architectures from dpkg-architecture. Autoconf's more advanced architecture-aware code (invoked when the AC_CANONICAL family of macros are used, which is always the case if libtool is being used, for example) also needs two extra helper scripts: config.sub and config.guess. These two helpers, config.guess and config.sub, need to be constantly updated to address new architectures. Unfortunately, Debian's automake package has failed to provide such timely updates regularly, and even if it did, most developers are not even aware of the need for such updates. Their upstream often isn't, either. The result ends up as serious bugs filled in the BTS by porters. Given the amount of packages using autoconf and automake in Debian, we are talking about a rather large number of bugs every time we start supporting a new architecture. There is currently no way around an updated config.sub file. Even if one supplies to a GNU autoconf (configure) script the build and target host system name, config.sub is needed to get the canonical GNU host system name for the build. In fact, an outdated config.sub script in that case might break any new architecture in Debian just the same. This package gives you tools to fix this problem, as well as some of the accumulated knowledge from dealing with the GNU autotools and their effect on upstream, on Debian auto-builders, and on version-controlled source trees. Please read this file entirely before you package something that uses GNU config, autoconf and automake. If your package uses libtool at all, you must also read the documentation that comes with the Debian libtool package. Basic summary of packaging source that uses autotools: ------------------------------------------------------ You have three choices for packaging upstream source which uses automake and autoconf and contains generated files. 1. Patch the autotools files (*.in, *.am) using the method of your choice when needed. Remove (or move away) all auto-generated files in the clean (and possibly in other targets), and have the autotools suite rebuild the build system (e.g. using autoreconf). This is called "retooling". dh_autoreconf can help automate retooling at package build time. Please refer to its manpage for details. You should consider using the extend-diff-ignore option in debian/source/options to get dpkg-source to ignore changes to the autogenerated files, see dpkg-source(1) for details. Don't forget to make sure all the build dependencies are correct for a full re-autotoolization (hint: conflicting with autoconf2.13 is always a good idea if you're not using autoconf 2.13 and automake 1.4). This means that the auto-builders will have to rerun the entire thing, and so will anyone else building the package. But it also means there will be no surprise breakages at bad times (e.g. during a security update) as the build system will be always fresh and throughout tested by all build daemons on all supported arches. This method is the current best practice, and you're well advised to use it on your packages. 2. Tolerate the big diff size, and retool the autotools stuff before you create the Debian source package. Modern dpkg will take care of avoiding the time-stamp skew issues this method sometimes caused. Nowadays, there is no reason to do this. You are strongly advised to use method 1 above, instead. This method is extremely prone to cause time-stamp skew issues, and it is a continuous source of build breakage. 3. Live with whatever unknown release upstream used, but update config.sub and config.guess to (hopefully) not break new arches. This is always a very bad choice. It can easily break anyway, and cause very annoying issues if anyone ever needs to update the autotools-generated files only to find out upstream did something funny. You do *not* have the choice of using this method when libtool is being used in the build. Use the method 1 above, instead. You can use dh_update_autotools_config from debhelper to help automate the config.sub and config.guess update at package build time. This document contains enough information to properly implement any of the three choices. But make no mistake, method 1 is the best practice, and it is the one you should use in your packages. Running and build-depending on autoconf and automake: ----------------------------------------------------- The autoconf2.13 package diverts autoconf, autoheader and autoreconf from the autoconf package to autoconf2.50, autoheader2.50 and autoreconf2.50. You must make sure you are not depending on the existence of the diverted scripts. The automake packages provide alternatives for the automake command, with increased priority for newer versions. There is one exception: the automake1.4 package provides a higher priority alternative for the automake command than the newer automake packages. These are only issues if you need to run autoconf or automake during build. To make sure you do not get build failures, or difficult-to-track problems, please follow these guidelines: For automake 1.4 and autoconf 2.13: - Build-Depends: autoconf2.13, automake1.4 - Call the autoconf suite using simple names ("autoconf", "autoheader", etc.) - Call the automake suite using simple names ("automake", "aclocal", etc) - autoreconf is probably buggy as all hell, don't use it - PLEASE upgrade your package to use the newest autoconf/automake, and send the fixes upstream - If you need to override prefix during make install, you should set the other directories relative to it, as in: ./configure --prefix=/usr --mandir=\$${prefix}/share/man [...] $(MAKE) prefix=$(CURDIR)/debian/pkg/usr install For automake 1.6 and above, and newest autoconf: - Build-Depends: autoconf, automake - Build-Conflicts: autoconf2.13, automake1.4 - Call the autoconf suite using simple names ("autoconf", "autoheader", etc.) - Call the automake suite using *versioned* names ("automake-1.9", etc), OR use autoreconf, but set the environment variables accordingly to the versioned names. Otherwise, you may get a higher version than what you expected. If you know your Makefile.am files are very well behaved, and will not break with a newer automake, versioned names are optional. - Use DESTDIR on the make install line to get the files installed elsewhere Libtool warning: *never* override prefix during make install when using libtool, as the .la files will hardcode the wrong path information. If you absolutely have to do so, fix the .la files after the make install file. Overriding DESTDIR is fine, it does not break libtool. Why should I help upstream upgrade their autotools scripts: ----------------------------------------------------------- Because after they are sanitized well enough, you probably can get away with always using the latest automake, autoconf, libtool and gettext, without errors. Using these tools as packaged by Debian gives us real benefits, such as a sane building environment with bug fixes that are often quite important. Using the latest libtool and gettext *do* *make* *a* *difference*, as in critical functionality fixes. This is especially true for libtool. Libtool and gettext have minimum autoconf version requirements, so you will have to switch to new autoconf sooner or later anyway, if you are using gettext or libtool. The sooner, the better for everyone. That means it is actually better to refresh the autotools, libtool and gettext build environment with whatever is packaged in Debian over whatever upstream is using. Calling GNU configure properly: ------------------------------- Autoconf always use the host/build architecture information, even if the AC_CANONICAL macros are not being used, and config.sub and config.guess are not required. That means there are no exceptions to the rules in this section: you must always tell configure the architectures to use. You should not[1] simply call ./configure with the proper prefix and package-dependent options and be done with it (regardless of the fact that many Debian packages do so). [1] As in: it is a bug if you do so, see Debian policy 3.6, section 4.3. CDBS automates the configure invocation with the proper parameters. debhelper offers dh_auto_configure (which is used automatically by the dh wrapper) to do the same. Please refer to their documentation for details. Calling GNU configure manually: Unless you know better, and your package has very specific needs (hint: you build several sub-arch variants, such as with MMX, i586-optimized, i686-optimized... binary packages AND you use the config.guess output to detect your build target), you should call configure this way (for autoconf 2.52): export DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) export DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) # FOR AUTOCONF 2.52 AND NEWER ONLY ifeq ($(DEB_BUILD_GNU_TYPE), $(DEB_HOST_GNU_TYPE)) confflags += --build $(DEB_HOST_GNU_TYPE) else confflags += --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE) endif ./configure $(confflags) \ ....... (other configure options) ...... This will tell configure to build for the proper architecture, even if you do not use dpkg-buildpackage, or manually override DEB_HOST_GNU_TYPE and DEB_BUILD_GNU_TYPE for cross-compiling. It requires GNU make and dpkg-dev, but those packages are build-essential in Debian, so that should not be a problem. YOU STILL NEED AN UP-TO-DATE config.sub FILE TO AVOID BREAKAGE IN NEW ARCHITECTURES, EVEN WITH THIS SETUP, IF YOUR BUILD USES AC_CANONICAL MACROS OR LIBTOOL. BTW, autoconf 2.52+ should enter cross-compiling mode if --host is specified. It will build in cross-compiling mode even if build and host type are the same (this information comes directly from autoconf upstream). This goes against what is in the docs of autoconf 2.59, and it may be a bug somewhere. One could remove (or fail to update) config.guess when using the above setup, since it is usually not called by configure anymore. I prefer to keep both files updated and in place just in case, and for completeness. When building cross-compilers, you may need to inform --target as well (if you are cross-compiling a cross-compiler(!)). For packages that use the outdated autoconf 2.13, one should use: # FOR AUTOCONF 2.13 ONLY ifeq ($(DEB_BUILD_GNU_TYPE), $(DEB_HOST_GNU_TYPE)) confflags += $(DEB_HOST_GNU_TYPE) else $(error Cannot cross-compile this package out-of-the-box) endif cross-compiling with autoconf 2.13 is an Evil art, which shall turn sane and bright minds into quivering wrecks upon contact with the madness that lies in that path. It is very easy to know which version of autoconf was used to build a configure script. Just use "head configure" and read the comments. If your package is still using autoconf 2.13, please seriously consider upgrading it to autoconf 2.58 or newer, and sending the changes upstream. The outdated GNU config.{sub,guess} problem and solutions: ---------------------------------------------------------- GNU autotools need the config.sub (and sometimes config.guess) file when the AC_CANONICAL_HOST, AC_CANONICAL_SYSTEM macros are used, and also when libtool is in use. It might need them in other circumstances, so don't hold your breath. An out-of-date config.sub will break builds in any architecture it doesn't know about. The same goes for config.guess, if it is used (so, don't use it, read "Calling GNU configure properly" above and call configure properly). To fix this particular problem automatically, we need both a source of fresh copies for config.{sub,guess} in Debian, as well as a way to get them inside the packages during their build. autotools-dev provides the fresh copies of config.{sub,guess} for any programs that might need them (in /usr/share/misc), including automake and libtool. The best way to get the updated files into the package is to remove all auto-generated files in the clean target, and to use autoreconf and related commands to regenerate the build system at build time. The reasons for this being the preferred method to deal with this have already been explained in a previous section of this document. You could also use something like autoreconf --force, to update the build system (including config.sub and config.guess). If you are also upstream for a package, this is probably what you should be doing. Make sure you have set the proper environment variables and package build-dependencies so as to get an autotools environment that will work for the package after autoreconf is run. See the autoreconf man/info page for more details. Packages calling autoreconf, automake, autoconf, aclocal and their friends must do so in a sane environment. This is explained elsewhere in this document. When updating auto-generated files, always remove them all in the clean target, and recreate/update them in a prerequisite of the build targets. Trying to autoreconf or freshen config.sub and config.guess on the clean target would cause a lot of useless cruft to migrate to the Debian diff, and can interact badly with the new patch-aware dpkg source formats. This can often be avoided if special care is taken to inform dpkg-source of all files it should ignore (using debian/source/options and the extend-diff-ignore option). Should autoreconfing be impossible, at least config.sub and config.guess should always be updated at build time. To do that, the two files should be removed on the clean target. On a prerequisite target of the build target, before configure runs, they should be copied from (or symlinked to) the fresh copies provided by autotools-dev in /usr/share/misc. This requires a build-dependency on autotools-dev, of course. Using the debhelper helpers: If you use the "dh" command provided by debhelper at version 9.20160115 or later, then it will automatically call dh_update_autotools_config to update config.sub and config.guess for the build. You should also check the dh-autoreconf documentation to automate retooling on build time, which is preferred over just updating config.sub and config.guess. Be careful with mixing the "autoreconf" and "autotools-dev" dh sequences as they are order sensitive. Please consider simply removing the "autotools-dev" sequence and rely on debhelper (>= 9.20160115) built-in functionality. dh_update_autotools_config can be used directly without relying on "dh". Refer to its manpage for more information. CDBS-based packages: For CDBS-using packages, if your package Build-Depends on autotools-dev, CDBS will automatically update config.guess and config.sub without needing to take further action. Reducing the annoyance factor with diffs: By adding a suitable extend-diff-ignore option to debian/source/options, one can instruct dpkg-source to ignore config.sub, config.guess, and any other autogenerated files (such as Makefile when autoconf is used, Makefile and Makefile.in when automake is used, etc). For example, you could use: extend-diff-ignore="(^|/)(config\.sub|config\.guess|Makefile)$" The problem with time-stamp skews and Debian source packages: ------------------------------------------------------------- File time-stamp skew can cause "make" and similar tools to decide that a file has to be regenerated when files are modified out of dependency order. This is a major problem when it happens in an unexpected build environment, and it does include the build system auto-generated files created by the autotools. Time-stamp skew is not a problem for the packages that employ the current best practice, which is to regenerate all auto-generated files on every build (this is not restricted to, but it does include most of the files generated by the autotools). You can use dh_autoreconf to retool an autotools-based build system at build time. File time-stamp skews are often caused by patching or modifying files at build time, unless special care is taken to ensure strict ordering. dpkg will set the time-stamps on all files it patches at source unpack time to the same value, but unless you patch all files in the tooling dependency graph, things will still break. For example, if you patch configure.in and configure, but not Makefile.in, Makefile.am and Makefile, only configure.in and configure will have their time-stamps updated by dpkg. This will cause Makefile to attempt to run configure for the initial "make clean". Should this happen before config.sub and config.guess are updated, the build will break. Depending on which files were patched, it is possible that autotools would try to rerun autoconf, automake, etc. When regenerating a specific auto-generated file is strongly undesirable, you must ensure that time stamp ordering is enforced by debian/rules where required (e.g. using "touch foo && touch bar && touch baz"). To remove time stamp skews in an entire file tree, use "touch -r" or "touch -d" to set the files to exactly the same time stamp. GNU config, autoconf/automake/autoheader, libtool, gettext and version control systems: -------------------------------------------------- This text applies to any machine-generated build-time files, such as those one gets when one runs gettextize (now called autopoint), libtoolize, autoconf, automake, autoheader, aclocal, bison... Experience shows that such files must NOT be kept under version control in the work tree. From a distro (not end-user!) point of view, such files are supposed to be regenerated at every build, from up-to-date, fresh copies of their master files. "But upstream keeps such stuff in his CVS/SVN/bzr/hg/git/whatever repository!" you could say. Well, your upstream probably doesn't know better. I know for a fact some of mine didn't, either. Nor did I at first, for that matter. By regenerating the files, you're making sure that the entire package can be fully built from the source files, and that the build is not going to break for untraceable reasons (such as changes done by hand by upstream to files that were supposed to be auto-generated). When such files are auto-generated properly at every build, outdated config.{sub,guess} (as well as the outdated gettext, outdated libtool, outdated automake...) are not an issue anymore. Also, the auto-builders will help you catch any problems early, before they creep in a stable release only to blow up on a security update. Build time will be a little longer, but that's an acceptable compromise. We do not want whatever upstream has for gettext, autoconf and friends causing issues for Debian builds, automated or otherwise. The gettext, libtool and autoconf Debian maintainers take great pain to make sure their packages are up-to-date, bug-free and work sanely on all Debian-supported architectures. All that work goes down the drain if you leave some old config.sub or libtool.sh from an unknown variant of some weird distribution of 3 years ago just because your upstream happens to like it. The usual way to automatically regenerate the build system is to create an autogen.sh script, that one runs from debian/rules as a pre-requisite to the build targets. This script should call aclocal, autoheader, autoconf, automake, libtoolize, gettextize in the proper order, and with the proper parameters (or better yet, call autoreconf with the proper environment variables). Just don't run it on the clean target, run it before the actual build. One can also have it all inside debian/rules, but that's a little less clear, and less useful if you are also the upstream maintainer. Also, here's a free hint: make sure your VCS ignores all the automatically generated files. That can make adding a new upstream release a lot easier. Use of dpkg-source's extend-diff-ignore option to ignore all auto-generated files will make sure any changes done at build time are not going to propagate to the source package. This can be permanently set on debian/source/options. -- Henrique de Moraes Holschuh Fabian Greffrath