debci (2.0) unstable; urgency=medium [ Paul Gevers ] * docs/MAINTAINERS.md: add lxc 3 syntax and consistent sudo usage [ Antonio Terceiro ] * debci-data: import and export package data * debci-shell: require debci/job from the start * add missing dependency on rsync * backends/qemu: use autopkgtest-build-qemu to build images * backends/lxc: detect bridge interface on lxc 3 * docs/HACKING.md: improve setup instructions * docs/HACKING.md: drop mention to vagrant setup * links: require link target to exist during build * debian/rules: remove unused, override_dh_clean * debian/control: deduplicate Build-Depends and Depends * debci localtest: skip --shell-fail it not on a tty * docs/HACKING.md: be explicit about installing build dependencies * Make mirror setting consistent across system (Closes: #923436) * Add -m/--mirror parameter to most scripts * debian/tests/backend-lxc: update lxc networking config for lxc >= 3 * debian/tests/backend-lxc: add new test dependency on lxc-templates -- Antonio Terceiro Fri, 01 Mar 2019 20:57:12 -0300 debci (1.15.1) unstable; urgency=medium * debci setup-chdist: fix debootstrap usage * debci collector: make sure a single file is used as exitcode * Bump Standards-Version to 4.3.0; no changes needed otherwise -- Antonio Terceiro Wed, 06 Feb 2019 22:56:52 -0200 debci (1.15) unstable; urgency=medium [ Paul Gevers ] * Proper support for autopkgtest exit status 14: erroneous package with skipped tests [ Antonio Terceiro ] * Debci::Job: order pending jobs by creation date * config: fix setting debci_data_retention_days * Debci::Config: replace conditionals with hash lookup * Debci::DB: add support for ActiveRecord 5.2 (Closes: #918179) * Add missing build dependency on ruby-rack-test * Add missing autopkgtest dependency on ruby-rack-test * Debci::Config: don't break config values containing "=" * config: add timeout to default database config -- Antonio Terceiro Mon, 07 Jan 2019 13:54:47 -0300 debci (1.14) unstable; urgency=medium * lxc/create-testbed: avoid cleaning up valid containers * README.md: point people to debian-ci@l.d.o and #debci (only) * docs/MAINTAINERS.md: add two new FAQs * debci-autopkgtest: make it work when run standalone * debci-localtest: remove last component from artifacts directory * debci-autopkgtest: don't hardcode autopkgtest arguments * Bump Standards-Version to 4.2.1 (no changes needed) * Move moreutils from Recommends: to Depends: (Closes: #910918) -- Antonio Terceiro Sat, 03 Nov 2018 12:50:26 -0300 debci (1.13) unstable; urgency=medium [ Antonio Terceiro ] * debci-generate-feeds: fix link to autopkgtest log * api: make API self-documenting * config.ru: serve compressed logs inline in development * debci-worker: declare results queue only once * backends/fake: drop rmadison call * debci-job: rename `finished` to `import` and move logic to Debci::Job * api: don't crash on failed authentication * api: don't display retry button to unauthenticated users [ Paul Gevers ] * Add --no-apt-fallback to autopkgtest options * .gitlab-ci.yml: use backports for stable testing -- Paul Gevers Thu, 13 Sep 2018 15:34:35 +0200 debci (1.12) unstable; urgency=medium [ Antonio Terceiro ] * ui: add status name back in package history page * lxc, schroot: don't make `debci` a system user * tools/init-dev.sh: reduce number of suites * tools/init-dev.sh: separate config files from test requests * debci-collector: replace pointless loop with conditional * Escape triggers when enqueuing test jobs (Closes: #902337) [ Paul Gevers ] * Treat autopkgtest exit status 8 as a neutral result (Closes: #901846) * debci.(pre|post)(inst|rm): call dpkg-maintscript-helper to adapt for change of directory to link in previous version * backends/debci-setup-chdist: add contrib and non-free (Closes: #825488) * backends: add debug archive to lxc and schroot (Closes: #817080) -- Paul Gevers Fri, 10 Aug 2018 20:06:49 +0200 debci (1.11) unstable; urgency=medium [ Antonio Terceiro ] * ui: pending jobs: add columns for suite and architecture * debci-generate-index: make sure to always run * Remove all handling of "blame" * Remove generation of britney hints file * Fix debci-localtest * Build-Depends: autopkgtest * debci-enqueue: log suite and architecture * debian/tests/control: install autopkgtest to run test suite * Debci::API: accept only package names starting with [a-z0-9] * ui: Optimize HTML generation process * Use FontAwesome via its standard API * lxc: name containers ci-$EPOCH (Closes: #893595, #898875) * Implement test priorities * ui: improve "Status Alerts" table * ui: add status page for slow-running tests * Bump Standards-Version to 4.1.4, no changes needed [ Paul Gevers ] * README.md: fix unsecure http URL's and fix e-mail list address [ Rafael Laboissiere ] * d/control: Change Maintainer to Debian CI team. Antonio Terceiro, the former Maintainer, is now an Uploader. -- Antonio Terceiro Wed, 13 Jun 2018 09:51:09 -0300 debci (1.10) unstable; urgency=medium * api: enable retry with client certificate and add GET response * Minor packaging updates * Thanks to Chris Lamb for the patches of bugs 880804, 879655 and 879654. -- Paul Gevers Wed, 25 Apr 2018 15:10:36 +0200 debci (1.9.1) unstable; urgency=medium * api: accept + in package names -- Antonio Terceiro Sat, 07 Apr 2018 15:45:26 -0300 debci (1.9) unstable; urgency=medium [ Paul Gevers ] * Make building debci reproducible (Closes: #880804) * Move from http://http.debian.net to http://deb.debian.org (Closes: #879654) * Document the use of a local mirror (Closes: #879655) * Document the need for running chdist * docs: fix typo [ Antonio Terceiro ] * debci-setup: mention suite in the logs * tools/build.sh: tell rerun that `make` exits * tools/init-dev.sh: initialize database * debci-worker: refuse invalid package names * api: reject invalid package names * Use salsa.debian.org in Vcs-* fields -- Antonio Terceiro Sat, 07 Apr 2018 15:24:55 -0300 debci (1.8) unstable; urgency=medium [ Antonio Terceiro ] * This release adds two major features that were developed to support triggering tests to speed up Debian testing migrations: * Added support for "migration tests", which are tests that run on testbeds of a given suite (e.g. testing) with some packages from another suite (e.g. unstable). * Added an HTTP API for requesting test runs. This will be used by britney to test package versions from unstable against testing. * debci setup: pass --suite to testbed creation scripts * debci test: Extract autopkgtest calls into a central helper program * backends/lxc: drop the unstable → sid renaming * Add script to run a Ruby shell * Move enqueing process to Ruby for performance reasons * debian/control * replace build dependency on dh-systemd with debhelper (>= 9.20160709) * Bump Standards-Version to 4.1.3; no changes needed * remove unnecessary Testsuite: field [ Paul Gevers ] * backends/fake: use month instead of minutes in the generated package name for non-existent packages * backends/fake: use rmadison get more up-to-date version numbers * Initial version for a page where one can see which jobs have pending results * Add retrigger end point to the API [ Lucas Kanashiro ] * Update git repository URL to point to salsa * UI: sort package status history based on date in descending order * define pipeline via gitlab-ci.yml -- Antonio Terceiro Thu, 15 Mar 2018 14:52:34 -0300 debci (1.7.1) unstable; urgency=medium * debian/control: - debci: recommend `ntp | time-daemon` instead of just `ntp` (Closes: #870739) * debian/debci-worker.cron.daily: also bail out immediately if debci is not installed (Closes: 822896) -- Antonio Terceiro Sun, 10 Sep 2017 13:35:21 -0300 debci (1.7) unstable; urgency=medium [ Martin Pitt ] * Move from obsolete adt-run to autopkgtest CLI (Closes: #866867) [ Antonio Terceiro ] * debian/NEWS: document changes to base testbed names due to the move to `autopkgtest` from `adt-run`. * debci-setup: don't mask failure of the backend create-testbed script (Closes: #809652) * backends/lxc: cleanup containers if testbed creation fails, making sure that calls to debci-setup will always exit cleanly. * New program: debci localtest. It can be used by users to reproduce locally the same test environment used in ci.debian.net. - autopkgtest calls make by backends/*/test-package will now use the --no-built-binaries option. * Add debci(1) manpage and support `debci --help` (Closes: #867891) * Add an experiental `qemu` backend -- Antonio Terceiro Wed, 26 Jul 2017 17:44:47 -0300 debci (1.6) unstable; urgency=medium * docs/INSTALL.md: update rabbitmq-related instructions * Debci::HTML: record template filename * Debci::Graph: assume empty array when there is no data * debci-expire: control data retention * Drop unnecessary symlinks to autopkgtest logs * web UI: don't link to removed files * debian/rules: skip running tests during build (Closes: #841708, #848054) * debci-worker.cron.daily: skip if debci-worker is not installed (Closes: #822896) * debian/*.service: add Documentation entry, pointing to online debci docs -- Antonio Terceiro Sun, 28 May 2017 17:22:50 -0300 debci (1.5.1) unstable; urgency=medium [ Brandon Fairchild ] * TUTORIAL.md: Add links to DDPO and package tracker [ Antonio Terceiro ] * docs/TUTORIAL.md: add link to DMD as well * debci-setup-chdist: - use $MIRROR as default mirror * debci-generate-index: - handle sub-directories of the artifacts/ directory produced by autopkgtest. [ Paul Gevers ] * Simplify the instructions to enable sudo to work on lxc-* now that debci ships an sudoers.d/debci file -- Antonio Terceiro Thu, 26 Jan 2017 10:27:45 -0200 debci (1.5) unstable; urgency=medium * debci setup - add -f/--force option to force an update, ignoring the testbed timestamp * web UI - fix links to packages in "status alerts" page * packaging: + debci-worker: move `apt-cacher-ng` from Recommends: to Suggests:. In practice, workers should usually be close to a good mirror; having a local cache increases the maintaince burden as the disk usage might go too high. * lxc backend: + drop "adt-sid-amd64-" prefix from container names. In practice, container names longer than 64 characters cause problems. * setup-chdist: + drop usage of timestamp to control when to call `apt-get update`. Just update always. an `apt-get update` on a updated tree should be quick enough. -- Antonio Terceiro Thu, 27 Oct 2016 19:38:44 -0200 debci (1.4) unstable; urgency=medium [ Antonio Terceiro ] * bin/debci-migrate.d/002_compress_artifacts: also remove directories after compressing artifacts * Web UI: - include version numbers almost everywhere. * lib/debci/html/package.erb: avoid crashing when installing a fresh system with no data, but with packages already blacklisted. * debian/debci.postrm: remove /var/log/debci on purge (Closes: #810701) * Add support for running tests for multiple suites (See: #831975). . Not that we couldn't do that already, but it would require specific workers for each suite. With these changes, a worker node for a given architecture can now run tests on any suite for that architecture. . - debci-worker will now expect the suite name as a second argument on test jobs (defaulting to whatever is in $debci_suite). - debci-enqueue will now send the suite name as a second argument on each test job - Added debci-update-worker as a wrapper for callling debci-setup for each supported suite. * backends/lxc: - retry `apt-get update` until it works. Most of the time when the testbed setup fails on production is during `apt-get update`. * "Platform-Specific issues" will now not consider tmpfail as an issue. Those are already being tracked by "Status alerts". * test/test_shell_best_practices.sh: start checking scripts with shellcheck; not failing on issues just yet. [ Gordon Ball ] * Add the concept of failure states, to distinguish between * tests which have never passed * tests which passed for a previous version, but never for the current version * tests which previously passed for the current version -- Antonio Terceiro Sun, 04 Sep 2016 18:27:51 -0300 debci (1.3) unstable; urgency=medium * debci-collector: - remove creation of .timestamp files under data/autopkgtest-incoming/, which are no longer used for anything * bin/debci-batch: - use one tmp_dir per package/suite/architecture * bin/debci-generate-index - use one tmp_dir per package/suite/architecture - compress artifacts (except log) in a single tarball so that instead of N files per test run, we only keep 2 files per test run: log.gz and artifacts.tar.gz. This should significantly reduce the usage of inodes. -- Antonio Terceiro Fri, 03 Jun 2016 16:26:11 -0300 debci (1.2) unstable; urgency=medium [ Antonio Terceiro ] * debian/debci-worker@.service: also bump LimitNPROC to infinity to avoid the `Cannot fork` issue. * backends/lxc/test-package: use adt-virt-lxc --name option to facilitate mapping from a running test to package names. This requires autopkgtest (>= 3.20). * bin/debci-collector, bin/debci-worker, lib/functions.sh#report_status: display suite and architecture together with package names * HTML UI: * add page listing Platform-specific issues, i.e. packages that passes on one suite/arch but fails on another suite/arch. * fix Status page to not hide all charts when any platform does not have enough data * rewrite Status charts page. In special, instead of loading the full history via AJAX, which takes quite a while, embed a reduced view of the data in the HTML itself. * remove search functionality: it was too heavy/slow. If one day the UI stops being only static HTML, we can a search feature again. * Drop debian/tests/worker-max-processes. The original issue was caused by (and fixed in) systemd >= 228 (see bug #823530), and the test ended up not being very reliable and keeps flapping on CI. * bin/debci-generate-index: remove call to debci-generate-html, since debci-update now call both debci-generate-index and debci-generate-html [ Jakub Wilk ] * public/style.css: Set correct font for alert, chart, rss, and tracker icons. (Closes: #821932) [ Brandon Fairchild ] * HTML UI: fix final copyright year in the footer -- Antonio Terceiro Mon, 23 May 2016 10:31:19 -0300 debci (1.1.2) unstable; urgency=medium * debian/tests/control: restrict backend-schroot to only running on VMs (i.e. not on containers) * debian/debci-worker@.service: make sure there is no maximum number of processes; otherwise when using the lxc backend, we might get random failures saying `Cannot fork: resource temporarily unavailable` with systemd from stretch onwards. - added debian/tests/worker-max-processes as a test case * Add new backend `null`, which just uses the `null` virtualization regime in adt-run. This will be useful for running tests, but can also be used for running tests with zero overhead on trusted packages. * backends/fake: add support for executing an specific command instead of producing a random result if $DEBCI_FAKE_COMMAND is set. -- Antonio Terceiro Tue, 12 Apr 2016 11:23:17 -0300 debci (1.1.1) unstable; urgency=medium * debian/sudoers.d/debci: adjust to work with autopkgtest >= 3.19.3, which will also call `timeout` under sudo. * backends/lxc/create-testbed: initialize $GUEST_PROXY before use, for the case when no proxy is detected. * Add integration tests for schroot and lxc backends * debian/tests/integration-test: wait until test jobs have been processed, to avoid failures due to not waiting long enough on slow machines. * debian/tests/control: force throwing away used testbed after integration-test and multiarch-integration-test -- Antonio Terceiro Sun, 27 Mar 2016 22:42:29 -0300 debci (1.1.0) unstable; urgency=medium New features: * debci-config: added option to output only values and not keys * Proper support for multiple architectures. - list of architectures is declared in the configuration file using the `debci_arch_list` variable. - new script debci-update will call debci-generate-index for each of the architectures in $debci_arch_list, and them call debci-generate-html which was already architecture-independent. - systemd unit files debci-generate-index.{service,timer} renamed to debci-update.{service,timer}. debci-update.service and changed to call `debci update` instead of `debci generate-index`. Bug fixes: * remove unused debian/po directory (Closes: #813447) * test_helper: stop rabbitmq-server the right way Minor packaging updates: * remove deprecated and unused tools * debian/control: Use https URLs in Vcs-* fields * debian/control: Bump Standars-Version to 3.9.7; no changes needed -- Antonio Terceiro Sat, 26 Mar 2016 18:59:52 -0300 debci (1.0.2) unstable; urgency=medium [ Martin Pitt ] * Fix some excessive amd64 hardcoding in the test suite. Thanks Colin Watson! * test/test_blame.sh: Sleep for one second between process() calls, to ensure that timestamps in run IDs are different. (Closes: #809265) [ Brian Murray ] * Include extra information in HTML tags, such as package name, architecture, suite, etc (Closes: #811563) -- Antonio Terceiro <terceiro@debian.org> Mon, 25 Jan 2016 18:57:03 -0200 debci (1.0.1) unstable; urgency=medium * debian/debci-worker.cron.daily: exit immediately if debci is no longer installed (Closes: #807360) * Drop Debconf templates * bin/debci-batch: make sure package status directory exists before creating queue marker -- Antonio Terceiro <terceiro@debian.org> Mon, 14 Dec 2015 15:17:30 -0200 debci (1.0) unstable; urgency=medium * This is a major overhaul of the debci infrastructure. debci now works as a distributed system, with one or more workers that process test runs, and one central node that collects tests results and generates the HTML web UI. The nodes communicate using an AMQP server (usually rabbitmq-server), and test requests can be submitted from any host that can connect to the AMQP server. One will usually have one specific node, perhaps the one that runs the result collection/web UI, running `debci batch` to schedule tests. * New packages: + debci-worker: contains the systemd service units necessary to run the worker daemons; must be installed on worker nodes. + debci-collector: contains the systemd service unit necessary to run the debci collector daemon: will receive test results, and generate the HTML web interface; must be installed on a single node in the network * The `debci` package contains the debci core, and is still the package that needs to be installed for developers trying out debci locally as instructed the documentation. * Change default backend to `lxc` [ Martin Pitt ] * Initial implementation of the master/worker setup * Add support for importing data from test runs executed elsewhere [ Antonio Terceiro ] * Finished implementation of the master/worker setup * schroot/create-testbed: find usable union-type (Closes: #799760) * debian/debci.postrm: remove /var/lib/debci on purge (Closes: #769766) * added debian/tests/integration-test: full-system smoke test [ Chris Lamb ] * Document how to get started without a virtualisation regime (Closes: #777531) [ Brandon Fairchild ] * Several documentation and frontend updates + Including displaying correct failing architectures (Closes: #797198) -- Antonio Terceiro <terceiro@debian.org> Thu, 03 Dec 2015 22:15:30 -0200 debci (0.13) unstable; urgency=medium * debci hint: generate a britney hints file -- Antonio Terceiro <terceiro@debian.org> Tue, 18 Aug 2015 16:52:06 +0200 debci (0.12) unstable; urgency=medium [ Brandon Fairchild ] * Debci::Package#failures: Determine the test status that occurred before a tmpfail [ Antonio Terceiro ] * web UI: read head.html and footer.html from config dir -- Antonio Terceiro <terceiro@debian.org> Wed, 17 Dec 2014 08:45:20 -0200 debci (0.11) unstable; urgency=medium [ Antonio Terceiro ] * debci-migrate: added a data migration mechanism that is also automatically executed during postinst. * Switch data storage strategy to compress all autopkgtest logs. This will save huge amounts of storage space. - tools/server.sh adjusted to make lighttpd serve *.log.gz files as regular text/plain files, but with the propers headers to tell browsers that they are compressed so they will do the right tihng. * debci-batch: will now stop even in the middle of a iteration when told to go offline. * backends/schroot/create-testbed, debci-setup-chdist: add buildd suites to sources.list so we detect newly uploaded packages faster * Removed embedded copy of Bootstrap, now depending on libjs-bootstrap [ Brandon Fairchild ] * Several web UI improvements * Documentation update on hacking the web interface -- Antonio Terceiro <terceiro@debian.org> Sat, 15 Nov 2014 14:56:14 -0200 debci (0.10.3) unstable; urgency=medium [ Brandon Fairchild ] * handle old test run data that didn't have a run_id. [ Antonio Terceiro ] * debci-status: fix generation of packages status file which was incorrectly repeating the status for the very first package over and over. -- Antonio Terceiro <terceiro@debian.org> Tue, 14 Oct 2014 18:55:27 -0300 debci (0.10.2) unstable; urgency=medium [ Antonio Terceiro ] * lib/functions.sh: accept any value of Testsuite matching `autopkgtest` insted of only exact matches when looking for source packages with test suites. This will allow detecting packages with specialized values such as autopkgtest-pkg-ruby and autopkgtest-pkg-perl. [ Brandon Fairchild ] * Detect JSON parsing errors on history files instead of silently failing and leaving a blank history (Closes: #759569) * Avoid also listing lib* packages under l/. -- Antonio Terceiro <terceiro@debian.org> Wed, 08 Oct 2014 08:53:37 -0300 debci (0.10.1) unstable; urgency=medium * Fix documentation build -- Antonio Terceiro <terceiro@debian.org> Tue, 30 Sep 2014 17:29:48 -0300 debci (0.10) unstable; urgency=medium [ Martin Pitt ] * Fix Makefile to be idempotent, previously it failed on already existing symlinks. * Add missing dctrl-tools build dependency, Makefile is calling grep-dctrl. * Makefile: Don't call "checkdeps" on "make check", the test suite doesn't actually need autopkgtest or the libjs packages (which are for the UI only). * test/test_helper.sh: Don't hardcode "amd64", to fix test failure when building on other architectures. [ Antonio Terceiro ] * debci-generate-index: update packages.json atomically * web UI improvements: - remove limit of number of items to display in the search (Closes: #762213) - trigger search immediately when there is text in the search box (this will happen if you reload the page) - store search query in the URL, and honor that when opening any page that contains a search box -- Antonio Terceiro <terceiro@debian.org> Fri, 26 Sep 2014 18:22:06 -0300 debci (0.9) unstable; urgency=medium [ Antonio Terceiro ] * schroot/create-testbed: update existing chroots * debci-generate-feeds: add description to feeds * Incorporate results of the Google Summer of Code 2014 project by Brandon Fairchild, providing uncountable improvements to the web interface. http://lists.alioth.debian.org/pipermail/soc-coordination/2014-August/002295.html [ Brandon Fairchild ] * web UI was improved to support multiple suites and architectures, reduce the dependency the UI had on Javascript, and provide more information to package maintainers. The documentation was also worked on to provide information to people interested in working on debci (specifically the web UI) -- Antonio Terceiro <terceiro@debian.org> Sat, 23 Aug 2014 18:10:04 -0300 debci (0.8.1) unstable; urgency=medium * Documentation updates * Fix obtaining the finish date of tests. Parsing the adt-run log started producing bad dates when adt-run changed to displaying only the time without the date. debci now uses the timestamp of when adt-run finished instead. * debci-batch: store offline mark in persistent storage to avoid going back online after a reboot. * debci-test: revert to updating only package-specific data; it turns out that updating global data after every package is too expensive. * debci-batch: update global data every 1 hour or so * debci-generate-index: avoid crash when regenerating data for extenal consumption from scratch. This process is not completely idempotent yet, so it needs more work. -- Antonio Terceiro <terceiro@debian.org> Mon, 18 Aug 2014 08:25:23 -0300 debci (0.8) unstable; urgency=medium * debci-test: update global data files, not only package-specific data file, after each successfull test run. * debci-generate-index: ensure that no two instances can run at the same time. -- Antonio Terceiro <terceiro@debian.org> Thu, 31 Jul 2014 11:55:27 -0300 debci (0.7.3) unstable; urgency=medium * debci-generate-feeds: - always update the global feed. - Point main link of feed entries to the debci log instead of the package page. * debci-generate-index: - report duration of each test run together with the status * debci-test: - fix regression with new packages caused by not mkdir'ing the full path to the adt-run output directory. `fake` backend adapted to emulate the behaviour of adt-run so that the tests would now fail without the fix. -- Antonio Terceiro <terceiro@debian.org> Wed, 18 Jun 2014 14:29:46 +0200 debci (0.7.2) unstable; urgency=medium * Fix calculation of test elapsed time. We can't store the beginning timestamp at the adt-run output directory because that will be wiped out by adt-run before running the test. * public/index.html: fix source code URL to point to the collab-main repository which is the canonical one. -- Antonio Terceiro <terceiro@debian.org> Tue, 10 Jun 2014 17:30:41 -0300 debci (0.7.1) unstable; urgency=medium * Fix handling of blame with "unknown" as value. Will avoid crashing when generating indexes -- Antonio Terceiro <terceiro@debian.org> Mon, 09 Jun 2014 11:07:11 -0300 debci (0.7) unstable; urgency=medium [ Antonio Terceiro ] * debci-batch: put packages with fastest testsuites first in the queue * debci-generate-feeds: allow generating feeds for specific packages * debci-generate-index: update package feed right away when indexing individual packages * public/app.js: Adjust the "Pass percentage" Y axis labels so that chart is actually a percentage chart. * public/index.html: add link to global Atom feed in the home page. * debci-test, debci-generate-index: more robust way of calculating test run duration. Parsing the autopkgtest log is just not good enough. [ Martin Pitt ] * Fix report_status() to work for bash scripts. * debci-test: Add --print-output option to write output directory to stdout. With that the caller can easily check the result. * debci, tools/server.sh: Exec the target program instead of running in a subprocess, for efficiency and proper SIGTERM handling. * Drop tools/convert-data. We only needed it for version 0.6, and we're going to need a different conversion script after the next data/ dir reorganization. * Reorganize data directory to move the "categories" packages/, autopkgtest/, status/, and feeds/ to the top level; they contain subdirs for all releases and architectures. This makes autopkgtest/ fully rsyncable and also makes it easier to the web UI to browse/present data for multiple releases and architectures. Also split release-arch/ into release/arch/. * Add tools/convert-data-0.7: Convert data directory to changed format from above. * Fix debci-status -l if there is just one package. * Add tests for debci-batch, covering skipping/re-running of tests under appropriate conditions and showing the cause for re-run in the log. * Replace test/json_validity (in ruby) with test/test_json.sh (shell with just calling ruby for the actual JSON validation). With that we can re-use test_helper's test bed setup instead of having to replicate it all in ruby. * Make test/runall.sh executable for convenience, and make it only run test_* scripts. * debci-batch: remove reading configuration from /etc/default/debci * lib/environment.sh: read configuration from $debci_basedir/config/debci.conf * Add debci_mirror config variable to configure an archive mirror. Use it for chdist and schroot setup. * Drop unnecessary faketime build dependency. * Add LXC backend. -- Antonio Terceiro <terceiro@debian.org> Tue, 03 Jun 2014 22:36:44 -0300 debci (0.6.3) unstable; urgency=medium * debian-status: avoid generating invalid JSON in the presence of packages without status files -- Antonio Terceiro <terceiro@debian.org> Thu, 29 May 2014 12:48:48 -0300 debci (0.6.2) unstable; urgency=medium * debci-generate-index: don't try merging dependencies diff when no package has been blamed before. -- Antonio Terceiro <terceiro@debian.org> Wed, 28 May 2014 18:14:06 -0300 debci (0.6.1) unstable; urgency=medium * Fix chdist calls on stable (See #736504) -- Antonio Terceiro <terceiro@debian.org> Thu, 15 May 2014 12:51:25 -0300 debci (0.6) unstable; urgency=medium [ Martin Pitt ] * Move schroot creation from bin/debci-setup to backends/schroot/create-testbed, as it is schroot backend specific and other backends will need different setup. * process-package: Call backend test-package with second argument that specifies the adt-run output directory. Implement that in schroot backend with adt-run --output-dir, and fake backend when passing $DEBCI_FAKE_DEPS. This will be used to collect and present log files, artifacts, package/version lists, etc. * Eliminate list-base-system and use autopkgtest's testbed-packages output file instead. This will also work with remote workers and other virt servers. * Eliminate check_version() and take this information from autopkgtest's output. * Move report_status() into lib/functions.sh as we'll need it in more places than just process-package. * Move "run needed" policy from process-package to bin/debci. With this, debci-test can be called manually at any time to retry tests or get called by e. g. britney which already evaluates when to run tests. Remove the "run test at most once a day" check completely as package updates can happen several times a day. As a by-product this also fixes debci's -f option to force a test run. * backends/schroot/list-dependencies: Ignore individual apt-get failures as that gets called for binaries which don't exist on the selected architecture. Fixes operation for e. g. glib2.0. * Use chdist instead of the local schroot for package queries: - Add scripts/setup-chdist to create/update a chdist for the selected release. - Use that in list-dependencies instead of querying the schroot. Move that script to scripts/ as it is now not backend specific any more. - Update lib/functions.sh to use chdist instead of the schroot. - Add devscripts dependency for chdist, and demote schroot dependency to recommends. This allows us to use other backends, like QEMU or remote ones. * Reorganize data directory: - Split out autopkgtest output into ${debci_data_dir}/autopkgtest/: This now contains a full directory for each test run with all of adt-run's --output-dir files. This provides access to individual test stdout/err logs, the package version lists, and additional artifacts that tests produce. With this, autopkgtest/ becomes eligible for being on a remote file system and can be synced/rebuilt independently of the metadata. - Use run IDs with format YYYYMMDD_HHMMSS[machine or tags] instead of the old YYYYMMDD-n as the former works for a distributed file system and remote workers. - base.txt is now replaced with adt-run's testbed-packages output file. - test-packages.txt gets dropped as there is no need to keep it around permanently. Build it from the per-test -packages logs from adt-run. - Move status.json into status/ directory and remove the redundant latest.json. - Move packages.json into status/ directory. * Add tools/convert-data: Convert data/ directory to changed format from above. This can be dropped again after running it on the production debci instance. * app.js: Filter out query strings from URL for constructing the .json URLs. Fixes wrong URLs after using the package search field. * backends/schroot/test-package: Drop unused temporary dir. * backends/*/test-package: Write exit code into "exitcode" file in output dir, so that we retain it even when we switch to asynchronous/remote processing. * backends/fake/test-package: Write log file into output dir, like adt-run. * Move test result evaluation, log and meta data creation from debci-test to debci-generate-indexes. This prepares the way for running adt-run asynchronously or remotely, and picking up the results from the autopkgtest/ data directory. * Reduce the per-test run <timestamp>.log files to the debci metadata (triggers, package versions, package diffs, overall result), as the actual autopkgtest log is now kept in data/*/autopkgtest/. This avoids duplicating the log contents. * app.js: In the per-package test list, add link to latest autopkgtest log and artifacts (other output files). * debci-status --all: Output a list also if there is only one package. * debci-status --all: Sort package names so that they appear sorted in the web UI. * Avoid some error messages noise when calling generate-index without any existing test data. * Drop debci-test --reason option and make debci-batch put the reason into the status directory directly. * Fold debci-cleanup into debci-generate-index. [ Antonio Terceiro ] * schroot backend: pass --session-id to adt-virt-schroot. This makes it easier to debug test suite that leave rogue schroot sessions behind. * rename `debci` binary to `debci-batch`. `debci` stays reserved for providing a entry point binary like `git` that will invoke subcommands that are kept out of $PATH to avoid cluttering the namespace. * debci-batch: make batch locking suite- and architecture-specific * debci-batch: add --offline and --online options. --offline will prevent new test runs from being started, and --online will revert to normal operation. * Rename scripts/*: - scripts/process-package: absorbed by the previously almost empty bin/debci-test - scripts/list-dep8-packages: absorbed by the previously almost empty bin/debci-list-packages - scripts/list-dependencies → bin/debci-list-dependencies - scripts/setup-chdist → bin/debci-setup-chdist - scripts/blame → bin/debci-blame - scripts/cleanup → bin/debci-cleanup - scripts/generate-index → bin/debci-generate-index * add `debci` binary to not clutter the namespace. When you call `debci COMMAND` it will set PATH to include the debci bin/ and then call `debci-COMMAND` with the supplied arguments. * debci-blame: replace usage of deprecated IO#lines method with IO#each_line * debci-generate-index: accept duration as command line switch * backends/schroot/create-testbed: drop usage of eatmydata to avoid breaking test suites that actually depend on a real fsync(). To compensate the performance loss when installing packages, set dpkg `--force-unsafe-io` option in a configuration file (Closes: #747308). * Added Ruby bindings and the corresponding documentation * Dropped usage of faketime in the test suite (Closes: #747715) * backends/fake: output a timestamp like the ones adt-run outputs * Added new debci-config, a helper to output value of debci configuration values * debci-generate-feeds: added generation of per-package and global Atom feeds. Called from debci-batch * public: now using FontAwesome from fonts-font-awesome package * public/app.js: add a link to the per-package Atom feed * public/style.css: make the status indicators use FontAwesome icons -- Antonio Terceiro <terceiro@debian.org> Thu, 15 May 2014 12:34:33 -0300 debci (0.5.2) unstable; urgency=medium * scripts/process-package: skip blame check without a dependencies diff * lib/*.sh: guard `set -eu` to avoid doing that on interactive shells * lib/functions.sh: fix list_binaries to work against the sources index - This fixes listing binaries coming from packages that were binNMUed -- Antonio Terceiro <terceiro@debian.org> Sat, 05 Apr 2014 20:03:22 -0300 debci (0.5.1) unstable; urgency=medium * etc/schroot/debci/fstab: improve schroot isolation - do not share /tmp with host - mount tmpfs at /dev/shm as quite some software needs it * test_shell_best_practices.sh: skip when checkbashisms is not available. - You don't want to install devscripts and all its dependencies when running as-installed tests - checking for best practices is only useful in development environments. * Use unified diff in "Change in the base system since last run" * debian/source/options: fix to include publib/data in the source tarball, but not data/* -- Antonio Terceiro <terceiro@debian.org> Tue, 01 Apr 2014 12:23:15 -0300 debci (0.5.0) unstable; urgency=medium [ Martin Pitt ] * Add missing dctrl-tools dependency. * Use "dpkg --print-architecture" instead of dpkg-architecture, to avoid a dependency on dpkg-dev and the (undeclared) dependency on gcc. * Fix grep_packages() to actually use the configured architecture instead of the host architecture. * Add --help output to debci-{list-packages,setup,test}, instead of failing with an "usage: not found" error. [ Antonio Terceiro ] * Initial upload to Debian (Closes: #736416) The API is not set yet, but I am uploading anyway to allow for early testing, improve contribution opportunities, to get a BTS etc. * Updated documentation - instructions for setting up a development instance. - Point traffic at debian-qa * modularize behaviour of a backend - existing implementation extracted as the `schroot` backend - added a `fake` backend that helps with testing * schroot backend optimization: always run with eatmydata * Add blame support: when tests of a package start failing after some dependencies were updated, those dependencies will be "blamed" for the failure. * improvements on autopkgtest usage: - debci now uses the autopkgtest support for downloading the source package instead of downloading manually and them calling adt-run - debci now differentiates between the non-zero adt-run exit codes, so that skipped tests will not count as failures. Also added a `tmpfail` status, to be used when for some reason the tests could not be run for some external failure (e.g. mirror failures, broken dependencies etc). * debci now keeps track of the base system. Tests will be run again every 2 weeks even if no explicit dependency changes. Test logs will also display the changes in the base system since the last run. * Add DEP-8 support to debci itself \o/ * extra safety: add `set -u` to all shell scripts -- Antonio Terceiro <terceiro@debian.org> Mon, 24 Mar 2014 20:06:56 -0300 debci (0.4.0) unstable; urgency=medium * Package renamed to debci * Require a newer autopkgtest. - In general we should always be using a backport of the most recent version in unstable. * Visual facelift - embedding a copy of bootstrap 3 for now :-( * Add backend support for multiple architectures. For now unstable/amd64 is hardcoded, but reading them from the command line or from the environment will not require a large change. -- Antonio Terceiro <terceiro@debian.org> Tue, 28 Jan 2014 14:40:22 -0300 dep8 (0.3.1) unstable; urgency=medium * Explicitly set PATH in crontab -- Antonio Terceiro <terceiro@debian.org> Sun, 12 Jan 2014 22:52:00 -0300 dep8 (0.3.0) unstable; urgency=medium * record duration of each run, and record more tham one status entry per day. -- Antonio Terceiro <terceiro@debian.org> Sun, 12 Jan 2014 19:40:23 -0300 dep8 (0.2.3) unstable; urgency=medium * Makefile: fix creating symlinks for JS files -- Antonio Terceiro <terceiro@debian.org> Sat, 11 Jan 2014 15:47:00 -0300 dep8 (0.2.2) unstable; urgency=medium * check_version: handle package with more than 1 version -- Antonio Terceiro <terceiro@debian.org> Sat, 11 Jan 2014 15:41:00 -0300 dep8 (0.2.1) unstable; urgency=medium * environment.sh: print error when dep8_base_dir is not set * cron-wrapper: change to base directory before loading environment -- Antonio Terceiro <terceiro@debian.org> Mon, 06 Jan 2014 16:22:18 -0300 dep8 (0.2.0) unstable; urgency=medium * keep track of the archive-wide status * process-package: set directory for schroot invocation * generate-index: better naming for status entries * cron-wrapper: fix user detection * Move web UI to public/ subdirectory * add test to check for bashisms * major reorganization for reuse and consolidation * webui: display global status in time series chart * Use explicit directory for gnupg home * Also install a config directory in /etc * ignore new top-level directories -- Antonio Terceiro <terceiro@debian.org> Mon, 06 Jan 2014 15:44:29 -0300 dep8 (0.1.1) unstable; urgency=medium * Add basic test suite * add `set -e` to all shell scripts -- Antonio Terceiro <terceiro@debian.org> Sat, 04 Jan 2014 14:20:53 -0300 dep8 (0.1.0) unstable; urgency=low * Initial Release. -- Antonio Terceiro <terceiro@debian.org> Wed, 01 Jan 2014 01:35:14 -0300