debci (0.10.3) unstable; urgency=medium [ Brandon Fairchild ] * handle old test run data that didn't have a run_id. [ Antonio Terceiro ] * debci-status: fix generation of packages status file which was incorrectly repeating the status for the very first package over and over. -- Antonio Terceiro Tue, 14 Oct 2014 18:55:27 -0300 debci (0.10.2) unstable; urgency=medium [ Antonio Terceiro ] * lib/functions.sh: accept any value of Testsuite matching `autopkgtest` insted of only exact matches when looking for source packages with test suites. This will allow detecting packages with specialized values such as autopkgtest-pkg-ruby and autopkgtest-pkg-perl. [ Brandon Fairchild ] * Detect JSON parsing errors on history files instead of silently failing and leaving a blank history (Closes: #759569) * Avoid also listing lib* packages under l/. -- Antonio Terceiro Wed, 08 Oct 2014 08:53:37 -0300 debci (0.10.1) unstable; urgency=medium * Fix documentation build -- Antonio Terceiro Tue, 30 Sep 2014 17:29:48 -0300 debci (0.10) unstable; urgency=medium [ Martin Pitt ] * Fix Makefile to be idempotent, previously it failed on already existing symlinks. * Add missing dctrl-tools build dependency, Makefile is calling grep-dctrl. * Makefile: Don't call "checkdeps" on "make check", the test suite doesn't actually need autopkgtest or the libjs packages (which are for the UI only). * test/test_helper.sh: Don't hardcode "amd64", to fix test failure when building on other architectures. [ Antonio Terceiro ] * debci-generate-index: update packages.json atomically * web UI improvements: - remove limit of number of items to display in the search (Closes: #762213) - trigger search immediately when there is text in the search box (this will happen if you reload the page) - store search query in the URL, and honor that when opening any page that contains a search box -- Antonio Terceiro Fri, 26 Sep 2014 18:22:06 -0300 debci (0.9) unstable; urgency=medium [ Antonio Terceiro ] * schroot/create-testbed: update existing chroots * debci-generate-feeds: add description to feeds * Incorporate results of the Google Summer of Code 2014 project by Brandon Fairchild, providing uncountable improvements to the web interface. http://lists.alioth.debian.org/pipermail/soc-coordination/2014-August/002295.html [ Brandon Fairchild ] * web UI was improved to support multiple suites and architectures, reduce the dependency the UI had on Javascript, and provide more information to package maintainers. The documentation was also worked on to provide information to people interested in working on debci (specifically the web UI) -- Antonio Terceiro Sat, 23 Aug 2014 18:10:04 -0300 debci (0.8.1) unstable; urgency=medium * Documentation updates * Fix obtaining the finish date of tests. Parsing the adt-run log started producing bad dates when adt-run changed to displaying only the time without the date. debci now uses the timestamp of when adt-run finished instead. * debci-batch: store offline mark in persistent storage to avoid going back online after a reboot. * debci-test: revert to updating only package-specific data; it turns out that updating global data after every package is too expensive. * debci-batch: update global data every 1 hour or so * debci-generate-index: avoid crash when regenerating data for extenal consumption from scratch. This process is not completely idempotent yet, so it needs more work. -- Antonio Terceiro Mon, 18 Aug 2014 08:25:23 -0300 debci (0.8) unstable; urgency=medium * debci-test: update global data files, not only package-specific data file, after each successfull test run. * debci-generate-index: ensure that no two instances can run at the same time. -- Antonio Terceiro Thu, 31 Jul 2014 11:55:27 -0300 debci (0.7.3) unstable; urgency=medium * debci-generate-feeds: - always update the global feed. - Point main link of feed entries to the debci log instead of the package page. * debci-generate-index: - report duration of each test run together with the status * debci-test: - fix regression with new packages caused by not mkdir'ing the full path to the adt-run output directory. `fake` backend adapted to emulate the behaviour of adt-run so that the tests would now fail without the fix. -- Antonio Terceiro Wed, 18 Jun 2014 14:29:46 +0200 debci (0.7.2) unstable; urgency=medium * Fix calculation of test elapsed time. We can't store the beginning timestamp at the adt-run output directory because that will be wiped out by adt-run before running the test. * public/index.html: fix source code URL to point to the collab-main repository which is the canonical one. -- Antonio Terceiro Tue, 10 Jun 2014 17:30:41 -0300 debci (0.7.1) unstable; urgency=medium * Fix handling of blame with "unknown" as value. Will avoid crashing when generating indexes -- Antonio Terceiro Mon, 09 Jun 2014 11:07:11 -0300 debci (0.7) unstable; urgency=medium [ Antonio Terceiro ] * debci-batch: put packages with fastest testsuites first in the queue * debci-generate-feeds: allow generating feeds for specific packages * debci-generate-index: update package feed right away when indexing individual packages * public/app.js: Adjust the "Pass percentage" Y axis labels so that chart is actually a percentage chart. * public/index.html: add link to global Atom feed in the home page. * debci-test, debci-generate-index: more robust way of calculating test run duration. Parsing the autopkgtest log is just not good enough. [ Martin Pitt ] * Fix report_status() to work for bash scripts. * debci-test: Add --print-output option to write output directory to stdout. With that the caller can easily check the result. * debci, tools/server.sh: Exec the target program instead of running in a subprocess, for efficiency and proper SIGTERM handling. * Drop tools/convert-data. We only needed it for version 0.6, and we're going to need a different conversion script after the next data/ dir reorganization. * Reorganize data directory to move the "categories" packages/, autopkgtest/, status/, and feeds/ to the top level; they contain subdirs for all releases and architectures. This makes autopkgtest/ fully rsyncable and also makes it easier to the web UI to browse/present data for multiple releases and architectures. Also split release-arch/ into release/arch/. * Add tools/convert-data-0.7: Convert data directory to changed format from above. * Fix debci-status -l if there is just one package. * Add tests for debci-batch, covering skipping/re-running of tests under appropriate conditions and showing the cause for re-run in the log. * Replace test/json_validity (in ruby) with test/test_json.sh (shell with just calling ruby for the actual JSON validation). With that we can re-use test_helper's test bed setup instead of having to replicate it all in ruby. * Make test/runall.sh executable for convenience, and make it only run test_* scripts. * debci-batch: remove reading configuration from /etc/default/debci * lib/environment.sh: read configuration from $debci_basedir/config/debci.conf * Add debci_mirror config variable to configure an archive mirror. Use it for chdist and schroot setup. * Drop unnecessary faketime build dependency. * Add LXC backend. -- Antonio Terceiro Tue, 03 Jun 2014 22:36:44 -0300 debci (0.6.3) unstable; urgency=medium * debian-status: avoid generating invalid JSON in the presence of packages without status files -- Antonio Terceiro Thu, 29 May 2014 12:48:48 -0300 debci (0.6.2) unstable; urgency=medium * debci-generate-index: don't try merging dependencies diff when no package has been blamed before. -- Antonio Terceiro Wed, 28 May 2014 18:14:06 -0300 debci (0.6.1) unstable; urgency=medium * Fix chdist calls on stable (See #736504) -- Antonio Terceiro Thu, 15 May 2014 12:51:25 -0300 debci (0.6) unstable; urgency=medium [ Martin Pitt ] * Move schroot creation from bin/debci-setup to backends/schroot/create-testbed, as it is schroot backend specific and other backends will need different setup. * process-package: Call backend test-package with second argument that specifies the adt-run output directory. Implement that in schroot backend with adt-run --output-dir, and fake backend when passing $DEBCI_FAKE_DEPS. This will be used to collect and present log files, artifacts, package/version lists, etc. * Eliminate list-base-system and use autopkgtest's testbed-packages output file instead. This will also work with remote workers and other virt servers. * Eliminate check_version() and take this information from autopkgtest's output. * Move report_status() into lib/functions.sh as we'll need it in more places than just process-package. * Move "run needed" policy from process-package to bin/debci. With this, debci-test can be called manually at any time to retry tests or get called by e. g. britney which already evaluates when to run tests. Remove the "run test at most once a day" check completely as package updates can happen several times a day. As a by-product this also fixes debci's -f option to force a test run. * backends/schroot/list-dependencies: Ignore individual apt-get failures as that gets called for binaries which don't exist on the selected architecture. Fixes operation for e. g. glib2.0. * Use chdist instead of the local schroot for package queries: - Add scripts/setup-chdist to create/update a chdist for the selected release. - Use that in list-dependencies instead of querying the schroot. Move that script to scripts/ as it is now not backend specific any more. - Update lib/functions.sh to use chdist instead of the schroot. - Add devscripts dependency for chdist, and demote schroot dependency to recommends. This allows us to use other backends, like QEMU or remote ones. * Reorganize data directory: - Split out autopkgtest output into ${debci_data_dir}/autopkgtest/: This now contains a full directory for each test run with all of adt-run's --output-dir files. This provides access to individual test stdout/err logs, the package version lists, and additional artifacts that tests produce. With this, autopkgtest/ becomes eligible for being on a remote file system and can be synced/rebuilt independently of the metadata. - Use run IDs with format YYYYMMDD_HHMMSS[machine or tags] instead of the old YYYYMMDD-n as the former works for a distributed file system and remote workers. - base.txt is now replaced with adt-run's testbed-packages output file. - test-packages.txt gets dropped as there is no need to keep it around permanently. Build it from the per-test -packages logs from adt-run. - Move status.json into status/ directory and remove the redundant latest.json. - Move packages.json into status/ directory. * Add tools/convert-data: Convert data/ directory to changed format from above. This can be dropped again after running it on the production debci instance. * app.js: Filter out query strings from URL for constructing the .json URLs. Fixes wrong URLs after using the package search field. * backends/schroot/test-package: Drop unused temporary dir. * backends/*/test-package: Write exit code into "exitcode" file in output dir, so that we retain it even when we switch to asynchronous/remote processing. * backends/fake/test-package: Write log file into output dir, like adt-run. * Move test result evaluation, log and meta data creation from debci-test to debci-generate-indexes. This prepares the way for running adt-run asynchronously or remotely, and picking up the results from the autopkgtest/ data directory. * Reduce the per-test run .log files to the debci metadata (triggers, package versions, package diffs, overall result), as the actual autopkgtest log is now kept in data/*/autopkgtest/. This avoids duplicating the log contents. * app.js: In the per-package test list, add link to latest autopkgtest log and artifacts (other output files). * debci-status --all: Output a list also if there is only one package. * debci-status --all: Sort package names so that they appear sorted in the web UI. * Avoid some error messages noise when calling generate-index without any existing test data. * Drop debci-test --reason option and make debci-batch put the reason into the status directory directly. * Fold debci-cleanup into debci-generate-index. [ Antonio Terceiro ] * schroot backend: pass --session-id to adt-virt-schroot. This makes it easier to debug test suite that leave rogue schroot sessions behind. * rename `debci` binary to `debci-batch`. `debci` stays reserved for providing a entry point binary like `git` that will invoke subcommands that are kept out of $PATH to avoid cluttering the namespace. * debci-batch: make batch locking suite- and architecture-specific * debci-batch: add --offline and --online options. --offline will prevent new test runs from being started, and --online will revert to normal operation. * Rename scripts/*: - scripts/process-package: absorbed by the previously almost empty bin/debci-test - scripts/list-dep8-packages: absorbed by the previously almost empty bin/debci-list-packages - scripts/list-dependencies → bin/debci-list-dependencies - scripts/setup-chdist → bin/debci-setup-chdist - scripts/blame → bin/debci-blame - scripts/cleanup → bin/debci-cleanup - scripts/generate-index → bin/debci-generate-index * add `debci` binary to not clutter the namespace. When you call `debci COMMAND` it will set PATH to include the debci bin/ and then call `debci-COMMAND` with the supplied arguments. * debci-blame: replace usage of deprecated IO#lines method with IO#each_line * debci-generate-index: accept duration as command line switch * backends/schroot/create-testbed: drop usage of eatmydata to avoid breaking test suites that actually depend on a real fsync(). To compensate the performance loss when installing packages, set dpkg `--force-unsafe-io` option in a configuration file (Closes: #747308). * Added Ruby bindings and the corresponding documentation * Dropped usage of faketime in the test suite (Closes: #747715) * backends/fake: output a timestamp like the ones adt-run outputs * Added new debci-config, a helper to output value of debci configuration values * debci-generate-feeds: added generation of per-package and global Atom feeds. Called from debci-batch * public: now using FontAwesome from fonts-font-awesome package * public/app.js: add a link to the per-package Atom feed * public/style.css: make the status indicators use FontAwesome icons -- Antonio Terceiro Thu, 15 May 2014 12:34:33 -0300 debci (0.5.2) unstable; urgency=medium * scripts/process-package: skip blame check without a dependencies diff * lib/*.sh: guard `set -eu` to avoid doing that on interactive shells * lib/functions.sh: fix list_binaries to work against the sources index - This fixes listing binaries coming from packages that were binNMUed -- Antonio Terceiro Sat, 05 Apr 2014 20:03:22 -0300 debci (0.5.1) unstable; urgency=medium * etc/schroot/debci/fstab: improve schroot isolation - do not share /tmp with host - mount tmpfs at /dev/shm as quite some software needs it * test_shell_best_practices.sh: skip when checkbashisms is not available. - You don't want to install devscripts and all its dependencies when running as-installed tests - checking for best practices is only useful in development environments. * Use unified diff in "Change in the base system since last run" * debian/source/options: fix to include publib/data in the source tarball, but not data/* -- Antonio Terceiro Tue, 01 Apr 2014 12:23:15 -0300 debci (0.5.0) unstable; urgency=medium [ Martin Pitt ] * Add missing dctrl-tools dependency. * Use "dpkg --print-architecture" instead of dpkg-architecture, to avoid a dependency on dpkg-dev and the (undeclared) dependency on gcc. * Fix grep_packages() to actually use the configured architecture instead of the host architecture. * Add --help output to debci-{list-packages,setup,test}, instead of failing with an "usage: not found" error. [ Antonio Terceiro ] * Initial upload to Debian (Closes: #736416) The API is not set yet, but I am uploading anyway to allow for early testing, improve contribution opportunities, to get a BTS etc. * Updated documentation - instructions for setting up a development instance. - Point traffic at debian-qa * modularize behaviour of a backend - existing implementation extracted as the `schroot` backend - added a `fake` backend that helps with testing * schroot backend optimization: always run with eatmydata * Add blame support: when tests of a package start failing after some dependencies were updated, those dependencies will be "blamed" for the failure. * improvements on autopkgtest usage: - debci now uses the autopkgtest support for downloading the source package instead of downloading manually and them calling adt-run - debci now differentiates between the non-zero adt-run exit codes, so that skipped tests will not count as failures. Also added a `tmpfail` status, to be used when for some reason the tests could not be run for some external failure (e.g. mirror failures, broken dependencies etc). * debci now keeps track of the base system. Tests will be run again every 2 weeks even if no explicit dependency changes. Test logs will also display the changes in the base system since the last run. * Add DEP-8 support to debci itself \o/ * extra safety: add `set -u` to all shell scripts -- Antonio Terceiro Mon, 24 Mar 2014 20:06:56 -0300 debci (0.4.0) unstable; urgency=medium * Package renamed to debci * Require a newer autopkgtest. - In general we should always be using a backport of the most recent version in unstable. * Visual facelift - embedding a copy of bootstrap 3 for now :-( * Add backend support for multiple architectures. For now unstable/amd64 is hardcoded, but reading them from the command line or from the environment will not require a large change. -- Antonio Terceiro Tue, 28 Jan 2014 14:40:22 -0300 dep8 (0.3.1) unstable; urgency=medium * Explicitly set PATH in crontab -- Antonio Terceiro Sun, 12 Jan 2014 22:52:00 -0300 dep8 (0.3.0) unstable; urgency=medium * record duration of each run, and record more tham one status entry per day. -- Antonio Terceiro Sun, 12 Jan 2014 19:40:23 -0300 dep8 (0.2.3) unstable; urgency=medium * Makefile: fix creating symlinks for JS files -- Antonio Terceiro Sat, 11 Jan 2014 15:47:00 -0300 dep8 (0.2.2) unstable; urgency=medium * check_version: handle package with more than 1 version -- Antonio Terceiro Sat, 11 Jan 2014 15:41:00 -0300 dep8 (0.2.1) unstable; urgency=medium * environment.sh: print error when dep8_base_dir is not set * cron-wrapper: change to base directory before loading environment -- Antonio Terceiro Mon, 06 Jan 2014 16:22:18 -0300 dep8 (0.2.0) unstable; urgency=medium * keep track of the archive-wide status * process-package: set directory for schroot invocation * generate-index: better naming for status entries * cron-wrapper: fix user detection * Move web UI to public/ subdirectory * add test to check for bashisms * major reorganization for reuse and consolidation * webui: display global status in time series chart * Use explicit directory for gnupg home * Also install a config directory in /etc * ignore new top-level directories -- Antonio Terceiro Mon, 06 Jan 2014 15:44:29 -0300 dep8 (0.1.1) unstable; urgency=medium * Add basic test suite * add `set -e` to all shell scripts -- Antonio Terceiro Sat, 04 Jan 2014 14:20:53 -0300 dep8 (0.1.0) unstable; urgency=low * Initial Release. -- Antonio Terceiro Wed, 01 Jan 2014 01:35:14 -0300