diffoscope (291) unstable; urgency=medium
[ Chris Lamb ]
* Make two required adjustments for the new version of the src:file package:
- file(1) version 5.46 now emits "XHTML document" for .xhtml files, such as
those found nested within our .epub tests. Therefore, match this string
when detecting XML files. This was causing an FTBFS due to inconsistent
indentation in diffoscope's output.
- Require the new, upcoming, version of file(1) for a quine-related
testcase after adjusting the expected output. Previous versions of
file(1) had a duplicated "last modified, last modified" string for some
Zip archives that has now been removed.
* Add a missing subprocess import.
* Bump Standards-Version to 4.7.2.
-- Chris Lamb <lamby@debian.org> Fri, 21 Mar 2025 14:11:47 +0000
diffoscope (290) unstable; urgency=medium
[ Chris Lamb ]
* Also consider .aar files as APK files for the sake of not falling back to a
binary diff. (Closes: #1099632)
* Ensure all calls to out_check_output in the ELF comparator have the
potential CalledProcessError exception caught. (Re: #398)
* Ensure a potential CalledProcessError is caught in the OpenSSL comparator
as well.
* Update copyright years.
[ Eli Schwartz ]
* Drop deprecated and no longer functional "setup.py test" command.
-- Chris Lamb <lamby@debian.org> Fri, 21 Mar 2025 09:06:58 +0000
diffoscope (289) unstable; urgency=medium
[ Chris Lamb ]
* Catch CalledProcessError when calling html2text.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 21 Feb 2025 12:39:56 +0000
diffoscope (288) unstable; urgency=medium
[ Chris Lamb ]
* Add 'asar' to DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS. (Closes: #1095057)
* Update minimal 'black' version.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 07 Feb 2025 10:11:11 +0000
diffoscope (287) unstable; urgency=medium
[ Chris Lamb ]
* Drop an unused subprocess import.
* Update copyright years.
[ fridtjof ]
* Add ASAR comparator.
-- Chris Lamb <lamby@debian.org> Fri, 31 Jan 2025 11:22:37 +0000
diffoscope (286) unstable; urgency=medium
[ Chris Lamb ]
* Bug fixes:
- When passing files on the command line, don't call specialize(..) before
we've checked that the files are identical. In the worst case, this was
resulting in spinning up binwalk and extracting two entire filesystem
images merely to confirm that they were indeed filesystem images..
before simply concluding that they were identical anyway.
- Do not exit with a traceback if paths are inaccessible, either directly,
via symbolic links or within a directory. (Closes: #1065498)
- Correctly identify changes to only the line-endings of files; don't mark
them as "Ordering differences only".
- Use the "surrogateescape" mechanism of str.{decode,encode} to avoid a
UnicodeDecodeError and crash when decoding zipinfo output that is not
valid UTF-8. (Closes: #1093484)
* Testsuite changes:
- Don't mangle newlines when opening test fixtures; we want them untouched.
- Move to assert_diff in test_text.py.
* Misc:
- Remove unnecessary return value from check_for_ordering_differences in
the Difference class.
- Drop an unused function in iso9600.py
- Inline a call/check of Config().force_details; no need for an additional
variable.
-- Chris Lamb <lamby@debian.org> Fri, 24 Jan 2025 12:07:52 +0000
diffoscope (285) unstable; urgency=medium
[ Chris Lamb ]
* Validate --css command-line argument. Thanks to Daniel Schmidt @ SRLabs for
the report. (Closes: #396)
* Prevent XML entity expansion attacks through vulnerable versions of
pyexpat. Thanks to Florian Wilkens @ SRLabs for the report. (Closes: #397)
* Print a warning if we have disabled XML comparisons due to a potentially
vulnerable version of pyexpat.
* Remove (unused) logging facility from a few comparators.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 17 Jan 2025 09:39:49 +0000
diffoscope (284) unstable; urgency=medium
[ Chris Lamb ]
* Simplify tests_quines.py::test_{differences,differences_deb} to use
assert_diff and not mangle the expected test output.
* Update some tests to support file(1) version 5.46.
(Closes: reproducible-builds/diffoscope#395)
-- Chris Lamb <lamby@debian.org> Fri, 06 Dec 2024 10:37:15 +0000
diffoscope (283) unstable; urgency=medium
[ Martin Abente Lahaye ]
* Fix crash when objdump is missing when checking .EFI files.
-- Chris Lamb <lamby@debian.org> Fri, 08 Nov 2024 12:06:11 -0800
diffoscope (282) unstable; urgency=medium
[ Chris Lamb ]
* Ignore errors when listing .ar archives. (Closes: #1085257)
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 25 Oct 2024 10:16:04 -0700
diffoscope (281) unstable; urgency=medium
[ Chris Lamb ]
* Don't try and test with systemd-ukify within Debian stable.
[ Jelle van der Waa ]
* Add support for UKI files.
-- Chris Lamb <lamby@debian.org> Fri, 18 Oct 2024 10:09:08 -0700
diffoscope (280) unstable; urgency=medium
[ Chris Lamb ]
* Drop Depends on deprecated python3-pkg-resources. (Closes: #1083362)
-- Chris Lamb <lamby@debian.org> Fri, 11 Oct 2024 08:07:17 -0700
diffoscope (279) unstable; urgency=medium
[ Chris Lamb ]
* Drop removal of calculated basename from readelf output.
(Closes: reproducible-builds/diffoscope#394)
-- Chris Lamb <lamby@debian.org> Mon, 07 Oct 2024 12:26:08 -0700
diffoscope (278) unstable; urgency=medium
[ Chris Lamb ]
* Temporarily remove procyon-decompiler from Build-Depends as it was removed
from testing (#1057532). (Closes: #1082636)
* Add a helpful contextual message to the output if comparing Debian .orig
tarballs within .dsc files without the ability to "fuzzy-match" away the
leading directory. (Closes: reproducible-builds/diffoscope#386)
* Correctly invert "X% similar" value and do not emit "100% similar".
(Closes: reproducible-builds/diffoscope#391)
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 27 Sep 2024 08:08:56 -0700
diffoscope (277) unstable; urgency=medium
[ Sergei Trofimovich ]
* Don't crash when attempting to hashing symlinks with targets that point to
a directory.
-- Chris Lamb <lamby@debian.org> Fri, 30 Aug 2024 09:51:59 +0100
diffoscope (276) unstable; urgency=medium
[ Chris Lamb ]
* Also catch RuntimeError when importing PyPDF so that PyPDF or, crucially,
its transitive dependencies do not cause diffoscope to traceback at runtime
and build time. (Closes: #1078944, reproducible-builds/diffoscope#389)
* Factor out a method for stripping ANSI escapes.
* Strip ANSI escapes from the output of Procyon. Thanks, Aman Sharma!
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Sun, 18 Aug 2024 18:41:14 +0100
diffoscope (275) unstable; urgency=medium
[ Chris Lamb ]
* Update the test_zip.py text fixtures and definitions to support new changes
to IO::Compress. (Closes: #1078050)
* Do not call marshal.loads(...) of precompiled Python bytecode as it is
inherently unsafe. Replace, at least for now, with a brief summary of the
code section of .pyc files. (Re: reproducible-builds/diffoscope#371)
* Don't bother to check the Python version number in test_python.py: the
fixture for this test is deterministic/fixed.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 16 Aug 2024 11:07:34 +0100
diffoscope (274) unstable; urgency=medium
[ Chris Lamb ]
* Add support for IO::Compress::Zip >= 2.212. (Closes: #1078050)
* Don't include debug output when calling dumppdf(1).
* Append output from dumppdf(1) in more cases.
(Closes: reproducible-builds/diffoscope#387)
* Update copyright years.
[ Mattia Rizzolo ]
* Update the available architectures for test dependencies.
-- Chris Lamb <lamby@debian.org> Fri, 09 Aug 2024 12:52:36 +0100
diffoscope (273) unstable; urgency=medium
[ Chris Lamb ]
* Factor out version detection in test_jpeg_image. (Re:
reproducible-builds/diffoscope#384)
* Ensure that 'convert' is from Imagemagick 6.x; we will need to update a
few things with IM7. (Closes: reproducible-builds/diffoscope#384)
* Correct import of identify_version after refactoring change in 037bdcbb0.
[ Mattia Rizzolo ]
* tests:
+ Add OpenSSH key test with a ed25519 key.
+ Skip the OpenSSH test with DSA key if openssh is >> 9.7
+ Support ffmpeg >= 7 that adds some extra context to the diff
* Do not ignore testing in gitlab-ci.
* debian:
+ Temporarily remove aapt, androguard and dexdump from the build/test
dependencies as they are not available in testin/trixie. Closes: #1070416
+ Bump Standards-Version to 4.7.0, no changes needed.
+ Adjust options to make sure not to pack the python s-dist directory
into the debian source package.
+ Adjust the lintian overrides.
-- Mattia Rizzolo <mattia@debian.org> Wed, 31 Jul 2024 15:31:22 +0900
diffoscope (272) unstable; urgency=medium
[ Chris Lamb]
* Move away from using DSA OpenSSH keys in tests; support has been removed
in OpenSSH 9.8p1. (Closes: reproducible-builds/diffoscope#382)
* Move to assert_diff helper in test_openssh_pub_key.py
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 05 Jul 2024 07:16:14 -0700
diffoscope (271) unstable; urgency=medium
[ Chris Lamb]
* Drop Build-Depends on liblz4-tool. Thanks, Chris Peterson.
(Closes: #1072575)
* Update tests to support zipdetails version 4.004 shipped with Perl 5.40.
(Closes: reproducible-builds/diffoscope#377)
-- Chris Lamb <lamby@debian.org> Fri, 07 Jun 2024 09:33:33 +0100
diffoscope (270) unstable; urgency=medium
[ Chris Lamb ]
* No-change release due to broken version 269 tarballs.
-- Chris Lamb <lamby@debian.org> Tue, 04 Jun 2024 09:16:41 +0100
diffoscope (269) unstable; urgency=medium
[ Chris Lamb ]
* Allow Debian testing continuous integration builds to fail right now.
[ Sergei Trofimovich ]
* Amend 7zip version test for older versions that include the "[64]" string.
(Closes: reproducible-builds/diffoscope#376)
-- Chris Lamb <lamby@debian.org> Fri, 31 May 2024 09:10:35 +0100
diffoscope (268) unstable; urgency=medium
[ Chris Lamb ]
* Drop apktool from Build-Depends; we can still test our APK code
via autopkgtests. (Closes: #1071410)
* Fix tests for 7zip version 24.05.
* Add a versioned dependency for at least version 5.4.5 for the xz
tests; they fail under (at least xz 5.2.8).
(Closes: reproducible-builds/diffoscope#374)
[ Vagrant Cascadian ]
* Relax Chris' versioned xz test dependency (5.4.5) to also allow
version 5.4.1.
-- Chris Lamb <lamby@debian.org> Fri, 24 May 2024 08:29:57 +0100
diffoscope (267) unstable; urgency=medium
[ Chris Lamb ]
* Include "xz --verbose --verbose" (ie. double --verbose) output, not just
the single --verbose. (Closes: #1069329)
* Only include "xz --list" output if the xz has no other differences.
-- Chris Lamb <lamby@debian.org> Fri, 17 May 2024 08:37:23 +0100
diffoscope (266) unstable; urgency=medium
[ Chris Lamb ]
* Use "xz --list" to supplement the output when comparing .xz archives;
essential when some underlying metadata differs. (Closes: #1069329)
* Actually append the xz --list after the container differences, as it
simplifies tests and the output.
* Add 7zip to <!nocheck> Build-Depends in debian/control.
* Update copyright years.
[ James Addison ]
* Maintain an in-header boolean state to determine whether to drop
from-file/to-file lines. This fixes an issue where HTML differences were
being inadvertendly neglected. (Closes: reproducible-builds/diffoscope#372)
-- Chris Lamb <lamby@debian.org> Fri, 10 May 2024 13:49:03 +0100
diffoscope (265) unstable; urgency=medium
[ Chris Lamb ]
* Ensure that tests with ">=" version constraints actually print the
corresponding tool name. (Closes: reproducible-builds/diffoscope#370)
* Prevent odt2txt tests from always being skipped due to an impossibly new
version requirement. (Closes: reproducible-builds/diffoscope#369)
* Avoid nested parens-in-parens when printing "skipping…" messages
in the testsuite.
-- Chris Lamb <lamby@debian.org> Fri, 19 Apr 2024 10:15:50 +0100
diffoscope (264) unstable; urgency=medium
[ Chris Lamb ]
* Don't crash on invalid zipfiles, even if we encounter 'badness'
halfway through the file. (Re: #1068705)
[ FC (Fay) Stegerman ]
* Fix a crash when there are (invalid) duplicate entries in .zip files.
(Closes: #1068705)
* Add note when there are duplicate entries in ZIP files.
(Closes: reproducible-builds/diffoscope!140)
[ Vagrant Cascadian ]
* Add an external tool reference for GNU Guix for zipdetails.
-- Chris Lamb <lamby@debian.org> Fri, 12 Apr 2024 09:38:55 +0100
diffoscope (263) unstable; urgency=medium
[ Chris Lamb ]
* Add support for the zipdetails(1) tool included in the Perl distribution.
Thanks to Larry Doolittle et al. for the pointer to this tool.
* Don't use parenthesis within test "skipping…" messages; PyTest adds its own
parenthesis, so we were ending up with double nested parens.
* Fix the .epub tests after supporting zipdetails(1).
* Update copyright years and debian/tests/control.
[ FC (Fay) Stegerman ]
* Fix MozillaZipContainer's monkeypatch after Python's zipfile module changed
to detect potentially insecure overlapping entries within .zip files.
(Closes: reproducible-builds/diffoscope#362)
-- Chris Lamb <lamby@debian.org> Fri, 05 Apr 2024 12:21:10 +0100
diffoscope (262) unstable; urgency=medium
[ Chris Lamb ]
* Factor out Python version checking in test_zip.py. (Re: #362)
* Also skip some zip tests under 3.10.14 as well; a potential regression may
have been backported to the 3.10.x series. The underlying cause is still to
be investigated. (Re: #362)
-- Chris Lamb <lamby@debian.org> Fri, 29 Mar 2024 09:43:00 +0000
diffoscope (261) unstable; urgency=medium
[ Chris Lamb ]
* Don't crash if we encounter an .rdb file without an equivalent .rdx file.
(Closes: #1066991)
* In addition, don't identify Redis database dumps (etc.) as GNU R database
files based simply on their filename. (Re: #1066991)
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 22 Mar 2024 09:42:15 +0000
diffoscope (260) unstable; urgency=medium
[ Chris Lamb ]
* Actually test 7z support in the test_7z set of tests, not the lz4
functionality. (Closes: reproducible-builds/diffoscope#359)
* In addition, correctly check for the 7z binary being available
(and not lz4) when testing 7z.
* Prevent a traceback when comparing a contentful .pyc file with an
empty one. (Re: Debian:#1064973)
-- Chris Lamb <lamby@debian.org> Fri, 08 Mar 2024 11:07:49 +0000
diffoscope (259) unstable; urgency=medium
[ Chris Lamb ]
* Don't error-out with a traceback if we encounter "struct.unpack"-related
errors when parsing .pyc files. (Closes: #1064973)
* Fix compatibility with PyTest 8.0. (Closes: reproducible-builds/diffoscope#365)
* Don't try and compare rdb_expected_diff on non-GNU systems as %p formatting
can vary. (Re: reproducible-builds/diffoscope#364)
-- Chris Lamb <lamby@debian.org> Fri, 01 Mar 2024 09:34:23 +0000
diffoscope (258) unstable; urgency=medium
[ Chris Lamb ]
* Use the 7zip package (over p7zip-full) after package transition.
(Closes: #1063559)
* Update debian/tests/control.
[ Vagrant Cascadian ]
* Fix a typo in the package name field (!) within debian/changelog.
-- Chris Lamb <lamby@debian.org> Fri, 23 Feb 2024 11:31:52 +0000
diffoscope (257) unstable; urgency=medium
[ James Addison ]
* Parse the header and hunksize of diffs strictly before parsing the context
below. (Closes: reproducible-builds/diffoscope#363)
* Reformat code to comply with the latest version of Black (24.1.1).
[ Chris Lamb ]
* Expand the previous changelog entry to include the CVE number that was
subsequently assigned.
* Bump the miniumum Black requirement to run the "Black clean" test and make
test_zip.py Black clean.
-- Chris Lamb <lamby@debian.org> Mon, 12 Feb 2024 10:08:35 -0800
diffoscope (256) unstable; urgency=high
* CVE-2024-25711: Use a determistic name when extracting content from GPG
artifacts instead of trusting the value of gpg's --use-embedded-filenames.
This prevents a potential information disclosure vulnerability that could
have been exploited by providing a specially-crafted GPG file with an
embedded filename of, say, "../../.ssh/id_rsa".
Many thanks to Daniel Kahn Gillmor <dkg@debian.org> for reporting this
issue and providing feedback.
(Closes: reproducible-builds/diffoscope#361)
* Temporarily fix support for Python 3.11.8 re. a potential regression
with the handling of ZIP files. (See reproducible-builds/diffoscope#362)
-- Chris Lamb <lamby@debian.org> Fri, 09 Feb 2024 12:22:37 -0800
diffoscope (255) unstable; urgency=medium
[ Vekhir ]
* Add/fix compatibility for Python progressbar 2.5 & 3.0 etc.
[ Chris Lamb ]
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 26 Jan 2024 08:44:05 -0800
diffoscope (254) unstable; urgency=medium
[ Chris Lamb ]
* Reflow some code according to black.
[ Seth Michael Larson ]
* Add support for comparing the 'eXtensible ARchive' (.XAR/.PKG) file format.
[ Vagrant Cascadian ]
* Add external tool on GNU Guix for 7z.
-- Chris Lamb <lamby@debian.org> Fri, 19 Jan 2024 09:32:28 -0800
diffoscope (253) unstable; urgency=medium
* Improve DOS/MBR extraction by adding support for 7z.
(Closes: reproducible-builds/diffoscope#333)
* Process objdump symbol comment filter inputs as the Python "bytes" type
(and not str). (Closes: reproducible-builds/diffoscope#358)
* Add a missing RequiredToolNotFound import.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 08 Dec 2023 12:25:12 +0000
diffoscope (252) unstable; urgency=medium
* As UI/UX improvement, try and avoid printing an extended traceback if
diffoscope runs out of memory. This may not always be possible to detect.
* Mark diffoscope as stable in setup.py (for PyPI.org). Whatever diffoscope
is, at least, not "alpha" anymore.
-- Chris Lamb <lamby@debian.org> Fri, 17 Nov 2023 08:18:10 +0000
diffoscope (251) unstable; urgency=medium
* If the equivalent of `file -i` returns text/plain, fallback to comparing
this file as a text file. This especially helps when file(1) miscategorises
text files as some esoteric type. (Closes: Debian:#1053668)
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 13 Oct 2023 08:59:12 +0100
diffoscope (250) unstable; urgency=medium
[ Chris Lamb ]
* Fix compatibility with file 5.45. (Closes: reproducible-builds/diffoscope#351)
[ Vagrant Cascadian ]
* Add external tool references for GNU Guix (for html2text and ttx).
-- Chris Lamb <lamby@debian.org> Fri, 08 Sep 2023 12:10:31 -0700
diffoscope (249) unstable; urgency=medium
[ FC Stegerman ]
* Add specialize_as() method, and use it to speed up .smali comparison in
APKs. (Closes: reproducible-builds/diffoscope!108)
[ Chris Lamb ]
* Add documentation for the new specialize_as, and expand the documentation
of `specialize` too. (Re: reproducible-builds/diffoscope!108)
* Update copyright years.
[ Felix Yan ]
* Correct typos in diffoscope/presenters/utils.py.
-- Chris Lamb <lamby@debian.org> Fri, 01 Sep 2023 10:41:51 -0700
diffoscope (248) unstable; urgency=medium
[ Greg Chabala ]
* Merge Docker "RUN" commands into single layer.
-- Chris Lamb <lamby@debian.org> Fri, 25 Aug 2023 08:20:33 -0700
diffoscope (247) unstable; urgency=medium
[ Chris Lamb ]
* Fix compataibility with file(1) version 5.45.
* Use assert_diff in test_uimage and test_cpio.
[ Roland Clobus ]
* xb-tool has moved in Debian bookworm.
-- Chris Lamb <lamby@debian.org> Fri, 04 Aug 2023 09:37:54 +0100
diffoscope (246) unstable; urgency=medium
[ Gianfranco Costamagna ]
* Add support for LLVM 16.
-- Chris Lamb <lamby@debian.org> Fri, 28 Jul 2023 08:57:05 +0100
diffoscope (245) unstable; urgency=medium
[ Chris Lamb ]
* Don't include file size in image metadata; it is, at best, distracting and
it is already in the directory metadata.
* Move to using assert_diff in ICO and JPEG tests.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 21 Jul 2023 08:57:41 +0100
diffoscope (244) unstable; urgency=medium
[ Chris Lamb ]
* Address compatibility with python-libarchive-c version 5.
(Closes: reproducible-builds/diffoscope#344)
* Testsuite changes:
- Mark that test_dex::test_javap_14_differences requires procyon.
- Fix "test skipped" textual reason generation in the case of a required
version being outside of the required range.
- Temporarily mark some Android-related as XFAIL due to Debian bugs
#1040941 and #1040916.
-- Chris Lamb <lamby@debian.org> Fri, 14 Jul 2023 13:07:47 +0100
diffoscope (243) unstable; urgency=medium
[ Chris Lamb ]
* Drop Jenkins build reference in README.rst.
[ Ed Maste ]
* Update FreeBSD package names
[ Mattia Rizzolo ]
* Improve the documentation on to produce that binary blob that in the arsc
comparator.
-- Chris Lamb <lamby@debian.org> Fri, 23 Jun 2023 17:11:25 +0100
diffoscope (242) unstable; urgency=medium
* If the binwalk Python module is not available, ensure the user knows they
may be missing more differences in, for example, concatenated .cpio
archives.
* Factor out routine to generate a human-readable comments when
Python modules are missing.
-- Chris Lamb <lamby@debian.org> Fri, 05 May 2023 12:05:28 -0700
diffoscope (241) unstable; urgency=medium
[ Chris Lamb ]
* Add a missing 'raise' statement dropped in 2d95ae41e. Thanks, Mattia!
[ Mattia Rizzolo ]
* document sending out an email upon release
-- Chris Lamb <lamby@debian.org> Fri, 21 Apr 2023 09:26:15 +0100
diffoscope (240) unstable; urgency=medium
[ Holger Levsen ]
* Update Lintian override info format in debian/source/lintian-overrides.
* Add Lintian overrides for some "very long lines" in test cases.
* Update Lintian overrides for tests being tagged source-is-missing or
prebuilt.
* Add Lintian override for very long lines for debian/tests/control.
* Re-add two Lintian overrides about (well-known) source-is-missing
instances.
[ Mattia Rizzolo ]
* Drop the use of include_package_data=True in setup.py.
-- Chris Lamb <lamby@debian.org> Fri, 31 Mar 2023 09:05:24 +0100
diffoscope (239) unstable; urgency=medium
[ Chris Lamb ]
* Fix compatibility with pypdf 3.x, and correctly restore test data.
(Closes: reproducible-builds/diffoscope#335)
* Rework PDF annotations processing into a separate method.
-- Chris Lamb <lamby@debian.org> Fri, 17 Mar 2023 08:31:17 +0000
diffoscope (238) unstable; urgency=medium
* autopkgtest: fix tool name in the skippable list.
-- Mattia Rizzolo <mattia@debian.org> Wed, 01 Mar 2023 20:17:05 +0100
diffoscope (237) unstable; urgency=medium
* autopkgtest: only install aapt and dexdump on architectures where they are
available. (Closes: #1031297)
* compartors/pdf:
+ Drop backward compatibility assignment.
+ Fix flake warnings, potentially reinstating PyPDF 1.x support (untested).
-- Mattia Rizzolo <mattia@debian.org> Tue, 28 Feb 2023 18:16:09 +0100
diffoscope (236) unstable; urgency=medium
[ FC Stegerman ]
* Update code to match latest version of Black. (Closes: #1031433)
[ Chris Lamb ]
* Require at least Black version 23.1.0 to run the internal Black tests.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 17 Feb 2023 08:48:41 -0800
diffoscope (235) unstable; urgency=medium
[ Akihiro Suda ]
* Update .gitlab-ci.yml to push versioned tags to the container registry.
(Closes: reproducible-builds/diffoscope!119)
[ Chris Lamb ]
* Fix compatibility with PyPDF2. (Closes: reproducible-builds/diffoscope#331)
* Fix compatibility with ImageMagick 7.1.
(Closes: reproducible-builds/diffoscope#330)
[ Daniel Kahn Gillmor ]
* Update from PyPDF2 to pypdf. (Closes: #1029741, #1029742)
[ FC Stegerman ]
* Add support for Android resources.arsc files.
(Closes: reproducible-builds/diffoscope!116)
* Add support for dexdump. (Closes: reproducible-builds/diffoscope#134)
* Improve DexFile's FILE_TYPE_RE and add FILE_TYPE_HEADER_PREFIX, and remove
"Dalvik dex file" from ApkFile's FILE_TYPE_RE as well.
[ Efraim Flashner ]
* Update external tool for isoinfo on guix.
(Closes: reproducible-builds/diffoscope!124)
-- Chris Lamb <lamby@debian.org> Fri, 10 Feb 2023 10:31:52 -0800
diffoscope (234) unstable; urgency=medium
[ FC Stegerman ]
* test_text_proper_indentation requires at least file version 5.44.
(Closes: reproducible-builds/diffoscope#329)
-- Chris Lamb <lamby@debian.org> Mon, 30 Jan 2023 09:29:37 -0800
diffoscope (233) unstable; urgency=medium
[ FC Stegerman ]
* Split packaging metadata into an extras_require.json file instead of using
the pep517 and the pip modules directly. This was causing build failures if
not using a virtualenv and/or building without internet access.
(Closes: #1029066, reproducible-builds/diffoscope#325)
[ Vagrant Cascadian ]
* Add an external tool reference for GNU Guix (lzip).
* Drop an external tool reference for GNU Guix (pedump).
[ Chris Lamb ]
* Split inline Python code in shell script to generate test dependencies to a
separate Python script.
* No need for "from __future__ import print_function" import in setup.py
anymore.
* Comment and tidy the new extras_require.json handling.
-- Chris Lamb <lamby@debian.org> Fri, 20 Jan 2023 08:56:22 -0800
diffoscope (232) unstable; urgency=medium
[ Chris Lamb ]
* Allow ICC tests to (temporarily) fail.
* Update debian/tests/control after the addition of PyPDF 3 support.
[ FC Stegerman ]
* Update regular expression for Android .APK files.
[ Sam James ]
* Support PyPDF version 3.
-- Chris Lamb <lamby@debian.org> Fri, 13 Jan 2023 07:05:01 +0000
diffoscope (231) unstable; urgency=medium
* Improve "[X] may produce better output" messages. Based on a patch by
Helmut Grohne. (Closes: #1026982)
-- Chris Lamb <lamby@debian.org> Fri, 06 Jan 2023 09:02:57 +0000
diffoscope (230) unstable; urgency=medium
[ Chris Lamb ]
* Fix compatibility with file(1) version 5.43; thanks, Christoph Biedl.
[ Jelle van der Waa ]
* Support Berkeley DB version 6.
-- Chris Lamb <lamby@debian.org> Fri, 30 Dec 2022 06:35:04 +0000
diffoscope (229) unstable; urgency=medium
[ Chris Lamb ]
* Skip test_html.py::test_diff if html2text is not installed.
(Closes: #1026034)
[ Holger Levsen ]
* Bump standards version to 4.6.2, no changes needed.
-- Chris Lamb <lamby@debian.org> Tue, 20 Dec 2022 23:12:16 +0000
diffoscope (228) unstable; urgency=medium
[ FC Stegerman ]
* As an optimisation, don't run apktool if no differences are detected before
the signing block. (Closes: reproducible-builds/diffoscope!105)
[ Chris Lamb ]
* Support both the python3-progressbar and python3-progressbar2 Debian
packages, two modules providing the "progressbar" Python module.
(Closes: reproducible-builds/diffoscope#323)
* Ensure we recommend apksigcopier. (Re: reproducible-builds/diffoscope!105)
* Make the code clearer around generating the Debian substvars and tidy
generation of os_list.
* Update copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 02 Dec 2022 08:10:44 +0000
diffoscope (227) unstable; urgency=medium
[ Chris Lamb ]
* Don't attempt to attach text-only differences notice if there are no
differences to begin with. (Closes: #1024171, #1024349)
* Don't run Python decompiling tests on Python bytecode that both file(1)
cannot yet detect and Python 3.11 cannot demarshall. (Closes: #1024335)
-- Chris Lamb <lamby@debian.org> Fri, 18 Nov 2022 09:38:34 +0000
diffoscope (226) unstable; urgency=medium
[ Christopher Baines ]
* Add an lzip comparator with tests.
[ Chris Lamb ]
* Add support for comparing the "text" content of HTML files using html2text.
(Closes: #1022209, reproducible-builds/diffoscope#318)
* Misc/test improvements:
* Drop the ALLOWED_TEST_FILES test; it's mostly just annoying.
* Drop other copyright notices from lzip.py and test_lzip.py.
* Use assert_diff helper in test_lzip.py.
* Pylint tests/test_source.py.
[ Mattia Rizzolo ]
* Add lzip to debian dependencies.
-- Chris Lamb <lamby@debian.org> Fri, 04 Nov 2022 06:51:43 +0000
diffoscope (225) unstable; urgency=medium
[ Chris Lamb ]
* Add support for detecting ordering-only differences in XML files.
(Closes: #1022146)
* Fix an issue with detecting ordering differences. (Closes: #1022145)
* Add support for ttx(1) from fonttools.
(Re: reproducible-builds/diffoscope#315)
* Test improvements:
- Tidy up the JSON tests and use assert_diff over get_data and manual
assert in XML tests.
- Rename order1.diff to json_expected_ordering_diff for consistency.
- Temporarily allow the stable-po pipeline to fail in the CI.
* Use consistently capitalised "Ordering" everywhere we use the word in
diffoscope's output.
-- Chris Lamb <lamby@debian.org> Fri, 28 Oct 2022 08:29:53 -0700
diffoscope (224) unstable; urgency=medium
[ Mattia Rizzolo ]
* Fix rlib test failure with LLVM 15. Thanks to Gianfranco Costamagna
(locutusofborg) for the patch.
-- Chris Lamb <lamby@debian.org> Fri, 07 Oct 2022 09:35:04 -0700
diffoscope (223) unstable; urgency=medium
[ Chris Lamb ]
* The cbfstools utility is now provided in Debian via the coreboot-utils
Debian package, so we can enable that functionality within Debian.
(Closes: #1020630)
[ Mattia Rizzolo ]
* Also include coreboot-utils in Build-Depends and Test-Depends so it is
available for the tests.
[ Jelle van der Waa ]
* Add support for file 5.43.
-- Chris Lamb <lamby@debian.org> Fri, 30 Sep 2022 08:25:31 -0700
diffoscope (222) unstable; urgency=medium
[ Mattia Rizzolo ]
* Use pep517 and pip to load the requirements. (Closes: #1020091)
* Remove old Breaks/Replaces in debian/control that have been obsoleted since
bullseye
-- Chris Lamb <lamby@debian.org> Fri, 23 Sep 2022 09:05:12 +0100
diffoscope (221) unstable; urgency=medium
* Don't crash if we can open a PDF file with PyPDF but cannot parse the
annotations within. (Closes: reproducible-builds/diffoscope#311)
* Depend on the dedicated xxd package, not vim-common.
* Update external_tools.py to reflect xxd/vim-common change.
-- Chris Lamb <lamby@debian.org> Fri, 19 Aug 2022 07:32:23 -0700
diffoscope (220) unstable; urgency=medium
* Support Haskell 9.x series files and update the test files to match. Thanks
to Scott Talbert for the relevant info about the new format.
(Closes: reproducible-builds/diffoscope#309)
* Fix a regression introduced in diffoscope version 207 where diffoscope
would crash if one directory contained a directory that wasn't in the
other. Thanks to Alderico Gallo for the report and the testcase.
(Closes: reproducible-builds/diffoscope#310)
-- Chris Lamb <lamby@debian.org> Fri, 29 Jul 2022 11:03:24 -0700
diffoscope (219) unstable; urgency=medium
* Don't traceback if we encounter an invalid Unicode character in Haskell
versioning headers. (Closes: reproducible-builds/diffoscope#307)
* Update various copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 15 Jul 2022 06:45:55 +0100
diffoscope (218) unstable; urgency=medium
* Improve output of Markdown and reStructuredText to use code blocks with
syntax highlighting. (Closes: reproducible-builds/diffoscope#306)
-- Chris Lamb <lamby@debian.org> Fri, 08 Jul 2022 08:45:11 +0100
diffoscope (217) unstable; urgency=medium
* Update test fixtures for GNU readelf 2.38 (now in Debian unstable).
* Be more specific about the minimum required version of readelf (ie.
binutils) as it appears that this "patch" level version change resulted in
a change of output, not the "minor" version. (Closes: #1013348)
* Don't leak the (likely-temporary) pathname when comparing PDF documents.
-- Chris Lamb <lamby@debian.org> Thu, 23 Jun 2022 07:43:21 +0100
diffoscope (216) unstable; urgency=medium
* Print profile output if we were called with --profile and we receive a
TERM signal.
* Emit a warning if/when we are handling a TERM signal.
* Clarify in the code in what situations the main "finally" block gets
called, especially in relation to handling TERM signals.
* Clarify and tidy some unconditional control flow in diffoscope.profiling.
-- Chris Lamb <lamby@debian.org> Fri, 10 Jun 2022 07:00:54 +0100
diffoscope (215) unstable; urgency=medium
[ Chris Lamb ]
* Bug fixes:
- Also catch IndexError (in addition to ValueError) when parsing .pyc
files. (Closes: #1012258)
- Strip "sticky" etc. from "x.deb: sticky Debian binary package […]".
Thanks to David Prévot for the report. (Closes: #1011635)
- Correctly package diffoscope's scripts/ directory, fixing the extraction
of vmlinuz kernel images. (Closes: reproducible-builds/diffoscope#305)
- Correct the logic for supporting different versions of argcomplete in
debian/rules.
* New features:
- Support both PyPDF 1.x and 2.x.
* Codebase improvements:
- Don't call re.compile and then call .sub on the result; just call
re.sub directly.
- Clarify the logic around the difference between --usage and --help.
* Testsuite improvements:
- Integrate test coverage with GitLab's concept of artifacts.
- Re-enable Gnumeric tests as its now available again.
- Test --help and --usage, and additionally test that --help includes
the programmatically-generated file format list as well.
[ Holger Levsen ]
* Bump Standards Version field in debian/control to 4.6.1.
-- Chris Lamb <lamby@debian.org> Fri, 03 Jun 2022 06:54:48 +0100
diffoscope (214) unstable; urgency=medium
[ Chris Lamb ]
* Support both python-argcomplete 1.x and 2.x.
[ Vagrant Cascadian ]
* Add external tool on GNU Guix for xb-tool.
-- Chris Lamb <lamby@debian.org> Fri, 27 May 2022 06:53:04 +0100
diffoscope (213) unstable; urgency=medium
* Don't mask differences in .zip/.jar central directory extra fields.
* Don't show a binary comparison of .zip/.jar files if we have at least
one observed nested difference.
* Use assert_diff in test_zip over get_data and separate assert.
-- Chris Lamb <lamby@debian.org> Fri, 20 May 2022 08:15:29 -0700
diffoscope (212) unstable; urgency=medium
* Add support for extracting vmlinuz/vmlinux Linux kernel images.
(Closes: reproducible-builds/diffoscope#304)
* Some Python .pyc files report as "data", so support ".pyc" as a
fallback extension.
-- Chris Lamb <lamby@debian.org> Fri, 13 May 2022 08:19:57 -0700
diffoscope (211) unstable; urgency=medium
[ Mattia Rizzolo ]
* Drop mplayer from the Build-Depends, it was add likely by accident and it's
not needed.
* Disable gnumeric tests in Debian because it's not currently available.
-- Chris Lamb <lamby@debian.org> Thu, 28 Apr 2022 08:20:23 -0700
diffoscope (210) unstable; urgency=medium
[ Mattia Rizzolo ]
* Make sure that PATH is properly mangled for all diffoscope actions, not
just when running comparators.
-- Chris Lamb <lamby@debian.org> Fri, 15 Apr 2022 09:02:25 +0100
diffoscope (209) unstable; urgency=medium
* Update R test fixture for R 4.2.x series. (Closes: #1008446)
* Update minimum version of Black to prevent test failure on Ubuntu jammy.
-- Chris Lamb <lamby@debian.org> Sun, 27 Mar 2022 09:23:02 +0100
diffoscope (208) unstable; urgency=medium
[ Brent Spillner ]
* Add graceful handling for UNIX sockets and named pipes.
(Closes: reproducible-builds/diffoscope#292)
* Remove a superfluous log message and reformatt comment lines.
[ Chris Lamb ]
* Reformat various files to satisfy current version of Black.
-- Chris Lamb <lamby@debian.org> Fri, 25 Mar 2022 08:24:33 +0000
diffoscope (207) unstable; urgency=medium
* Fix a gnarly regression when comparing directories against non-directories.
(Closes: reproducible-builds/diffoscope#292)
* Use our assert_diff utility where we can within test_directory.py
-- Chris Lamb <lamby@debian.org> Fri, 04 Mar 2022 12:55:26 +0000
diffoscope (206) unstable; urgency=medium
* Also allow "Unicode text, UTF-8 text" as well as "UTF-8 Unicode text" to
match for .buildinfo files too.
* Add a test for recent file(1) issue regarding .changes files.
(Re: reproducible-builds/diffoscope#291)
* Drop "_PATH" suffix from some module-level globals that are not paths.
-- Chris Lamb <lamby@debian.org> Fri, 25 Feb 2022 08:03:04 +0000
diffoscope (205) unstable; urgency=medium
* Fix a file(1)-related regression where .changes files that contained
non-ASCII text were not identified as being .changes files, resulting in
seemingly arbitrary packages on tests.reproducible-builds.org and elswhere
not comparing the package at all. The non-ASCII parts could have been in
the Maintainer or in the upload changelog, so we were effectively
penalising anyone outside of the Anglosphere.
(Closes: reproducible-builds/diffoscope#291)
* Don't print a warning to the console regarding NT_GNU_BUILD_ID changes in
ELF binaries.
-- Chris Lamb <lamby@debian.org> Sat, 19 Feb 2022 14:58:57 -0800
diffoscope (204) unstable; urgency=medium
[ Chris Lamb ]
* Don't run the binwalk comparator tests as root (or fakeroot) as the
latest version of binwalk has some security protection against doing
precisely this.
* If we fail to scan a file using binwalk, return 'False' from
BinwalkFile.recognizes rather than raise a traceback.
* If we fail to import the Python "binwalk" module, don't accidentally report
that we are missing the "rpm" module instead.
[ Mattia Rizzolo ]
* Use dependencies to ensure that "diffoscope" and "diffoscope-minimal"
packages always have the precise same version.
-- Chris Lamb <lamby@debian.org> Fri, 11 Feb 2022 10:39:27 -0800
diffoscope (203) unstable; urgency=medium
[ Chris Lamb ]
* Improve documentation for --timeout due to a few misconceptions.
Add an allowed-to-fail test regarding a regression in directory handling.
* Tidy control flow in Difference._reverse_self a little.
[ Alyssa Ross ]
* Fix diffing CBFS names that contain spaces.
-- Chris Lamb <lamby@debian.org> Fri, 04 Feb 2022 08:13:13 -0800
diffoscope (202) unstable; urgency=medium
[ Chris Lamb ]
* Don't fail if comparing a nonexistent file with a .pyc file (and add test).
(Closes: #1004312)
* Drop a reference in the manual page which claims the ability to compare
non-existent files on the command-line. This has not been possible since
version 32 which was released in September 2015. (Closes: #1004182)
* Add experimental support for incremental output support with a timeout.
Passing, for example, --timeout=60 will mean that diffoscope will not
recurse into any sub-archives after 60 seconds total execution time has
elapsed and mark the diff as being incomplete. (Note that this is not a
fixed/strict timeout due to implementation issues.)
(Closes: reproducible-builds/diffoscope#301)
* Don't return with an exit code of 0 if we encounter device file such as
/dev/stdin with human-readable metadata that matches literal, non-device,
file contents. (Closes: #1004198)
* Correct a "recompile" typo.
[ Sergei Trofimovich ]
* Fix/update whitespace for Black 21.12.
-- Chris Lamb <lamby@debian.org> Fri, 28 Jan 2022 08:17:21 -0800
diffoscope (201) unstable; urgency=medium
[ Chris Lamb ]
* If the debian.deb822 module raises any exception on import, re-raise it as
an ImportError instead. This should fix diffoscope on some Fedora systems.
Thanks to Mattia Rizzolo for suggesting this particular solution.
(Closes: reproducible-builds/diffoscope#300)
[ Zbigniew Jędrzejewski-Szmek ]
* Fix json detection with file-5.41-3.fc36.x86_64.
-- Chris Lamb <lamby@debian.org> Fri, 21 Jan 2022 08:35:36 -0800
diffoscope (200) unstable; urgency=medium
* Even if a Sphinx .inv inventory file is labelled "The remainder of this
file is compressed using zlib", it might not actually be. In this case,
don't traceback, and simply return the original content.
(Closes: reproducible-builds/diffoscope#299)
* Update "X has been modified after NT_GNU_BUILD_ID has been applied" message
to, for instance, not duplicating the full filename in the primary
diffoscope's output.
-- Chris Lamb <lamby@debian.org> Fri, 14 Jan 2022 09:15:19 +0000
diffoscope (199) unstable; urgency=medium
[ Chris Lamb ]
* Support both variants of "odt2txt", including the one provided by unoconv.
(Closes: reproducible-builds/diffoscope#298)
[ Jelle van der Waa ]
* Add external tool reference on Arch Linux for xb-tool.
-- Chris Lamb <lamby@debian.org> Fri, 07 Jan 2022 08:46:42 +0000
diffoscope (198) unstable; urgency=medium
[ Chris Lamb ]
* Support showing "Ordering differences only" within .dsc field values.
(Closes: #1002002, reproducible-builds/diffoscope#297)
* Support OCaml versions 4.11, 4.12 and 4.13. (Closes: #1002678)
* Add support for XMLb files. (Closes: reproducible-builds/diffoscope#295)
* Also add, for example, /usr/lib/x86_64-linux-gnu to our internal PATH.
[ Mattia Rizzolo ]
* Also recognize "GnuCash file" files as XML.
-- Chris Lamb <lamby@debian.org> Fri, 31 Dec 2021 08:52:04 +0000
diffoscope (197) unstable; urgency=medium
[ Chris Lamb ]
* Drop unnecessary has_same_content_as logging calls.
[ Mattia Rizzolo ]
* Ignore the new "binary-with-bad-dynamic-table" Lintian tag.
* Support pgpdump 0.34 in the tests. Thanks to Michael Weiss
<dev.primeos@gmail.com> for reporting and testing the fix.
-- Chris Lamb <lamby@debian.org> Fri, 17 Dec 2021 09:56:04 +0000
diffoscope (196) unstable; urgency=medium
[ Roland Clobus ]
* Add a comment/annotation when the GNU_BUILD_ID field has been modified.
[ Brent Spillner ]
* Fix the "Black" version detection.
[ Chris Lamb ]
* Replace "token" with anonymous variable "x" in order to remove extra lines.
-- Chris Lamb <lamby@debian.org> Fri, 10 Dec 2021 08:05:10 -0800
diffoscope (195) unstable; urgency=medium
[ Chris Lamb ]
* Don't use the runtime platform's native endianness when unpacking .pyc
files to fix test failures on big-endian machines.
-- Chris Lamb <lamby@debian.org> Sun, 05 Dec 2021 09:30:08 -0800
diffoscope (194) unstable; urgency=medium
[ Chris Lamb ]
* Don't traceback when comparing nested directories with non-directories.
(Closes: reproducible-builds/diffoscope#288)
-- Chris Lamb <lamby@debian.org> Fri, 26 Nov 2021 10:01:05 -0800
diffoscope (193) unstable; urgency=medium
[ Chris Lamb ]
* Don't duplicate file lists at each directory level.
(Closes: #989192, reproducible-builds/diffoscope#263)
* When pretty-printing JSON, mark the difference as such, additionally
avoiding including the full path.
(Closes: reproducible-builds/diffoscope#205)
* Codebase improvements:
- Update a bunch of %-style string interpolations into f-strings or
str.format.
- Import itertools top-level directly.
- Drop some unused imports.
- Use isinstance(...) over type(...) ==
- Avoid aliasing variables if we aren't going to use them.
[ Brandon Maier ]
* Fix missing diff output on large diffs.
[ Mattia Rizzolo ]
* Ignore a Python warning coming from a dependent library (triggered by
supporting Python 3.10)
* Document that support both Python 3.9 and 3.10.
-- Chris Lamb <lamby@debian.org> Fri, 19 Nov 2021 07:35:10 -0800
diffoscope (192) unstable; urgency=medium
* Update .epub test methodology after improving XML file parsing.
-- Chris Lamb <lamby@debian.org> Fri, 12 Nov 2021 08:17:14 -0800
diffoscope (191) unstable; urgency=medium
[ Chris Lamb ]
* Detect XML files as XML files if either file(1) claims if they are XML
files, or if they are named .xml.
(Closes: #999438, reproducible-builds/diffoscope#287)
* Don't reject Debian .changes files if they contain non-printable
characters. (Closes: reproducible-builds/diffoscope#286)
* Continue loading a .changes file even if the referenced files inside it do
not exist, but include a comment in the diff as a result.
* Log the reason if we cannot load a Debian .changes file.
[ Zbigniew Jędrzejewski-Szmek ]
* Fix inverted logic in the assert_diff_startswith() utility.
-- Chris Lamb <lamby@debian.org> Fri, 12 Nov 2021 06:43:57 -0800
diffoscope (190) unstable; urgency=medium
[ Chris Lamb ]
* Don't raise a traceback if we cannot de-marshal Python bytecode to support
Python 3.7 loading newer .pyc files.
(Closes: reproducible-builds/diffoscope#284)
* Fix Python tests under Python 3.7 with file 5.39+.
[ Vagrant Cascadian ]
* Skip Python bytecode testing when "file" is older than 5.39.
[ Roland Clobus ]
* Detect whether the GNU_BUILD_ID field has been modified.
-- Chris Lamb <lamby@debian.org> Fri, 05 Nov 2021 08:47:27 +0000
diffoscope (189) unstable; urgency=medium
[ Chris Lamb ]
* Try some alternative suffixes (eg. ".py") to support distributions that
strip or retain them. (Closes: reproducible-builds/diffoscope#283)
* Skip Python bytecode testing where we do not have an expected diff.
(Closes: reproducible-builds/diffoscope#284)
* Refactor the find_executable utility into an explicit method.
* Split out a custom call to assert_diff to support a .startswith equivalent.
* Use skipif instead of manual conditionals in some tests.
[ Vagrant Cascadian ]
* Add an external tool reference for Guix to support ppudump and dumppdf.
[ Sergei Trofimovich ]
* Update uImage test output for file(1) version 5.41.
[ Jelle van der Waa ]
* Add Arch Linux as CI test target.
* Add external tools on Arch Linux for ffmpeg, openssl and ocalobjinfo.
-- Chris Lamb <lamby@debian.org> Fri, 29 Oct 2021 09:47:04 +0100
diffoscope (188) unstable; urgency=medium
* Add support for Python Sphinx inventory files, usually named objects.inv.
* Fix Python bytecode decompilation tests with Python 3.10+.
(Closes: reproducible-builds/diffoscope#278)
-- Chris Lamb <lamby@debian.org> Fri, 22 Oct 2021 09:08:55 +0100
diffoscope (187) unstable; urgency=medium
* Add support for comparing .pyc files. Thanks to Sergei Trofimovich.
(Closes: reproducible-builds/diffoscope#278)
-- Chris Lamb <lamby@debian.org> Fri, 08 Oct 2021 09:01:52 +0100
diffoscope (186) unstable; urgency=medium
[ Chris Lamb ]
* Don't call close_archive when garbage-collecting Archive instances unless
open_archive returned successfully. This prevents, amongst others, an
AttributeError traceback due to PGPContainer's cleanup routines assuming
that its temporary directory had been created.
(Closes: reproducible-builds/diffoscope#276)
* Ensure that the string "RPM archives" exists in the package description,
regardless of whether python3-rpm is installed or not at build time.
[ Jean-Romain Garnier ]
* Fix the LVM Macho comparator for non-x86-64 architectures.
-- Chris Lamb <lamby@debian.org> Fri, 01 Oct 2021 09:02:48 +0100
diffoscope (185) unstable; urgency=medium
[ Mattia Rizzolo ]
* Fix the autopkgtest in order to fix testing migration: the androguard
Python module is not in the python3-androguard Debian package
* Ignore a warning in the tests from the h5py package that doesn't concern
diffoscope.
[ Chris Lamb ]
* Bump Standards-Version to 4.6.0.
-- Chris Lamb <lamby@debian.org> Tue, 21 Sep 2021 16:15:17 +0100
diffoscope (184) unstable; urgency=medium
[ Chris Lamb ]
* Fix the semantic comparison of R's .rdb files after a refactoring of
temporary directory handling in a previous version.
* Support a newer format version of R's .rds files.
* Update tests for OCaml 4.12. (Closes: reproducible-builds/diffoscope#274)
* Move diffoscope.versions to diffoscope.tests.utils.versions.
* Use assert_diff in tests/comparators/test_rdata.py.
* Reformat various modules with Black.
[ Zbigniew Jędrzejewski-Szmek ]
* Stop using the deprecated distutils module by adding a version
comparison class based on the RPM version rules.
* Update invocations of llvm-objdump for the latest version of LLVM.
* Adjust a test with one-byte text file for file(1) version 5.40.
* Improve the parsing of the version of OpenSSH.
[ Benjamin Peterson ]
* Add a --diff-context option to control the unified diff context size.
(reproducible-builds/diffoscope!88)
-- Chris Lamb <lamby@debian.org> Fri, 17 Sep 2021 09:10:38 +0100
diffoscope (183) unstable; urgency=medium
[ Chris Lamb ]
* Add support for extracting Android signing blocks.
(Closes: reproducible-builds/diffoscope#246)
* Format debug messages for elf sections using our
diffoscope.utils.format_class utility.
* Clarify a comment about the HUGE_TOOLS dict in diffoscope.external_tools.
[ Felix C. Stegerman ]
* Clarify output around APK Signing Blocks and remove an accidental duplicate
"0x" prefix.
-- Chris Lamb <lamby@debian.org> Fri, 03 Sep 2021 11:14:32 +0100
diffoscope (182) unstable; urgency=medium
[ Chris Lamb ]
* Also ignore, for example, spurious "fwGCC: (Debian ... )" lines in output
from strings(1).
* Only use "java -jar /path/to/apksigner.jar" if we have a .jar as newer
versions of apksigner use a shell wrapper script which will obviously be
rejected by the JVM. Also mention in the diff if apksigner is missing.
* Pass "-f" to apktool to avoid creating a strangely-named subdirectory and
to simplify code.
* If we specify a suffix for temporary file or directory, ensure it starts
with a "_" to make the generated filenames more human-readable.
* Drop an unused File import.
* Update the minimum version of the Black source code formatter.
[ Santiago Torres Arias ]
* Support parsing the return value of squashfs versions which discriminate
between fatal and non-fatal errors.
-- Chris Lamb <lamby@debian.org> Fri, 27 Aug 2021 09:11:44 +0100
diffoscope (181) unstable; urgency=medium
[ Chris Lamb ]
* New features and bug fixes:
- Don't require apksigner in order to compare .apk files using apktool.
- Add a special-case to squshfs image extraction to not fail if we aren't
root/superuser. (Closes: #991059)
- Reduce the maximum line length to avoid O(n^2) Wagner-Fischer algorithm,
which meant that diff generation took an inordinate amount of time.
(Closes: reproducible-builds/diffoscope#272)
- Include profiling information in --debug output if --profile is not set.
- Don't print an orphan newline when the Black source code formatter
self-test passes.
* Tests:
- Update test to check specific contents of squashfs listing, otherwise it
fails depending on the test systems uid-to-username mapping in passwd(5).
- Assign "seen" and "expected" values to local variables to improve
contextual information in/around failed tests.
* Misc changes:
- Print the size of generated HTML, text (etc.) reports.
- Profile calls to specialize and diffoscope.diff.linediff.
- Update various copyright years.
-- Chris Lamb <lamby@debian.org> Fri, 20 Aug 2021 10:03:35 +0100
diffoscope (180) unstable; urgency=medium
* Don't include specific ".debug"-like lines in the output, as it invariably
a duplicate of the debug ID that exists in a better form in the readelf(1)
differences for this file.
* Also ignore include short "GCC" lines that differs on a single prefix byte
too. These are distracting, not very useful and are simply the strings(1)
command's idea of the build ID, which, again, is displayed nearby in the
file's diff.
* Update the invocation arguments and tests for the latest version of
odt2txt.
-- Chris Lamb <lamby@debian.org> Fri, 06 Aug 2021 13:57:31 +0100
diffoscope (179) unstable; urgency=medium
* Ensure that various LLVM tools are installed, even when testing whether
a MacOS binary has zero differences when compared to itself.
(Closes: reproducible-builds/diffoscope#270)
-- Chris Lamb <lamby@debian.org> Fri, 30 Jul 2021 09:05:01 +0100
diffoscope (178) unstable; urgency=medium
[ Chris Lamb ]
* Don't traceback on an broken symlink in a directory.
(Closes: reproducible-builds/diffoscope#269)
* Rewrite the calculation of a file's "fuzzy hash" to make the control
flow cleaner.
[ Balint Reczey ]
* Support .deb package members compressed with the Zstandard algorithm.
(LP: #1923845)
[ Jean-Romain Garnier ]
* Overhaul the Mach-O executable file comparator.
* Implement tests for the Mach-O comparator.
* Switch to new argument format for the LLVM compiler.
* Fix test_libmix_differences in testsuite for the ELF format.
* Improve macOS compatibility for the Mach-O comparator.
* Add llvm-readobj and llvm-objdump to the internal EXTERNAL_TOOLS data
structure.
[ Mattia Rizzolo ]
* Invoke gzip(1) with the short option variants to support Busybox's gzip.
(Closes: reproducible-builds/diffoscope#268)
-- Chris Lamb <lamby@debian.org> Fri, 16 Jul 2021 15:37:59 +0100
diffoscope (177) unstable; urgency=medium
[ Keith Smiley ]
* Improve support for Apple "provisioning profiles".
* Fix ignoring objdump tests on MacOS.
-- Chris Lamb <lamby@debian.org> Fri, 04 Jun 2021 10:03:04 +0100
diffoscope (176) unstable; urgency=medium
* Update ffmpeg tests to work with ffmpeg 4.4.
(Closes: reproducible-builds/diffoscope#258)
-- Chris Lamb <lamby@debian.org> Fri, 28 May 2021 10:52:25 +0100
diffoscope (175) unstable; urgency=medium
* Use the actual filesystem path name (instead of diffoscope's concept of the
source name) to correct APK filename filtering when an APK file is in
another container -- we need to filter the auto-generated "1.apk" instead
of "original-filename.apk". (Closes: reproducible-builds/diffoscope#255)
* Don't call os.path.basename twice.
* Correct grammar in a fsimage.py debug message.
* Add a comment about stripping filenames.
-- Chris Lamb <lamby@debian.org> Fri, 14 May 2021 10:43:10 +0100
diffoscope (174) unstable; urgency=medium
[ Chris Lamb ]
* Check that we are parsing an actual Debian .buildinfo file, not just
a file with that extension.
(Closes: #987994, reproducible-builds/diffoscope#254)
* Support signed .buildinfo files again -- file(1) reports them as
"PGP signed message".
[ Mattia Rizzolo ]
* Make the testsuite pass with file(1) version 5.40.
* Embed some short test fixtures in the test code itself.
* Fix recognition of compressed .xz files with file(1) 5.40.
-- Chris Lamb <lamby@debian.org> Fri, 07 May 2021 10:48:54 +0100
diffoscope (173) unstable; urgency=medium
[ Chris Lamb ]
* Add support for showing annotations in PDF files.
(Closes: reproducible-builds/diffoscope#249)
* Move to assert_diff in test_pdf.py.
[ Zachary T Welch ]
* Difference.__init__: Demote unified_diff argument to a Python "kwarg".
-- Chris Lamb <lamby@debian.org> Fri, 30 Apr 2021 08:51:21 +0100
diffoscope (172) unstable; urgency=medium
* If zipinfo(1) shows a difference but we cannot uncover a difference within
the underlying .zip or .apk file, add a comment and show the binary
comparison. (Closes: reproducible-builds/diffoscope#246)
* Make "error extracting X, falling back to binary comparison E" error
message nicer.
-- Chris Lamb <lamby@debian.org> Fri, 02 Apr 2021 08:49:24 +0100
diffoscope (171) unstable; urgency=medium
[ Mattia Rizzolo ]
* Do not list as a "skipping reason" tools that do exist.
* Drop the "compose" tool from the list of required tools for these tests,
since it doesn't seem to be required.
-- Chris Lamb <lamby@debian.org> Fri, 26 Mar 2021 09:35:37 +0000
diffoscope (170) unstable; urgency=medium
[ Chris Lamb ]
* Avoid frequent long lines in RPM header outputs that cause very very slow
HTML outputs. (Closes: reproducible-builds/diffoscope#245)
* Fix test_libmix_differences on openSUSE Tumbleweed.
(Closes: reproducible-builds/diffoscope#244)
* Move test_rpm to use the assert_diff utility helper.
[ Hans-Christoph Steiner ]
* Add a diffoscope.tools.get_tools() method to support programmatically
fetching Diffoscope's config.
[ Roland Clobus ]
* Become tolerant of malformed Debian .changes files.
-- Chris Lamb <lamby@debian.org> Fri, 19 Mar 2021 10:30:53 +0000
diffoscope (169) unstable; urgency=medium
[ Chris Lamb ]
* Optimisations:
- Use larger buffer/block sizes when extracting files from libarchive-
based archives.
- Use a much-shorter CSS class (instead of "diffponct") to dramatically
reduce uncompressed HTML output.
* Logging improvements:
- Don't emit "Unable to stat file" warning/debug messages; we have
entirely-artificial directory entries such as ELF sections which, of
course, never exist as filesystem files.
- Don't emit a "Returning a FooContainer" logging message - we already emit
"Instantiating a FooContainer" one and are unlikely to fail in the
middle.
- Format the report size logging messages when generating HTML reports.
- Add the target directory when logging which directory we are extracting
containers to.
* Miscellaneous:
- Ignore "--debug" and similar arguments when creating a (hopefully useful)
temporary directory.
- Ensure all internal temporary directories have useful names.
- Clarify a comment regarding diffoscope not extracting excluded files.
[ Vagrant Cascadian ]
* Skip a DEX-related test if the "procyon" tool is unavailable.
-- Chris Lamb <lamby@debian.org> Fri, 12 Mar 2021 15:33:54 +0000
diffoscope (168) unstable; urgency=medium
* Don't call difflib.Differ.compare with very large inputs; it is at least
O(n^2) and makes diffoscope appear to hang.
(Closes: reproducible-builds/diffoscope#240)
* Don't use "Inheriting PATH of X" in debug log message; use "PATH is X".
* Correct the capitalisation of jQuery.
-- Chris Lamb <lamby@debian.org> Sat, 27 Feb 2021 09:34:37 +0000
diffoscope (167) unstable; urgency=medium
* Temporary directory handling:
- Ensure we cleanup our temporary directory by avoiding confusion between
the TemporaryDirectory instance and the underlying directory.
(Closes: #981123)
- Use a potentially-useful suffix to our temporary directory based on the
command-line passed to diffoscope.
- Fix some tempfile/weakref interaction in Python 3.7 (ie. Debian buster).
(Closes: reproducible-builds/diffoscope#239)
- If our temporary directory does not exist anymore (eg. it has been
cleaned up in tests, signal handling or reference counting), make sure
we recreate it.
* Bug fixes:
- Don't rely on magic.Magic(...) to have an identical API between file's
magic.py and PyPI's "python-magic" library.
(Closes: reproducible-builds/diffoscope#238)
- Don't rely on dumpimage returning an appropriate exit code; check that
the file actually exists after we call it.
* Codebase changes:
- Set a default Config.extended_filesystem_attributes.
- Drop unused Config.acl and Config.xattr attributes.
- Tidy imports in diffoscope/comparators/fit.py.
* Tests:
- Add u-boot-tools to test dependencies so that salsa.debian.org pipelines
actually test the new FIT comparator.
- Strip newlines when determining Black version to avoid "requires black
>= 20.8b1 (18.9b0\n detected)" in test output (NB. embedded newline).
- Gnumeric is back in testing so re-add to test dependencies.
- Use assert_diff (over get_data, etc.) in the FIT and APK comparators.
- Mark test_apk.py::test_android_manifest as being allowed to fail for now.
- Fix the FIT tests in buster and unstable.
-- Chris Lamb <lamby@debian.org> Fri, 19 Feb 2021 09:06:42 +0000
diffoscope (166) unstable; urgency=medium
[ Chris Lamb ]
* New features and bugfixes:
- Explicitly remove our top-level temporary directory.
(Closes: #981123, reproducible-builds/diffoscope#234)
- Increase fuzzy matching threshold to 130 ensure that we show more
differences. (Closes: reproducible-builds/diffoscope#232)
- Save our sys.argv in our top-level temporary directory in case it
helps debug current/errant temporary directories.
- Prefer to use "magic.Magic" over the "magic.open" compatibility
interface. (Closes: reproducible-builds/diffoscope#236)
- Reduce fuzzy threshold to 110 to prevent some test failures.
(Closes: reproducible-builds/diffoscope#233)
* Output improvements:
- Show fuzzyness amount in percentage terms, not out of the
rather-arbitrary "400".
- Improve the logging of fuzzy matching.
- Print the free space in our temporary directory when we create it, not
from within diffoscope.main.
* Codebase improvements:
- Tidy the diffoscopecomparators.utils.fuzzy module.
- Update my copyright years.
- Clarify the grammar of a comment.
- Clarify in a comment that __del__ is not always called, so temporary
directories are not neccessary removed the *moment* they go out of scope.
[ Conrad Ratschan ]
* Fix U-Boot Flattened Image Tree ("FIT") image detection for larger image
files. (MR: reproducible-builds/diffoscope!75)
-- Chris Lamb <lamby@debian.org> Sat, 30 Jan 2021 12:31:22 +0000
diffoscope (165) unstable; urgency=medium
[ Dimitrios Apostolou ]
* Introduce the --no-acl and --no-xattr arguments [later collapsed to
--extended-filesystem-attributes] to improve performance.
* Avoid calling the external stat command.
[ Chris Lamb ]
* Collapse --acl and --xattr into --extended-filesystem-attributes to cover
all of these extended attributes, defaulting the new option to false (ie.
to not check these very expensive external calls).
[ Mattia Rizzolo ]
* Override several lintian warnings regarding prebuilt binaries in the
* source.
* Add a pytest.ini file to explicitly use Junit's xunit2 format.
* Ignore the Python DeprecationWarning message regarding the `imp` module
deprecation as it comes from a third-party library.
* debian/rules: filter the content of the d/*.substvars files
-- Chris Lamb <lamby@debian.org> Sat, 23 Jan 2021 08:57:13 +0000
diffoscope (164) unstable; urgency=medium
[ Chris Lamb ]
* Truncate jsondiff differences at 512 bytes lest they consume the entire page.
* Wrap our external call to cmp(1) with a profile (to match the internal
profiling).
* Add a note regarding the specific ordering of the new
all_tools_are_listed test.
[ Dimitrios Apostolou ]
* Performance improvements:
- Improve speed of has_same_content by spawning cmp(1) less frequently.
- Log whenever the external cmp(1) command is spawn.ed
- Avoid invoking external diff for identical, short outputs.
* Rework handling of temporary files:
- Clean up temporary directories as we go along, instead of at the end.
- Delete FIFO files when the FIFO feeder's context manager exits.
[ Mattia Rizzolo ]
* Fix a number of potential crashes in --list-debian-substvars, including
explicitly listing lipo and otool as external tools.
- Remove redundant code and let object destructors clean up after themselves.
[ Conrad Ratschan ]
* Add a comparator for Flattened Image Trees (FIT) files, a boot image format
used by U-Boot.
-- Chris Lamb <lamby@debian.org> Fri, 08 Jan 2021 13:00:55 +0000
diffoscope (163) unstable; urgency=medium
[ Chris Lamb ]
* Bug fixes & new features:
- Normalise "ret" to "retq" in objdump output to support multiple versions
of obdump(1). (Closes: #976760, reproducible-builds/diffoscope#227)
- Don't show progress indicators when running zstd(1).
(Closes: reproducible-builds/diffoscope#226)
- Move the slightly-confusing behaviour of loading an "existing diff" if a
single file is passed to diffoscope to the new --load-existing-diff
command.
* Output improvements:
- Use pprint.pformat in the JSON comparator to serialise the differences
from jsondiff to make the output render better.
- Correct tense in --debug log output.
* Code quality:
* Don't use an old-style "super" call.
- Rewrite the filter routine for post-processing output from readelf(1).
- Update my copyright years.
- Remove unncessary PEP 263 encoding lines (replaced via PEP 3120).
- Use "minimal" instead of "basic" as a variable name to match the
underlying package name.
- Add comment regarding Java tests for diffoscope contributors who are not
using Debian. (Re: reproducible-builds/diffoscope!58)
* Debian packaging:
- Update debian/copyright to match copyright notices in source-tree.
(Closes: reproducible-builds/diffoscope#224)
- Ensure the new "diffoscope-minimal" package has a different short
description from the main "diffoscope" one.
[ Jean-Romain Garnier ]
* Add tests for OpenJDK 14.
[ Conrad Ratschan ]
* Add comparator for "legacy" uboot uImage files.
(MR: reproducible-builds/diffoscope!69)
-- Chris Lamb <lamby@debian.org> Fri, 11 Dec 2020 10:07:28 +0000
diffoscope (162) unstable; urgency=medium
[ Chris Lamb ]
* Don't depends on radare2 in the Debian autopkgtests as it will not be in
bullseye due to security considerations (#950372). (Closes: #975313)
* Avoid "Command `s p a c e d o u t` failed" messages when creating an
artificial CalledProcessError instance in our generic from_operation
feeder creator.
* Overhaul long and short descriptions.
* Use the operation's full name so that "command failed" messages include
its arguments.
* Add a missing comma in a comment.
[ Jelmer Vernooij ]
* Add missing space to the error message when only one argument is passed to
diffoscope.
[ Holger Levsen ]
* Update Standards-Version to 4.5.1.
[ Mattia Rizzolo ]
* Split the diffoscope package into a diffoscope-minimal package that
excludes the larger packages from Recommends. (Closes: #975261)
* Drop support for Python 3.6.
-- Chris Lamb <lamby@debian.org> Fri, 27 Nov 2020 09:41:32 +0000
diffoscope (161) unstable; urgency=medium
[ Chris Lamb ]
* Fix failing testsuite: (Closes: #972518)
- Update testsuite to support OCaml 4.11.1. (Closes: #972518)
- Reapply Black and bump minimum version to 20.8b1.
* Move the OCaml tests to the assert_diff helper.
[ Jean-Romain Garnier ]
* Add support for radare2 as a disassembler.
[ Paul Spooren ]
* Automatically deploy Docker images in the continuous integration pipeline.
-- Chris Lamb <lamby@debian.org> Tue, 20 Oct 2020 15:52:28 +0100
diffoscope (160) unstable; urgency=medium
* Check that pgpdump is actually installed before attempting to run it.
Thanks to Gianfranco Costamagna (locutusofborg). (Closes: #969753)
* Add some documentation for the EXTERNAL_TOOLS dictionary.
* Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump
against all files that are recognised by file(1) as "data".
-- Chris Lamb <lamby@debian.org> Fri, 11 Sep 2020 10:08:38 +0100
diffoscope (159) unstable; urgency=medium
[ Chris Lamb ]
* Show "ordering differences only" in strings(1) output.
(Closes: reproducible-builds/diffoscope#216)
* Don't alias output from "os.path.splitext" to variables that we do not end
up using.
* Don't raise exceptions when cleaning up after a guestfs cleanup failure.
[ Jean-Romain Garnier ]
* Make "Command" subclass a new generic Operation class.
-- Chris Lamb <lamby@debian.org> Fri, 04 Sep 2020 11:12:52 +0100
diffoscope (158) unstable; urgency=medium
* Improve PGP support:
- Support extracting of files within PGP signed data.
(Closes: reproducible-builds/diffoscope#214)
- pgpdump(1) can successfully parse some unrelated, non-PGP binary files,
so check that the parsed output contains something remotely sensible
before identifying it as a PGP file.
* Don't use Python's repr(...)-style output in "Calling external command"
logging output.
* Correct a typo of "output" in an internal comment.
-- Chris Lamb <lamby@debian.org> Fri, 28 Aug 2020 11:53:10 +0100
diffoscope (157) unstable; urgency=medium
[ Chris Lamb ]
* Try obsensibly "data" files named .pgp against pgpdump to determine whether
they are PGP files. (Closes: reproducible-builds/diffoscope#211)
* Don't raise an exception when we encounter XML files with "<!ENTITY>"
declarations inside the DTD, or when a DTD or entity references an external
resource. (Closes: reproducible-builds/diffoscope#212)
* Temporarily drop gnumeric from Build-Depends as it has been removed from
testing due to Python 2.x deprecation. (Closes: #968742)
* Codebase changes:
- Add support for multiple file extension matching; we previously supported
only a single extension to match.
- Move generation of debian/tests/control.tmp to an external script.
- Move to our assert_diff helper entirely in the PGP tests.
- Drop some unnecessary control flow, unnecessary dictionary comprehensions
and some unused imports found via pylint.
* Include the filename in the "... not identified by any comparator"
logging message.
-- Chris Lamb <lamby@debian.org> Fri, 21 Aug 2020 12:24:25 +0100
diffoscope (156) unstable; urgency=medium
[ Chris Lamb ]
* Update PPU tests for compatibility with Free Pascal versions 3.2.0 or
greater. (Closes: #968124)
* Emit a debug-level logging message when our ppudump(1) version does not
match file header.
* Add and use an assert_diff helper that loads and compares a fixture output
to avoid a bunch of test boilerplate.
[ Frazer Clews ]
* Apply some pylint suggestions to the codebase.
-- Chris Lamb <lamby@debian.org> Fri, 14 Aug 2020 10:04:26 +0100
diffoscope (155) unstable; urgency=medium
[ Chris Lamb ]
* Bump Python requirement from 3.6 to 3.7 - most distributions are either
shipping3.5 or 3.7, so supporting 3.6 is not somewhat unnecessary and also
more difficult to test locally.
* Improvements to setup.py:
- Apply the Black source code reformatter.
- Add some URLs for the site of PyPI.org.
- Update "author" and author email.
* Explicitly support Python 3.8.
[ Frazer Clews ]
* Move away from the deprecated logger.warn method logger.warning.
[ Mattia Rizzolo ]
* Document ("classify") on PyPI that this project works with Python 3.8.
-- Chris Lamb <lamby@debian.org> Fri, 07 Aug 2020 12:09:50 +0100
diffoscope (154) unstable; urgency=medium
[ Chris Lamb ]
* Add support for F2FS filesystems.
(Closes: reproducible-builds/diffoscope#207)
* Allow "--profile" as a synonym for "--profile=-".
* Add an add_comment helper method so don't mess with our _comments list
directly.
* Add missing bullet point in a previous changelog entry.
* Use "human-readable" over unhyphenated version.
* Add a bit more debugging around launching guestfs.
* Profile the launch of guestfs filesystems.
* Correct adding a comment when we cannot extract a filesystem due to missing
guestfs module.
-- Chris Lamb <lamby@debian.org> Fri, 31 Jul 2020 10:26:15 +0100
diffoscope (153) unstable; urgency=medium
[ Chris Lamb ]
* Drop some legacy argument styles; --exclude-directory-metadata and
--no-exclude-directory-metadata have been replaced with
--exclude-directory-metadata={yes,no}.
* Code improvements:
- Make it easier to navigate the main.py entry point.
- Use a relative import for get_temporary_directory in diffoscope.diff.
- Rename bail_if_non_existing to exit_if_paths_do_not_exist.
- Rewrite exit_if_paths_do_not_exist to not check files multiple times.
* Documentation improvements:
- CONTRIBUTING.md:
- Add a quick note about adding/suggesting new options.
- Update and expand the release process documentation.
- Add a reminder to regenerate debian/tests/control.
- README.rst:
- Correct URL to build job on Jenkins.
- Clarify and correct contributing info to point to salsa.debian.org.
-- Chris Lamb <lamby@debian.org> Fri, 24 Jul 2020 12:03:51 +0100
diffoscope (152) unstable; urgency=medium
[ Chris Lamb ]
* Bug fixes:
- Don't require zipnote(1) to determine differences in a .zip file as we
can use libarchive directly.
* Reporting improvements:
- Don't emit "javap not found in path" if it is available in the path but
it did not result in any actual difference.
- Fix "... not available in path" messages when looking for Java
decompilers; we were using the Python class name (eg. "<class
'diffoscope.comparators.java.Javap'>") over the actual command we looked
for (eg. "javap").
* Code improvements:
- Replace some simple usages of str.format with f-strings.
- Tidy inline imports in diffoscope.logging.
- In the RData comparator, always explicitly return a None value in the
failure cases as we return a non-None value in the "success" one.
[ Jean-Romain Garnier ]
* Improve output of side-by-side diffs, detecting added lines better.
(MR: reproducible-builds/diffoscope!64)
* Allow passing file with list of arguments to ArgumentParser (eg.
"diffoscope @args.txt"). (MR: reproducible-builds/diffoscope!62)
-- Chris Lamb <lamby@debian.org> Sat, 18 Jul 2020 11:00:13 +0100
diffoscope (151) unstable; urgency=medium
[ Chris Lamb]
* Improvements and bug fixes:
- Pass the absolute path when extracting members from SquashFS images as we
run the command with our working directory set to the temporary
directory. (Closes: #964365, reproducible-builds/diffoscope#189)
- Increase the minimum length of the output from strings(1) to 8 characters
to avoid unnecessary diff noise. (Re. reproducible-builds/diffoscope#148)
* Logging improvements:
- Fix the compare_files message when the file does not have a literal name.
- Reduce potential log noise by truncating the has_some_content messages.
* Codebase changes:
- Clarify use of a "null" diff in order to remember an exit code.
- Don't alias a variable when don't end up it; use "_" instead.
- Use a "NullChanges" file to represent missing data in the Debian package
comparator.
- Update some miscellaneous terms.
-- Chris Lamb <lamby@debian.org> Fri, 10 Jul 2020 10:52:56 +0100
diffoscope (150) unstable; urgency=medium
[ Chris Lamb ]
* Don't crash when listing entries in archives if they don't have a listed
size (such as hardlinks in .ISO files).
(Closes: reproducible-builds/diffoscope#188)
* Dump PE32+ executables (including EFI applications) using objdump.
(Closes: reproducible-builds/diffoscope#181)
* Tidy detection of JSON files due to missing call to File.recognizes that
checks against the output of file(1) which was also causing us to attempt
to parse almost every file using json.loads. (Whoops.)
* Drop accidentally-duplicated copy of the new --diff-mask tests.
* Logging improvements:
- Split out formatting of class names into a common method.
- Clarify that we are generating presenter formats in the opening logs.
[ Jean-Romain Garnier ]
* Remove objdjump(1) offsets before instructions to reduce diff noise.
(Closes: reproducible-builds/diffoscope!57)
-- Chris Lamb <lamby@debian.org> Fri, 03 Jul 2020 11:04:46 +0100
diffoscope (149) unstable; urgency=medium
[ Chris Lamb ]
* Update tests for file 5.39. (Closes: reproducible-builds/diffoscope#179)
* Downgrade the tlsh warning message to an "info" level warning.
(Closes: #888237, reproducible-builds/diffoscope#29)
* Use the CSS "word-break" property over manually adding U+200B zero-width
spaces that make copy-pasting cumbersome.
(Closes: reproducible-builds/diffoscope!53)
* Codebase improvements:
- Drop some unused imports from the previous commit.
- Prevent an unnecessary .format() when rendering difference comments.
- Use a semantic "AbstractMissingType" type instead of remembering to check
for both "missing" files and missing containers.
[ Jean-Romain Garnier ]
* Allow user to mask/filter reader output via --diff-mask=REGEX.
(MR: reproducible-builds/diffoscope!51)
* Make --html-dir child pages open in new window to accommodate new web
browser content security policies.
* Fix the --new-file option when comparing directories by merging
DirectoryContainer.compare and Container.compare.
(Closes: reproducible-builds/diffoscope#180)
* Fix zsh completion for --max-page-diff-block-lines.
[ Mattia Rizzolo ]
* Do not warn about missing tlsh during tests.
-- Chris Lamb <lamby@debian.org> Fri, 26 Jun 2020 15:57:41 +0100
diffoscope (148) unstable; urgency=medium
[ Daniel Fullmer ]
* Fix a regression in the CBFS comparator due to changes in our_check_output.
[ Chris Lamb ]
* Add a remark in the deb822 handling re. potential security issue in the
.changes, .dsc, .buildinfo comparator.
-- Chris Lamb <lamby@debian.org> Fri, 19 Jun 2020 11:38:20 +0100
diffoscope (147) unstable; urgency=medium
[ Chris Lamb ]
* New features:
- Add output from strings(1) to ELF binaries. It is intended this will
expose expose build paths that are hidden somewhere within the objdump(1)
output. (Closes: reproducible-builds/diffoscope#148)
- Add basic zsh shell tab-completion support.
(Closes: reproducible-builds/diffoscope#158)
* Bug fixes:
- Prevent a traceback when comparing a PDF document that does not contain
any metadata, ie. it is missing a PDF "/Info" stanza.
(Closes: reproducible-builds/diffoscope#150)
- Fix compatibility with jsondiff 1.2.0 which was causing a traceback and
log the version of jsondiff we are using to aid debugging in the future.
(Closes: reproducible-builds/diffoscope#159
- Fix an issue in GnuPG keybox handling that left filenames in the diff.
- Don't mask an existing test name; ie. ensure it is actually run.
* Reporting:
- Log all calls to subprocess.check_output by using our own wrapper utility.
(Closes: reproducible-builds/diffoscope#151)
* Code improvements:
- Replace references to "WF" with "Wagner-Fischer" for clarity.
- Drop a large number of unused imports (list_libarchive,
ContainerExtractionError, etc.)
- Don't assign exception to a variable that we do not use.
- Compare string values with the equality operator, not via "is" identity.
- Don't alias an open file to a variable when we don't use it.
- Don't alias "filter" builtin.
- Refactor many small parts of the HTML generation, dropping explicit
u"unicode" strings, tidying the generation of the "Offset X, Y lines
modified" messages, moving to PEP 498 f-strings where appropriate, etc.
- Inline a number of single-used utility methods.
-- Chris Lamb <lamby@debian.org> Thu, 11 Jun 2020 12:50:34 +0100
diffoscope (146) unstable; urgency=medium
[ Chris Lamb ]
* Refactor .changes and .buildinfo handling to show all details (including
the GPG header and footer components), even when referenced files are not
present. (Closes: reproducible-builds/diffoscope#122)
* Normalise filesystem stat(2) "birth times" (ie. st_birthtime) in the same
way we do with stat(1)'s "Access:" and "Change:" times to fix a
nondetermistic build failure on GNU Guix.
(Closes: reproducible-builds/diffoscope#74)
* Drop the (default) subprocess.Popen(shell=False) keyword argument so that
the more unsafe shell=True is more obvious.
* Ignore lower vs. upper-case when ordering our file format descriptions.
* Don't skip string normalisation in Black.
[ Mattia Rizzolo ]
* Add a "py3dist" override for the rpm-python module (Closes: #949598)
* Bump the debhelper compat level to 13 and use the new
execute_after_*/execture_before_* style rules.
* Fix a spelling error in changelog.
[ Daniel Fullmer ]
* Mount GuestFS filesystem images readonly.
[ Jean-Romain Garnier ]
* Prevent an issue where (for example) LibarchiveMember's has_same_content
method is called regardless of the actual type of file.
-- Chris Lamb <lamby@debian.org> Sat, 30 May 2020 12:36:51 +0100
diffoscope (145) unstable; urgency=medium
[ Chris Lamb ]
* Improvements:
- Add support for Apple Xcode mobile provisioning .mobilepovision files.
(Closes: reproducible-builds/diffoscope#113)
- Add support for printing the signatures via apksigner(1).
(Closes: reproducible-builds/diffoscope#121)
- Use SHA256 over MD5 when generating page names for the HTML directory
presenter, validate checksums for files referenced in .changes files
using SHA256 too, and move to using SHA256 in "Too much input for diff"
output too. (Closes: reproducible-builds/diffoscope#124)
- Don't leak the full path of the temporary directory in "Command [..]
exited with 1". (Closes: reproducible-builds/diffoscope#126)
- Identify "iOS App Zip archive data" files as .zip files.
(Closes: reproducible-builds/diffoscope#116)
* Bug fixes:
- Correct "differences" typo in the ApkFile handler.
(Closes: reproducible-builds/diffoscope#127)
* Reporting/output improvements:
- Never emit the same id="foo" TML anchor reference twice, otherwise
identically-named parts will not be able to linked to via "#foo".
(Closes: reproducible-builds/diffoscope#120)
- Never emit HTML with empty "id" anchor lements as it is not possible to
link to "#" (vs "#foo"). We use "#top" as a fallback value so it will
work for the top-level parent container.
- Clarify the message when we cannot find the "debian" Python module.
- Clarify "Command [..] failed with exit code" to remove duplicate "exited
with exit" but also to note that diffoscope is interpreting this as an
error.
- Add descriptions for the 'fallback' Debian module file types.
- Rename the --debugger command-line argument to --pdb.
* Testsuite improvements:
- Prevent CI (and runtime) apksigner test failures due to lack of
binfmt_misc on Salsa CI and elsewhere.
* Codebase improvements:
- Initially add a pair of comments to tidy up a slightly abstraction level
violating code in diffoscope.comparators.mising_file and the
.dsc/.buildinfo file handling, but replace this later by by inlining
MissingFile's special handling of deb822 to prevent leaking through
abstraction layers in the first place.
- Use a BuildinfoFile (etc.) regardless of whether the associated files
such as the orig.tar.gz and the .deb are present, but don't treat them as
actual containers. (Re: reproducible-builds/diffoscope#122)
- Rename the "Openssl" command class to "OpenSSLPKCS7" to accommodate other
commands with this prefix.
- Wrap a docstring across multiple lines, drop an inline pprint import and
comment the HTMLPrintContext class, etc.
[ Emanuel Bronshtein ]
* Avoid build-cache in building the released Docker image.
(Closes: reproducible-builds/diffoscope#123)
[ Holger Levsen ]
* Wrap long lines in older changelog entries.
-- Chris Lamb <lamby@debian.org> Sat, 23 May 2020 09:31:26 +0100
diffoscope (144) unstable; urgency=medium
[ Chris Lamb ]
* Improvements:
- Print the amount of free space that we have available in our temporary
directory as a debugging message.
- Remove (broken) fuzzy matching of JSON files as file 5.35 (in buster,
released 2018-10-18) supports recognising JSON data.
(Closes: reproducible-builds/diffoscope#106)
- Don't pretty-print the JSON output by default as it will usually be so
complicated to be unreadable by the human eye and it can be easily
replaced by piping to "| jq".
- Don't print a traceback if we pass a single, missing argument to
diffoscope (eg. a JSON diff to re-load).
* Reporting/output improvements:
- Reduce the default number of maximum standard error lines printed from 50
to 25; usually the error is obvious by that point.
- Clarify the message when we truncate the number of lines to standard
error.
- Clarify when an external command emits for both files, otherwise it can
look like diffoscope is repeating itself when it is actually being run
twice.
- Don't repeat "stderr from {}" if both commands emit the same thing.
* Dockerfile improvements:
- Use ARG instead of ENV for DEBIAN_FRONTEND so we do not set this
environment variable at runtime.
(Closes: reproducible-builds/diffoscope#103)
- Run diffoscope as a non-root user in the runtime container.
(Closes: reproducible-builds/diffoscope#102)
- Add a .dockerignore file to whitelist files we need in our container.
Thanks to Emanuel Bronshtein for the original idea.
(Closes: reproducible-builds/diffoscope#105)
- Install/remove the build-essential package during build so we can install
the recommended packages from Git.
* Testsuite improvements:
- Include the Black output in the assertion failure too.
- Update the Black self-test; we don't care about the length of the black
output, rather whether it has some or, preferably, not.
* Codebase improvements:
- Bump the officially required version of Python from 3.5 to 3.6.
(Closes: reproducible-builds/diffoscope#117)
- Drop an unused shlex import.
- Instruct linters to pass over a bare try-except when cleaning up
temporary files used to extract archives.
- Format diffoscope/comparators/utils/command.py according to Black
19.10b0-3.
- Drop entries from Uploaders that have not uploaded in over three years
with esteemed thanks for their previous work.
- Drop .travis.yml; we are using Salsa now and likely would not give
support for running on Travis CI at this point.
[ Jelle van der Waa ]
* Update LLVM diff for LLVM version 10.
[ Vagrant Cascadian ]
* Add external tool reference on openssl for GNU Guix.
-- Chris Lamb <lamby@debian.org> Thu, 14 May 2020 16:16:06 +0100
diffoscope (143) unstable; urgency=medium
* Add support for .p7c and .p7b certificates.
(Closes: reproducible-builds/diffoscope#94)
* Add "pdftotext" as a requirement to run the PDF test_metadata text. Special
thanks to Chocimier/Piotr for the debugging work.
(Closes: reproducible-builds/diffoscope#99)
* Improve the documentation of FALLBACK_FILE_TYPE_HEADER_PREFIX and
FILE_TYPE_HEADER_PREFIX to note that only the first 16 bytes are used.
-- Chris Lamb <lamby@debian.org> Mon, 27 Apr 2020 13:14:07 +0100
diffoscope (142) unstable; urgency=medium
[ Chris Lamb ]
* Render multiline difference comments in a way to show indentation.
(Closes: reproducible-builds/diffoscope#101)
* Correct parsing of "./setup.py test --pytest-args ...".
* Capitalise "Ordering differences only" in text comparison comments.
* Don't include the JSON similarity percentage if it is 0.0%.
[ Ben Hutchings ]
* Document how --exclude arguments are matched against filenames.
[ Vagrant Cascadian ]
* Add external tool reference for h5dump on Guix.
-- Chris Lamb <lamby@debian.org> Wed, 22 Apr 2020 16:35:14 +0100
diffoscope (141) unstable; urgency=medium
[ Chris Lamb ]
* Dalvik .dex files can also serve as APK containers. Restrict the narrower
identification of .dex files to files ending with this extension, and widen
the identification of APK files to when file(1) discovers a Dalvik file.
(Closes: #884095, reproducible-builds/diffoscope#28)
* Explicitly list python3-h5py in debian/tests/control.in to ensure that we
have this module installed during an autopkgtest run to generate the test
fixture & regenerate debian/tests/control from debian/tests/control.in
to match.
* Drop unnecessary and unused assignment to "diff" variable.
* Strip paths from the output of zipinfo(1) warnings.
(re. reproducible-builds/diffoscope#97)
[ Michael Osipov ]
* Revert to using zipinfo(1) directly instead of piping input via /dev/stdin
for BSD portability. (Closes: reproducible-builds/diffoscope#97)
[ Jelle van der Waa ]
* Add an external tool for h5dump on Arch.
-- Chris Lamb <lamby@debian.org> Wed, 15 Apr 2020 19:15:42 +0100
diffoscope (140) unstable; urgency=medium
* apktool 2.5.0 changed the handling of output of XML schemas, so update and
restrict the corresponding test to match.
(Closes: reproducible-builds/diffoscope#96)
* Add support for Hierarchical Data Format (HD5) files.
(Closes: reproducible-builds/diffoscope#95)
-- Chris Lamb <lamby@debian.org> Sat, 11 Apr 2020 19:10:57 +0100
diffoscope (139) unstable; urgency=medium
[ Mattia Rizzolo ]
* Fix Build-Depends on python3-pdfminer. (Closes: #955645)
-- Chris Lamb <lamby@debian.org> Fri, 03 Apr 2020 21:29:22 +0100
diffoscope (138) unstable; urgency=medium
* New features & bug fixes:
- Use dumppdf from python3-pdfminer if we do not see any other differences
from pdftext, etc. (reproducible-builds/diffoscope#92)
- Prevent a traceback when comparing two .rdx files directly as get_member
will return a file even if the file is missing and not raise a KeyError
exception; we therefore explicitly test for the existence of the file.
* Don't allow errors within "Rscript" deserialisation to cause the entire
operation to fail (for example if an external library cannot be loaded).
(reproducible-builds/diffoscope#91)
* Reporting improvements:
- Inject the supported file formats into the package long description and
remove any duplicate comparator descriptions when formatting.
(#reproducible-builds/diffoscope#90)
- Print a potentially-helpful message if the PyPDF2 module is not installed
(#reproducible-builds/diffoscope#92)
- Weaken "Install the X package to get a better output" claim to "... may
produce a better output" as this is not guaranteed.
* Code improvements:
- Inline the RequiredToolNotFound.get_package method; only used once.
- Drop deprecated "py36 = [..]" argument in pyproject.toml.
- Factor out generation of comparator descriptions.
- Add an upstream metadata file.
-- Chris Lamb <lamby@debian.org> Mon, 30 Mar 2020 16:16:28 +0100
diffoscope (137) unstable; urgency=medium
* Also extract classes2.dex, classes3.dex etc. from .apk files.
(Closes: reproducible-builds/diffoscope#88)
* Add generalised support for "ignoring" returncodes and move special-casing
of returncodes in the zip comparator to this.
* Accommodate sng returning with a UNIX exit code of 1 even if there minor
errors in the file (discovered via #950806).
-- Chris Lamb <lamby@debian.org> Tue, 18 Feb 2020 17:15:59 +0000
diffoscope (136) unstable; urgency=medium
[ Chris Lamb ]
* Improvements:
- Support external build tools. (Closes: reproducible-builds/diffoscope#87)
- Fallback to the regular .zip container format for .apk files if apktool
is not available.
- Clarify that "install X" in difference comment messages actually refer to
a system/distribution package.
- Drop the --max-report-size-child and --max-diff-block-lines-parent
options; both deprecated and scheduled for removal in January 2018.
* Bug fixes:
- No longer raise a KeyError exception if we request an invalid member from
a directory container.
* Logging improvements:
- Log a debug-level message if we cannot open a file as container due to a
missing tool in order to assist diagnosing issues.
- If we don't know the HTML output name, don't emit an enigmatic "html
output for" message.
- Add the current PATH environment variable to the "Normalising locale..."
debug-level message.
- Print the "Starting diffoscope $VERSION" line as the first line.
- Correct a debug-level message for compare_meta calls to quote the
arguments correctly.
* Refactoring:
- Add support for alternative container types for a file, allowing for
runtime (vs import time) control of fallbacks such as adding comments.
and append a comment to a difference if we fallback to an inferior
container format due to missing a tool.
- Factor-out the generation of "foo not available in path" difference
comment messages as a helper method in the exception that represents
them.
* Code improvements:
- Tidy diffoscope.main's configure method, factoring out the set of the
Config() global out of the run_diffoscope method and inlining the
functionality of maybe_set_limit, etc.
- Rename diffoscope.locale module to diffoscope.environ as we are modifying
things beyond just the locale (eg. calling tzset(), etc.)
- Drop unused "Difference" import from the APK comparator.
- Drop an assertion that is guaranteed by parallel "if" conditional.
- Add a "noqa" line to avoid a false-positive flake8 "unused import"
warning.
- Turn down the "volume" for a recommendation in a comment.
* Release/source-code management:
- Add a .git-blame-ignore-revs file to improve the output of git-blame(1) by
ignoring large changes when introducing the Black source code reformatter
and update the CONTRIBUTING.md guide on how to optionally use it locally.
- Convert CONTRIBUTING.rst to CONTRIBUTING.md and include it in the
PyPI.org release.
* Test improvements
- Refresh and update the fixtures for the .ico tests to match the latest
version of Imagemagick in Debian unstable.
[ Holger Levsen ]
* Bump Standards Version to 4.5.0, no changes needed.
[ Marc Herbert ]
* Search for expected keywords in the output of cbfstool tests and not a
specific output. (Closes: reproducible-builds/diffoscope!42)
-- Chris Lamb <lamby@debian.org> Fri, 24 Jan 2020 15:38:57 +0000
diffoscope (135) unstable; urgency=medium
[ Chris Lamb ]
* Extract resources.arsc files (as well as classes.dex) from Android .apk
files to ensure that we at least show differences there. (Closes: #916359)
* Always pass a filename with a ".zip" extension to zipnote otherwise it
will return with an exit code of 9 and revert to a binary diff for the
entire file.
* Don't exhaustively output the entire HTML report when testing the
regression for #875281; parsing the JSON and pruning the tree should be
enough.
* Factor out running all of our zipinfo variants into a new method and use
this routine when comparing .apk files, thus now also displaying any
differences exposed by bsdtar.
* Testsuite improvements:
- Always allow fixtures called "output*".
- Actually test the output of the test_html_regression_875281 test.
* Add a note to the "Contributing" page to suggest enable concurrency when
running the tests locally.
[ Marc Herbert ]
* Fix a misplaced debug "Loading diff from stdin" logging message.
* Add new "Testing" section to CONTRIBUTING.rst.
-- Chris Lamb <lamby@debian.org> Tue, 14 Jan 2020 14:26:07 +0000
diffoscope (134) unstable; urgency=medium
[ Chris Lamb ]
* Ensure that autopkgtests are run with our pyproject.toml present for the
correct "black" source code formatter settings. (Closes: #945993)
* Tidy some unnecessary boolean logic in the ISO9660 tests.
* Rename the "text_option_with_stdiout" to "text_option_with_stdout".
* Include the libarchive file listing for ISO images to ensure that
timestamps (not just dates) are visible in any difference.
(Closes: reproducible-builds/diffoscope#81)
[ Eli Schwartz ]
* Fix an exception in the progressbar handler.
[ Vagrant Cascadian ]
* Add an external tool reference for zstd on GNU Guix.
-- Chris Lamb <lamby@debian.org> Fri, 27 Dec 2019 19:17:16 +0000
diffoscope (133) unstable; urgency=medium
* Correct the substitution/filtering of paths in ELF output to avoid
unnecessary differences depending on the path name provided on the
commandline. (Closes: #945572)
* Silence/correct a SyntaxWarning message due to incorrectly comparing an
integer by identity (is) over equality (==). (Closes: #945531)
-- Chris Lamb <lamby@debian.org> Thu, 28 Nov 2019 10:03:33 +0000
diffoscope (132) unstable; urgency=medium
* Allow all possible .zip variations to return with non-zero exit codes, not
just known types we can explicitly identify (eg. Java .jmod and .jar
files). (Closes: reproducible-builds/diffoscope#78)
* Also permit UTF-8 encoded .dsc and .changes files.
(Re: reproducible-builds/diffoscope#78)
* Rework a long string of "or" statements into a loop with a "break".
-- Chris Lamb <lamby@debian.org> Tue, 26 Nov 2019 10:31:44 +0000
diffoscope (131) unstable; urgency=medium
* Clarify in the HTML and text outputs that the limits are per output format,
not global. (Closes: #944882, reproducible-builds/diffoscope#76)
- Bump the previous "max_page_size" limit from 400 kB to 4 MB.
* Update an Android manifest test to reflect that parsed XML attributes are
returned in a sorted or otherwise novel manner under Python 3.8.
* Update code to reflect version 19.10b0 of the black source code
reformatter and don't run our self-test for versions earlier than this as
it will generate different results and thus fail.
* Limit .dsc and .buildinfo matching; don't attempt to compare them as
ELF sections or similar. (Closes: reproducible-builds/diffoscope#77)
* Add a comment that the text_ascii{1,2} test fixture files are used in
multiple places so are not trivial to generate on the fly.
-- Chris Lamb <lamby@debian.org> Sat, 23 Nov 2019 15:22:51 -0500
diffoscope (130) unstable; urgency=medium
[ Chris Lamb ]
* debian/tests/basic-command-line:
Move from deprecated ADTTMP to AUTOPKGTEST_TMP.
* Correct the matching of R .rds files by also detecting newer versions
of this file format.
* Drop unused BASE_DIR global in the tests.
* Try and ensure that new test data files are generated dynamically, ie.
at least no new ones are added without "good" reasons.
* Truncate the tcpdump expected diff to 8KB (from ~600KB).
* Refresh OCaml test fixtures to support OCaml >= 4.08.1. Closes: #944709
* Correct reference to the ".rdx" extension in a comment.
* Update XML test for Python 3.8+.
* Don't use line-base dbuffering when communucating with subprocesses
in binary mode. (Closes: reproducible-builds/diffoscope#75)
[ Jelle van der Waa ]
* Add support for comparing .zst files are created by zstd.
(Closes: reproducible-builds/diffoscope!34)
-- Mattia Rizzolo <mattia@debian.org> Thu, 14 Nov 2019 11:22:20 +0100
diffoscope (129) unstable; urgency=medium
* Call R's "deparse" function to ensure that we do not error out and revert
to a binary diff when processing .rdb files with internal "vector" types as
they do not automatically coerce to strings.
* Add the ability to pass Python bytestrings to external commands and
pass our long script to parse R .rdb files using this new method over a
long command-line argument
* Use Rscript's --vanilla option over --no-environ as this also enables
--no-save, --no-restore, --no-site-file and --no-init-file.
* Improve command-line error messages:
- Split out formatting into a separate utility function.
- Truncate very long lines when displaying them as an external source
of data.
- When printing an error from a command, format the command for the user.
* Use "exit code" over "return code" when referring to UNIX error codes in
displayed differences.
-- Chris Lamb <lamby@debian.org> Mon, 28 Oct 2019 11:15:18 +0000
diffoscope (128) unstable; urgency=medium
* Query the container for the full path of the parallel R .rdx file for a
.rdb file as well as looking in the same directory. This ensures that
comparing two .deb/.changes files shows a varying path introduced in
version 127.
-- Chris Lamb <lamby@debian.org> Fri, 25 Oct 2019 09:21:40 +0100
diffoscope (127) unstable; urgency=medium
[ Chris Lamb ]
* Move build-dependency on python-argcomplete to the Python 3.x version to
facilitate Python 2.x removal. (Closes: #942967)
* Overhaul the handling of GNU R .rdb files:
- Rework and refactor the handling of .rdb files specifically with respect
to locating the parallel .rdx file prior to inspecting the file to ensure
that we do not add files to the user's filesystem in the case of directly
comparing two .rdb files or, worse, overwriting a file in is place.
- Use a ("""-formatted) docstring for our internal R script to dump
variables.
- Mask/hide standard errors; R will often produce noisy output that is not
useful to us.
- Don't include a useless and misleading "NULL".
- Include all R object names are displayed, including ones beginning with a
fullstop (".").
- Sort package fields when dumping data for output stability.
- Format package contents as "foo = bar" rather than using ugly and
misleading brackets.
- Include the object's type when dumping package contents.
- Never read the site or user's R location environment configuration to
ensure output stability.
- Expose absolute paths in the semantic/human-readable output, preventing
falling back to a useless hexdump.
* Improve the formatting of command lines:
- Ensure newlines and other metacharacters appear escaped as "\n", etc.
- Use the newline (etc.) escaped version of the commandline being executed
in logging/debug output.
- When displaying standard error, ensure use the escaped version too.
* Add support for easily masking the standard error of commands and use this
in the ffprobe comparator.
* To match the libarchive container, raise a KeyError exception if we request
an invalid member from a directory container.
* Correct string representation output in the traceback when we cannot locate
a specific item in a container.
[ Mattia Rizzolo ]
* Run Debian autopkgtests against all Python versions.
-- Chris Lamb <lamby@debian.org> Thu, 24 Oct 2019 15:59:34 +0100
diffoscope (126) unstable; urgency=medium
[ Chris Lamb ]
* Track and report on missing Python modules.
(Closes: reproducible-builds/diffoscope#72)
* Drop some unnecessary control flow.
* Drop explicit inheriting from 'object' class; unnecessary in Python 3.
* Bump Standards-Version to 4.4.1.
[ Mattia Rizzolo ]
* Exit with 2 instead of 1 in case of no disk space.
-- Chris Lamb <lamby@debian.org> Mon, 14 Oct 2019 12:29:47 -0700
diffoscope (125) unstable; urgency=medium
* The test_libmix_differences ELF test requires xxd. (Closes: #940645)
-- Chris Lamb <lamby@debian.org> Wed, 18 Sep 2019 11:44:57 +0200
diffoscope (124) unstable; urgency=medium
[ Chris Lamb ]
* Also conditionally skip the identification and "no differences" tests as we
require the Ocaml compiler to be present when building the test files
themselves. (Closes: #940471)
[ Mattia Rizzolo ]
* Permit all sorts of version suffixes when checking the Debian package
version matches setup.py, not just for backports. (Closes: #939387)
[ Vagrant Cascadian ]
* Add external tools on GNU Guix for odt2txt, sng and pgpdump.
[ Marc Herbert ]
* Remove StaticLibFile in the ELF comparator -- ArFile.compare_details() is
superior. (Closes: reproducible-builds/diffoscope#64)
-- Chris Lamb <lamby@debian.org> Tue, 17 Sep 2019 12:19:07 +0200
diffoscope (123) unstable; urgency=medium
[ Chris Lamb ]
* Build OCaml test input files on-demand rather than shipping them with the
package in order to prevent test failures with OCaml 4.08.
(Closes: #939386)
* Update test skipping messages:
- When skipping tests due to the lack of installed tool, print the package
that may provide it.
- Update "requires foo module" messages to clarify that they are regarding
Python modules, not packages.
* Rebuild the test squashfs images to exclude the character device as they
requires root or fakeroot to extract. (reproducible-builds/diffoscope#65)
* Remove accidentally-committed test fixture generation code from tests.
* Set long_description_content_type to 'text/x-rst' in setup.py to appease
the PyPI.org linter.
[ Mattia Rizzolo ]
* Don't crash when the progress is requested but the Python module is
missing. (Closes: #939085)
[ Vagrant Cascadian ]
* Add external tools for gifbuild, javap and kbxutil in GNU Guix.
[ Marc Herbert ]
* In the cbfs tests, add a "Comp" column test to support Coreboot
utils > 4.6.
-- Chris Lamb <lamby@debian.org> Sat, 07 Sep 2019 14:16:31 +0100
diffoscope (122) unstable; urgency=medium
[ Chris Lamb ]
* Apply patch from László Böszörményi to update the squashfs test output and
bump the required version for the test itself. (Closes: #935684)
* Skip calls to unsquashfs when we are not root or fakeroot.
(Re: reproducible-builds/diffoscope#63)
* Include either standard error or standard output (and not just the latter)
if/when an external command fails.
* Fix a few unicode/bytes issues:
- Avoid a possible traceback caused by a str/bytes confusion when handling
the output of failing external commands.
- Ensure that all of our artificially-created subprocess.CalledProcessError
instances have `output` instances that are bytes objects, not str.
* Improve debugging output:
* Add the containing module name to the (eg.) "Using StaticLibFile for ..."
* Improve and condense output when creating our Comparator object types.
* Correct a reference to `parser.diff` as `diff` in this context is a Python
function in the module, not the actual output returned from diff(1).
* Add the "wabt" Debian package to the test dependencies so that we run the
wasm tests.
[ Mattia Rizzolo ]
* Now that we test-require wabt, expect that its tools to be available during
autopkgtests.
-- Chris Lamb <lamby@debian.org> Fri, 30 Aug 2019 08:52:25 +0100
diffoscope (121) unstable; urgency=medium
* Don't fallback to a (useless) raw hexdump when readelf(1) reports an minor
issue in a section in an ELF binary. For example, when the "frames" section
is of the "NOBITS" type, its contents are apparently "unreliable" and thus
readelf(1) exits with a returncode of 1. (Closes: #849407, #931962)
* Add support to Difference.from_command_exc and friends to optionally ignore
specified returncodes from the called program and treat them as "no"
difference.
* Simplify the parsing of the optional "command_args" argument to the
from_command and from_command_exc methods in the Difference class.
-- Chris Lamb <lamby@debian.org> Fri, 16 Aug 2019 09:49:07 -0700
diffoscope (120) unstable; urgency=medium
* No-change sourceful after accidentally uploading binaries in order to
migration to testing.
-- Chris Lamb <lamby@debian.org> Mon, 29 Jul 2019 17:17:46 -0300
diffoscope (119) unstable; urgency=medium
[ Chris Lamb ]
* If a command fails to execute but does not print anything to standard
error, try and include the first line of standard output in the message we
include in the diff. This was motivated by readelf(1) returning its error
messages on stdout. (Closes: #931963)
* Add support for Java ".jmod" modules. Not all versions of file(1) support
detection of Jmod files yet, so we perform a manual comparison instead.
(Closes: #933308)
* Re-add "return code" noun to "Command `foo` exited with X" error messages.
* Factor out the ability to ignore the exit codes of "zipinfo" and
"zipinfo -v" in the presence of non-standard headers.
* Only override the exit code from our special-cased calls to zipinfo(1) if
they are 1 or 2 to avoid potentially masking real errors.
* Also add missing textual description entries for ZipFile and
MozillaZipFile file types.
* Skip extra newline in "Output:\n<none>".
* Cease ignoring test failures in stable-backports.
* Bump debhelper compat level to 12.
[ Marc Herbert ]
* Catch failures to disassemble and rescue all other differences.
(Closes: reproducible-builds/diffoscope!29)
-- Chris Lamb <lamby@debian.org> Mon, 29 Jul 2019 15:22:24 -0300
diffoscope (118) unstable; urgency=medium
* Don't fail in autopkgtests when, for example, we do not have sufficiently
newer or older version of file. (Closes: #931881)
* Also include python3-tlsh in test dependencies.
* Tidy handling of DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS:
- Merge two previously-overlapping environment variables.
- Use repr(..)-style output when printing test status.
- Add some explicit return values to appease pylint.
-- Chris Lamb <lamby@debian.org> Sat, 13 Jul 2019 10:23:29 -0300
diffoscope (117) unstable; urgency=medium
[ Chris Lamb ]
* Add support for file 5.37. Thanks again to Christoph Biedl for the heads-up
in advance. (Closes: reproducible-builds/diffoscope/#57)
* Apply patch from Gianfranco Costamagna to fix autopkgtest failures in
Debian. (Closes: #931709)
[ Holger Levsen ]
* debian/control: Bump Standards-Version to 4.4.0.
-- Chris Lamb <lamby@debian.org> Tue, 09 Jul 2019 10:24:54 -0300
diffoscope (116) unstable; urgency=medium
[ Chris Lamb ]
* Upload to unstable after the release of Debian "buster".
* README.rst & manual page generation:
- Strip out manpage-only parts of the README rather than using "only"
reStructuredText directives in order to support the demands of the latest
PyPI website.
- Use "real" reStructuredText comments instead of using the "raw" directive.
* Dockerfile:
- Build the Docker image from the current Git checkout, not the Debian
archive. (reproducible-builds/diffoscope#56)
- Use the ENTRYPOINT directive with the JSON syntax instead so we pass all
arguments to the underlying diffoscope executable.
* Document that run_diffoscope should not be considered a stable API.
[ Mattia Rizzolo ]
* Rename a test function to prevent shadowing a previous one with the
same name.
* Add ffmpeg to the list of Debian build-dependencies for the testsuite.
[ Vagrant Cascadian ]
* Add support for known external tools in GNU Guix.
-- Chris Lamb <lamby@debian.org> Sun, 07 Jul 2019 11:54:29 -0300
diffoscope (115) experimental; urgency=medium
[ Chris Lamb ]
* Fix execution of symbolic links that point to the "bin/diffoscope" entry
point by fully resolving the location.
* Add a Dockerfile.
* Update contributing messages with instructions regarding the Docker image.
[ Mattia Rizzolo ]
* tests:
+ Allow specifying which tools are known missing using
DIFFOSCOPE_TESTS_MISSING_TOOLS, to override a _FAIL_ON_MISSING_TOOLS.
+ With DIFFOSCOPE_TESTS_FAIL_ON_MISSING_TOOLS=1, actually fail only tests
that are missing the required tools.
* debian:
+ Add black to the test dependencies.
+ Enforce the "fail on missing tools" only when testing with all the
recommended packages.
+ Ack some missing tools during autopkgtest.
+ Reinstate oggvideotools and procyon-decompiler test dependencies,
now that are not buggy anymore.
-- Mattia Rizzolo <mattia@debian.org> Tue, 21 May 2019 16:02:02 +0200
diffoscope (114) experimental; urgency=medium
[ Chris Lamb ]
* Add support for GnuPG "keybox" files. Thanks to Daniel Kahn Gillmor for the
suggestion. (Closes: #871244, reproducible-builds/diffoscope#23)
* Always warn if tlsh module is not available (not just if a specific
fuzziness threshold is specified) to match the epilog of the --help output.
This prevents missing support for file rename detection.
(Closes: #888237, reproducible-builds/diffoscope#29)
* Treat missing tools on Debian autopkgtests as individual test failures by
checking whether a new DIFFOSCOPE_TESTS_FAIL_ON_MISSING_TOOLS environment
variable is exported. (Closes: #905885, reproducible-builds/diffoscope#35)
* Require that "-" is explicitly specified to read a single diff from
standard input to avoid non-intuitive behaviour when diffoscope is called
without any arguments. (Closes: reproducible-builds/diffoscope#54)
* Make --use-dbgsym a ternary operator to make it easier to totally disable.
Thanks to Mattia Rizzolo for the suggestion.
* Consolidate on "e" as the aliased exception name.
[ Milena Boselli Rosa ]
* Various fixes to the HTML markup to prevent validation warnings/errors:
- Prevent empty values for the "name" attribute name on HTML anchor tags,
and add an "id" to its parent "div" container.
- Fix "table column x established by element 'col' has no cells beginning
in it" warnings.
- Fix "Text run is not in Unicode Normalization Form C".
- Remove the "type" HTML attribute from <style> elements.
[ Vibhu ]
* Fail more gracefully when running out of diskspace. (Closes: #874582)
-- Chris Lamb <lamby@debian.org> Sun, 28 Apr 2019 17:21:46 +0100
diffoscope (113) unstable; urgency=medium
* Replace over 8 MB of Android boot ROM test suite fixtures with 14 KB
equivalents. (Closes: #894334, reproducible-builds/diffoscope#13)
* Compare .asc PGP signatures as text, not as a hexdump. (Closes: #908991,
reproducible-builds/diffoscope#7)
* Improve the displayed comment when falling back to a binary diff to include
the file type. (Closes: reproducible-builds/diffoscope#49)
* Explicitly mention when the guestfs module is missing at runtime and we are
falling back to a binary diff. (Closes: reproducible-builds/diffoscope#45)
* Provide explicit help when the libarchive system package is missing or
"incomplete". (Closes: reproducible-builds/diffoscope#50)
* Improve the --help outout:
* Indent and wrap the list of supported file formats.
* Include links to the diffoscope homepage and bug tracker.
* Refer to the Debian package names when indicating how to obtain the tlsh
and argcomplete modules.
* Drop "DOS/MBR" source string test.
* Correct a "recurse" typo.
* Adopt the "black" <https://black.readthedocs.io> source code formatter:
- Add an initial configuration in a PEP 518 pyproject.toml file and update
MANIFEST.in to include pyproject.toml in future release tarballs.
- Run the formatter against the source.
- Test that the source code satisfies the formatter.
-- Chris Lamb <lamby@debian.org> Mon, 04 Mar 2019 23:00:38 +0100
diffoscope (112) unstable; urgency=medium
[ Chris Lamb ]
* New features:
- Add a --exclude-directory-metadata=recursive option to support ignoring
timestamp differences (etc.) in nested archives/containers.
(Closes: #907600, reproducible-builds/diffoscope#36)
- Add support for comparing metadata in MP3 files.
(Closes: reproducible-builds/diffoscope#43)
- Add support for comparing .crx Chrome browser extensions.
(Closes: reproducible-builds/diffoscope#41)
* Bug fixes:
- Catch tracebacks when mounting invalid filesystem images under guestfs.
- Ensure both WASM test data files are actually unique.
(Closes: reproducible-builds/diffoscope#42)
- Replace literal xxd(1) output in tests/data/hello.wasm with its binary
equivalent. (Closes: reproducible-builds/diffoscope#47)
* Misc improvements:
- Rework and comment logic determining a default for
exclude_directory_metadata.
- Fix a number of Ghostscript-related test issues regarding the update of
this package from 9.20 to 9.26 in Debian "stable".
[ Mattia Rizzolo ]
* Make test_ps.test_text_diff pass with ghostscript 9.26.
[ Ed Maste ]
* Include relocation information in objdump disassembly.
(Closes: reproducible-builds/diffoscope#48)
[ Graham Christensen ]
* Clarify notice if no file-specific diff caused fallback. (MR: !19)
-- Chris Lamb <lamby@debian.org> Tue, 19 Feb 2019 09:56:29 +0100
diffoscope (111) unstable; urgency=medium
* Fix a regression introduced via #920701 where we stopped using the -dbgsym
packages when comparing .buildinfo or .changes files. (Closes:
reproducible-builds/diffoscope#46)
-- Chris Lamb <lamby@debian.org> Sun, 10 Feb 2019 13:09:26 +0100
diffoscope (110) unstable; urgency=medium
[ Chris Lamb ]
* Don't look for adjacent -dbgsym package files automatically anymore to
align better with default user expectations. The existing behaviour can be
re-enabled by specifying the new `--use-dbgsym` flag. (Closes: #920701)
* Don't crash with if we were unable to successfully extract a
"guestfs"-based file. (Closes: #901982)
* Avoid clumsy profiling title length calculations by moving to Markdown
syntax, from reStructuredText.
* Drop printing out dpkg-query output when running tests: it's rather noisy
and has not helped yet in reproducing an error.
* Re-enable gnumeric as a Build-Depends.
* debian/rules: Use str.format over "+" for string concatenation.
[ Jelle van der Waa ]
* comparators.wasm: Remove an unused "re" import. (MR: !18)
[ Mattia Rizzolo ]
* comparators/elf: Apply flake8.
-- Chris Lamb <lamby@debian.org> Wed, 06 Feb 2019 17:33:09 +0100
diffoscope (108) unstable; urgency=medium
[ Chris Lamb ]
* test_icc: Calculate the path to a test .icc file using data() rather than
using the pytest fixture itself to avoid a "Fixtures are not meant to be
called directly" warning/error. (Closes: #916226)
* Fix a test_mozzip_compressed_files test failure under Alpine Linux.
(Closes: #916353, MR: !8)
* Output improvements:
* Prefer to comment that files are identical rather than having a
"fuzziness score" of zero.
* Expand the padding/spacing for "N calls" in the profiling output; having
99,999+ calls to cmp(1) is not uncommon for ISOs.
* Add a note to the "Files similar despite different names" comment to
clarify that a lower score is more similar.
* Use File.file_header to tidy JSON matching in the json, ppu and wasm
comparators.
* Drop debbindiff Breaks/Replaces; was removed in 2015, never part of a
stable release, etc.
* Correct a "positives" typo.
[ Joachim Breitner ]
* Add support for comparing WebAssembly modules. (MR: !17)
[ Holger Levsen ]
* Bump Standards-Version to 4.3.0
* Clarify that (upstream) issues should be reported via the
issue tracker on salsa.debian.org.
[ Mattia Rizzolo ]
* Try matching for MozillaZipFile before ZipFile.
-- Chris Lamb <lamby@debian.org> Sun, 13 Jan 2019 19:21:05 +0000
diffoscope (107) unstable; urgency=medium
* Compare .zip file comments with zipnote. (Closes: #901757)
* Don't emit a large number of warnings if getfacl(1) is not available. This
makes the behaviour consistent with if lsattr(1) is unavailable.
(Closes: #902369)
-- Chris Lamb <lamby@debian.org> Thu, 06 Dec 2018 22:42:54 +0100
diffoscope (106) unstable; urgency=medium
[ Chris Lamb ]
* comparators:
+ elf: Don't assume all files called ".a" are ELF binaries. Closes: #903446
+ pdf:
- Display the reason when cannot extract metadata from PDF files.
- Prevent tracebacks when obtaining PDF metadata from files with multiple
PDF dictionary definition entries. Closes: #913315
[ Marius Gedminas ]
* Add a python_requires to setup.py.
[ Mattia Rizzolo ]
* debian:
+ Disable depends on gnumeric and procyon as they are RC buggy.
+ Override the new lintian's public-upstream-key-in-native-package.
-- Mattia Rizzolo <mattia@debian.org> Fri, 23 Nov 2018 11:09:28 +0100
diffoscope (105) unstable; urgency=medium
[ Chris Lamb ]
* tests:
+ Prevent test failures when running under stretch-backports by checking
the ocaml version number. Closes: #911846
[ Mattia Rizzolo ]
* debian: Reinstate apktool Build-Depends and Test-Depends.
* Fix some flake8 and deprecation warnings.
[ Daniel Shahaf ]
* comparators/pcap:
+ Fix recognition with the upcoming file(1) 5.35. Closes: #912756
Thanks to Christoph Biedl for the heads-up in advance.
[ Will Thompson ]
* Add a new command line flag --list-missing-tools. MR: !14
-- Mattia Rizzolo <mattia@debian.org> Thu, 08 Nov 2018 22:27:23 +0100
diffoscope (104) unstable; urgency=medium
[ Chris Lamb ]
* comparators:
+ macho: Prevent a traceback if the filename cannot be found on the line.
Closes: #910540
+ ocaml New comparator OCaml files using ocamlobjinfo. Closes: #910542
+ pdf: Add support for comparing metadata using PyPDF2. Closes: #911446
* debian/tests/control.in: Add note on how to regenerate this file.
[ Mattia Rizzolo ]
* d/control:
+ Build-Depend on linux-image-amd64 [amd64] to allow squashfs tests to run.
+ Add a Build-Conflicts against graphicsmagick-imagemagick-compat.
See: #910652
* Move the list of optional python packages from d/rules into setup.py, and
have dh_python3 check the generated requires.txt.
* Also add the optional python modules to the autopkgtest dependencies.
* Temporarily drop Build-Depends and Test-Depends on apktool and
oggvideotools, as they are not available in buster at the moment.
* comparators:
+ java:
- Rename the tool procyon-decompiler to procyon.
- Properly fall back from procyon to javap also when procyon exists
but doesn't return any output.
* Declare in setup.py that diffoscope is good to go with Python 3.7.
[ Marek Marczykowski-Górecki ]
* comparators:
+ fsimage: Handle FAT filesystems. MR: !13
-- Mattia Rizzolo <mattia@debian.org> Thu, 25 Oct 2018 14:55:41 +0200
diffoscope (103) unstable; urgency=medium
[ Mattia Rizzolo ]
* In Debian, list liblz4-tool as an alternative to the lz4 package.
stretch only has the former.
[ Chris Lamb ]
* Strip trailing whitespace from ssconvert(1) output to support
gnumeric 1.12.43.
-- Mattia Rizzolo <mattia@debian.org> Wed, 03 Oct 2018 16:35:59 +0200
diffoscope (102) unstable; urgency=medium
[ Chris Lamb ]
* Fix tests under colord >= 1.4.3. Closes: #908900
[ Xavier Briand ]
* Add an "Add a comparator" section in CONTRIBUTING. MR: !9
[ Mattia Rizzolo ]
* debian: Use the new debhelper-compat(=11) build dep and drop d/compat.
[ Marek Marczykowski-Górecki ]
* comparators/json: Try fuzzy matching for non-text files too.
This avoids loading very large file just to discover they aren't JSON.
Closes: #909122
-- Mattia Rizzolo <mattia@debian.org> Sun, 23 Sep 2018 10:43:40 +0200
diffoscope (101) unstable; urgency=medium
* Fix testsuite under LLVM >= 7.0. (Closes: #908074)
* Substvar generation:
- Ensure Debian substvar generation is deterministic regardless of
installed packages. (Closes: #908072)
- Clarify distinction between tools (eg. gpg) and packages (eg. gnupg) when
generating Debian substvars.
- Revert updating generated debian/tests/control made in diffoscope 100.
* Disable binwalk's own user configuration for predictable results and to
ensure it does not create unnecessary directories. (Closes: #903444)
* Ensure we return "bytes" objects from Command.filter to avoid a
"TypeError: Unicode-objects must be encoded before hashing" traceback.
* Don't print GPG output in diffoscope.changes.
-- Chris Lamb <lamby@debian.org> Sat, 15 Sep 2018 16:37:45 +0100
diffoscope (100) unstable; urgency=medium
[ Alexis Murzeau ]
* Correct matching of .deb archive members. (Closes: #903565)
[ Chris Lamb ]
* Support .deb archives that contain an uncompressed control.tar and data.tar
Thanks to Roderich Schupp <roderich.schupp@gmail.com>.
(Closes: #903391, #903401)
* Wrap jsondiff calls with try-except to prevent fatal errors.
(Closes: #903447, #903449)
* Don't include the filename in llvm-bcanalyzer results. (Closes: #905598)
* Update generated debian/tests/control.
* Bump Standards-Version to 4.2.1.
[ Daniel Kahn Gillmor ]
* Avoid line eraser error on dumb terminals. (Closes: #906967)
* Correct spelling of ereser to eraser.
[ Mattia Rizzolo ]
* On Debian, do not require apktool on ppc64el and s390x, ogvideotools on
s390x and fp-utils on ppc64el and s390x as they are not installable there.
* Explicitly add `file` to the dependencies of autopkgtest to have the tests
triggered whenever `file` changes
[ Ricardo Gaviria ]
* Handle errors with encrypted archive files. (Closes: #904685)
-- Chris Lamb <lamby@debian.org> Sat, 01 Sep 2018 09:09:48 +0100
diffoscope (99) unstable; urgency=medium
[ Xavier Briand ]
* Add lz4 comparator. Closes: #901548; !4
[ Paul Wise ]
* Clear the progress bar after completion. Closes: #901758
Handle terminals that do not support erasing the line by filling the
terminal with spaces. Ignore output devices that are not terminals.
* Do not delete the current terminal line for every progress bar update.
The erasure was also causing the progress bar to flicker.
[ Mattia Rizzolo ]
* Add a gitlab CI script. !8
* tempfiles:
+ Set the base directory only if the caller didn't specify one already.
+ Do not append '_diffoscope' to the temporary file names, as they are now
all in their namespaced directory already.
+ Empty the list of known tempfiles after removing them. Closes: #902709
* external_tools:
+ Fix package name, s/xmlutils/xmlbeans/.
* tests/test_tools: fix test if /sbin contains a directory. MR: !2
Thanks to Chris Lamb <lamby@debian.org> for the patch.
* logging:
+ Move the computation of the terminal line eraser into logging.py
+ Always clean the line before printing a log message
* main:
+ Clean the terminal line before printing a traceback.
+ Be sure to print 'Keyboard Interrupt' if that happens.
* comparators:
+ Do not shadow the original import errors. MR: !7
* d/control:
+ Bump Standards-Version to 4.1.5, no changes needed.
+ Add Build-Dependency on procyon-decompiler, to run the tests.
* d/tests:
+ Autogenerate d/tests/control with all the recommends listed as
dependencies, so autopkgtest is appropriately triggered whenever one of
those packages changes. Closes: #902920
[ anthraxx ]
* comparators.utils:libarchive:
+ Add compatibility code for python-libarchive >= 2.8. MR: !6
-- Mattia Rizzolo <mattia@debian.org> Sun, 08 Jul 2018 20:07:06 +0200
diffoscope (98) unstable; urgency=medium
* Fix compatibility with Python 3.7. (Closes: #902650)
-- Chris Lamb <lamby@debian.org> Fri, 29 Jun 2018 08:19:25 +0100
diffoscope (97) unstable; urgency=medium
* Create all temporary directories within a top-level dir. (Closes: #902627)
* tests/conftest.py: Fix compatibility with pytest 3.6.2-1, currently in
Debian experimental.
-- Chris Lamb <lamby@debian.org> Thu, 28 Jun 2018 21:38:50 +0100
diffoscope (96) unstable; urgency=medium
[ Chris Lamb ]
* Drop dependency on pdftk as it relies on GCJ, relying on the pdftotext
fallback. (Closes: #893702)
* Change the "No file format specific differences found inside, yet data
differs" message to be clearer that diffoscope "knows" about this file
format yet could not be helpful in this case.
* Don't append a rather useless "(data)" suffix from file(1).
* Comply with a number of PEP8 recommendations:
- E226 - Add missing whitespaces around operators.
- E241 - Fix extraneous whitespaces around keywords.
- E251 - Remove whitespace around parameter '=' signs.
- E302 - Add missing 2 blank lines.
- E501 - Try to make lines fit to length.
- E502 - Remove extraneous escape of newline.
- E731 - Don't assign lambda expressions.
- E121, E122, E126, E128 - Fix badly indented lines.
[ Xavier Briand ]
* Add merge request details to contributing documentation.
-- Chris Lamb <lamby@debian.org> Sat, 16 Jun 2018 22:19:28 +0200
diffoscope (95) unstable; urgency=medium
[ Mattia Rizzolo ]
* tests:
+ test_binary: Don't capture unused output from subprocess.
+ test_git: Fix test failure on FreeBSD. Closes: #872826
Thanks to Ximin Luo <infinity0@debian.org> for the initial patch
* Fix handling of filesnames with non-unicode chars. Closes: #898022
* diff: Use bytes as much as possible, to prevent possible encoding issues.
* d/control: Make the dependency on python3-distutils an alternative on the
previous versions of libpython3.*-stdlib that used to ship it.
Closes: #898683
* d/watch: Update URL to the new archive.
[ Chris Lamb ]
* The Git repository has been migrated to salsa, update all the references.
* Drop extra whitespace in supported file format output.
-- Mattia Rizzolo <mattia@debian.org> Sun, 20 May 2018 17:46:57 +0200
diffoscope (94) unstable; urgency=medium
[ Chris Lamb ]
* Update tests to prevent FTBFS under file 5.33. (Closes: #897099)
* Remove all __pycache__ directories to avoid source-contains-empty-directory
Lintian warning.
* Remove unused test1.txt test data file.
* Bump Standards-Version to 4.1.4.
[ anthraxx ]
* Add gnumeric to the list of ArchLinux tools.
[ Paul Wise ]
* Sort the Debian dependencies.
-- Chris Lamb <lamby@debian.org> Sat, 05 May 2018 22:44:44 -0700
diffoscope (93) unstable; urgency=medium
* Only append the file formats if --help is passed, otherwise we spam
#debian-reproducible-changes et al. (re. #893443)
* doc/Makefile: Reinstate accidental version parsing change; my sed call
was over-eager.
-- Chris Lamb <lamby@debian.org> Thu, 29 Mar 2018 10:03:03 +0100
diffoscope (92) unstable; urgency=medium
[ Chris Lamb ]
* New features:
- Show the list of supported file types in the manual and --help output. We
can copy-paste this into the website, etc. Thanks to Jonathan Dowland
for the suggestion. (Closes: #893443)
- Add support for comparing Gnumeric spreadsheets. (Closes: #893311)
* Bug fixes:
- Additionally extract the classes.dex file from .apk files; apktool does
not do this by default which was preventing us from recursing into it to
find differences. Thanks to Hans-Christoph Steiner for the report.
(Closes: #890904)
- Recursively reset the permissions of temporary directories prior to
deletion to ensure that non-writable permissions such as 0555 are removed
and do not cause a traceback. (Closes: #891363)
- Support the case where the python3-xattr package is installed but
python3-pyxattr is not. Based on an patch by Doug Freed; thanks!
(Closes: #892240)
- Update Java tests for openjdk-9. (Closes: #893183)
* Output:
- Print a nicer error message if you only specify one file to compare.
- Don't show progress bar if we passed --debug as it just gets in the way.
* Code tidying:
- Avoid some necessary indentation around unconditional control flow.
- Tidy unnecessary assignments.
- Move the documentation for maybe_decode into a docstring on the method
itself.
- Import LooseVersion as LooseVersion.
- Use more Pythonic `old_level` variable names (over `oldLabel`) and avoid
using ambiguous `l` variable names.
- Add whitespace around operators and ensure 4-line indentation throughout.
* debian/*:
- Move DOS/MBR check into the testsuite.
- Add explicit runtime dependency on python3-distutils as it was dropped in
src:python3.6 3.6.5~rc1-2.
* Misc:
- Clarify that the Reproducible Builds project is not just about Debian.
- Drop executable bit on doc/Makefile.
- Use our bin/diffoscope wrapper in manpage generation to ensure we are
using the local version of diffoscope.
[ Mattia Rizzolo ]
* Update terminology used in docs about exclusion options. Thanks to Paul
Wise for the idea. (Closes: #893324)
* Don't try to decode a string in comparators.utils.file. (Closes: #891903)
* Save a bunch of system calls by resetting tempfile permissions when
we actually need to do it.
-- Chris Lamb <lamby@debian.org> Thu, 22 Mar 2018 10:28:44 -0400
diffoscope (91) unstable; urgency=medium
[ Mattia Rizzolo ]
* d/clean: Remove .cache/.
* diff: Wrap long regular expression for improved readability.
* comparators.json: Fix UnicodeDecodeError with a non-UTF8 locale.
* d/copyright: Update for the new year.
* d/control: Bump Standards-Version to 4.1.3, no changes needed.
* tests/android:
+ Skip tests involving abootimg on BE archs (see #725729).
[ Daniel Shahaf ]
* diffoscope.diff: Document feeders. Closes: #863880
* Optimize the common case of feeders. Closes: #863879
[ Juliana Oliveira ]
* {tempfiles, utils.archive}: catch possible FileNotFoundError.
* comparators.java: add support for procyon-decompiler. Closes: #849386
* {command, feeders, diff}: replaces subprocess.Popen() by .run().
This is done in preparation for work on multiprocessing (#842837).
+ {command, zip, feeders}: replaces .wait by the new .returncode property.
+ comparators.utils.command: replaces .stderr_content by .stderr.
* tests:
+ test_progress: fix test case for some broken versions of
python-progressbar. Closes: #877726
+ utils.tools: add support for modules on skip_unless* annotations.
[ Chris Lamb ]
* comparators:
+ utils/compare:
- Show extended filesystem metadata even when directly comparing two
files, not just when we specify two directories. Closes: #888402
+ macho:
- If the If the LLVM disassembler does not work, try the
internal one. Closes: #886736
- Always strip the filename, not just when by itself.
+ json:
- Do some cheap fuzzy parsing to detect JSON files not named .json.
- Also match unicode JSON files.
- Optionally compare JSONs with the jsondiff module. Closes: #888112
+ directory:
- Report differences in extended file attributes when comparing files.
Closes: #888401
+ xsb:
- Add support for comparing XMLBeans binary schemas.
+ barkeley_db:
- Add support for comparing Berkeley DB files. Closes: #890528
* Misc code cleaup.
* tests:
+ comparators.test_elf: Return '0' if we can't parse the readelf
version number. Closes: #886963
* debian:
+ Explicitly build-depend and recommend e2fsprogs. Closes: #887180
[ Ximin Luo ]
* Partially revert the changes done for #888402 to maintain the current.
behaviour of --exclude-directory-metadata.
* Refactor how the configuration is loaded:
+ Move the defaults into a Config.reset() method.
+ reset() the configuration at the end of main(), to help the testsuite.
-- Mattia Rizzolo <mattia@debian.org> Thu, 01 Mar 2018 13:40:48 +0100
diffoscope (90) unstable; urgency=medium
[ Juliana Oliveira ]
* tests:
+ comparators/test_rlib: Fix tests for llvm >= 5.0. Closes: #877727
+ Skip some tests if 'xz' is not present.
-- Mattia Rizzolo <mattia@debian.org> Tue, 26 Dec 2017 19:49:30 +0100
diffoscope (89) unstable; urgency=medium
[ Mike Hommey ]
* comparators:
+ elf:
- Add fallbacks to elf code section disassembly. Closes: #879003
+ utils/libarchive:
- Extract libarchive members with the file extension.
[ Ximin Luo ]
* Auto-generate manpage using help2man, so it's no longer out-of-date.
* difference:
+ Add a Difference.from_command_exc() to help distinguish excluded
commands from commands returning an empty diff.
* comparators:
+ utils/command:
- Replace the feed_stdin() feeder into the simpler stdin() function,
which provides (if needed) a simple file opject instead of a feeder.
+ utils/container:
- Add a perform_fuzzy_matching() method (wrapping the already existing
method) so that comparators can easily override it.
+ deb:
- Loose matching for .deb archive members, so it's possible to e.g.
compare deb with different control.tar.{gz,xz}. Closes: #881937
+ elf:
- Improve disassembly fallbacks with Difference.from_command_exc().
+ png:
- Cater for the feed_stdin() removal.
+ zip:
- Run zipinfo on /dev/stdin instead of a variable path to avoid including
the temporary file name in the output. Closes: #879011
- Cater for the feed_stdin() removal.
* presenters/formats:
+ Allow non-text formats to output an empty diff.
* debian/clean:
+ Remove the egg file when cleaning.
[ Chris Lamb ]
* comparators:
+ utils/file:
- Handle case where a file to be "fuzzy" matched does not contain enough
entropy despite being over 512 bytes. Closes: #882981
+ android:
- Add support for Android ROM boot.img introspection. Closes: #884557
+ symlink:
- Make cleanup of placeholders idempotent.
[ Juliana Oliveira Rodrigues ]
* tests:
+ utils/tools:
- Add a skip_if_tool_version_is() function.
+ elf:
- Skip some tests if readelf is version '2.29', where its behaviour
was accidentally changed to exit with code 1 when it previously
didn't. Closes: #877728
[ Holger Levsen ]
* debian/control: Bump Standards-Version to 4.1.2. No changes required.
[ Mattia Rizzolo ]
* debian/control: Set Rules-Requires-Root: no.
* Bump Debhelper compat version to 11.
-- Mattia Rizzolo <mattia@debian.org> Mon, 25 Dec 2017 14:49:22 +0100
diffoscope (88) unstable; urgency=medium
[ Holger Levsen ]
* Bump standards version to 4.1.1, no changes needed.
[ Mattia Rizzolo ]
* tests/comparators:
+ dtb: compatibility with version 1.4.5. Closes: #880279
[ Chris Lamb ]
* comparators:
+ binwalk: improve names in output of "internal" members. Closes: #877525
+ Omit misleading "any of" prefix when only complaining about one module
in ImportError messages.
* Don't crash on malformed md5sums files. Closes: #877473
* tests/comparators:
+ ps: ps2ascii > 9.21 now varies on timezone, so skip this test for now.
+ dtby: only parse the version number, not any "-dirty" suffix.
* debian/watch: use HTTPS URI.
[ Ximin Luo ]
* comparators:
+ utils/file: diff container metadata centrally. Closes: #797759
This fixes a last remaining bug in fuzzy-matching across containers.
+ Fix all the affected comparators after the above change.
-- Mattia Rizzolo <mattia@debian.org> Sat, 04 Nov 2017 16:26:09 +0100
diffoscope (87) unstable; urgency=medium
[ Juliana Oliveira Rodrigues ]
* comparators:
+ utils:
- compare: fix comparison between container types.
- container: add get_path_name() method, aiming at preventing hitting
OS' max filename length.
+ gzip: replace dest_path generation with self.get_path_name().
+ bzip2: replace dest_path generation with self.get_path_name().
* tests:
+ presenters:
- skip html_visuals test if 'sng' binary is not available.
+ comparators:
- test_cointainers: new tests to test cross-container comparisons.
- add test_containers.py.
[ Chris Lamb ]
* comparators:
+ utils:
- file: correct reference to path_apparent_size.
+ fonts: loosen matching of file(1)'s output to ensure we correctly also
match TTF files under file 5.32.
+ binwalk:
- new comparator to deal with (e.g.) concatenated CPIO archives using
the new optional python module 'binwalk'. Closes: #820631
* diffoscope.diff: Correct reference to self.buf.
* debian/rules: run flake8 during the tests, and fail the build in case the
"underfined name" flake8's check fails.
* tests/comparators:
+ test_cpio: check we identify all CPIO fixtures.
* main:
+ print a debugging message if we are reading diff from stdin.
* presenters:
+ html: use logging.py's lazy argument interpolation.
* debian/control: bump Standards-Version to 4.1.0.
* Code style improvements across all of the codebase.
[ Mattia Rizzolo ]
* debian/changelog: retroactively close a bug in an old changelog entry.
* setup.py: add a check to prevent people from installing diffoscope under
python < 3.5 (i.e. python2) and give them a usable error message otherwise.
* Code style improvements across all of the codebase.
[ Ximin Luo ]
* comparators:
+ add a fallback_recognizes() method to work around file(1)'s bug #876316.
Closes: #875282
+ add a --force-details flag for debugging.
+ deb:
- if --force-details then don't skip files with identical md5sums either.
* presenters:
+ html:
- prune all descendants properly. Closes: #875281
- don't show pointer-cursor when jquery is disabled.
* config:
+ force-set a value if it must be less than another, and it was not set on
purpose. Closes: #875451
* readers: convert bytes to str in the right place.
* diff: use diff_split_lines everywhere.
* difference:
+ in fmap/map_lines, don't forget about self._visuals.
+ also copy self._comment properly, compare self._visuals in equals().
* tests/comparators:
+ test_deb: make test_md5sums less brittle.
-- Mattia Rizzolo <mattia@debian.org> Fri, 22 Sep 2017 18:53:05 +0200
diffoscope (86) unstable; urgency=medium
[ Mattia Rizzolo ]
* tests:
+ binary: skip a test if the 'distro' module is not available.
+ iso9660: avoid an overly-catching exception.
* debian/copyright: coalesce some file paragraphs and update information.
[ Guangyuan Yang ]
* tests:
+ iso9660: support both cdrtools' genisoimage's versions of isoinfo.
[ Chris Lamb ]
* comparators:
+ xml: Use ``name`` attribute over ``path`` to avoid leaking comparison
full path in output.
* Tidy diffoscope.progress a little.
[ Ximin Luo ]
* Add a --tool-prefix-binutils CLI flag. Closes: #869868
* On non-GNU systems, prefer some tools that start with "g". Closes: #871029
* presenters:
+ html:
- Don't traverse children whose parents were already limited.
Closes: #871413
[ Santiago Torres-Arias ]
* diffoscope.progress:
+ Support the new fork of python-progressbar. Closes: #873157
-- Mattia Rizzolo <mattia@debian.org> Sun, 03 Sep 2017 22:23:06 +0200
diffoscope (85) unstable; urgency=medium
[ Mattia Rizzolo ]
* tools:
+ move from the deprecated platform.linux_distribution() to the external
python package "distro". Add it as an optioanl dependency, as without it
distribution detection (i.e. `diffoscope --list-tools`) won't work.
Also add it as a Debian (hard) dependency, as it's a very lightway
package, and the usefulness it brings are great.
+ add a get_package_provider() function, returning the package name
that best matches the system.
* exc:
+ in RequiredToolNotFound.get_package(), just call the new
get_package_provider()
* debian/rules:
+ recommends the defusedxml python package, to avoid using the python3's
standard xml library and its security holes.
[ Chris Lamb ]
* comparators:
+ sqlite:
- Simplify file detection by rewriting manual `recognizes` call
with a `Sqlite3Database.RE_FILE_TYPE` definition.
+ xml:
- Fix EPUB "missing file" tests; they ship a META-INF/container.xml file.
[ Ximin Luo ]
* comparators:
+ factor common logic from various comparators into File.recognizes.
+ more tidying up and making names consistent.
+ directory:
- make stat(1) warning textually like the other warnings.
- bump stat(1) warning into an error.
- use getfacl(1) before lsattr(1) as it's more general.
+ apk:
- less kludgy way of detecting APKs. Closes: #868486
* main, logging:
+ restore old logger settings to avoid pytest fail in certain situations.
* debian/rules:
+ add a check to prevent additions of "DOS/MBR" file type.
* feeder:
+ force a flush when writing output to diff. Closes: #870049
* tests/comparators:
+ directory:
- be less strict about the expected test output, to cope with a missing
`getfacl`. Closes: #868534
[ Juliana Oliveira Rodrigues ]
* comparators:
+ Add new XML comparator. Closes: #866120
The comparator will use defusedxml if this is installed, to avoid
falling in known security holes coming from the XML specification.
* tests/comparators:
+ apk:
- fix the tests after the addition of the XML comparator.
+ image:
- fix test_ico_image for identify >= 6.9.8.
[ Guangyuan Yang ]
* tests/comparators:
+ device:
- fix 2 cases for FreeBSD.
-- Mattia Rizzolo <mattia@debian.org> Mon, 31 Jul 2017 22:27:45 +0200
diffoscope (84) unstable; urgency=medium
[ Ximin Luo ]
* comparators:
+ directory:
- raise warning for getfacl and remove a redundant try-clause.
- add a --exclude-directory-metadata option. Closes: #866241
* diff:
+ Restore artificial limit when calculating linediff, to prevent memory
blow up. Closes: #865660
* presenters:
+ text:
- fix recursive header display in the text output, deeply-nested headers
were not getting indented far enough.
+ add a PartialString class.
+ html:
- in html-dir mode, put css/icon in separate files to avoid duplication.
- split index pages up if they get too big.
- add a size-hint to the diff headers and lazy-load buttons.
- diffcontrol UI tweaks:
* don't toggle buttons for diffs that haven't been loaded yet;
* make the diffcontrol headers clickable.
- more intuitive "limit" flags, with some backwards-incompatible changes:
--max-report-size:
Old: in --html-dir this limited only the parent page
New: in --html-dir this applies across all pages
--max-diff-block-lines:
Old: in --html-dir 4 * this number applied across all pages
New: in --html-dir this applies across all pages
--max-page-size:
New flag
Applies to the sole --html page, or the top-level --html-dir page
--max-report-child-size
Renamed to
--max-page-size-child:
No behavioural changes
--max-diff-block-lines-parent
Renamed to
--max-page-diff-block-lines:
Old: Only applied to the top-level --html-dir page
New: Applies to the sole --html page or the top-level --html-dir page
* main:
+ temporarily add old flags back for backwards-compatibility.
+ warn if loading a diff but also giving diff-calculation flags.
+ implement proper boolean flags to fix test failure in previous commit.
* tests:
+ handle existing but not importable modules.
This fixes the tests during the Python 3.6 transition, where some
modules (like rpm) exists but are not yet rebuilt for 3.6.
+ fix progressbar failure that was exposed by the previous importing fixes.
+ presenters:
- skip visual test if tools aren't installed. Closes: #865625
+ comparators
- sqlite: test for sqlite 3.19.
- fsimage: move the guestfs cache out of the default /var/tmp/ into a
"more temporary" directory.
[ Mattia Rizzolo ]
* debian/control: fix Vcs-Git field.
[ Chris Lamb ]
* debian/control: bump Standards-Version to 4.0.0.
[ Daniel Shahaf ]
* Fix markup in the man page synopsis. Closes: #866577
Thanks to Niels Thykier for the report.
-- Mattia Rizzolo <mattia@debian.org> Wed, 12 Jul 2017 12:28:11 +0200
diffoscope (83) unstable; urgency=medium
[ Chris Lamb ]
* New features:
* Add comparator for Fontconfig cache files.
* Bug fixes:
- Don't fail when run under perversely recursive input files. See
<https://research.swtch.com/zip> for more information. (Closes: #780761)
- Prevent a traceback when using --new-file with containers by passing
progress_name of the Member and not the member itself. (Closes: #861286)
- Drop passing has_differences around, fixing an issue with generating
files called '-'.
* Cleanups & refactoring:
- Reorder and rename FIFOFeeder arguments to prevent tools incorrectly
parsing as a SyntaxError.
- Refactor HTML and text presenters so they fit the same interface.
- Refactor Presenter to a singleton manager.
- Split output and configuration of presenters.
- Prevent abstraction-level violation by defining visual diff support on
the Presenter classes.
- Split diffoscope.difference into a feeders module.
- Document various variables.
- PEP8-ify and tidy a lot of files.
[ Ximin Luo ]
* New features:
- Add --max-container-depth CLI option.
- Add various traverse_* methods to Difference.
- Weigh elements in progress bar by their size.
- Add a reader for the JSON format.
- Add a --exclude-command CLI for filtering out long-running commands like
"readelf --debug-dump=info".
- Don't show +/- controls for differences with no children.
- Use unicode chars for the +/- controls instead of punctuation.
* Bug fixes:
- Fix --exclude control.tar.gz.
- Make the progress bar play nicely with --debug.
- When enforcing max-container-depth, show which internal files differ,
without showing details.
- Fix JSON presenter to handle recursion properly.
- Avoid a ZeroDivisionError in the progress bar
- Fix create_limited_print_func.
* Tests:
- Fix failing tests due to logging global state.
- Add tips about running with TMPDIR=/run/shm.
* Cleanups & refactoring:
- Remove unnecessary "dest" args to argparse.
- Refactor DirectoryContainer to be more similar to Container.
- Refactor Container abstract method names.
- Remove unused imports and tidy up Container.comparisons().
- rename get_{filtered => adjusted}_members_sizes for consistency.
- Move tests/comparators/utils up one directory.
- html-dir: show/hide diff comments which can be very large.
- Refactor html-dir presenter to a class instance avoiding global state.
- Move side-by-side and linediff algorithms to difference.py.
- difference: has_children -> has_visible_children, and take into account
comments.
- Move ydiff/linediff from diffoscope.{difference => diff} to group
unified_diff-related things together
[ Maria Glukhova ]
* New features:
- Add visual comparisons for JPEG, ICO, PNG and static GIF images.
(Closes: #851359)
* Test improvements:
- Test that external tools providers are being returned when tool is not
found.
- Add tests for OutputParserError and ContainerExtractionError.
* Cleanups & refactoring:
* Ignore text difference if we have a visual one.
* Fix link formatting and typo in README.
* Rename html_output to compute_visual_diffs and explain its purpose.
* Removed duplicated functions from diff.py.
[ Mattia Rizzolo ]
* Export junit-xml style test report when building on Jenkins.
[ anthraxx ]
* Extend external Arch Linux tools list.
-- Chris Lamb <lamby@debian.org> Sun, 18 Jun 2017 15:34:59 +0800
diffoscope (82) experimental; urgency=medium
[ Chris Lamb ]
* New features:
- Add support for comparing Pcap files. (Closes: #858867)
- Add support for .docx and .odt files via docx2txt & odt2txt.
(Closes: #859056)
- Add support for PGP files via pgpdump. (Closes: #859034)
- Add support for comparing Ogg Vorbis files.
* Bug fixes:
- Don't crash on invalid archives; print a useful error instead.
(Closes: #833697)
- Ensure tests and the runtime environment can locate binaries in
/usr/sbin (eg. tcpdump)
* Tests:
- Ensure that PATH is modified.
- Ensure @tool_required raises RequiredToolNotFound.
- Don't assume that /sbin/init exists; locate any /sbin binary manually and
then test for that. This should prevent FTBFS on travis-ci.org.
- Show packages installed in report output.
* Misc:
- comparators.bzip2: Don't print error output from bzip2 call.
- comparators.pcap: Show the delta, not the absolute time.
- Use /usr/share/dpkg/pkg-info.mk over manual calls to dpkg-parsechangelog
in debian/rules.
- Document PYTHONPATH usage when running tests in README.Source.
- Add internal documentation for @tool_required decorator.
[ beuc@beuc.net ]
* Display differences in zip platform-specific timestamps. (Closes: #859117)
[ Ximin Luo ]
* Add support for R .rds and .rdb object files.
[ Vagrant Cascadian ]
* Add support for .dtb (device tree blob) files (Closes: #861109).
-- Chris Lamb <lamby@debian.org> Wed, 26 Apr 2017 16:17:57 +0200
diffoscope (81) experimental; urgency=medium
[ Chris Lamb ]
* Correct meaningless "1234-content" metadata when introspecting files
within archives. This was a regression since #854723 due to the use of
auto-incrementing on-disk filenames. (Closes: #858223)
* Refactor get_compressed_content_name.
[ Ximin Luo ]
* Improve ISO9660/DOS/MBR check.
-- Chris Lamb <lamby@debian.org> Mon, 27 Mar 2017 16:21:17 +0100
diffoscope (80) experimental; urgency=medium
* Ensure that we really are using ImageMagick and not, for example, the
GraphicsMagick compatibility layer installed by
graphicsmagick-imagemagick-compat. (Closes: #857940)
* Factor out the unicode decoding of the identify -version output.
* travis.yml: Don't build tags.
-- Chris Lamb <lamby@debian.org> Thu, 16 Mar 2017 17:06:45 +0100
diffoscope (79) experimental; urgency=medium
[ Chris Lamb ]
* Extract SquashFS images in one go rather than per-file, speeding up (eg.)
Tails ISO comparison by ~10x.
* Support newer versions of cbfstool to avoid test failures.
(Closes: #856446)
* Skip icc test that varies on endian if the Debian-specific patch is not
present. (Closes: #856447)
* Compare GIF images using gifbuild. (Closes: #857610)
* Also interpret "DOS/MBR boot sector" files as ISO images as they may have
been processed by isohybrid.
* Progress bar:
- Hide bar if we are running with --debug mode.
- Update prior to working on an item so the displayed filename is correct.
[ Maria Glukhova ]
* Improve AndroidManifest.xml comparison for APK files.
- Indicate the AndroidManifest.xml type. (Closes: #850758)
-- Chris Lamb <lamby@debian.org> Thu, 16 Mar 2017 09:38:16 +0100
diffoscope (78) unstable; urgency=medium
[ Brett Smith ]
* comparators.json:
+ Catch bad JSON errors on Python pre-3.5. Closes: #855233
[ Chris Lamb ]
* tests:
+ Move normalize_zeros to more generic `utils.data` module.
+ Fix tests that call xxd fail on jessie due to output change.
Closes: #855239
[ Ed Maste ]
* comparators.directory:
+ Assume BSD-style stat(1) on FreeBSD. Closes: #855169
-- Mattia Rizzolo <mattia@debian.org> Sun, 26 Feb 2017 17:26:48 +0100
diffoscope (77) unstable; urgency=medium
[ Chris Lamb ]
* tests/comparators/utils:
+ Correct logic of module_exists, ensuring we correctly skip in case of
modules containing a dot in their name. Closes: #854745
* comparators/utils/libarchive:
+ No need to track archive directory locations.
* Add --exclude option. Closes: #854783
* Add PyPI badge to README.rst.
* Update .travis.yml from http://travis.debian.net.
[ Mattia Rizzolo ]
* Add CVE reference to the changelog of v76.
* Add my key to debian/upstream/signing-key.asc.
[ Ximin Luo ]
* comparators/utils/libarchive:
+ When extracting archives, try to keep directory sizes small.
-- Mattia Rizzolo <mattia@debian.org> Mon, 13 Feb 2017 16:25:02 +0100
diffoscope (76) unstable; urgency=medium
[ Chris Lamb ]
* Extract archive members using an auto-incrementing integer, avoiding the
need to sanitise filenames and avoiding writes to arbitrary locations.
(Closes: #854723 - CVE-2017-0359)
[ Ximin Luo ]
* Simplify call to subprocess.Popen
-- Chris Lamb <lamby@debian.org> Fri, 10 Feb 2017 11:37:52 +1300
diffoscope (75) unstable; urgency=medium
[ Chris Lamb ]
* Fix ImportError in Debian comparator tests. This was caused by not
handling the case where ``importlib.find_spec`` was testing for a submodule
(ie. ``debian.Deb822``) where it will attempt to import the ``debian``
module and raise an exception if it does not exist. Thanks to Iain Lane for
initial patches. (Closes: #854670)
[ Ximin Luo ]
* Remove pointless use of a thread
-- Chris Lamb <lamby@debian.org> Fri, 10 Feb 2017 09:28:47 +1300
diffoscope (74) unstable; urgency=medium
* Add missing Recommends for comparators. This was a regression introduced in
version 71 due to lazily-importing them; they were then not available when
we called "--list-tools=debian" during package build. (Closes: #854655)
-- Chris Lamb <lamby@debian.org> Thu, 09 Feb 2017 16:58:28 +1300
diffoscope (73) unstable; urgency=medium
* debian/tests/pytest: Remove spurious clipboard contents.
-- Chris Lamb <lamby@debian.org> Thu, 09 Feb 2017 13:24:59 +1300
diffoscope (72) unstable; urgency=medium
* Fix autopkgtest failures when Recommends are not installed.
(Closes: #854593)
* Specify <html lang="en"> in HTML output. (re. #849411)
* Tests:
- Add a "@skip_unless_module_exists" decorator.
- Show local variables in tracebacks.
-- Chris Lamb <lamby@debian.org> Thu, 09 Feb 2017 12:49:40 +1300
diffoscope (71) unstable; urgency=medium
[ Chris Lamb ]
* New features:
- Add a machine-readable JSON output format. (Closes: #850791)
- Show results from debugging packages last. (Closes: #820427)
- Add a --max-text-report-size option. (Closes: #851147)
* Bug fixes:
- Clean all temp files in signal handler thread instead of attempting to
bubble exception back to the main thread. (Closes: #852013)
- Prevent FTBFS by loading fixtures as UTF-8 in case surrounding terminal
is not Unicode-aware. (Closes: #852926)
- Fix errors when comparing directories with non-directories.
(Closes: #835641)
- Fix behaviour of setting report maximums to zero (ie. no limits)
* Tests:
- Test the RPM "fallback" comparison.
- Test the Deb{Changes,Buildinfo,Dsc} fallback comparisons.
- Test --progress and --status-fd output.
- Add tests for symlinks differing in destination.
- When comparing two empty directories, ensure that the mtime of the
directory is consistent to avoid non-deterministic failures.
- Smoke test profiling output.
- Ensure we ignore invalid JSON files correctly.
- Ensure 2nd source of a Difference is a string, not just the 1st.
- Don't report on test coverage for some internal error messages.
* Misc:
- Add docs about releasing signed tarballs.
- Drop the incomplete list of external tools from README.rst.
- Add debian/watch file with cryptographic signature verification.
- Drop CpioContent command now that we use libarchive.
- Use a singleton to manage our Comparator classes.
- Many small optimisations and code cleanups.
[ Brett Smith ]
* diffoscope.diff: Improve FIFO writing robustness.
[ Ximin Luo ]
* Fix bug introduced in commit 36d1c964 that only worked "accidentally".
* Fix lazy expression; filter is lazy in Python 3.
[ Mattia Rizzolo ]
* Override the debian-watch-file-in-native-package lintian tag.
[ anthraxx ]
* Arch package changed from cdrkit to cdrtools.
[ Holger Levsen ]
* Restore history section in README, explaining this was started in Debian.
-- Chris Lamb <lamby@debian.org> Wed, 08 Feb 2017 13:02:01 +1300
diffoscope (70) unstable; urgency=medium
[ Mattia Rizzolo ]
* comparators
+ haskell: add a comment describing the file header.
Thanks to James Clarke <jrtc27@debian.org> for all the investigation done.
* tests:
+ Skip two more tests requiring a x86-64-capable binutils.
This fixes the tests on ppc64el.
* CONTRIBUTING: misc updates, clearer info about how to submit a Debian bug.
[ James Clarke ]
* comparators:
+ haskell: Properly extract version from interface files.
What the code did before was just totally wrong, and worked only by
chance (and only on little endian systems).
This also fixes the test suite when run on big endian systems.
[ Chris Lamb ]
* comparators:
+ haskell: Also catch CalledProcessError, not just OSError.
* presenters:
+ Move text presenter to use Visitor pattern.
+ Add markdown output support. Closes: #848141
+ Add RestructuredText output format.
+ Instantiate our presenter classes directly instead of wrapping a method.
+ Use an optimised indentation routine throughout all text presenters.
+ text: Remove superfluous empty newlines from diff.
* tests:
+ Split main and presenter tests.
+ Actually compare the output of text/ReST/markdown formats to fixtures.
+ Drop output_* calls that are inconsistently applied to differences.
+ Add tests for HTML output.
+ Add a test comparing two empty directories.
+ Test --text-color output format.
+ Test that no arguments (beyond the filenames) prints the text output.
+ Don't warn about coverage lines that raise NotImplementedError.
+ Increase coverage by adding "# noqa" in relevant parts.
* Add build status to README.rst.
[ Brett Smith ]
* diffoscope:
+ Specify choices for --list-tools switch.
+ Improve --help output. Closes: #852015
* CONTRIBUTING: Refresh instructions for contributing to diffoscope.
[ anthraxx ]
* tools: switch Arch Linux dependency for pedump to mono.
-- Mattia Rizzolo <mattia@debian.org> Thu, 26 Jan 2017 16:39:10 +0100
diffoscope (69) unstable; urgency=medium
[ Chris Lamb ]
* Skip tests if binutils can't handle the object file format. Based on a
patch by Mattia Rizzolo. (Closes: #851588)
* Move external tool definitions out of misleading "exceptions" module.
* Save some complicated logic by setting default RE_FILE_{EXTENSION,TYPE}
* Test --html-dir option.
* Misc:
* Add missing `data` imports.
- Inherit GzipFile from File, not object.
- Remove unused imports in comparator tests.
- Consistently space out environment exports in debian/rules.
[ Mattia Rizzolo ]
* If both RE_FILE_TYPE and RE_FILE_EXTENSION are defined, AND them
* Use the path attribute of the specialized file instead of the original name
in tests.
* tests/main:
* Shorten argument lists by unpacking common arguments.
* Disable jQuery while testing --htmldir so tests can run without it.
* tests/comparators:
* Refactor into packages with smaller modules.
* Rename load_fixture() to init_fixture().
* Add a load_fixture() function wrapping both init_fixture() and data().
-- Chris Lamb <lamby@debian.org> Tue, 17 Jan 2017 11:33:54 +1100
diffoscope (68) unstable; urgency=medium
[ Chris Lamb ]
* Don't blow up if directory containing ELF debug symbols already exists.
(Closes: #850807)
* Fix .APK extration when provided with absolute filenames.
(Closes: #850485)
* Support comparing .ico files using img2txt. (Closes: #850730)
* comparators.utils.file: If we don't have an archive-extraction tool (eg.
apktool), don't blow up when attempting to unpack it.
* Include magic file type when we know the file format but can't find
file-specific details. (Closes: #850850)
* Ensure fake "APK metadata" file appears first, fixing non-deterministic
tests/output.
* Correctly escape value of href="" elements (re. #849411)
* Optimisations:
- Disable profiling entirely (unless enabled) for a 2%+ optimisation
- Compile APK filename regex instead of generating it each loop.
* Logging:
- Log tempfile cleanup process
- Log when we add a progress observer.
- Drop milliseconds from log output
* Misc:
- Many unused import removals, indentation changes, etc.
- Fix duplicated word and long line errors in debian/changelog.
- Suggest some promotion in post-release documentation.
[ Maria Glukhova ]
* comparators/device: don't crash when comparing a non-device against a
device (Closes: #850055)
* Remove archive name from apktool.yml and rename it. (Closes: #850501)
* Zipinfo included in APK files comparison. (Closes: #850502)
- Add some tests for APK comparator.
* Add image metadata comparison. (Closes: #849395)
* Ensure imagemagick version is new enough for image metadata tests.
[ Mattia Rizzolo ]
* Skip the openssh_pub_key test if the version of ssh is < 6.9.
* comparators/icc: rename RE_FILE_EXTENSION to RE_FILE_TYPE, as that's what
the regular expression is looking for.
* Make use of a new mechanism to remove a bunch of recognizes() methods
dealing with simple RE_FILE_TYPE matching.
-- Chris Lamb <lamby@debian.org> Mon, 16 Jan 2017 11:24:22 +1100
diffoscope (67) unstable; urgency=medium
[ Chris Lamb ]
* Optimisations:
- Avoid multiple iterations over archive by unpacking once for an ~8X
runtime optimisation.
- Avoid unnecessary splitting and interpolating for a ~20X optimisation
when writing --text output.
- Avoid expensive diff regex parsing until we need it, speeding up diff
parsing by 2X.
- Alias expensive Config() in diff parsing lookup for a 10% optimisation.
* Progress bar:
- Show filenames, ELF sections, etc. in progress bar.
- Emit JSON on the status file descriptor output instead of a custom
format.
* Logging:
- Use more-Pythonic logging functions and output based on __name__, etc.
- Use Debian-style "I:", "D:" log level format modifier.
- Only print milliseconds in output, not microseconds.
- Print version in debug output so that saved debug outputs can standalone
as bug reports.
* Profiling:
- Also report the total number of method calls, not just the total time.
- Report on the total wall clock taken to execute diffoscope, including
cleanup.
* Tidying:
- Rename "NonExisting" -> "Missing".
- Entirely rework diffoscope.comparators module, splitting as many separate
concerns into a different utility package, tidying imports, etc.
- Split diffoscope.difference into diffoscope.diff, etc.
- Update file references in debian/copyright post module reorganisation.
- Many other cleanups, etc.
* Misc:
- Clarify comment regarding why we call python3(1) directly. Thanks to
Jérémy Bobbio <lunar@debian.org>.
- Raise a clearer error if trying to use --html-dir on a file.
- Fix --output-empty when files are identical and no outputs specified.
- Wrap changelog line to avoid Lintian warning.
[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)
[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to
"jsbeautifier".
[ siamezzze ]
* Fixed no newline being classified as order-like difference.
-- Chris Lamb <lamby@debian.org> Wed, 04 Jan 2017 16:56:36 +0000
diffoscope (66) unstable; urgency=medium
[ Chris Lamb ]
* Update dex_expected_diffs and test requirement to ensure test compatibility
with enjarify >= 1.0.3. (Closes: #849142)
* Print the detected version in @skip_unless_tool_is_at_least test utility.
[ Maria Glukhova ]
* Add detection of order-only difference in plain text format.
(Closes: #848049)
[ anthraxx ]
* Add OpenSSH Arch package to Recommends.
-- Chris Lamb <lamby@debian.org> Sat, 24 Dec 2016 19:18:30 +0000
diffoscope (65) unstable; urgency=medium
[ Chris Lamb ]
* Avoid unpacking packages twice when comparing .changes. (Closes: #843531)
* Add a simple profiling framework (enabled via --profile) which tracks:
- Container extraction, opening and closing times.
- The "compare_files" top-level/recursive method.
- The time taken to produce the various output formats.
- All "has_same_content_as" and "recognizes" methods.
- External commands that wrap "@tool_required".
- External commands that use "make_feeder_from_command".
- The internal/external "cmp" methods.
- Some manual calls to subprocess.check_output().
* Tidy log messages:
* Clarify the meaning of the compare_files and Binary.has_same_content
debug messages.
* Skip low-value "X is already specialized" message.
* Correct "instantiating" typo.
[ Emanuel Bronshtein ]
* Use ssh-keygen for comparing OpenSSH public keys
* Remove inline styles from col elements
-- Chris Lamb <lamby@debian.org> Fri, 23 Dec 2016 11:58:15 +0000
diffoscope (64) unstable; urgency=medium
[ Chris Lamb ]
* Avoid shelling out to colordiff by implementing highlighting in Python
directly.
* Split out trydiffoscope client and binary package from main diffoscope
repository so that it can be more-easily released on PyPI.
* Memoize calls to ``distutils.spawn.find_executable`` to avoid excessive
stat(1) syscalls.
* Remove temporary profiling file introduced in 97bddd18a.
* comparators.directory:
- Correct `source` attribute of lsattr-based Difference instances.
- Add comment regarding why difficult to optimise calls to lsattr
* Internally guarantee to all progress observers that we will be processing
something.
* bin/diffoscope:
- Ensure that running from Git always uses that checkout's modules.
- Use os.path.join "properly" as we are already using it.
- Use typical sys.path.insert(0, ...) method to modify sys.path.
- Use immutable tuple over mutable list for clarity.
* comparators.debian: No need to keep .buildinfo file descriptor open.
* Suggest command-line for signing of PyPI uploads.
* Improve documentation of new tests/conftest.py file.
[ Maria Glukhova ]
* Remove test data causing problems during build with Python 3.4.
* Change icc-recognizing regexp to reflect changes in file type description.
(Closes: #848814)
[ Brett Smith ]
* set_locale should call tzset. (Closes: #848249)
* Ensure set_locale fixture runs before all tests.
[ Emanuel Bronshtein ]
* Use js-beautify as JavaScript code beautifier for .js files (with tests).
* CSS & HTML changes:
- Disable referrer and window.opener leakage.
- Disable compatibility mode usage in IE.
- Use double quotes for HTML attributes.
- Fix unclosed element span.
- CSS optimizations.
- Add fallback color to rgba usage.
- Fix CSS markup warnings.
* Change all HTTP URLs to HTTPS where applicable.
[ Baptiste Daroussin ]
* comparators/directory: add compatibily with FreeBSD's getfacl.
* Improve portability by not relying on /dev/fd.
[ Reiner Herrmann ]
* Improved support for Android apk files.
* Fix wording in Comparators.get_members comment.
[ Mattia Rizzolo ]
* presenters/html: Namespace the diffoscope CSS class.
-- Chris Lamb <lamby@debian.org> Thu, 22 Dec 2016 10:37:36 +0000
diffoscope (63) unstable; urgency=medium
* Add myself to Uploaders.
* Greatly improve speed for large archives by fixing O(n^2) complexity for
archive member lookup.
- There is still O(n^2) complexity for archive member extraction, but this
is less noticeable for various reasons and would require more complexity
to fix, so for now is left as a task for the future.
* Text output: add coloured diff support via colordiff(1).
* Html-dir output: add +/- buttons to fold sub-diffs (i.e. toggle their
visibility) as well as the whole diff itself. As with similar features in
other programs, the effect affects all descendants if you shift-click.
-- Ximin Luo <infinity0@debian.org> Fri, 02 Dec 2016 12:10:04 +0100
diffoscope (62) unstable; urgency=medium
[ Chris Lamb ]
* progress: Pass the fileobj to the print(..) function, not str.format.
Thanks to Marius Gedminas <marius@gedmin.as>.
* Move diffoscope.Config to a more-standard and simpler singleton pattern
and validate constraints on every __setattr__.
[ Maria Glukhova ]
* tests/{dex,java}: Skip the java tests if javap is older than 1.8.
This fixes test_java and test_dex failing in jessie-backports. See #838984
[ Michel Messerschmidt ]
* comparators/zip: Add rudimentary support for OpenDocumentFormat files.
[ Mattia Rizzolo ]
* debian: bump debhelper compat level to 10, no changes needed.
* comparators/ppu:
+ don't do run a full ppudump while only looking for PPU file version.
+ ignore decoding errors from ppudump while filtering the output.
* tests/ppu: skip some PPU tests if ppudump is < 3.0.0. Closes: #838984
* tests: skip test using a format accepted by older pytest. Closes: #841146
[ Daniel Shahaf ]
* comparators/json: detect order-only differences and print them nicely.
Closes: #839538
-- Mattia Rizzolo <mattia@debian.org> Tue, 01 Nov 2016 15:33:45 +0000
diffoscope (61) unstable; urgency=medium
[ Mattia Rizzolo ]
* Skip rlib tests if the "nm" tool is missing. (Closes: #837742)
* Use dedicated xxd binary package instead of vim-common.
* autopkgtest: Allow stderr for the basic-command-line test.
* d/rules: make it possible to backport without changing the "upstream"
version.
[ Maria Glukhova ]
* Catch an error when we can't read files in has_same_content_as().
(Closes: #835642)
[ Chris Lamb ]
* Add the ability track the (approximate) progress throughout diffoscope run:
- Add a graphical progressbar and show by default if STDOUT is a TTY.
- Add a --status-fd argument to output machine-readable status to the
specified file descriptor.
[ Ximin Luo ]
* Add a --output-empty option, useful for batch scripts.
* Improve --help text and group the optional arguments based on topic.
* Add a script to check sizes of dependencies.
* tests/basic-command-line: check exit code and use a more complex example.
* Provide better advice about what environment variable to set to make the
console work.
-- Chris Lamb <lamby@debian.org> Wed, 28 Sep 2016 11:44:25 +0100
diffoscope (60) unstable; urgency=medium
[ Chris Lamb ]
* debian/rules:
+ Run tests with python3-coverage and generate reports post-build.
* CONTRIBUTING: Add PyPI reference.
* Comparators:
+ Avoid ugly DRY violations in diffoscope.comparators.__init__ by
dynamically importing classes via a single list.
+ git: Specify paths as strings as required since b7830b5cb.
+ elf:
- Mask more warnings (eg. "readelf: Error: the dynamic segment offset" or
"size exceeds the size of the file").
- Specify string format arguments as logging function parameters, not
using interpolation.
+ json: Correct behaviour of JSON comparator when file does not exist.
* Presenters:
+ html:
- Use html.escape over xml.sax.saxutils.escape.
- Don't use unsafe cgi.escape method as its ``quote`` kwarg — which
enables escaping of quotation marks — is False by default.
* Code clean-up:
+ Tidy imports.
+ Split exceptions out into dedicated diffoscope.exc module.
+ Use try-except idiom instead of ``if`` in RequiredToolNotFound.get_package
+ Prefer kwargs[x] = kwargs.pop(x, 'default') idiom over ``if`` statements.
* tests:
+ Factor out ugly os.path.join to a utility method.
+ Drop unused try_except in tests/conftest.
+ Add GitIndex tests to bring coverage of comparators/git.py to 100%.
+ Add tests for Haskell comparator.
- Skip Haskell tests if GHC is not installed.
- Skip Haskell tests if GHC version does not match. Closes: #835055
+ Add test in JSON comparator for non-existing files.
+ Rewrite lengthy "[..] skipIf [..] tool_missing" with a
@skip_unless_tools_exist decorator.
+ Add tests for skip_unless_tools_exist helper.
+ Rename tests/comparators/conftest.py -> tests/comparators/utils.py.
+ Avoid Don't-Repeat-Yourself violations by factoring out fixture loading.
+ Avoid more DRY violations by factoring-out non-existing file tests.
+ Remove some other unused imports from comparator tests.
+ Check rendering of text and HTML presenters when checking non-existing
files.
+ Use `pytest.xfail` over `assert False`.
[ Mattia Rizzolo ]
* debian/control:
+ Reinstated build profiles, now that dh-python is fixed (see #834238).
+ Version the build-dependency on dh-python, so it supports build profiles.
+ Add more nocheck build profiles.
+ Bump dependencies on debhelper and dpkg-dev to assure build profile
support is available.
+ Add Build-Dependency on fontforge-extras for showttf (run during tests)
+ Alternate Build-Dependency on default-jdk-headless|default-jdk, to ease
backporting to older Debian releases.
* debian/rules:
+ Don't install '/usr/lib/python3/dist-packages/.coverage'.
+ Run the basic-command-line test during package build.
+ Be even more verbose about failing tests.
+ Allow building a mismatching version between diffoscope and the debian
changelog if the target distribution is UNRELEASED.
* Comparators
+ ppu:
- Unbreak ppu_version detection.
- Put code out of the context manager, it doesn't need to be within.
* trydiffoscope: add copyright/license header.
* autopkgtest/basic-command-line:
+ Enable running basic-command-line test without ADTTMP being set.
+ Be a bit more verbose about what is being tested.
+ Also test return code when passing an invalid command line flag.
+ Force LC_ALL=C.UTF-8, so that diffoscope can always output something.
* test:
+ Factor out a tools_missing() function.
+ Rewrite tool_older_than() into skip_unless_tool_is_older_than().
+ Skip test_squashfs.py.test_superblock if squashfs is too old.
+ Normalize xxd's output so that we can compare jessie's xxd with stretch's.
* add default-jdk to the alternate packages for javap for Debian;
default-jdk-headless is not available in older Debian releases.
[ Satyam Zode ]
* Ignore differences in .buildinfo if all the other files listed in .changes
are identical. In those cases any differences in .buildinfo files are
irrelevant, and are actually a feature (as it indicates two reproducible
builds with different environment).
[ Jérémy Bobbio ]
* Properly skip tests requiring python-debian when unavailable.
[ Ximin Luo ]
* Show the timestamp when logging, to detect which steps take longer.
* Add a --no-max flag to disable all limits.
* Have --max-report-size honour 0 to mean "no limit".
* Comparators:
+ cbfs: only scan whole file when the filename ends with ".rom".
+ binary: Suppress decimal line numbers for xxd output.
* Increase default --max-diff-input-lines to 2^20 (1048576) from 100000.
* Rename --no-max-limits to --no-default-limits, a more accurate name.
* presenters/html: html-dir output:
+ Split large diffs across several child pages.
Also add some flags so html-dir behaviour is easier to reason about.
+ Use 4 times the given --max-diff-block-lines.
This way, '--html x.html --html-dir y' gives reasonable output for both.
* difference:
+ Fix behaviour for max-diff-block-limit=1 and don't report when 0 lines
are removed.
+ Don't truncate diff output by default, instead add a separate flag
--max-diff-block-lines-saved (infinite by default) to do so.
-- Mattia Rizzolo <mattia@debian.org> Sun, 11 Sep 2016 16:14:33 +0000
diffoscope (59) unstable; urgency=medium
[ Chris Lamb ]
* SquashFS test_testing:
- Don't assume that user and group are 5 characters long in squashfs
test_listing.
- Drop test; simply too unreliable and/or has too many requirements to
satisfy (eg. shorter usernames) for it to not be skipped.
* Difference class:
- Ensure that we pass str instances to our Difference class as paths,
otherwise we can't be sure we can render them. For example, passing a
class here breaks the HTML renderer.
- Add test for type checking of path1/path2/source.
- Pass a str for the path in the JSON comparator, fixing the HTML reporting
of JSON output.
* When looping over JQUERY_SYSTEM_LOCATIONS, actually use the variable we are
looping with when performing the symlink(2) call.
* trydiffoscope:
- Add manpage for trydiffoscope.
- Write trydiffoscope console output as UTF-8, even if STDOUT claims it
doesn't support it.
* debian/rules:
- Make separate calls to dh_python3 for diffoscope and trydiffoscope to
avoid both getting the same Recommends substvar.
- Use a pattern-based rule for rst2man-based manpages instead of
hard-coding "diffoscope.1".
- Dynamically calculate manpages to generate over hardcoding.
* .gitignore: Ignore all generated manpages, not just diffoscope.1.
[ Holger Levsen ]
* ./CONTRIBUTING:
- Add new file.
- Explain what to look out for when uploading diffoscope.
-- Chris Lamb <lamby@debian.org> Mon, 15 Aug 2016 09:54:48 +0100
diffoscope (58) unstable; urgency=medium
* Shorten Description of trydiffoscope to appease Lintian.
* Make squashfs test_listing test more robust:
- Add ".in" suffix to squashfs listing expected diff as we are about to
template it properly.
- Replace $USER with the current user with uid 1000.
- Move test_listing replacements to a loop so we can easily add more.
- Don't assume that the test environment as 4 CPUs in squashfs.test_listing
- Don't assume default group is the same name as the user in test_listing.
-- Chris Lamb <lamby@debian.org> Sun, 14 Aug 2016 10:22:19 +0100
diffoscope (57) unstable; urgency=medium
[Chris Lamb]
* Add myself to Uploaders.
* Add try.diffoscope.org command-line client in new binary package
"trydiffoscope" to save installation of dependencies:
- Open files in binary mode to support comparing binary files.
- Optionally open URL in web browser.
- Optionally poll locally until result is available.
- Move to version 2 of API, don't continue to poll on error.
- Correct and improve help text.
* Code cleanups:
- Tidy PROVIDERS dict in diffoscope/__init__.py.
- Tidy OS_NAMES dict.
- Correct indentation levels in various files.
- Drop unused imports.
- platform.get_linux_distribution() will break in Python 3.7.
- Don't name first argument of Difference.from_command "cls" as it's not a
classmethod.
- comparators/directory.py: Call logger.warning over logger.warn.
- Rename diffoscope.__main__ to diffoscope.main as its not a "real" magic
name.
- Raise NotImplementedError() instances over NotImplemented classes.
- Add short option for trydiffoscope --url.
- Rewrite trydiffoscope to match diffoscope's command-line API.
- Use keepalive in trydiffoscope client to avoid new HTTP requests.
- Highlight lines via HTML/CSS when hovering over a line to make it easier
to visually track.
- Rename debian/install -> debian/diffoscope.install now we have multiple
binary packages.
* Add a JSON comparator.
- Add tests for JSON comparator.
- Move parsing to JSONFile.recognizes
- Ensure decode errors in JSON comparator are not fatal.
* Ignore /.*eggs and /.cache - they get created when running tests.
* Reflow long dh_python3 call in debian/rules.
* Use dict.get fallback to shorten OS_NAMES lookup in --list-tools.
* --list-tools:
- Output --list-tools in RFC822 format to make parsing less brittle.
- Print "no package mapping" in --list-tools to stderr, not stdout.
- Remove unused ``reduce`` import.
- Reflow long PROVIDERS lookup.
* Add .travis.yml from http://travis.debian.net/
* Don't skip squashfs test_listing test; we can workaround it, as long as we
have a uid 1000.
* Drop "Testsuite: autopkgtest" in debian/control; it is added automatically
by dpkg-source version 1.17.11.
[Mattia Rizzolo]
* autopkgtest: be more verbose when running the tests
* debian/control:
- Mark build-dependency needed only for tests with a <!nocheck> build
profile (reverted as they break dh-python)
- add more build-dependencies to be able to run more tests at build time
- default-jdk-headless is enough to have javap, instead of all of
default-jdk
* debian/rules:
- be more verbose when running the tests, in particular, show why tests are
being skipped
- teach pybuild where to put diffoscope's files
* do not try to execute the command if the command is not available
[Ximin Luo]
* Add support for reading LLVM bitcode files
- Ignore line numbers because different llvm-dis versions output extra
metadata
- Disable the llvm-dis test if the version is lower than 3.8
* Add support for reading Rust LLVM object files
* Add test for rlib files, helping also to test ArFile, LlvmBitCodeFile and
RustObjectFile
* Fix failing directory test
* Add support for reading the symbol table to ArFile
* Fix typo in tool_older_than
- Remove debugging print from tool_older_than
[anthraxx]
* skip rlib test if llvm-dis tool is missing
-- Chris Lamb <lamby@debian.org> Sat, 13 Aug 2016 23:46:42 +0100
diffoscope (56) unstable; urgency=medium
[ Jérémy Bobbio ]
* Fix maybe_decode() so that it actually works.
[ Chris Lamb ]
* Mask EPIPE to avoid ugly tracebacks when eg. prematurely terminating $PAGER.
* Add git index file comparator.
* Drop unused imports in Ar comparator.
[ Ximin Luo ]
* Add support for AR archives (including Rust .rlib files).
* Add some more documentation for ArFile/ArContainer.
* RequiredToolNotFound.PROVIDERS dict values are package names, not binary
names.
-- Reiner Herrmann <reiner@reiner-h.de> Thu, 28 Jul 2016 00:02:56 +0200
diffoscope (55) unstable; urgency=medium
[ anthraxx ]
* Fix sqlite3 magic recognition when using file >= 5.27. (Closes: #830434)
[ Satyam Zode ]
* Add argument completion feature to diffoscope. (Closes: #826711)
[ Chris Lamb ]
* Improve message when data differs. (Closes: #827981)
[ Reiner Herrmann ]
* Clarify the input arguments. (Closes: #826894)
[ Holger Levsen ]
* Add link to homepage in HTML output.
[ Rainer Müller ]
* python-magic >=0.4.12 does not require decode.
-- Reiner Herrmann <reiner@reiner-h.de> Fri, 15 Jul 2016 21:50:29 +0200
diffoscope (54) unstable; urgency=medium
* Fix syntax in RequiredToolNotFound.PROVIDERS. This caused --list-tools to
miss entries, and by it missing Recommends in the Debian package.
-- Mattia Rizzolo <mattia@debian.org> Fri, 03 Jun 2016 16:16:02 +0000
diffoscope (53) unstable; urgency=medium
[ Reiner Herrmann ]
* In the directory comparator, fall back to current directory instead of
empty name. (Closes: #820624)
* Improve deb(5) support. (Closes: #818414)
[ Mattia Rizzolo ]
* debian/control: add myself to Uploaders.
* Clean up temporary debuglink files.
* Fix FileNotFoundError with broken symlinks. (Closes: #818856)
* Decode Md5sumsFile in utf-8. (Closes: #823874)
* Always suffix temporary files with '_diffoscope'.
* Rewrite sanity check for the version to always run every time debian/rules
is invoked, not only during a binary build.
[ Ed Maste ]
* Add FreeBSD packages for required tools.
[ Ceridwen ]
* Add README.rst to MANIFEST.in.
-- Mattia Rizzolo <mattia@debian.org> Fri, 03 Jun 2016 14:23:16 +0000
diffoscope (52) unstable; urgency=medium
[ Mattia Rizzolo ]
* debian/gbp.conf: add a conf to have gbp tag the releases how we like them.
[ Holger Levsen ]
* Drop transitional debbindiff package.
* debian/control: Drop XS-Prefix from Testsuite: field, thanks lintian.
* Mention --max-report-size only once in the manpage. (Closes: #819956)
[ Satyam Zode ]
* Fix typo in documentation.
[ Reiner Herrmann ]
* Bump Standards-Version to 3.9.8, no changes required.
* Let objdump demangle symbols for better readability.
* Install bin/diffoscope instead of auto-generated script. (Closes: #821777)
-- Reiner Herrmann <reiner@reiner-h.de> Wed, 20 Apr 2016 20:28:03 +0200
diffoscope (51) unstable; urgency=medium
* Team upload.
* Re-upload, the last uploaded tarball was broken, missing test files.
-- Mattia Rizzolo <mattia@debian.org> Sun, 28 Feb 2016 11:56:35 +0000
diffoscope (50) unstable; urgency=medium
[ Jérémy Bobbio ]
* Remove myself from uploaders
[ Reiner Herrmann ]
* Show line ordering variation in deb md5sums
* Reset mtimes of test directories and ignore ctimes (Closes: #815171)
* Add comparator for Postscript files
-- Holger Levsen <holger@debian.org> Sat, 27 Feb 2016 19:37:20 +0100
diffoscope (49) unstable; urgency=medium
[ Jérémy Bobbio ]
* Don't crash when we can't find the debug file in a matching debug package.
This repairs diffoscope for xserver-xorg and other packages.
* Use libarchive to read metadata from ar archives. The output is more
precise and less dependent on binutils version. The command line `ar` tool
is not used any more so remove it from the required tools.
* Split readelf --debug-dump output when examining ELF files. Based on a
patch by Dhole.
* Keep both .debug_str and .zdebug_str as ElfSection. Thanks to Niels Thykier
for noticing the problem.
* Fix a logic error in _install_debug_symbols. Thanks anthraxx for the report
and the fix.
* Use recursive containers for directory and only look at files with ending
in .deb when looking for Build IDs. Both avoid looking at too many files
when searching for matching debug packages. Thanks Helmut Grohne and Steven
Chamberlain for the reports and tests. (Closes: #813052)
* Add support for ICC profiles. This adds a Recommends on colord in Debian.
* Harmonize spaces and commas in package list.
* Update PPU files test data for FPC 3.0.0.
* Update exepcted javap output for OpenJDK 1.8.
* Stop specializing files from directories earlier than necessary.
[ anthraxx ]
* Adding cd-iccdump package reference for Arch Linux.
[ Mattia Rizzolo ]
* Use HTTPS in Vcs-Git.
[ Holger Levsen ]
* Bump standards version to 3.9.7, no changes needed.
* Use /git/ instead /cgit/ in Vcs-Browser.
-- Jérémy Bobbio <lunar@debian.org> Wed, 17 Feb 2016 18:12:45 +0100
diffoscope (48) unstable; urgency=medium
* Open debian/control file in binary mode (Closes: #812524)
* Fix recognizes() for files ending in .buildinfo but without a
Checksums-Sha256 field. (Closes: #812534)
* Fix finding debug packages with multiple Build-Ids.
* Cleanup how arguments are given to readelf for sections.
* Only pass --decompress to readelf when the option is supported. This
restores compatibility with older versions of binutils.
* Skip dbgsym tests if the debian Python module is not available.
-- Jérémy Bobbio <lunar@debian.org> Tue, 26 Jan 2016 12:13:57 +0100
diffoscope (47) unstable; urgency=low
* Don't show a difference for identical but long diff inputs (e.g. objdump
output). This should make diffoscope run longer for large diff inputs, but
more correct reports worths it.
* Fix symbol ignore regexp for ELF files.
* Ask readelf to decompress sections before dumping their content.
* Don't redefine .name() in Archive when its already defined in File.
* All files can now belong to a container. This will help us implement
cross-file analysis.
* Update requirements for ELF tests.
* Sort file order when comparing directories.
* Use Python OrderedDict instead of keeping a list of sections in ELF
container.
* Install detached symbols from debug .deb before comparing ELF files.
This means objdump output should have line numbers for Debian packages
built with recent debhelper as long as the associated debug package is in
the same directory.
* Add support for Debian .buildinfo files.
-- Jérémy Bobbio <lunar@debian.org> Sat, 23 Jan 2016 18:08:33 +0000
diffoscope (46) unstable; urgency=low
[ Dhole ]
* Improve ELF comparisons by performing comparisons section by section.
(Closes: #808197)
[ Jérémy Bobbio ]
* Further split readelf output when examining ELF files.
* Skip dumping ELF sections that are already covered by previous readelf
calls to avoid redundant output. Thanks Mike Hommey for the report.
(Closes: #808267)
* Fix ELF comparisons against non-existing files.
* Filter symbols for IP-relative ops from ELF disassembled output as
they will create differences that don't really exist. Thanks Mike Hommey
for the report and test cases. (Closes: #808207)
* Add forgotten requirements for tests with non-existing files.
* Improve tests for comparisons against non-existing files.
* Respect order of containers when performing comparisons. It makes the
output somewhat nicer, especially for things that are expected to be in a
certain order, like ELF sections.
* Fix comparisons of directory with broken symlinks. Thanks Tuomas Tynkkynen
for reporting the issue. (Closes: #810825)
-- Jérémy Bobbio <lunar@debian.org> Tue, 19 Jan 2016 16:46:47 +0100
diffoscope (45) unstable; urgency=medium
[ Rainer Müller ]
* Fix typo in Mach-O comparator.
[ Jérémy Bobbio ]
* Ensure that we always get path names from libarchive as str.
(Closes: #808541)
* Read ext4 reference diff using UTF-8 to ensure that the test suite
works when run under various locales.
* Replace file path by a fixed placeholder in readelf and objdump output.
This will work better when comparing ELF files given on the command line.
* Read lines using an iterator instead of loading a full list in memory.
This greatly help diffoscope when running on large binaries. Thanks Mike
Hommey for the report and good test case. (Closes: #808120)
* Properly report lines we can't parse in squashfs listing.
* Correctly parse squashfs device entries with device minors larger than 3
digits. Thanks sajolida for providing the samples.
* Use libarchive to list files in tar and cpio archives. The output of cpio
and tar is too hard to control properly and tend to vary over time.
(Closes: #808809)
* Ensure files extracted from a squashfs image are deleted after being
compared instead being done examining the image.
-- Jérémy Bobbio <lunar@debian.org> Mon, 04 Jan 2016 16:24:50 +0100
diffoscope (44) unstable; urgency=medium
[ Mike Hommey ]
* Tell readelf to use wide output. (Closes: #808103)
* Add support for Mozilla optimized Zip files. (Closes: #808002)
* Use <ins/> and <del/> in HTML output.
* Fix line numbering in HTML diff output (Closes: #808199)
[ Jérémy Bobbio ]
* Fix .deb comparisons when md5sums is wrong or missing. (Closes: #808104)
* Remove some quadratic string concatenations in HTML presenter.
* Remove useless <span>s in HTML output. (Closes: #808121)
* Replace calling find by using os.walk in directory comparator.
* Sort the file list in directory comparator. (Closes: #808003)
* Ensure the 'lines skipped' message gets written when lines are skipped
at the end of a diff.
* Make sure to read HTML reports as utf-8 in tests.
[ Esa Peuha ]
* Convert HTML character entity references to UTF-8 characters to save space.
-- Jérémy Bobbio <lunar@debian.org> Fri, 18 Dec 2015 15:42:20 +0000
diffoscope (43) unstable; urgency=low
[ Reiner Herrmann ]
* Add comparator for Android dex files.
* Add comparator for filesystem images using libguestfs.
[ Chris Lamb ]
* Add comparator for icons and JPEG images using libcaca.
[ Clemens Lang ]
* Add comparator for OS X binary files (also known as Mach-O). Requires the
otool(1) and lipo(1) which are mostly likely found on OS X.
[ Baptiste Daroussin ]
* Use libarchive for tar instead of Python included library. We already
have libarchive as a mandatory dependency, and libarchive is faster
and more reliable.
[ Mike McQuaid ]
* Support older versions of GNU diff. OS X 10.11 ships with an older GPLv2
version (`2.8.1`) which does not support the `diff -u7` flag but instead
wants to use `diff -U7` which appears to be forward compatible.
[ Ed Maste ]
* Use more portable LC_CTYPE suggestion. The C.UTF-8 locale is a Debian
thing.
[ Levente Polyak ]
* Force ppudump to output time in UTC using the TZ variable. Support for the
TZ environment variable has been implemented in fpc r32369.
* Make PPU comparator version aware by checking the magic version expected by
ppudump itself.
[ Joachim Breitner ]
* Add support for multi-file HTML output with on-demand loading of long
diffs. This is available through the --html-dir option. Location of the
jQuery JavaScript library can be specified through the --jquery option.
(Closes: #806891)
[ Jérémy Bobbio ]
* Return the target path as string when extracting from a Zip. Regression
introduced in diffoscope 41. (Closes: #805774)
* Use pass_fds from Popen instead of reimplementing it. Thanks Baptiste
Daroussin for the tip.
* Fix typo in debug message. Thanks Clemens Lang!
* Benefit from dictionary keys() behaving as set when comparing md5sums.
* Prevent overwriting the destination when extracting with libarchive.
Thanks Baptiste Daroussin noticing and initial implementation.
* Fix PPU detection and associated tests.
* Use lazy extraction instead of explicit bracketing. We now rely on
Python garbage collector to delete file once we are done comparing.
This will make parallel processing easier to implement.
* Make container class available on demand instead of bracketing.
This cleanups boilerplate code, and should make parallel processing
easier to implement.
* Always compare container content if applicable. This removes more
boilerplate code.
* Call xxd like other commands.
* Analyze md5sums file in .deb before working on data.tar. This makes
the code more robust and is required to properly skip files listed as
identical in md5sums while doing parallel processing.
* Add support for package names for OS other than Debian. The --list-tools
command-line option now supports an optional argument to display a given
operating system. When --list-tools is called without
an optional argument and the current operating system is unknown,
the list of packages for all known operating systems will be displayed.
Initial implementation by Mattia Rizzolo, then greatly enhanced by
Levente Polyak.
* Update list of contributors.
* Add contact info to README.
* Add python3-guestfs to the list of Recommends.
* Display a better error message when guestfs can't be started.
* Set libguestfs memory allocation to 128 MiB in autopkgtest as more is
likely to fail with adt-virt-qemu default settings.
* Prevent CBFS comparator from scanning big files needlessly. Thanks Mike
Hommey for the detailed bug report and suggestions. (Closes: #807997)
-- Jérémy Bobbio <lunar@debian.org> Tue, 15 Dec 2015 15:54:55 +0000
diffoscope (42) unstable; urgency=medium
* Add missing Depends on python3-pkg-resources.
* Add another autopkgtest to test command line without recommends.
* Make comparison of zip archives with utf-8 file names more robust.
(Closes: #805418)
* Stop crashing on misencoded readelf output. (Closes: #804061)
* Ignore encoding errors in diff output.
* Add binutils-multiarch to Build-Depends so the test suite can pass
on architectures other than amd64.
-- Jérémy Bobbio <lunar@debian.org> Fri, 20 Nov 2015 18:57:54 +0000
diffoscope (41) unstable; urgency=low
* Minor rewordings in README.
* Fix link in README. Thanks anthraxx.
* Also skip tests when very common tools are unavailable.
* Add git-buildpackage configuration in the hope it will help to get proper
source tarball.
* Allow commands to be called with specified environment variables.
* Force ppudump to output time in UTC by setting TZDIR in the environment.
-- Jérémy Bobbio <lunar@debian.org> Sat, 14 Nov 2015 14:23:09 +0100
diffoscope (40) unstable; urgency=low
* Add missing files in PyPI package.
* Use libarchive instead of debian.arfile to extract .deb.
* Make python-debian optional. This will help packaging diffoscope in other
distributions.
* Add 'Programming Language' to PyPI classifiers.
-- Jérémy Bobbio <lunar@debian.org> Fri, 13 Nov 2015 12:03:16 +0100
diffoscope (39) unstable; urgency=low
[ Reiner Herrmann ]
* Fix EPUB detection.
[ Paul Gevers ]
* Add diffoscope support for PPU files.
[ Jérémy Bobbio ]
* Split lines when they are recorded comments instead of waiting for the
presenters to do it. This helps when there are multiple lines on stderr
output.
* Switch homepage URL to HTTPS. Thanks to Let's Encrypt!
* Remove some Python 2 specific code in diffoscope.changes and
diffoscope.comparators.__init__.
* Make the tlsh module optional. python3-tlsh is now listed in Recommends.
* Make file detection compatible with the pyhon-magic module available on
PyPI.
* Fix fallback code used when rpm module is unavailable. python3-rpm is now
listed in Recommends.
* Convert README to reStructuredText and update it.
* Improve setup.py:
- Fix the non-working PyTest class.
- Update homepage URL.
- Improve short description.
- Add license field.
- Update install_requires.
* Update Build-Depends and Depends.
* Also run the test suite without Recommends installed in autopkgtest.
* Properly skip tests that requires external tools when they are not
available.
-- Jérémy Bobbio <lunar@debian.org> Thu, 12 Nov 2015 12:28:05 +0100
diffoscope (38) unstable; urgency=medium
* Fix dh_python3 package overrides so we get a correct versioned
Depends on python3-tlsh.
-- Jérémy Bobbio <lunar@debian.org> Sat, 17 Oct 2015 11:05:12 +0200
diffoscope (37) unstable; urgency=low
* Switch to incremental interface for TLSH. That means we won't load
800 MiB or more in memory to compute the fuzzy hash.
* Add support for CBFS images (Closes: #788364)
* Add support for .dsc files (Closes: #800359)
* Fix handling of malformed md5sums file. (Closes: #799901)
* Fix placeholder encoding issue when trimming stderr.
(Closes: #799863)
* Read text report as UTF-8 encoded in test_text_option_with_file.
(Closes: #801766)
* Read Haskell interfaces in binary mode (Closes: #801333)
* Read md5sums file using UTF-8.
* Add a test for --max-diff-block-lines.
* Handle closing stdin for Command implementing feed_stdin.
* Handle subprocess errors with no output when comparing with a command.
* Use name and not path when doing binary comparisons.
* Add a test for --list-tools.
* Fix main exception handling.
* Use text file comparison for .dsc and .changes when the referenced files
are missing.
* Do skip squashfs test if there's no user with uid 1000.
* Use super() instead of the old cumbersome form.
-- Jérémy Bobbio <lunar@debian.org> Thu, 15 Oct 2015 16:45:55 +0200
diffoscope (36) unstable; urgency=low
* Read Debian .changes file as utf-8 encoded.
* Add missing encode() for the 'too much input for diff' message.
-- Jérémy Bobbio <lunar@debian.org> Tue, 22 Sep 2015 11:24:37 +0200
diffoscope (35) unstable; urgency=low
[ Jérémy Bobbio ]
* Switch to Python 3. This means code has been changed slightly
here and there. Issues related to text encoding should now be
far less frequent. According to some preliminary benchmark,
diffoscope should run faster as well.
* The system tar is now used instead of the Python module to display Tar
archive metadata. The output should be slightly more accurate.
* Log exceptions raised while computing diffs as errors
* Avoid concatenating many strings in hexdump_fallback. This matters when
looking at big binary files with xxd unavailable.
* Let's ignore command errors when the file content is identical.
* Fix isoinfo arguments to read Joliet data on ISO9660 images.
* Ignore errors when reading Joliet and Rockridge data in ISO9660 images.
* Use threads instead of processes for multiprocessing when running diff.
There is no measurable performance impact and it helps to simplify the
code.
* Use the right argument order when creating CalledProcessError.
* Propagate failure of external commands feeding diff.
* Improve debugging log for command lines and exit codes.
* Remove she-bang from files not meant to be executed directly.
* Add more tests for main() behavior.
* Improve RPM header conversion.
* Fix a refactoring error in compare_commented_files().
[ Mattia Rizzolo ]
* Drop X-Python-Version field, not needed anymore for python3.
-- Jérémy Bobbio <lunar@debian.org> Mon, 21 Sep 2015 21:23:56 +0000
diffoscope (34) unstable; urgency=medium
* Fix path to executable in README. Thanks Niko Tyni for the patch.
* Encode file path when filtering it from readelf output. This fixes
diffoscope for gnome-clocks. (Closes: #798398)
* Include ELF test files in the source tarball.
* Fix required tool for zip tests.
* Fix expected `unsquashfs -s` output for squashfs-tools 4.3.
* Parse md5sums in .deb even if they are identical.
* Log ignore files count when comparing data.tar in .deb.
* Minor code improvemnts.
-- Jérémy Bobbio <lunar@debian.org> Thu, 17 Sep 2015 12:57:26 +0200
diffoscope (33) unstable; urgency=medium
* Fix path to diffoscope used to generate Recommends. (Closes: #797978)
-- Jérémy Bobbio <lunar@debian.org> Fri, 04 Sep 2015 10:20:45 +0200
diffoscope (32) unstable; urgency=medium
[ Jérémy Bobbio ]
* Add `--fuzzy-threshold` option. This allows one to specify the TLSH score
used as cut-off for fuzzy matching. Specifying 0 will disable
fuzzy-matching entirely. Thanks Jakub Wilk for prompting me to implement
this. (Closes: #797557)
* Add `--new-file` to treat absent files as empty. Thanks Jakub Wilk for the
suggestion. (Closes: #797560)
* Enable comparisons of symlinks and devices given on the command line.
(Closes: #796262)
* Fix comparisons of device files. (Closes: #796288)
* Perform comparisons of file metadata in directories using original path.
(Closes: #796202)
* Display default values in `--help`.
* Stop unpacking Tar directories. Thanks h01ger for the report.
(Closes: #797164)
* Make directory comparison work again.
* Fix and document required Python modules. Thanks plntyk for the feedback on
IRC.
* Pass `--force-local`. to cpio. We don't want filenames with ':' to trigger
remote copies.
* Mark that get_ar_content() as requiring the 'ar' executable.
* Rework how diffoscope main() is run. Thanks Yaroslav Halchenko for the
report and suggestions. (Closes: #796196)
* Assume UTF-8 output if stdin does not advertise any encoding.
* Give proper error message when run on non-existing files.
* Output differences in directory test to ease debugging.
* Update manpage. Thanks Jakub Wilk for reporting the issue.
(Closes: #797561)
* Properly handle SIGTERM and do cleanup temp files. Thanks Mattia Rizzolo
for reporting this and how he did work around this deficiency.
(Closes: #788568)
* Fix handling of GHC .hi file (Closes: #796039)
* Add a test on how Ctrl+C is handled.
* Minor code improvements.
[ Chris Lamb ]
* Correct "comment" on Device comparator.
-- Jérémy Bobbio <lunar@debian.org> Thu, 03 Sep 2015 20:00:45 +0200
diffoscope (31) unstable; urgency=low
[ Reiner Herrman ]
* Fix detection of jar archives.
* Make some hardcoded limits configurable. (Closes: #795292)
[ Mattia Rizzolo ]
* Don't print the traceback when killed by KeyboardInterrupt.
* Add a --debugger option to dump the user into pdb in case of crashes.
(Closes: #796204)
[ Jérémy Bobbio ]
* Replace ssdeep by tlsh
[ Chris Lamb ]
* Unify to 4 spaces everywhere.
* Leave string interpolation to logging.$level(..) calls.
* Add missing 'sys' import.
* Specify source of 'Files' correct; was using "Changes" leaked from for loop.
* Remove unused and duplicate imports.
* Support passing directories as "foo/" or "foo".
-- Jérémy Bobbio <lunar@debian.org> Fri, 21 Aug 2015 00:03:11 +0200
diffoscope (30) unstable; urgency=low
* Remove empty line at the beginning of HTML reports.
* Stop failing to run the tests when uid 1000 doesn't exist.
* Make .changes file matching more accurate.
* Move the sanity check for version number from clean to build rule.
* Remove leftover debug in squashfs comparator.
* Stop decompressing squashfs directories as we compare content already.
-- Jérémy Bobbio <lunar@debian.org> Sun, 09 Aug 2015 12:18:30 +0200
diffoscope (29) unstable; urgency=low
[ Reiner Herrmann ]
* Prevent empty line at the end of squashfs listing.
[ Jérémy Bobbio ]
* Rename to diffoscope. debbindiff has grown way beyond a being just a tool
to compare Debian packages. Let's rename it to better reflect this state of
things.
* Add a favicon to HTML reports.
* Always use pybuild in debian/rules.
-- Jérémy Bobbio <lunar@debian.org> Mon, 03 Aug 2015 15:55:07 +0200
debbindiff (28) unstable; urgency=low
* Improve code quality by storing comments for Difference as lists.
* Fix label for content of gzip, bzip2, and xz files.
* Handle when only the encoding is different when comparing text files.
(Closes: #794347)
-- Jérémy Bobbio <lunar@debian.org> Sun, 02 Aug 2015 17:21:52 +0200
debbindiff (27) unstable; urgency=low
[ Reiner Herrmann ]
* Add dh-python to Build-Depends.
[ Daniel Kahn Gillmor ]
* Add support for Mono PE executables.
[ Holger Levsen ]
* Add myself to Uploaders.
[ Jérémy Bobbio ]
* Add tests for tar comparator.
* Massive rearchitecturing.
* Now use libarchive to handle cpio archives and iso9660 images.
* find is now used to compare directory listings.
* Symlinks and devices can now be properly compared. (Closes: #789003)
* Files in squashfs are now extracted one by one.
* Text files are now compared after being decoded with the detected
encoding. Encoding differences are reported. (Closes: #785777)
* Pre-compile regexps for ELF archives to get a good performance gain
when comparing ELF objects.
* Display better stacktrace for functions and methods using decorators.
* Reset locale environment during tests.
* Use numeric uid/gid when listing cpio content.
* Set timezone to UTC when configuring locale.
* Return no differences when there's none even when a required tool is
unavailable.
* Fix rpm expected listing after cpio change.
* Skip tests when required tools are missing.
* Skip squashfs listing test until #794096 is solved.
* Analyze md5sums in .deb and skip identical files.
* Add support for sqlite3 databases.
* Implement fuzzy-matching of files in the same container using ssdeep.
* Add pydist-overrides for magic and rpm.
* When comparing .changes, match names without version numbers.
* Switch to pybuild as dh buildsystem.
-- Jérémy Bobbio <lunar@debian.org> Fri, 31 Jul 2015 08:38:10 +0000
debbindiff (26) unstable; urgency=low
* Declare the package autopkgtestable.
* Fix comparator for unknown files.
* Add tests for unknown files comparator.
-- Jérémy Bobbio <lunar@debian.org> Sun, 28 Jun 2015 09:22:21 +0000
debbindiff (25) unstable; urgency=low
* Comparators now have a test suite with 93% code coverage.
* Support autopkgtest to run the test suite.
* Properly strip path from javap output even when they are relative.
* Also remove 'Last modified' and 'MD5 checksum' lines from javap output.
* Factorize readelf commands.
* Filter archive name from readelf output.
* Filter archive name from objdump output.
* Fix charset handling of .mo files.
* Don't be quadratic when parsing .mo header.
* Skip archive name in zipinfo output.
* Fix destination path when decompressing gzip files not ending in .gz.
* Filter image name from squashfs superblock information.
* Fix comparison of files in cpio archives.
* Change how we handle a missing RPM module.
* Don't add empty Difference when comparing files with equal metadata in
directories.
* Filter access time from stat output.
* Rename .changes comparator.
* Rework .changes comparator.
* Comparators now return a single Difference instead of a list of Difference.
-- Jérémy Bobbio <lunar@debian.org> Sat, 27 Jun 2015 14:32:58 +0000
debbindiff (24) unstable; urgency=medium
* Fix for undefined variables introduced in last release.
-- Reiner Herrmann <reiner@reiner-h.de> Tue, 23 Jun 2015 00:15:28 +0200
debbindiff (23) unstable; urgency=medium
[ Helmut Grohne ]
* Speed up diff collection.
* Read lines in larger chunks.
[ Reiner Herrmann ]
* Don't follow symlinks, but print their targets.
-- Reiner Herrmann <reiner@reiner-h.de> Mon, 22 Jun 2015 23:19:17 +0200
debbindiff (22) unstable; urgency=low
* Add myself to uploaders
* Swap JDK dependencies, so openjdk is preferred (thanks mapreri)
-- Reiner Herrmann <reiner@reiner-h.de> Sun, 14 Jun 2015 23:05:49 +0200
debbindiff (21) unstable; urgency=low
* Non-maintainer upload.
* Add support for Java .class files.
* Add support for .ipk package files.
-- Reiner Herrmann <reiner@reiner-h.de> Sat, 13 Jun 2015 21:38:20 +0200
debbindiff (20) unstable; urgency=low
[ Reiner Herrmann ]
* info files can contain numbers in the extension
[ Jérémy Bobbio ]
* Fix diff parser when skipping until the end of a file without a newline.
* Use same encoding as stdin when output is not a TTY. (Closes: #785288)
-- Jérémy Bobbio <lunar@debian.org> Sun, 17 May 2015 16:05:14 +0200
debbindiff (19) unstable; urgency=low
[ Reiner Herrmann ]
* Fix info files comparator by specifying the correct parameter name
for the text comparator. (Closes: #784891)
-- Jérémy Bobbio <lunar@debian.org> Tue, 12 May 2015 10:35:48 +0200
debbindiff (18) unstable; urgency=low
[ Reiner Herrmann ]
* Use zip comparator for pk3 files.
* Use text comparator for info files.
* Drop mime type for info files.
* Remove limit of diff output again.
[ Jérémy Bobbio ]
* Assume tar member names are UTF-8 encoded. This allows debbindiff to
process drmips.
* Write a text report on stdout as default behavior.
* Allow both --html and --text to produce output in one run.
-- Jérémy Bobbio <lunar@debian.org> Sat, 09 May 2015 11:29:46 +0200
debbindiff (17) unstable; urgency=low
[ Jérémy Bobbio ]
* Add support for ISO9660 images.
* Catch a few more initrds as such.
* Reimplement are_same_binaries() using cmp.
[ Reiner Herrmann ]
* Use gzip comparator for .dz (dictzip) files.
* Added isoinfo to list of tool providers.
[ Yasushi SHOJI ]
* Disable 'Device' section diff in stat output. (Closes: #783792)
-- Jérémy Bobbio <lunar@debian.org> Thu, 30 Apr 2015 19:29:26 +0200
debbindiff (16) unstable; urgency=low
[ Reiner Herrmann ]
* Remove temporary directory when exception is raised. (Closes: #782253)
* Fix extraction when Zip member contain "../" in their path.
[ Jérémy Bobbio ]
* Fail with a better error message when unable to create a Queue.
(Closes: #782551)
-- Jérémy Bobbio <lunar@debian.org> Thu, 16 Apr 2015 11:07:58 +0200
debbindiff (15) unstable; urgency=low
* Terminate xxd if its output was too long.
* Handle broken zip files by falling back on binary comparison.
* Fix an over-matching jar/war filename regexp.
* Fix .hi comparator.
* Fix some file descriptor leaks.
-- Jérémy Bobbio <lunar@debian.org> Sat, 04 Apr 2015 18:30:48 +0200
debbindiff (14) unstable; urgency=low
* Remove diffutils from Recommends as it is Essential:yes.
* Fallback to Python hexlify if xxd is not available.
* Decode msgunfmt output using the actual PO charset.
* Stop buffering output of most external commands and
send it to diff as it comes. (Closes: #781373)
* Stop feeding input to diff after a certain amount of lines,
as GNU diff cannot cope with arbitrary large output.
* Fix newline and tab handling in HTML presenter.
-- Jérémy Bobbio <lunar@debian.org> Thu, 02 Apr 2015 16:16:59 +0200
debbindiff (13) unstable; urgency=low
[ Helmut Grohne ]
* Fix several imports for RequiredToolNotFound.
[ Jérémy Bobbio ]
* Remove dead code from HTML presenter.
* Fix tab handling in HTML presenter.
* Swallow stderr when running showttf. (Closes: #781374)
* Swallow stderr for readelf and objdump.
* Use pipes instead of temp files to feed diff.
* Stop processing diff output after too many lines.
* Use LINESIZE constant directly in HTML presenter.
* Better handle non-printable characters in HTML presenter.
* Cut long lines in HTML presenter.
-- Jérémy Bobbio <lunar@debian.org> Sat, 28 Mar 2015 18:53:29 +0100
debbindiff (12) unstable; urgency=low
[ Helmut Grohne ]
* Attempt at fixing unicode issues with --text. (Closes: #778641)
[ Jérémy Bobbio ]
* Try with utf-8 encoding when comparing text files and no common
encoding has been detected.
* Perform content comparison when finding differences instead of
waiting for the presentation phases. (Closes: #781280)
* Instead of using vim, the HTML view is created from diff output.
(Closes: #772029, #779476)
-- Jérémy Bobbio <lunar@debian.org> Fri, 27 Mar 2015 18:42:26 +0100
debbindiff (11) unstable; urgency=low
* Use latin-1 to decode showttf output.
* Normalize locale environment variables on startup and subsequently use
UTF-8 to decode output of most external tools. (Closes: #780863)
* Error out when encoding is not Unicode-compatible for text output.
(Closes: #778641)
-- Jérémy Bobbio <lunar@debian.org> Mon, 23 Mar 2015 22:00:15 +0100
debbindiff (10) unstable; urgency=low
[ Chris Lamb ]
* SVGZ files are gzip files.
* Use gzip comparator for .tgz files.
[ Reiner Herrmann ]
* Use zip comparator for Java web archives (.war).
* Use zip comparator for epub files.
* Don't fill up diffs with dots.
* Add support for squashfs images.
[ Jérémy Bobbio ]
* Output a nicer name for sub-directories.
* Also compare stat, acl, and xattrs when comparing directories.
* Check for availability of tools required by comparators (original work by
Reiner Herrmann). Most packages in Depends are now in Recommends.
* Ask objdump to display more sections.
* Drop stderr output when running cpio -t.
* Attempt to sort out multiple decoding and encoding issues.
(Closes: #778641)
* Fix source path when handling gzip, bzip2 and xz. (Closes: #779474)
* Don't crash when two .changes don't contain the same files.
(Closes: #779391)
* Properly split lines when comparing .changes.
* Add newlines if needed in text output.
* Remove most table borders in HTML output. (Closes: #779475)
* Properly close <span/> in HTML output.
* Add anchors in HTML output. (Closes: #778537)
* Update debian/copyright.
-- Jérémy Bobbio <lunar@debian.org> Fri, 20 Mar 2015 13:52:44 +0100
debbindiff (9) unstable; urgency=medium
[ Chris Lamb ]
* Add rpm-common to Build-Depends so that we can “import rpm" during build.
* Code improvements improvements.
[ Jérémy Bobbio ]
* Add plain text output for differences. Thanks Helmut Grohne for the
original patch. (Closes: #778423)
* Exit with 2 in case of errors. (Closes: #774983)
* Properly output diffs when one of the file is empty.
* Add support for comparing directories.
* Update debian/copyright.
-- Jérémy Bobbio <lunar@debian.org> Sun, 15 Feb 2015 12:13:46 +0100
debbindiff (8) unstable; urgency=medium
[ Peter De Wachter ]
* Use `diff -a` so debbindiff doesn't abort on binary files.
[ Jérémy Bobbio ]
* Verify that version in debian/changelog matches debbindiff version.
* Update debian/copyright.
* Recognize text/plain files.
[ Reiner Herrmann ]
* Increased line context of diff to further lower running time.
* Use text comparator for xml/xhtml files.
* Use text comparator for postscript files.
* Add support for cpio archives.
* Add support for rpm packages.
* Fix a typo in debbindiff name in source headers.
[ Chris Lamb ]
* Improve code quality in several places.
-- Jérémy Bobbio <lunar@debian.org> Tue, 10 Feb 2015 03:02:55 +0100
debbindiff (7) unstable; urgency=medium
[ Jérémy Bobbio ]
* Add support for PDF files.
* Add --max-report-size command-line option.
* Add extra mime type for .deb.
* Fallback on binary comparison when external tools fail. (Closes: #764140)
* Update copyrights.
[ Helmut Grohne ]
* Avoid unnecessary guess_mime_type calls.
[ Reiner Herrmann ]
* Speed up creation of diffs by removing extra lines before calling vim.
-- Jérémy Bobbio <lunar@debian.org> Sat, 10 Jan 2015 14:05:17 +0100
debbindiff (6) unstable; urgency=medium
* Fallback to binary comparison when text encoding gets misdetected.
* Skip full comparison when small files match.
-- Jérémy Bobbio <lunar@debian.org> Wed, 07 Jan 2015 12:01:11 +0100
debbindiff (5) unstable; urgency=medium
* Properly encode strings before writing them to tempfiles.
(Closes: #764254)
* Fallback on binary comparison when text encoding is unknown.
* Always use given source path in binary_fallback.
* Add support for .udeb.
* Add support for ttf and otf files.
* Add support for png files. (Closes: #773573)
-- Jérémy Bobbio <lunar@debian.org> Tue, 30 Dec 2014 12:34:13 +0100
debbindiff (4) unstable; urgency=medium
* Look harder for differences in zip files if needed.
* Avoid deadlock when calling vimdiff. (Closes: #764131, #764420)
* Add support for an external CSS. (Closes: #764470)
* Improve default CSS to keep tables from overflowing.
* Bump Standards-Version.
-- Jérémy Bobbio <lunar@debian.org> Sun, 09 Nov 2014 12:35:07 +0100
debbindiff (3) unstable; urgency=low
* Initial Debian release. (Closes: #763328)
-- Jérémy Bobbio <lunar@debian.org> Thu, 02 Oct 2014 13:52:55 +0200