- Created a batch file to build borg on windows
- Adjusted setup.py to be runnable on windows and build the windows
extension
- Extracted the free space check to a function in the platform module
- Created the minimal needed (dummy) functions for the windows platform
module
The build process can be controlled via environment variables
similar to other bundled libraries in borgbackup. The main difference
is probably that upstream does not provide a pkgconfig file for
xxhash.
Therefore borg will probably fail to detect the system-provided
version by default (tested on Fedora, seems like Debian and Ubuntu
do not ship a pkgconfig file either). I kept the pkgconfig lookup
code anyway to keep the code as similar as possible to
"setup_compress.py"/"setup_crypto.py".
Setting BORG_LIBXXHASH_PREFIX=/usr helps borg to detect xxhash
on my system (Fedora). You can force the use of the bundled
version of xxhash by setting BORG_USE_BUNDLED_XXHASH=1.
these are already used internally when the build system can not find
a system library (neither via pkginfo nor BORG_LIBXXX_PREFIX is given)
and then triggers usage of the bundled code via these env vars.
now they are also used to tell right from the beginning "use the
bundled code" and in that case it will not try to locate system libs
and headers.
1. BORG_*_PREFIX is checked (avoids lib detection via pkg-config).
2. pkg-config is tried
3. fallback to bundled C code (or failure in case of OpenSSL)
also:
- simplified code again
- removed (c) headers, nothing left of original code
- removed hardcoded lib search pathes
- to find system libs/headers, one must point these env vars to them:
BORG_OPENSSL_PREFIX
- moved some code from setup.py to setup_*.py
- removed hardcoded lib search pathes
- to find system libs/headers, one must point these env vars to them:
BORG_LIBLZ4_PREFIX, BORG_LIBZSTD_PREFIX, BORG_LIBB2_PREFIX
- moved some code from setup.py to setup_*.py
if you do not have python 3.6.x, you can still use borg 1.1.x or 1.0.x.
another option is to use the fat binary from github releases, which
includes python 3.6 and all other stuff needed.
MASTER BRANCH ONLY, DO NOT BACKPORT TO 1.1 or 1.0!
As there are no 0.4.x msgpack releases (under this new name, 0.4.x
used to be named "msgpack-python"), currently only 0.5.6 can be
recommended. See comments for other versions that are acceptable
under certain conditions.
We will add new msgpack versions later, after testing them.
Failure to use environment markers means the dependencies are
unconditionally added at build time based on the host instead of being
always present and evaluated at runtime on the target; e.g. wheels have
the wrong information.
Also use python_requires. This teaches pip how to natively comprehend
when the current version of python is not supported by borg.
Returns errors in the format:
borgbackup requires Python '>=3.5' but the running Python is $oldver
we temporarily used the updated, bundled lz4 and zstd code for testing
purposes, but now going back to using system provided libs by default.
(cherry picked from commit 66084c7234)
update build process to use newer Cython/setuptools APIs
Cython.Distutils.build_ext has been deprecated, and the recommended replacement
is Cython.Build.cythonize.
setuptools.Extension will convert the file extension .pyx -> .c so there's no
need to duplicate the functionality. This allows the removal of the special
Sdist logic when cython is available.
cythonize will compile .pyx -> .c, but the resulting Extension's files won't be
included in sdist unless the Extension is in ext_modules. We don't want the
platform extensions in ext_modules though because that will break compilation,
so the platform .c files were added to the manifest.
- removed some unnecessary distutils imports and moved some imports for better
visibility
- stop cythonize from running in some common use cases like 'clean'
maybe this is the easiest way for us to deal with msgpack compatibility.
0.5.0 release had some troubles:
- FutureWarning on stderr disturbing other output there, breaking tests
- pip install -U broken due to a pip issue with the transisition pkg
which was needed due to the package rename (ImportError for msgpack)
- some linux dists not packaging the transition pkg
setup_zstd.py modified so it is just amending the Extension() kwargs,
but the Extension is initialized by the caller.
this way, amending can happend multiple times (e.g. for multiple
compression algorithms).
also:
- move include/library dirs processing for system-library case
- move system zstd prefix detection to setup_zstd module
- cosmetic: setup.py whitespace fixes
- prefer system zstd option, document zstd min. requirement
based on willyvmm's work in PR #3116, but some changes:
- removed any mulithreading changes
- add zstandard in setup.py install_requires
- tests
- fix: minimum compression level is 1 (not 0)
- use 3 for the default compression level
- use ID 03 00 for zstd
- only convert to bytes if we don't have bytes yet
- move zstd code so that code blocks are ordered by ID
- other cosmetic fixes
