Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support fetching dependencies bundled with .tar.zst extension #14299

Closed
Tracked by #14265
andrewrk opened this issue Jan 13, 2023 · 3 comments · Fixed by #17552
Closed
Tracked by #14265

support fetching dependencies bundled with .tar.zst extension #14299

andrewrk opened this issue Jan 13, 2023 · 3 comments · Fixed by #17552
Labels
contributor friendly This issue is limited in scope and/or knowledge of Zig internals. enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Milestone

Comments

@andrewrk
Copy link
Member

andrewrk commented Jan 13, 2023

Extracted from #14265.
Depends on #14183.

zig build should support fetching via an URL like this:

    .url = "https://example.com/foo.tar.zst",

zig/src/Package.zig

Lines 357 to 376 in 7cb2f92

if (mem.endsWith(u8, uri.path, ".tar.gz")) {
// I observed the gzip stream to read 1 byte at a time, so I am using a
// buffered reader on the front of it.
var br = std.io.bufferedReaderSize(std.crypto.tls.max_ciphertext_record_len, req.reader());
var gzip_stream = try std.compress.gzip.gzipStream(gpa, br.reader());
defer gzip_stream.deinit();
try std.tar.pipeToFileSystem(tmp_directory.handle, gzip_stream.reader(), .{
.strip_components = 1,
});
} else {
return reportError(
ini,
comp_directory,
uri.path.ptr,
"unknown file extension for path '{s}'",
.{uri.path},
);
}

Related:

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. contributor friendly This issue is limited in scope and/or knowledge of Zig internals. zig build system std.Build, the build runner, `zig build` subcommand, package management labels Jan 13, 2023
@andrewrk andrewrk added this to the 0.12.0 milestone Jan 13, 2023
@scheibo
Copy link
Contributor

scheibo commented Jan 13, 2023

should probably support .zst as well (or instead of .zstd) given thats the more common/standard extension for zstd

@andrewrk
Copy link
Member Author

Typo, thank you

@andrewrk andrewrk changed the title support fetching dependencies bundled with .tar.zstd extension support fetching dependencies bundled with .tar.zst extension Jan 13, 2023
@andrewrk andrewrk modified the milestones: 0.13.0, 0.12.0 Jan 16, 2023
@dweiller
Copy link
Contributor

dweiller commented Feb 27, 2023

I have a (mostly untested) branch that adds support for Zstandard compressed tarballs, but it first does some minor surgery on std.compress to make the high level decompession APIs more similar to try and use the current technique of making unpackTarball generic across formats. I've opened #14739 to get some input on how std.compress could be made more consistent across formats.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor friendly This issue is limited in scope and/or knowledge of Zig internals. enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants