-
-
Notifications
You must be signed in to change notification settings - Fork 508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for building images with buildkit #2482
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for testcontainers-go ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
I see there are some test failures related to logs output when building an image and I'll look into that. Meanwhile, I would like to hear your opinion about enabling buildkit automatically, as I do in this PR, or maybe it should be a build option that users could specify? |
@@ -877,6 +880,28 @@ var _ ContainerProvider = (*DockerProvider)(nil) | |||
func (p *DockerProvider) BuildImage(ctx context.Context, img ImageBuildInfo) (string, error) { | |||
buildOptions, err := img.BuildOptions() | |||
|
|||
const minBuildKitApiVersion = "1.39" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @silh, thanks for submitting this change. I need to talk about it more in depth with the docker-build team, as I have some concerns regarding the HTTP endpoints and the options that will be available for buildkit.
In the meantime, we have the BuildOptionsModifier
concept in the FromDockerfile
struct (see https://golang.testcontainers.org/features/build_from_dockerfile/#advanced-usage). I think client code could write a modifier function when buildkit is needed. Something like this:
// BuildKitOptionsModifier is a function that modifies the build options to use buildkit.
// It checks if the docker client supports buildkit and if it does, it creates a new build session.
// You can use this function as a BuildOptionsModifier in the ContainerRequest.FromDockerfile
// to build images using buildkit.
func BuildKitOptionsModifier(buildOptions *types.ImageBuildOptions) {
ctx := context.Background()
cli, err := testcontainers.NewDockerClientWithOpts(ctx)
if err != nil {
testcontainers.Logger.Printf("🛠️ Could not access the docker client: %s", err)
return
}
defer cli.Close()
clientApiVersion := cli.ClientVersion()
if versions.GreaterThanOrEqualTo(clientApiVersion, minBuildKitApiVersion) {
s, err := session.NewSession(ctx, "testcontainers", "")
if err == nil {
testcontainers.Logger.Printf("🛠️ Building using buildkit")
dialSession := func(ctx context.Context, proto string, meta map[string][]string) (net.Conn, error) {
return cli.DialHijack(ctx, "/session", proto, meta)
}
go func() {
if err := s.Run(ctx, dialSession); err != nil {
testcontainers.Logger.Printf("🛠️ Failed to run the build session: %s", err)
}
}()
defer s.Close()
buildOptions.SessionID = s.ID()
buildOptions.Version = types.BuilderBuildKit
} else {
testcontainers.Logger.Printf("🛠️ Could not create a build session, building without buildkit: %s", err)
}
}
}
From testcontainers-go standpoint, if this feature is widely used, we could create a separate Go module exposing that function, which would come with the benefit of not pushing the buildkit dependency to all consumers, only to those interested in using buildkit.
Wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something like this: https://github.com/mdelapenya/testcontainers-go/pull/new/builkit-module
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like modern docker doesn't require setting up the grpc cionnection to build with v2 builder. I'll try to update the PR in the near future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using buildkit via From FromDockerfile
causes image pulls to fail due to auth credentials seemingly not being processed. See the following for more details moby/moby#48112
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stevenh did you give it a try to my branch for buildkit options modifier? https://github.com/testcontainers/testcontainers-go/compare/main...mdelapenya:testcontainers-go:builkit-module?expand=1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately that fails as the session is closed before the build is complete resulting in errors like:
failed to resolve source metadata for docker.io/krakend/builder:latest: no active session for 1tcg31xrn1qyt6bmvq1ev9yoh: context deadline exceeded: failed to create container.
Had a play and got the basic flow seeming to work, but things like print output doesn't work. Looks like its going to pretty complex to get all buildkit functionality.
What does this PR do?
Automatically build images with buildkit instead of the old builder.
Why is it important?
Buildkit is since a few docker version a default building mechanism in Docker and many Dockerfiles are relying on features only available with Builkit support enabled. However the current testcontainers implementation doesn't support buildkit.
Related issues
How to test this PR
An autotest is added which fails when the added code in docker.go is disabled and succeeds when it is enabled.
Another way to test would be to use an old docker version (before API 1.39) - then the test TestBuildImageFromDockerfileBuildx will also fail.