Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for building images with buildkit #2482

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

silh
Copy link

@silh silh commented Apr 14, 2024

What does this PR do?

Automatically build images with buildkit instead of the old builder.

Why is it important?

Buildkit is since a few docker version a default building mechanism in Docker and many Dockerfiles are relying on features only available with Builkit support enabled. However the current testcontainers implementation doesn't support buildkit.

Related issues

How to test this PR

An autotest is added which fails when the added code in docker.go is disabled and succeeds when it is enabled.
Another way to test would be to use an old docker version (before API 1.39) - then the test TestBuildImageFromDockerfileBuildx will also fail.

@silh silh requested a review from a team as a code owner April 14, 2024 12:12
Copy link

netlify bot commented Apr 14, 2024

Deploy Preview for testcontainers-go ready!

Name Link
🔨 Latest commit b0b967f
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-go/deploys/661bc81ed8bc020008b4c135
😎 Deploy Preview https://deploy-preview-2482--testcontainers-go.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@silh
Copy link
Author

silh commented Apr 14, 2024

I see there are some test failures related to logs output when building an image and I'll look into that.

Meanwhile, I would like to hear your opinion about enabling buildkit automatically, as I do in this PR, or maybe it should be a build option that users could specify?

@@ -877,6 +880,28 @@ var _ ContainerProvider = (*DockerProvider)(nil)
func (p *DockerProvider) BuildImage(ctx context.Context, img ImageBuildInfo) (string, error) {
buildOptions, err := img.BuildOptions()

const minBuildKitApiVersion = "1.39"
Copy link
Member

@mdelapenya mdelapenya Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @silh, thanks for submitting this change. I need to talk about it more in depth with the docker-build team, as I have some concerns regarding the HTTP endpoints and the options that will be available for buildkit.

In the meantime, we have the BuildOptionsModifier concept in the FromDockerfile struct (see https://golang.testcontainers.org/features/build_from_dockerfile/#advanced-usage). I think client code could write a modifier function when buildkit is needed. Something like this:

// BuildKitOptionsModifier is a function that modifies the build options to use buildkit.
// It checks if the docker client supports buildkit and if it does, it creates a new build session.
// You can use this function as a BuildOptionsModifier in the ContainerRequest.FromDockerfile
// to build images using buildkit.
func BuildKitOptionsModifier(buildOptions *types.ImageBuildOptions) {
	ctx := context.Background()

	cli, err := testcontainers.NewDockerClientWithOpts(ctx)
	if err != nil {
		testcontainers.Logger.Printf("🛠️ Could not access the docker client: %s", err)
		return
	}
	defer cli.Close()

	clientApiVersion := cli.ClientVersion()

	if versions.GreaterThanOrEqualTo(clientApiVersion, minBuildKitApiVersion) {
		s, err := session.NewSession(ctx, "testcontainers", "")
		if err == nil {
			testcontainers.Logger.Printf("🛠️ Building using buildkit")
			dialSession := func(ctx context.Context, proto string, meta map[string][]string) (net.Conn, error) {
				return cli.DialHijack(ctx, "/session", proto, meta)
			}
			go func() {
				if err := s.Run(ctx, dialSession); err != nil {
					testcontainers.Logger.Printf("🛠️ Failed to run the build session: %s", err)
				}
			}()
			defer s.Close()
			buildOptions.SessionID = s.ID()
			buildOptions.Version = types.BuilderBuildKit
		} else {
			testcontainers.Logger.Printf("🛠️ Could not create a build session, building without buildkit: %s", err)
		}
	}
}

From testcontainers-go standpoint, if this feature is widely used, we could create a separate Go module exposing that function, which would come with the benefit of not pushing the buildkit dependency to all consumers, only to those interested in using buildkit.

Wdyt?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like modern docker doesn't require setting up the grpc cionnection to build with v2 builder. I'll try to update the PR in the near future.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using buildkit via From FromDockerfile causes image pulls to fail due to auth credentials seemingly not being processed. See the following for more details moby/moby#48112

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately that fails as the session is closed before the build is complete resulting in errors like:

failed to resolve source metadata for docker.io/krakend/builder:latest: no active session for 1tcg31xrn1qyt6bmvq1ev9yoh: context deadline exceeded: failed to create container.

Had a play and got the basic flow seeming to work, but things like print output doesn't work. Looks like its going to pretty complex to get all buildkit functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants