Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to Kafka using SASL authentication(Kafka认证问题) #1515

Open
zhanghangorg opened this issue Oct 17, 2024 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@zhanghangorg
Copy link

zhanghangorg commented Oct 17, 2024

环境:

crawlab 0.6.3 社区版
kubernetes部署

问题:

新建数据源,数据类型为kafka,配置host、port、user、password,kafka始终离线。
查看代码 https://github.sheincorp.cn/crawlab-team/crawlab/blob/main/core/utils/kafka.go ,没有找到关于认证部分代码,似乎未实现kafka认证部分。

需求:

增加kafka认证部分实现

提问:

尝试自行增加kafka认证实现,编译crawlab,复制到worker节点,并启动,但启动时报错。

cd crawlab/backend 
GOOS=linux GOARCH=amd64 go build -o crawlab-server main.go

kubectl cp crawlab-server crawlab/crawlab-worker-7d5796d9bd-52xtr:/opt/bin/

通过 
crawlab-server server 或   /bin/bash /app/bin/docker-init.sh 启动都报错。

请问正确的自行编译kafka认证实现的方式是怎样的?有没有自行编译部署的文档,查看0.5文档与当前程序流程不通用。
https://docs-v05.crawlab.cn/zh/Installation/Direct.html#%E7%9B%B4%E6%8E%A5%E9%83%A8%E7%BD%B2

# 报错信息
2024/10/17 14:46:14  info [GrpcClient] grpc client subscribed to remote server
2024/10/17 14:46:14  info [GrpcClient] start handling stream message...
panic: rpc error: code = Unknown desc = invalid character 'c' after top-level value

goroutine 10 [running]:
github.com/crawlab-team/crawlab/core/node/service.(*WorkerServiceV2).Register(0xc000150d80)
	/Users/Admin/Data/Code/golang/crawlab/core/node/service/worker_service_v2.go:89 +0x4bf
github.com/crawlab-team/crawlab/core/node/service.(*WorkerServiceV2).Start(0xc000150d80)
	/Users/Admin/Data/Code/golang/crawlab/core/node/service/worker_service_v2.go:51 +0x38
created by github.com/crawlab-team/crawlab/core/apps.(*ServerV2).Start in goroutine 9
	/Users/Admin/Data/Code/golang/crawlab/core/apps/server_v2.go:51 +0x145
@zhanghangorg zhanghangorg added the enhancement New feature or request label Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants
@zhanghangorg and others