Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create build for linux/arm64 platform to support Apple Silicon (M1, M2) #81

Open
benjaminsingleton opened this issue Mar 2, 2024 · 11 comments · May be fixed by #102
Open

Create build for linux/arm64 platform to support Apple Silicon (M1, M2) #81

benjaminsingleton opened this issue Mar 2, 2024 · 11 comments · May be fixed by #102
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@benjaminsingleton
Copy link

benjaminsingleton commented Mar 2, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe

There isn't a build that is compatible with modern Apple computers, which run on the platform linux/arm64.

$ docker pull --platform linux/arm64 ghcr.io/gnzsnz/ib-gateway:stable
stable: Pulling from gnzsnz/ib-gateway
no matching manifest for linux/arm64 in the manifest list entries

On my MacBook M2, I get odd errors when trying to run the image if I force pull it as a linux/amd64. The emulator that tries to run amd64 does not seem to be able to run ib-gateway.

2024-03-02 13:28:21 2024-03-02 18:28:21:668 IBC: Starting Gateway
2024-03-02 13:28:22 assertion failed [rem_idx != -1]: Unable to find existing allocation for shared memory segment to unmap
2024-03-02 13:28:22 (VMAllocationTracker.cpp:745 remove_shared_mem)
2024-03-02 13:28:22  /home/ibgateway/ibc/scripts/ibcstart.sh: line 504:    30 Trace/breakpoint trap   "$java_path/java" -cp "$ibc_classpath" $java_vm_options$autorestart_option $entry_point "$ibc_ini" ${mode}
2024-03-02 13:28:22 IBC returned exit status 133
2024-03-02 13:28:22 autorestart file not found
2024-03-02 13:28:22 
2024-03-02 13:28:22 Gateway finished
image

Describe the solution you'd like

A build that supports linux/arm64.

Describe alternatives you've considered

Tried to run as linux/amd64 but it didn't work.

Docker Desktop 4.27.2 (137060).
Docker version 25.0.3, build 4debf41
@wesget182
Copy link

wesget182 commented Mar 3, 2024 •
edited
Loading

Hey @benjaminsingleton, if you clone the aarch64 branch and build your own image locally, it should resolve this problem. I've successfully built and started my ibgateway using this method. Although, I'm currently having trouble connecting to it from within my api.

2024-03-03 11:00:03 2024-03-03 04:00:03:896 IBC: Login attempt: 1
2024-03-03 11:00:03 2024-03-03 04:00:03:909 IBC: Click button: Paper Log In
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: IBKR Gateway; event=Lost focus
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: IBKR Gateway; event=Deactivated
2024-03-03 11:00:06 2024-03-03 04:00:06:370 IBC: detected frame entitled: Loading...; event=Activated
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: detected frame entitled: Loading...; event=Focused
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: detected frame entitled: Loading...; event=Opened
2024-03-03 11:00:06 2024-03-03 04:00:06:373 IBC: Found Gateway main window
2024-03-03 11:00:06 2024-03-03 04:00:06:374 IBC: Got main window from future
2024-03-03 11:00:06 2024-03-03 04:00:06:374 IBC: Invoking config dialog menu
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Loading...; event=Lost focus
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Loading...; event=Deactivated
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Activated
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Focused
2024-03-03 11:00:06 2024-03-03 04:00:06:954 IBC: detected frame entitled: Connecting to server (trying for another 19 seconds)...; event=Opened
2024-03-03 11:00:09 LogModuleConfigurator-Init: Log4j Ver2.x found on classpath
2024-03-03 11:00:09 LogModuleConfigurator-Init: LogModuleConfigurator initialized with Log4j Verd.x
2024-03-03 11:00:10 2024-03-03 04:00:10:168 IBC: detected frame entitled: Starting application...; event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: DU7748944 Trader Workstation Configuration (Simulated Trading); event=Opened
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected frame entitled: Starting application...; event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: DU7748944 Trader Workstation Configuration (Simulated Trading); event=Activated
2024-03-03 11:00:10 2024-03-03 04:00:10:176 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Focused
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Got config dialog from future
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected frame entitled: Starting application...; event=Closed
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Login has completed
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: detected dialog entitled: Warning; event=Opened
2024-03-03 11:00:10 2024-03-03 04:00:10:177 IBC: Click button: I understand and accept
2024-03-03 11:00:10 2024-03-03 04:00:10:250 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:250 IBC: detected dialog entitled: Warning; event=Activated
2024-03-03 11:00:10 2024-03-03 04:00:10:251 IBC: detected dialog entitled: Warning; event=Focused
2024-03-03 11:00:10 2024-03-03 04:00:10:255 IBC: Setting ReadOnlyApi
2024-03-03 11:00:10 2024-03-03 04:00:10:300 IBC: Read-Only API checkbox is already set to: false
2024-03-03 11:00:10 2024-03-03 04:00:10:318 IBC: Configuration tasks completed
2024-03-03 11:00:10 2024-03-03 04:00:10:318 IBC: Click button: OK
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Lost focus
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Deactivated
2024-03-03 11:00:10 2024-03-03 04:00:10:392 IBC: detected dialog entitled: Warning; event=Closed
2024-03-03 11:00:10 2024-03-03 04:00:10:393 IBC: detected dialog entitled: Trader Workstation Configuration (Simulated Trading); event=Closed
2024-03-03 11:00:11 2024-03-03 04:00:11:795 IBC: detected dialog entitled: Pending Tasks; event=Closed

@gnzsnz
Copy link
Owner

gnzsnz commented Mar 3, 2024 •
edited
Loading

indeed aarch64 was an attempt to get this working on arm64/aarch64.

if you build localy ib-gateway using that branch, it should (kind of) work. there is no working GUI, but the API works. I have not fully tested the API, so i can't warranty anything.

the main problem is to get a working java environment for ibgateway/TWS. aarch64 branch is not too far way, but not there yet.

any help to get this working on arm64 is more than welcome.

@gnzsnz gnzsnz added the enhancement New feature or request label Mar 3, 2024
@gnzsnz
Copy link
Owner

gnzsnz commented Mar 3, 2024

@wesget182 , aarch64 is using an outdated "latest" version. updating latest/Dockerfile IB_GATEWAY_VERSION to latest version might help.

if this fix your issue please let me know. i can update the branch. I have not included this branch on the "master" branch because i don't consider it stable.

@wesget182
Copy link

wesget182 commented Mar 3, 2024 •
edited
Loading

Ahh actually I got everything working just fine. There were some internal issues with my code combined with my confusion regarding the remove Client *clientId* from the logs. Thanks!

@benjaminsingleton
Copy link
Author

Thanks for the pointer @wesget182 @gnzsnz! I was able to successfully build the image, run ib-gateway, and execute trades on my paper trading account. I wouldn't consider this super thorough testing, but it seems like it works as you'd expect. It would be great if you'd consider incorporating this in the master branch, so that a linux/arm64 build is deployed during CI/CD. Thanks again.

@gnzsnz
Copy link
Owner

gnzsnz commented Mar 4, 2024

@benjaminsingleton i'm not planning to merge aarch64 into master. At least not in it's current status, I don't consider it's current status as stable.

I welcome any pull request to improve it's current status, I have made some progress but not enough to merge this into master.

I would like to have:

  • working GUI for ibgateway
  • working TWS

@gnzsnz gnzsnz added the help wanted Extra attention is needed label Apr 21, 2024
@gnzsnz gnzsnz linked a pull request May 2, 2024 that will close this issue
Open
@gnzsnz gnzsnz linked a pull request May 2, 2024 that will close this issue
aarch64 support #102
Open
@gnzsnz gnzsnz mentioned this issue May 18, 2024
Closed
@Amir-Inbar
Copy link

@gnzsnz @benjaminsingleton

Solution Overview:

Port Mapping Adjustment:
The original docker-compose setup specified port mappings with 127.0.0.1 (e.g., 127.0.0.1:4001:4003). This strict IP binding caused issues on Apple Silicon. By changing it to broader exposure (e.g., 4001:4003 without 127.0.0.1), connectivity was resolved, allowing the ib-gateway container to function properly.

Platform Specification in Docker Compose: Adding platform: linux/amd64 to the docker-compose.yml file ensured the container consistently ran in linux/amd64 mode, improving stability on Apple Silicon.

Updated Image Version: Switching to the latest image (ghcr.io/gnzsnz/ib-gateway:latest) also helped as it provided additional compatibility updates.

VNC Configuration: We configured a VNC viewer to connect to the IB Gateway, allowing interaction with its GUI through port 5900. All incoming requests to IB Gateway worked as expected.

Code Example: Here’s the modified docker-compose.yml section:

services:
ib-gateway:
image: ghcr.io/gnzsnz/ib-gateway:latest
platform: linux/amd64
ports:
- '4001:4003'
- '4002:4004'
- '5900:5900'
# Remaining configuration remains the same

image

These changes allowed the container to run smoothly on Apple Silicon (M1/M2) Macs, enabling full functionality of IB Gateway.

Credit for a friend of my @ohadch

@gnzsnz
Copy link
Owner

gnzsnz commented Oct 27, 2024

@Amir-Inbar thanks for sharing this. i believe that you are not running the image in aarch64 architecture, but emulating amd64. which is fine but will not work on other aarch64 architectures like raspi or linux running on M1.

what branch #102 is trying to do is to build an image in aarch64 architecture.

@fuzihaofzh fuzihaofzh mentioned this issue Nov 9, 2024
Closed
@dannykliu
Copy link

@gnzsnz any recommended solutions to get this working on arm64 machines like AWS gravitron instances? Would you consider trying to rebuild the docker image using arm64 or using an emulator like qemu to translate instructions at the architecture level?

@wesget182
Copy link

wesget182 commented Nov 19, 2024 •
edited
Loading

@gnzsnz any recommended solutions to get this working on arm64 machines like AWS gravitron instances? Would you consider trying to rebuild the docker image using arm64 or using an emulator like qemu to translate instructions at the architecture level?

Add this to your docker compose and it should work.

     build:
            context: ./ib_gateway
            platforms:
                - "linux/arm64"

@dannykliu
Copy link

@gnzsnz any recommended solutions to get this working on arm64 machines like AWS gravitron instances? Would you consider trying to rebuild the docker image using arm64 or using an emulator like qemu to translate instructions at the architecture level?

Add this to your docker compose and it should work.

     build:
            context: ./ib_gateway
            platforms:
                - "linux/arm64"

are you locally building off the aarch64 branch? your suggestion on the main branch does not work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

aarch64 support gnzsnz/ib-gateway-docker
5 participants
@benjaminsingleton @gnzsnz @dannykliu @wesget182 @Amir-Inbar