Record of work on debugging the issue with malformed urls for RStudio in Codespaces

[eeholmes]

Another niche question: Codespaces, docker, and Jupyter rsession-server.
On my laptop I can run this Openscapes like docker image and open RStudio via the button in Jupyter Lab
docker run -p 8888:8888 ghcr.io/nmfs-opensci/container-images/py-rocket-base:4.4-3.10
and the RStudio button works. It goes to
http://127.0.0.1:8888/rstudio
But when I run the same docker cmd on a vanilla Codespace (with docker), the url is like so
https://expert-spork-ppr557gx42vw4-8888.app.github.dev/lab and that works fine. I do have to enter the token.
and clicking the RStudio button on the Jupyter Lab launcher goes to
https://expert-spork-ppr557gx42vw4-8888.app.github.dev:8888/rstudio/auth-sign-in?appUri=%2F
changing it to
https://expert-spork-ppr557gx42vw4-8888.app.github.dev/rstudio
just redirects back to the previous url.
So there is something about how rsession-proxy works in a Codespace.
Anyone have some insight about how ression-proxy works or how port forwarding in a Codespace works? (edited)

2 days ago
@Eli Holmes
reported in jupyterhub/jupyter-rsession-proxy#144
#144 RStudio launcher strangeness on Codespaces
Bug description

In Codespace, using jupyter-rsession-proxy to launch RStudio opens a window with a strange-looking URL.
For example,
Jupyter Lab URL: https://bookish-engine-7vjg45pq4xhpw4-8888.app.github.dev/lab
RStudio launched URL: https://bookish-engine-7vjg45pq4xhpw4-8888.app.github.dev:8888/rstudio/auth-sign-in?appUri=%2F
How to reproduce

1. Go to 'https://github.com/dddlab/rstudio/tree/update-jupyter-image'
2. Start Codespaces
3. image
4. Get Jupyter server token with jupyter server list in terminal
   Show more
   Labels
   bug
   Comments
   3
   https://github.com/[jupyterhub/jupyter-rsession-proxy](https://github.com/jupyterhub/jupyter-rsession-proxy)|jupyterhub/jupyter-rsession-proxyjupyterhub/jupyter-rsession-proxy | Nov 29th, 2023 | Added by GitHub

yuvipanda
2 days ago
there's a suggested workaround, if that works for you, report that (also if it does not)

Eli Holmes
2 days ago
wow. I didn't think to look at the ressions-proxy issues

Eli Holmes
2 days ago
That is weird. removing the :8888 twice works.

Eli Holmes
2 days ago
I have
jupyter-rsession-proxy 2.2.1
so version is not the issue. Sadly.

yuvipanda
2 days ago
@eli Holmes
I think jupyterhub/jupyter-rsession-proxy#124 is a fix
#124 Rewrite malformed root url
Fixes #121
Comments
8
https://github.com/jupyterhub/jupyter-rsession-proxy|jupyterhub/jupyter-rsession-proxyjupyterhub/jupyter-rsession-proxy | Mar 17th, 2022 | Added by GitHub

yuvipanda
2 days ago
@eli Holmes
if you can install rsession-proxy from that branch in your image (dynamically installing at runtime won't work), and see if that works...

Eli Holmes
2 days ago
I don't know how to install a python module from github

Eli Holmes
2 days ago
But I can figure that out. I'll report back

Eli Holmes
2 days ago
pip install git+https://github.com/juzdzema/jupyter-rsession-proxy@patch-1

yuvipanda
2 days ago
exactly right!

Eli Holmes
2 days ago
ok, the patch-1 didn't help. I still have to change the url 2x. After that, the button works.

Eli Holmes
2 days ago
Here is what I have to do.
Click the RStudio button and this url appears
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev:8888/rstudio/auth-sign-in?appUri=%2F
Remove the :8888 from the url and reload. That now looks like this
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev/rstudio/auth-sign-in?appUri=%2F
That will now redirect to. Notice the :8888 reappears
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev:8888/rstudio/
Now remove the :8888 again so url looks like
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev/rstudio/
Now that url will work (not redirect) and clicking the button also works and will continue to work.

Eli Holmes
2 days ago
Also I posted the details of the behavior along with my devcontainer.json to jupyterhub/jupyter-rsession-proxy#144

yuvipanda
2 days ago
@Eli Holmes
cool cool 🙂

yuvipanda
2 days ago
I think some variant of that PR should fix this issue

yuvipanda
2 days ago
unfortunately I definitely don't have time to look through that 😞
@carl Boettiger
how can we find people who may have time to help solve things like this?

yuvipanda
2 days ago
like, I'm fairly certain that this has to do with looking at the Origin header and the Host header, and something somewhere is picking the wrong one (one has port one does not) to redirect things to

yuvipanda
2 days ago
but we need people who feel comfortable hacking not on scientific code, but on http things (like this) to be able to do that. idk how to grow that community. it's a slightly different skillset.

Eli Holmes
2 days ago
I am checking out the PR changes to see if I can debug. jupyterhub/jupyter-rsession-proxy@e5bb37d

yuvipanda
2 days ago
\o/

Eli Holmes
2 days ago
BTW, all works fine in Binder.

yuvipanda
2 days ago
@eli Holmes
yeah, i remember doing some fiddling about around host vs origin there at some point in the past for this

yuvipanda
2 days ago
i suspect whatever reverse proxy copilot is up on isn't doing that

Eli Holmes
2 days ago
Another weirdness. For a brief moment
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev/rstudio
appears, then is quickly replaced with
https://crispy-space-memory-q97x6qjp59cx76p-8888.app.github.dev:8888/rstudio/auth-sign-in?appUri=%2F

Eli Holmes
2 days ago
Also you should work on other stuff. I'll putz around and see what I discover.

yuvipanda
2 days ago
❤️ thank you

yuvipanda
2 days ago
it's some fiddling in https://github.com/[jupyterhub/jupyter-rsession-proxy](https://github.com/jupyterhub/jupyter-rsession-proxy)/blob/216d9e5e65f0a16af34cf1bafc418c4daa1e08d6/jupyter_rsession_proxy/__init__.py#L34 for sure - given the help docstring there!
init.py
def rewrite_netloc(response, request):
https://github.com/jupyterhub/jupyter-rsession-proxy|jupyterhub/jupyter-rsession-proxyjupyterhub/jupyter-rsession-proxy | Added by GitHub

Eli Holmes
2 days ago
Update: rsession-proxy v 2.1.0 works fine so happened sometime afterwards.

yuvipanda
2 days ago
@Eli Holmes
so 2.1 was released in sep 7 2022, and jupyterhub/jupyter-rsession-proxy#134 was done sep 13 2022, and touches seemingly relevant bits so I suppose this is a bug introduced by that PR
#134 Fix netloc when rstudio-server inserts port.
Comments
1
https://github.com/jupyterhub/jupyter-rsession-proxy|jupyterhub/jupyter-rsession-proxyjupyterhub/jupyter-rsession-proxy | Sep 13th, 2022 | Added by GitHub

Eli Holmes
2 days ago
That's where I am looking.

yuvipanda
1 day ago
\o/

Eli Holmes
1 day ago
But that PR clearly fixed a number of reverse proxy issues.... jupyterhub/jupyter-rsession-proxy#121

yuvipanda
1 day ago
yeah, and i suppose it introduced some new ones 😄

Eli Holmes
1 day ago
Update: it only occurs in a special case and only after the PR associated with v2.2.0. I am running the Codespaces devcontainer with a Dockerfile with this line at the bottom. It is the same Dockerfile used in my JupyterHub.
EXPOSE 8888

CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--allow-root", "--no-browser"]
Using this image in a Codespace, doesn't automatically (for me at least) start jupyter lab on port 8888. Instead I run this from a terminal
jupyter lab --ip=0.0.0.0 --port=8888 --allow-root --no-browser --NotebookApp.token='' --NotebookApp.password=''
If I use --port=8888 I get the problem. If I use --port=8889, I don't. (edited)

Eli Holmes
1 day ago
@yuvipanda
Thanks for you help on this btw! For the time-being, I am going to call this solved although I don't know why this behavior happens. Changing to port=8889 works.
This is related to the "fledging" problem. I am trying to create "buttons" where you can open up the Openscapes JHub images and spin up the environment that looks just like what you see in the Openscapes JHub. Codespaces is almost there. Still a little clunky since you have to open a Codespace and think click in a weird place to get the Jupyter Lab page to open.