Problems in a Capacitor app using CapacitorHttp plugin

[posti85]

What do you want to do with Hls.js?

I'm developing an Android hybrid application with Capacitor and using hls.js to play video sources. The application uses CapacitorHttp plugin, which patches fetch and XMLHttpRequest to proxy the webview request to make with the native system. When a video is played, it fails due m3u8 and ts files failed requests.

I'm trying to play the Big Buck Bunny video: https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8

const sourceUrl = 'https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8';
const hls = new Hls();
hls.on(Hls.Events.MEDIA_ATTACHED, () => video.play());
hls.loadSource(sourceUrl);
hls.attachMedia(document.getElementById('video'));

What have you tried so far?

In my desktop in Google Chrome browser (normal execution), the request are:

https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8
https://test-streams.mux.dev/x36xhzz/url_0/193039199_mp4_h264_aac_hd_7.m3u8
https://test-streams.mux.dev/x36xhzz/url_0/url_462/193039199_mp4_h264_aac_hd_7.ts
...

When execute that code in the Android device, the first request to the m3u8 file is made successfully. The webview makes a local request which proxies to a native system HTTP request:

https://localhost/_capacitor_http_interceptor_?u=https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8

The problem is that the following request hls.js does is:

https://localhost/_capacitor_http_interceptor_?u=https://localhost/url_0/193039199_mp4_h264_aac_hd_7.m3u8

note that the query param refers to https://localhost/ (insead of https://test-streams.mux.dev/x36xhzz/, which should be the valid one).

I was able to fix it manipulating the url before the request is made (I let in comments the first url modifications to clarify):

let m3u8Path;
const hls = new Hls({
    xhrSetup: (xhr, url) => {
      if (url.startsWith('https://localhost/')) {
        let fixedUrl = url;

        if (fixedUrl.endsWith('.m3u8')) {
          //      url: https://localhost/url_0/193039199_mp4_h264_aac_hd_7.m3u8
          // fixedUrl: https://test-streams.mux.dev/x36xhzz/url_0/193039199_mp4_h264_aac_hd_7.m3u8
          fixedUrl = url.replace('https://localhost/', sourceUrl.replace(/[^\/]+?$/, ''))
          // Save the last downloaded m3u8 file location: https://test-streams.mux.dev/x36xhzz/url_0/
          m3u8Path = fixedUrl.replace(/[^\/]+?$/, '');
        } else if (fixedUrl.endsWith('.ts')) {
          //      url: https://localhost/url_462/193039199_mp4_h264_aac_hd_7.ts (note url_0/ is missing!)
          // fixedUrl: https://test-streams.mux.dev/x36xhzz/url_0/url_462/193039199_mp4_h264_aac_hd_7.ts
          fixedUrl = m3u8Path + url.replace('https://localhost/', '')
        }

        xhr.open('GET', fixedUrl, true);
      }
    }
  });

The video plays are first, but a few seconds later it ends showing weird frames.

I understand hls.js makes the requests 'based' on the first one, which was to https://localhost/. Is there any way to prevent this behaviour? are there any option to set that base url manually or similar...?

[robwalch]

I understand hls.js makes the requests 'based' on the first one, which was to https://localhost/. Is there any way to prevent this behaviour? are there any option to set that base url manually or similar...?

HLS.js uses the response URL as the base URL when resolving relative URLs in HLS playlists:

The base for media playlists is the response URL from the parent multi-variant playlist:

hls.js/src/loader/playlist-loader.ts

Lines 363 to 365 in bf0180c

	const url = getResponseUrl(response, context);
	const parsedResult = M3U8Parser.parseMasterPlaylist(string, url);

The base for media segments is the response URL from the parent media playlist:

hls.js/src/loader/playlist-loader.ts

Lines 454 to 463 in bf0180c

	const url = getResponseUrl(response, context);
	const levelId = Number.isFinite(level as number)
	? (level as number)
	: Number.isFinite(id as number)
	? (id as number)
	: 0;
	const levelType = mapContextToLevelType(context);
	const levelDetails: LevelDetails = M3U8Parser.parseLevelPlaylist(
	response.data as string,
	url,

This is the method that gets the response URL. There is no way to currently override it. We could accept a PR that adds an option to do this. The alternative would be to customize the loader so that you rewrite the response URL to match the context URL on complete.

hls.js/src/loader/playlist-loader.ts

Lines 51 to 62 in bf0180c

	function getResponseUrl(
	response: LoaderResponse,
	context: PlaylistLoaderContext,
	): string {
	let url = response.url;
	// responseURL not supported on some browsers (it is used to detect URL redirection)
	// data-uri mode also not supported (but no need to detect redirection)
	if (url === undefined || url.indexOf('data:') === 0) {
	// fallback to initial URL
	url = context.url;
	}
	return url;

I was able to fix it manipulating the url before the request is made (I let in comments the first url modifications to clarify):

The issue is that segment URLs in the media playlist are relative to their parent media playlist, not the multi-variant playlist.

[posti85]

Thank you very much @robwalch. Based on your suggestion:

The alternative would be to customize the loader so that you rewrite the response URL to match the context URL on complete.

I made a custom loader that replaces the response.url with the context.url, which is the correct one:

class ResponseUrlFixLoader extends Hls.DefaultConfig.loader {
  load(context, config, callbacks) {
    const originalSuccess = callbacks.onSuccess;

    callbacks.onSuccess = (response, stats, context, networkDetails) => {
      response.url = context.url;
      originalSuccess(response, stats, context, networkDetails);
    };

    super.load(context, config, callbacks);
  }
}

const hls = new Hls({
  loader: ResponseUrlFixLoader
});

And now the video plays! I hope it is usefull for someone else.

But I have found a new problem. I don't know if it's related to the fact of executing hls.js in a webview... or it might be related to other issue I should open.

The video plays well for 3 seconds but, after that, the video does like a zoom and only a top left region is shown (the red border belongs to the video tag):

The full frame of that scene:

What could be happening?

Edit: I was testing in an Android Emulator. I have tested the same in a real device and there it works! Maybe an issue related with the emulator hardware acceleration?

[robwalch]

What could be happening?

Have you looked at the page layout?

An HTMLMediaElement will resize according to the resolution of rendered bitrate variants, unless it and its parent containers are constrained using CSS.