react-native-video and media source extension API

[vuminhkh]

Hello,

I already used reac-native-video to display camera stream from remote URL (generated from Home Assistant backend), it's simple and it works great 👍, but it has some delays in this mode for live streaming. To improve the experience, I'm looking now in the source code of Frigate API which sends chunk of data via websocket and use MediaSource extension API to generate dynamic source for html video element on their web UI. I'm wondering it's there a way to do the same thing with react native video, it seems to me that react native video do not support media source extension api ? Is there a workaround to do the same thing ? Thank you in advance for your help.

[freeboub]

Looks like freegate supports rtsp. I added it 1 or 2 weeks ago (not sure it has been released). Did you try that ?

[freeboub]

Looks like freegate supports rtsp. I added it 1 or 2 weeks ago (not sure it has been released). Did you try that ?

[vuminhkh]

Hello, thanks, I'll check and test your rtsp implementation when it's released, hope that I'll have a way to get the rtsp stream via home assistant frigate integration 👍 I also tried to implement webrtc with react-native-webrtc in H264, it works great on some cameras my doorbell, my cheap wifi cameras, but on my POE annke cameras, webrtc stream freezes every 5 seconds or don't even work at all. That's why I looked into MSE in H265, it looks great, the stream is more compressed and MSE is supported by all browsers, I hope that it will be available on react native video some days.

[freeboub]

@vuminhkh There is another enhancement we can do is to add an "offsetFromLive" to bufferConfig in order to enhance latency:
See for android: https://developer.android.com/reference/kotlin/androidx/media3/exoplayer/source/DefaultMediaSourceFactory#setLiveTargetOffsetMs(long)
For ios: https://developer.apple.com/documentation/avfoundation/avplayeritem/3227883-recommendedtimeoffsetfromlive

Should be quite easy to prototype.

I don't know well WebRTC, but it seems that RTMP can also be used for communications, This is something which can be integrated on android. It seems that rtsp and rtmp are not supported by AVPlayer ...

[freeboub]

@vuminhkh I created : #3792
In order to be able to configure offset from live prop on android.
Let me know if it can fix your issue. I cn also do it on ios !

[vuminhkh]

@freeboub I'm not sure if your commit has anything with MSE extension, I just looked into the new props that you added to react native video, it looks like we can define playback speed / buffer, which is great because this is what's missing for me. When we check cameras recording usually I might event want to fast forward 16x. Anyway, for my little app, now I use react native video to watch recording, for streaming i'm using react native webrtc via TCP (UDP does not work with heavy WebRTC stream). I'll integrate the playback speed into my app to fast forward video recording. Thanks.

[freeboub]

Thank you for the feedback, lmet close the discussion 👍