(Will PR) Make native code super thin?

[fzyzcjy]

Hi thanks for the helpful package! I wonder whether it is possible to make native code super thin? Currently it is already not heavy, but bugs are still there (e.g. #1523) for the glue code.

Just a naive thought: Is it possible that, we move almost all logic from native side to Dart side. The native side only remains simplest primitive things.

Direct bugs that it can eliminiate:

• For example, when audio has finished playing, the onCompletion currently has some code on native side, and causes both fix(android): onPlayerStateChanged is not called when audio has completed playing #1523 bug
• ... and (Will PR) MediaPlayer on Android is *not* thread safe, but it is used in multiple threads, causing subtle bugs #1524 bug.
• The (Will PR) Android WrappedPlayer has concurrency bugs, since it is called in multiple threads, has shared fields, but without synchronizations #1525 can also disappear automatically if we move all logic to Dart (since Dart is single-threaded and no need to worry about synchronization).
• In addition, this can make state duplication less frequently happen. As we know when doing state management, if we are having a state that is duplicated multiple times, then it has a chance that multi replicas are not synchronized. For example, I did observe that, sometimes I tapped "pause" button, and audioplayers says it is paused, but the audio is still playing.

By the way, even if Dart->Java call is async, we can still ensure commands are executed serially, by ensuring one command is finished before sending second command.

Cons:

• Code latency will be larger. For example, originally the onCompletion will cause several operations on MediaPlayer by direct android code execution, but now it is implemented by back-and-forth between Dart and Android. This may or may not be a problem for low-latency scenarios. For example, if the critical path is not affected by this, then it is OK. We can also alleviate this by allowing one Dart-to-native call to execute multiple things.

[Gustl22]

Yes the thread stuff currently is a problem in Android. But don't know if it's worth it to use e.g. dart:ffi to interact with the native side. As then probably all modules need a rewrite and some parts are very different on each platform. I also wonder why we use different threads for Android in the first place. This may could be improved in the exoplayer rewrite without the need of using ffi.

[fzyzcjy]

But don't know if it's worth it to use e.g. dart:ffi to interact with the native side.

I am also not sure. Btw my proposal above suggests using the current way (or use pigeon if you like - but still the same way).

As then probably all modules need a rewrite and some parts are very different on each platform

Hopefully a "move" (from native to dart), not a "rewrite" ;)

I also wonder why we use different threads for Android in the first place

IMHO it may be caused un-intentionally.

1. most functions are called by Flutter channels, thus run on thread of flutter channels
2. MediaPlayer callbacks are called on main thread (b/c MediaPlayer cannot find a Looper on the flutter channel threads)

thus there are two threads.

[Gustl22]

I also thought about moving the continuous position update logic to the dart side, but then we would need to request the position every like 200ms (whose interval we also could control from the dart side) and I think it may would be delayed until it reaches the app, in contrast to pushing it from the native side (one call less). But that's something one needs to test.

We also need to define which components exactly can be moved to the dart side without loosing functionality on any platform. Each platform has a different API for their native players which probably can't run unified in dart code. All common elements may are already represented in the AudioPlayer logic.

[fzyzcjy]

I also thought about moving the continuous position update logic to the dart side

+1

but then we would need to request the position every like 200ms (whose interval we also could control from the dart side) and I think it may would be delayed until it reaches the app, in contrast to pushing it from the native side (one call less). But that's something one needs to test.

I have done some experiments (not about this specific API though) calling a function from Dart to native. I printed the timestamps to see the time. This may not be perfect, and looking forward to your experiments as well. The numbers are:

• dart starts call -> kotlin receives call: 1ms
• kotlin ends call -> dart recieves the end of call: 3-4ms

[fzyzcjy]

We also need to define which components exactly can be moved to the dart side without loosing functionality on any platform. Each platform has a different API for their native players which probably can't run unified in dart code. All common elements may are already represented in the AudioPlayer logic.

I guess we can write some different Dart code for each platform, just like how we write different native code today.

[Gustl22]

• dart starts call -> kotlin receives call: 1ms
• kotlin ends call -> dart recieves the end of call: 3-4ms

Do you also know the time from "dart starts call to -> kotlin... -> dart receives result", or is this meant by adding these two together?

I guess we can write some different Dart code for each platform, just like how we write different native code today.

This approach would need to change the general interface logic, as currently we have no dart component in the different platform packages, in contrast to like it's done in video_player. This also would mean to add much redundant code in each platform implementation on the dart side. But never say no, maybe it's worth it, but honestly I don't think so...

[fzyzcjy]

Do you also know the time from "dart starts call to -> kotlin... -> dart receives result", or is this meant by adding these two together?

What I measured was some long kotlin operations, so I do not report it here since it is unrelated to our scenario here. Yes, the total overhead should add the two IMHO.

This approach would need to change the general interface logic, as currently we have no dart component in the different platform packages, in contrast to like it's done in video_player. This also would mean to add much redundant code in each platform implementation on the dart side. But never say no, maybe it's worth it, but honestly I don't think so...

I see. Anyway I may try that approach :) I am using pigeon so the boilerplate seems to be minimal.

[fzyzcjy]

To be honest, I implemented what I described above today after posting and discussing here (and it also helps a bit for observations like #1524 (comment)). My implementation seems to be working up to now, so I want to say this may be a viable approach :) If you need any more details, feel free to ping me!

[Gustl22]

@fzyzcjy you can post your solution or make a PR to better see what you have done to achieve this and if it's applicable for the whole library, if you're interested :)

[fzyzcjy]

Sure! Half a month has passed and I have been moving to work on other parts, but I will try to find a bit of time open-sourcing my code.

The overall idea is very simple: Just do almost nothing on the native side. For example, the start() in Android will do nothing but a one-line exoplayer.start().

[Gustl22]

I understand the principle. I'm not sure, but I think this needs a separate package / interface to be handled, and due to our capacities I don't think we can contribute much to make it working on every platform. This must be pushed by the community :)

[jwinarske]

What about adopting pigeon? It should be similar to avfoundation video_player. It might be good to standardize, and it would decrease tech debt down the line. I have some time mid January.