Making a Dart interpreter inside official dart repo?

[fzyzcjy]

Hi, this project looks great and looks like a big amount of work! Excited to see it is still growing, and looking forward to seeing it in production :)

If I understand correctly, this looks like a Dart interpreter. Is it possible that we directly contribute to the dartlang repo and create an interpreter there? They use C++ so I guess the interpretation may run faster since we can control more low-level things.

Given that Dart already have JIT and AOT, adding an interpreted mode looks like something that will be accepted there.

Bytedance maintain a fork of Dart that does interpretation, for a similar reason; one could use that (I think it's public?). (by hixie in flutter/flutter#14330 (comment)

So technically speaking, it should also be implementable.

I really want to see hot-update become reality. That's why I am suggesting this. Because I am quite afraid many people will not choose an interpreter that is not in the dart repository.

Another advantage is that, we can utilize all infra of dart sdk internals. There must be a lot of optimizations etc already there that we may even be able to reuse :)

[ethanblake4]

Thanks for the kind words!

I'd love an interpreter in official Dart, but I have a few reservations about it:

1. Adding an interpreter to Dart means that the Dart team has to commit to supporting it, which they have expressed quite a few times that they do not want to do. If they change their mind on this, I do think we could reconsider.
2. From a technical standpoint, a typical Dart 'interpreter' would have an essentially self-contained runtime that runs the entire app. Eg, the interpreter is running not only your app code, but also all of the code in the Flutter framework, the code in dart:math and dart:core, etc. This might sound obvious because this is what all interpreters/VMs/JITs do, except that dart_eval does not do this, nor would you want this. For hot updates, ideally we want to run as much code in AOT as possible, and only having the updates be interpreted, to keep good performance. This could actually be a very hard problem, adding it might be a lot more work than just using the JIT version of Dart with the JIT part turned off (which is what I'd assume the Bytedance fork is doing). It also might be fairly easy, I can't say for sure, but I have already done the work of this part in dart_eval.
3. Although, as you say, the Dart compiler has a lot of optimizations that are useful, the Dart compiler is also huge and slow. Even running an app with the Dart JIT usually takes a second or two to start. This means that it would probably not be able to directly evaluate Dart code (like eval('2 + 2')), instead you would have to precompile to kernel snapshots (bytecode) which limits utility, those are also really large. dart_eval is also best if you precompile to bytecode, but because it is designed as a runtime solution the compiler is still really fast and doing something like eval('2 + 2') is possible. Also, dart_eval bytecode is much smaller because it only compiles your app code not Flutter and the core libs (same as point 2). It's questionable to me whether a native interpreter would even be much faster, because so much time in dart_eval is spent just transferring data back and forth between 'real Dart' and dart_eval that unless you are just doing math computations the speed hugely depends on how good the bridging implementation is.
4. IMO, iOS app store submission would be a lot more dicey with a C++-based interpreter (even though technically it might achieve the same thing)

Let me know what you think. I do think convincing the Dart team is probably the most insurmountable obstacle here :p

[fzyzcjy]

Thanks for the reply!

which they have expressed quite a few times that they do not want to do
I do think convincing the Dart team is probably the most insurmountable obstacle here :p

Could you please point out some discussions? Then I can see why they refuse.

I agree convincing them may be really hard :( But anyway I should see history discussions first to know why they refuse

interpreter ... that runs the entire app. Eg, the interpreter is running not only your app code, but also all of the code in the Flutter framework, the code in dart:math and dart:core, etc

Not sure about that. Indeed, when I am talking about Dart JIT on Android, I am still thinking it only runs app code in JIT mode (or even only changed app code) while flutter framework in AOT mode.

using the JIT version of Dart with the JIT part turned off (which is what I'd assume the Bytedance fork is doing).

Ah interesting solution - why I did not realize this! Then there is already interpreter inside official dart?

Btw did you check Bytedance fork? I have not found it and suspect it is not open source.

like eval('2 + 2')

That's one place where dart_eval is very helpful :)

It's questionable to me whether a native interpreter would even be much faster, because so much time in dart_eval is spent just transferring data back and forth between 'real Dart' and dart_eval that unless you are just doing math computations the speed hugely depends on how good the bridging implementation is.

Hmm good question. Originally when I looked at the implementation, I thought it was very slow, because, for example, the stack is implemented as a Dart list. Since Dart list is so high level I originally thought that is the source of slowness.

Could you please share a bit more insights about where the 12x slowness comes from? Maybe I can have some idea about optimizations.

iOS app store submission would be a lot more dicey with a C++-based interpreter (even though technically it might achieve the same thing)

Not sure about that, since React Native uses a C++ interpreter (the js interpreter) but no problem.

[fzyzcjy]

I am also worried about quality: The repo itself looks quite high quality already. However, if it has one bug in the whole repo, then someone will face big problem when using in real-world production app, because you know, a real-world app can be quite huge so touches each and every edge case. #59 may help somehow but I am not sure.

Anyway, good job!

[ethanblake4]

Thanks for your comments, these are definitely good questions :)

Could you please point out some discussions? Then I can see why they refuse.

I am mostly just referring to the ones in flutter/flutter#14330.

when I am talking about Dart JIT on Android, I am still thinking it only runs app code in JIT mode (or even only changed app code) while flutter framework in AOT mode.

Yes, this is ideal. But it may be very difficult - I will descibe this more below

Then there is already interpreter inside official dart?

I'd assume so :) I think most JITs have an interpreter component.

did you check Bytedance fork? I have not found it and suspect it is not open source.

No, I think you are right. I can't find it.

Could you please share a bit more insights about where the 12x slowness comes from?

Ok, I'm going to also describe here why it is hard to run app code in JIT and Flutter framework AOT.

First, a simple example:

var k = 1;
for (var i = 0; i < 100000; i++) {
  k = k * k + 1;
}

For this simple code, you are exactly right, that dart_eval is slower because of stuff like

The stack is implemented as a Dart list

In this case, an official Dart interpreter in C++ is easy to make, and would definitely be probably ~3-5x faster.

But now Flutter:

Wiget build(BuildContext context) {
  return Scaffold(
    appBar: AppBar(title: Text(widget.title)),
    body: Center(
      child: Column(
        mainAxisAlignment: MainAxisAlignment.center,
        crossAxisAlignment: CrossAxisAlignment.center,
        children: <Widget>[
          const Text('You have pushed the button this many times:'),
          Text('$_counter', style: Theme.of(context).textTheme.headline4),
        ],
      )
    );
}

Actually here, the speed of the stack and bytecodes etc doesn't even matter that much because it is a very tiny portion of the overall speed. Most of the slowness of running this code is because every time it needs to create one of the Widgets, it must call out from the interpreter loop into a bridged Dart function that creates the widget, wraps it in a data structure ($Value) that dart_eval can use, and returns back to the interpreter. It is kind of like dart:ffi.

This is also (potentially) hard to make in an official Dart interpreter. Effectively, now every function and class is a potential VM entrypoint (like main()). The compiler now has to decide where to put bridge boundaries, which isn't something Dart can currently do. In addition, Dart now has to decide whether to forego typechecks at these boundaries (risking a segfault), add typechecks everywhere (slowing down performance even for users not using bridging), or generate extra code that does typechecks only when bridging (making JIT even slower and increasing app size). Depending how it's implemented it may also not be faster than dart_eval - dart_eval has a pretty optimized implementation of this specific use case, and as previously mentioned the speed of bytecodes and stack etc is basically irrelevant for this. Hope that helped explain it.

Not sure about that, since React Native uses a C++ interpreter (the js interpreter) but no problem.

A little different because React is using JSCore which is built in to iOS I think...

[fzyzcjy]

You are right, that looks like a problem.

The compiler now has to decide where to put bridge boundaries, which isn't something Dart can currently do

For this concrete problem I may have two cents: Maybe let the user (e.g. Flutter framework, or end user) choose it. For example, we may define it as "all framework code are AOT and put bridge around it". So Dart compiler do not need to make decision at all.

Actually here, the speed of the stack and bytecodes etc doesn't even matter that much because it is a very tiny portion of the overall speed. Most of the slowness of running this code is because every time it needs to create one of the Widgets, it must call out from the interpreter loop into a bridged Dart function that creates the widget, wraps it in a data structure ($Value) that dart_eval can use, and returns back to the interpreter. It is kind of like dart:ffi.

I may look into the details and see whether there are some potential improveable locations, if you are interested (#61).

For example, not sure what "wrap" is (not check code yet), but if possible, we should avoid a wrapping, since that generate more objects, i.e. more indirection (slower) and more garbage (gc slower).

Not sure whether I have mentioned before - another workaround is that, for computationlly heavy things, users can choose to make it AOT, then done. No worries about this slowness.

[fzyzcjy]

Btw could you please share some profiling data about your previous experiments? Would be great to see where a real app janks by looking at real data (e.g. line 10 is slow).

[ethanblake4]

Yeah, I will do this at some point. But the reality is that for now, the speed of dart_eval is not really a problem, because it doesn't support enough features to make such complex apps that actually stress its performance. Once it has enough feature support then this will start to become a problem again.

[fzyzcjy]

the speed of dart_eval is not really a problem, because it doesn't support enough features to make such complex apps that actually stress its performance

I see. Look forward to the progress!