DifferentialEquations.jl v8

[ChrisRackauckas]

As a bit of an extension to the discussion in #1082 (comment), specifically on the DifferentialEquations.jl changes, we need to consider what we want DiffEq to look like. We can do this separately from anything else as just a v8 so... what should it be? I'm thinking the steps are:

• Move automated algorithm selection for DDEs to DelayDiffEq.jl
• Move automated algorithm selection of DAEs and DiscreteProblem to OrdinaryDiffEq.jl
• Move automated algorithm selection of SDEs and RODEs to StochasticDiffEq.jl
• Move automated algorithm selection of SteadyStateProblems to NonlinearSolve
• Move the default for BVPs to BoundaryValueDiffEq
• Fully remove Sundials, which was set to not re-export a few years ago already https://github.com/SciML/DifferentialEquations.jl/blob/master/src/DifferentialEquations.jl#L13

Those ones are clear. But now I think we have some choices to make. Here's one path I see:

Specialize DifferentialEquations.jl to just be ODEs, so:

• Make it just reexport OrdinaryDiffEq.jl
• Update each of the docs pages like https://docs.sciml.ai/DiffEqDocs/stable/tutorials/sde_example/ to tell people to grab the relevant solver package
• Remove the deps and importing on anything else
• Add specialized error hinting and warnings so if you build an SDEProblem it tells you to add StochasticDiffEq.jl if it's not loaded? (Maybe add this to SciMLBase itself?

So in that way, the whole system is still documented together as a cohesive set of differential equation solvers, but people are pushed to just get OrdinaryDiffEq.jl. This would wildly improve loading times for lots of people, since the vast majority just instinctively grab DifferentialEquations.jl because "it's for ODEs". This of course has drawbacks of forcing anyone beyond the simple ODEs to have to know to grab subpackages. Is that okay with a few helpers?

This really is then a question of what we think in 2025 the identity of DifferentialEquations.jl should be. I think with the package splitting and such people "in the know" has largely started to default to just grabbing specific OrdinaryDiffEq solvers, as they should. And people "in the half know" have largely started to just use OrdinaryDiffEq.jl directly. So what's the point of DifferentialEquations.jl? Is it giving an unnecessarily difficult loading time experience to new users given that no experienced user gravitates to it anymore? So should it and OrdinaryDiffEq.jl be effectively the same repo, and the rest is a documentation problem of how to make it look and feel cohesive in a world where these things are truly separated packages? Or is there still a role for newbies with the all batteries included form? Or is it a trap with too many batteries?

I'd like to get opinions from others, but at this point I think I'd lean towards making DiffEq v8 effectively be the same as OrdinaryDiffEq. It's OrdinaryDiffEq that's a bit redundant and not needed, since you either want to choose solvers or choose the easy thing, and the easy thing "should" be using DifferentialEquations. But I'd like second opinions since this is really nothing about a technical choice but more of how it should look and feel.

[termi-official]

This really is then a question of what we think in 2025 the identity of DifferentialEquations.jl should be.

I also struggled with this quite a bit in the past. Are there actually use cases where we need all the solvers under the sun loaded at once (other than compat testing)?

Another option which I would like to throw in is that DifferentialEquations.jl v8 could notify the user on package init that they should use the respective solver packages directly and effectively deprecate the full package.

[TorkelE]

I have moved towards never do using DifferentialEquations, and call the separate packages. Even if I quite frequently do using OrdinaryDiffEq, StochasticDiffEq, JumpPrcoesses, I don't mind it in this case as each have a very specific role.

Practically, we have purged using DifferentialEquations from Catalyst. So for a start just removing any cases of using DifferentialEquations from all docs should, long-term take us there. But then, some kind of deprications/warnings/just making DifferentialEquations=OrdinaryDiffEq might help force the change on the (considerable) userbase that exists.

[ChrisRackauckas]

But then, some kind of deprications/warnings/just making DifferentialEquations=OrdinaryDiffEq might help force the change on the (considerable) userbase that exists.

Yeah I think that's what I'm getting at. If everyone is doing this, then why not just make DifferentialEquations.jl be "the right thing" for people to gravitate towards? I don't think it's serving a purpose in its current form other than being a unified documentation source. What source should it serve? If it should be "the thing we point noobs to", then it should basically be OrdinaryDiffEqDefault with a few chosen algorithms and some error messages helping people find more things? It would lose the batteries-included feel, but it doesn't seem like that's the direction things are going anyways.

Are there actually use cases where we need all the solvers under the sun loaded at once (other than compat testing)?

Funnily enough, I'm probably the only one who needs that 😅. Testing and benchmarking.

Another option which I would like to throw in is that DifferentialEquations.jl v8 could notify the user on package init that they should use the respective solver packages directly and effectively deprecate the full package.

It could do that, point people to OrdinaryDiffEq.jl. But it already has the branding and the mindshare, so using the name for something useful seems like a good idea, but then yes it doubles with OrdinaryDiffEq...

[jonniediegelman]

Going from DifferentialEquations to OrdinaryDiffEq+DiffEqCallbacks to OrdinaryDiffEqLowOrderRK+DiffEqCallbacks has been such a huge quality of life improvement at each step. But I do feel like pointing new people towards more granular package loading like that can make things feel overwhelming.

I often wish for a more "batteries included" version of OrdinaryDiffEq that included default algorithm choice and the callback library so I could recommend that to people. But maybe that's also kinda my bias coming from the control systems world where periodic and other discrete callbacks are "obviously something everyone needs". To other disciplines the "batteries included" ODE solver package might include boundary-value problems and never need discrete callbacks.

I think one downside about moving toward more granular packages by default is that people won't seek out the comprehensive DifferentialEquations docs if they only see OrdinaryDiffEq or more granular packages loaded. I personally gained so much knowledge from those docs just by seeing something interesting in the side bar and clicking it. Like I had no idea stochastic differential equation solvers were even a thing. Or that I could take advantage of IMEX solvers when I have a huge system but only a small part of it is stiff so full implicit solvers were out of the question (that one's something I've actually used after I learned about it!).

[affans]

I don't have much to add to this discussion, but will comment that having end users load granular packages may make it a barrier to the overall Julia ecosystem. I believe, like Chris said, that to someone unfamiliar with the ecosystem using DifferentialEquations would be the bonafide package to solve, for example, system of first order differential equations. I, myself, would simply introduce DifferentialEquations to students who I want to push towards Julia.

I think the point I am trying to make is that if there is a possible solution of just 'using DifferentialEquations` then that's a better option from the lens of new users and non-CS folks to engage. If there is a way to simply load the required granular packages automatically, that would be ideal. Maybe it already does that. I've been away from the ecosystem for a few months/years now and actually just starting to get back into it this morning.

[isaacsas]

I think there is a difference in having unified docs that explain the broader ecosystem of packages, and how to use them in combination, vs. having DifferentialEquations.jl as a meta-package. Keeping the current docs that link many different components together, and shows their commonalities, would be nice. But I don't think that actually requires DifferentialEquations.jl to persist.

[kapple19]

FWIW, I would use DiffEq to check what automated solver would be chosen, then remove DiffEq and add OrdinaryDiffEq and explicitly call the solver to avoid the large DiffEq dependency.

And then after the solvers were split to their own packages, I enjoyed the smaller dependency for each solver.

My preference would be a lightweight dependency containing the automated OrdinaryDiffEq solver alg selector alg (I haven't toyed with this ecosystem for a while so I don't know if the automated alg selector has already been moved to OrdinaryDiffEq, but it seems to be, based on your discussion notes).

[Datseris]

My usage, both in research / day-job and in DynamicalSystems.jl is as follows: only solver-specific (sub)packages are ever used. With StochasticDiffEq.jl that is currently not an option so the whole suite is used as conditional dependency (the Pkg extension system) and directly as a dep in research.

Specialize DifferentialEquations.jl to just be ODEs, so:

But then what is the purpose of existence of both DifferentialEquations.jl and OrdinaryDiffeq.jl instead? I mean, why should two packages exist? Is it just for the purpose of deprecation and ease of transition? And the package would be outright deprecated fully in the subsequent major release? I would find it confusing as a newcomer for two packages to exist and have identical functionality. But I understand there is no simple way out. Besides, there is another important aspect: GitHub stars. Are they a strong metric of a package popularity? I am not sure. For a package that is so widely used as DifferentialEquations.jl, probably most of the users don't star it; likely most of them don't even know much about GitHub...?

I agree with most other comments on the automatic algorithm selections: they should be in the top-level specific packages like OrdinaryDiffEq.jl.

---

Can I ask what is the downside with having DifferentialEquations.jl be just a large re-exporter? No functionality, but does re-export all Delay/Stochastic/Ordinary diff eq + callbacks + anything that can be "clearly enough" differential equations. I personally like the existence of such big/central packages. DynamicalSystems.jl does this: it is just a re-exporter.

[TorkelE]

I think the argument against having DifferentialEquations.jl just exporting everything is that many people (likely for legacy reasons, or because of unfamiliarity) either have had it as the go-to package throughout the years, or just have heard of it, grabs it, it works, and they use that. In reality, these people should probably just use OrdinaryDiffEq or OrdinaryDiffEqDefault instead, and now expeirance way worse load times than they should.

If everyone had perfect information, having the current system would have been fine, but the question is if having the current version of DifferentialEquations.jl around unintentionally is detrimental to many people. Some kind of light deprication of it would quickly sort this out. The alternative is ensure everything is correctly documented everywhere and hope it sorts itself out with enough time and information

[isaacsas]

Related to the Github star issue that @Datseris mentions, is the paper citation issue. The DifferentialEquations.jl paper is (I assume) the most widely-cited SciML paper. Retiring the package may have an impact on citations to that paper (even if people using OrdinaryDiffEq.jl are told to cite that paper). I only mention this because such metrics can be useful when seeking funding, so we wouldn't want to take an action that could give the impression that use of the SciML ODE solvers is diminishing.

[Datseris]

I think the argument against having DifferentialEquations.jl just exporting everything is that many people (likely for legacy reasons, or because of unfamiliarity) either have had it as the go-to package throughout the years, or just have heard of it, grabs it, it works, and they use that. In reality, these people should probably just use OrdinaryDiffEq or OrdinaryDiffEqDefault instead, and now expeirance way worse load times than they should.

To me this sounds more like a communication problem than an infrastructure problem. Yet here we are discussing infrastructure changes, or at least this is where the major focus is.

My vote would be therefore to make sure there is a clear communication that DifferentialEquations.jl is a large re-exproter early on in the docs and in a central enough place. The tutorials can also show this by using the subpackages. For example, even though in my work I always just use DynamicalSystems directly, and I advertise the package as a whole always, all tutorials for the dedicated functionalities only use the subpackages (e.g., https://juliadynamics.github.io/DynamicalSystemsDocs.jl/attractors/stable/tutorial/#Tutorial-copy-pasteable-version ) .

Related to the Github star issue that @Datseris mentions, is the paper citation issue. The DifferentialEquations.jl paper is (I assume) the most widely-cited SciML paper. Retiring the package may have an impact on citations to that paper (even if people using OrdinaryDiffEq.jl are told to cite that paper). I only mention this because such metrics can be useful when seeking funding, so we wouldn't want to take an action that could give the impression that use of the SciML ODE solvers is diminishing.

Yeah, this is a problem as well. There is also a second nature: there comes a point where the users want to have more control over what they cite. If someone is using only delay differential equations, do they cite DifferentialEquations.jl or only DelayDiffEq.jl ? (I am using this example because I know it has a dedicated paper)

This is not clear to me and I am facing similar difficulties with DynamicalSYstems.jl. Personally I would prefer if DynamicalSystems.jl is always cited, and the same with DIfferentialEquations.jl, because having 1 paper with 1000 citations is better than 10 papers with 100 citations each. I understand that a user may not think like that... I am also not sure how to "force" or "nudge" a user to cite in a particular way, besides a statement in the docs.

(edit: ideally you want the user to cite both DifferentialEquations.jl and DelayDiffEq.jl, so I guess the focus should be how to achieve that)