Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create process for adding MIME-types to mapping. #2028

Open
lrhn opened this issue Mar 5, 2025 · 5 comments
Open

Create process for adding MIME-types to mapping. #2028

lrhn opened this issue Mar 5, 2025 · 5 comments

Comments

@lrhn
Copy link
Member

lrhn commented Mar 5, 2025

This package has a big map of mappings from extensions to mime-types, and reverse.

There is no policy for updating this mapping, which leads to numerours requests for extensions to be added
(probably around half of: https://github.com/dart-lang/tools/issues?q=is%3Aissue%20state%3Aopen%20label%3Apackage%3Amime).

This package should do one of three things:

  • Decide on a source of truth for mappings (maybe some IETF standard if there is one), follow that, and have regular updates if the standard changes.
  • Don't rely on an outside source, and create a process for adding extensions to the map, or rejecting requests to add extensions, in a predictable and consistent way. Then all these issues can start using that process to get their extensions added.
  • Add a way for users to override/add to the default mapping. Preferably without mutating global state (I would suggest a Zone-based way to add or override mappings, which won't clobber other code doing the same thing.)

See #2027 for a request to simly make the mapping mutable. I would not go that way.

(For the third option, I'd probably make some kind of a MimeRegistry class that you can configure, and install as the default lookup for code that it runs in a zone. There are many ways to make it configurable, and I don't mind if it's mutable, as long as it's whoever creates it that ops in to that.)

@devoncarew

@devoncarew
Copy link
Member

Decide on a source of truth for mappings (maybe some IETF standard if there is one), follow that, and have regular updates if the standard changes.

This looks reasonably canonical: https://www.iana.org/assignments/media-types/media-types.xhtml

Don't rely on an outside source, and create a process for adding extensions to the map, or rejecting requests to add extensions, in a predictable and consistent way. Then all these issues can start using that process to get their extensions added.

Here's another mime handler library that has a documented change process (what they will / won't do): https://docs.pantheon.io/mime-types; having a clear process ourselves makes sense.

Add a way for users to override/add to the default mapping. Preferably without mutating global state (I would suggest a Zone-based way to add or override mappings, which won't clobber other code doing the same thing.)

I'm not sure we need the sophistication of zones here. Introducing a MimeRegistry abstraction makes sense. MimeRegistry itself could be immutable, but could allow user extensions at creation time. And perhaps we have a single global instance people could use if they didn't want to create a new registry themselves.

We could do that in a 2.1.0 release (we're currently at 2.0.0) and deprecate the older API (lookupMimeType), or, deprecate in a new 3.0 release, whichever we think is less disruptive.

FYI, 325 packages depend on package:mime.

@lrhn
Copy link
Member Author

lrhn commented Mar 5, 2025

One issue is that the IANA list is icky mine types, not a napping to file extensions. I'm not sure there is a canonical mapping.
The point of using media types is to be unique, and file extensions are not. Media types are more like a replacement for extensions, as metadata that is not embedded in the file name.
(Which makes sense for data being shipped on the net, not stuck on a disc, and which may not even have a file name.)

Guessing a media type from a file extension is just a guess. It would be better to actually investigate the file contents. Also a job for a dedicated tool, not a small Dart library.

We can choose some existing comprehensive extension list, I think the Apache web server has one (... and there it was).
Android has another collection, which appears to be taken from Debian's media-types package originally.
I think using someone else's list, and saying no to any further additions, is a better strategy better than trying to maintain yet-another-list ourselves.
(It might be a big list, though.)

Or we could get out of the mapping-extensions-to-media-types game entirely. It does seem like something people want to do, though.

@devoncarew
Copy link
Member

I think using someone else's list, and saying no to any further additions, is a better strategy better than trying to maintain yet-another-list ourselves.

👍 That sounds great.

I did create a local branch to download and create a new mapping w/ the IANA list, but like you said above there were too many mime types.

@lrhn
Copy link
Member Author

lrhn commented Mar 6, 2025

Yet aother option is to be a vetted subset of another large list.
If something is not on that list, we won't add it either.
If it is, and we don't have it yet, we can add it if someone needs it.

That avoids including everything up front, but also makes it easier to say yes or no to requests.

@devoncarew
Copy link
Member

If we take the link above: https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types

and generate a new mapping from that, we see:

added:
  application/vnd.dart
  application/vnd.geogebra.slides
  audio/x-aac
  font/collection
  font/otf
  font/ttf
  font/woff
  image/jxl
  image/vnd.dvb.subtitle
  video/mp2t

removed:
  audio/aac
  application/dicom
  model/gltf-binary
  model/gltf+json
  image/heic
  image/heif
  model/vnd.mts
  application/x-font-otf
  application/toml
  application/x-font-ttf
  application/manifest+json
  application/x-font-woff

changed:
  aac: audio/aac => audio/x-aac
  mts: model/vnd.mts => video/mp2t
  otf: application/x-font-otf => font/otf
  ttc: application/x-font-ttf => font/collection
  ttf: application/x-font-ttf => font/ttf
  wmz: application/x-ms-wmz => application/x-msmetafile
  woff: application/x-font-woff => font/woff

Which is a relatively small number of diffs for changing our source of record for the mime types, and would move us to just pulling from some canonical list. I'm assuming the android or debian mappings would be similar in terms of diffs from our current state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants