Support LuaMetaTeX

[andreiborisov]

As discussed in #402, there are a couple of blockers to supporting LuaMetaTeX:

• expl3 no longer works with LuaMetaTeX: latex3/latex3#1518
• Selene Unicode package is unavailable in LuaMetaTeX

[Witiko]

Pull request #529 has removed the files markdownthemewitiko_tilde.tex, markdownthemewitiko_dot.sty, and markdownthemewitiko_graphicx_http.sty from the Markdown package. These files are now inlined in files markdown.tex and markdown.sty. This should simplify the distribution and the installation of the Markdown package in ConTeXt standalone.

[Witiko]

As discussed in #402, there are a couple of blockers to supporting LuaMetaTeX:

• expl3 no longer works with LuaMetaTeX: latex3/latex3#1518

@andreiborisov: LuaMetaTeX should now be compatible with expl3: latex3/latex3#1518.

[andreiborisov]

@andreiborisov: LuaMetaTeX should now be compatible with expl3: latex3/latex3#1518.

Wow, this is fantastic news!

Can we do something with Selene Unicode too?

[Witiko]

Can we do something with Selene Unicode too?

There are three functions from Selene Unicode that we use: unicode.utf8.char(), lower(), and match().

$ grep -F unicode.utf8 markdown.dtx

  return unicode.utf8.char(n)
  return unicode.utf8.char(n)
  return unicode.utf8.char(n)
      table.insert(char_table, unicode.utf8.char(code_point))
        local is_letter = unicode.utf8.match(char, "%a")
      if not unicode.utf8.match(char, "[%w_%-%.%s]") then
      if unicode.utf8.match(char, "[%s\n]") then
        char = unicode.utf8.lower(char)
        local is_letter = unicode.utf8.match(char, "%S")
      if not unicode.utf8.match(char, "[%w_%-%s]") then
      if unicode.utf8.match(char, "[%s\n]") then
        char = unicode.utf8.lower(char)
      local code = unicode.utf8.char(tonumber(codepoint, 16))
        if (c ~= nil) and (unicode.utf8.match(c, chartype)) then

The function unicode.utf8.char() encodes a Unicode codepoint in UTF8. We can easily replace it with the built-in function utf8.char() that is also available in LuaMetaTeX.

The function unicode.utf8.lower() lower-cases a UTF8-encoded string and can be replaced by the method uni_algos.case.casefold() from the Lua module lua-uni-algos, which we already use. Unlike Selene Unicode, lua-uni-algos is written in Lua, not in C, and can be installed together with the Markdown package without extending LuaMetaTeX itself.

The function unicode.utf8.match() finds regular expressions in a UTF8-encoded string and has no drop-in replacement. Currently, the file markdown-unicode-data.lua contains LPEG parsers for Unicode punctuation, generated from the file UnicodeData.txt. We would need to generate parsers for other classes of Unicode characters that we wish to recognize.

---

I can replace the functions unicode.utf8.char() and lower() within an hour. Replacing the function unicode.utf8.match() seems more difficult and will take me up to ~4 hours. I won't have time in January but I will schedule it for February.