Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Typst backend #48

Merged
merged 7 commits into from
Jun 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a"
Poppler_jll = "9c32591e-4766-534b-9725-b71a8799265b"
Rsvg = "c4c386cf-5103-5370-be45-f3a111cca3b8"
Typst_jll = "eb4b1da6-20f6-5c66-9826-fdb8ad410d0e"
Typstry = "f0ed7684-a786-439e-b1e3-3b82803b501e"
tectonic_jll = "d7dd28d6-a5e6-559c-9131-7eb760cdacc5"

[weakdeps]
Expand All @@ -35,7 +35,7 @@ Makie = "0.21.2"
Poppler_jll = "21.9, 22, 23"
Rsvg = "1"
julia = "1.9"
Typst_jll = "0"
Typstry = "0.2"
tectonic_jll = "0"

[extras]
Expand Down
2 changes: 2 additions & 0 deletions docs/src/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,12 @@ SVGDocument
PDFDocument
EPSDocument
TEXDocument
TypstDocument
```
### Cached document types
```@docs
CachedTEX
CachedTypst
CachedPDF
CachedSVG
CachedEPS
Expand Down
15 changes: 15 additions & 0 deletions docs/src/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,21 @@ scatter(fig[2, 1], rand(10), rand(10), marker=tex_document, markersize = 50)
fig
```

## Typst

```@example main
using MakieTeX, CairoMakie

typst_string = typst"$ integral_0^pi sin(x)^2 diff x $";
typst_document = TypstDocument(typst_string);
cached_typst = CachedTypst(typst_document);
cached_pdf = convert(CachedPDF, cached_typst);

fig = Figure(size=(100, 100));
LTeX(fig[1, 1], cached_pdf);
fig
```

## PDF

```@example main
Expand Down
6 changes: 2 additions & 4 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,7 @@ teximg(raw"""

Rendering can occur either to a bitmap (for GL backends) or to a Cairo surface (for CairoMakie). Both of these have APIs ([`rasterize`](@ref) and [`draw_to_cairo_surface`](@ref)).

Each rendering format has its own complexities, so the rendering pipelines are usually separate. SVG uses librsvg, PDF and EPS use Poppler directly, and TeX uses the available local TeX render (if not, `tectonic` is bundled with MakieTeX) to render to a PDF, which then follows the Poppler pipeline.

A hypothetical future Typst backend would likely also be a Typst -> PDF -> Poppler pipeline. [Typst_jll](https://github.com/JuliaBinaryWrappers/Typst_jll.jl) already exists, so it would be fairly easy to bundle.
Each rendering format has its own complexities, so the rendering pipelines are usually separate. SVG uses librsvg while PDF and EPS use Poppler directly. TeX uses the available local TeX renderer (if not, `tectonic` is bundled with MakieTeX) and Typst uses Typst_jll.jl to render to a PDF, which then each follow the Poppler pipeline.

### Makie

Expand All @@ -67,4 +65,4 @@ When rendering to Makie, MakieTeX rasterizes the document to a bitmap by default

```@raw html
</div>
```
```
5 changes: 3 additions & 2 deletions ext/MakieTeXCairoMakieExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ function CairoMakie.draw_marker(ctx, marker::MakieTeX.CachedSVG, pos, scale,
end


function CairoMakie.draw_marker(ctx, marker::Union{MakieTeX.CachedTEX, MakieTeX.CachedTeX, MakieTeX.CachedPDF}, pos, scale,
function CairoMakie.draw_marker(ctx, marker::Union{MakieTeX.CachedTEX, MakieTeX.CachedTeX, MakieTeX.CachedTypst, MakieTeX.CachedPDF},
pos, scale,
strokecolor #= unused =#, strokewidth #= unused =#,
marker_offset, rotation)
# get dimensions
Expand Down Expand Up @@ -259,4 +260,4 @@ function CairoMakie.draw_plot(scene::Makie.Scene, screen::CairoMakie.Screen, img

end

=#
=#
5 changes: 4 additions & 1 deletion src/MakieTeX.jl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module MakieTeX
using Makie
using Makie.MakieCore

using Colors, LaTeXStrings
using Colors, LaTeXStrings, Typstry
using Base64

# Patch for Makie.jl `@Block` macro error
Expand Down Expand Up @@ -36,19 +36,22 @@ include("layoutable.jl")

include("rendering/pdf_utils.jl")
include("rendering/tex.jl")
include("rendering/typst.jl")
include("rendering/pdf.jl")
include("rendering/svg.jl")

export Cached
export TeXDocument, CachedTeX
export TEXDocument, CachedTEX
export TypstDocument, CachedTypst
export PDFDocument, CachedPDF
export SVGDocument, CachedSVG
export dvi2svg, latex2dvi, rsvg2recordsurf, svg2rsvg
export teximg, teximg!, TeXImg
export LTeX

export LaTeXStrings, LaTeXString, latexstring, @L_str
export Typstry, TypstString, @typst_str

"Try to write to `engine` and see what happens"
function try_tex_engine(engine::Cmd)
Expand Down
1 change: 0 additions & 1 deletion src/recipe.jl
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,3 @@ function Makie.plot!(plot::TeXImg)
markerspace = plot.markerspace,
)
end

22 changes: 11 additions & 11 deletions src/rendering/pdf.jl
Original file line number Diff line number Diff line change
Expand Up @@ -86,15 +86,15 @@ end
# Rendering functions for the resulting Cairo surfaces and images

"""
page2img(tex::CachedTeX, page::Int; scale = 1, render_density = 1)
page2img(ct::Union{CachedTeX, CachedTypst}, page::Int; scale = 1, render_density = 1)

Renders the `page` of the given `CachedTeX` object to an image, with the given `scale` and `render_density`.
Renders the `page` of the given `CachedTeX` or `CachedTypst` object to an image, with the given `scale` and `render_density`.

This function reads the PDF using Poppler and renders it to a Cairo surface, which is then read as an image.
"""
function page2img(tex::Union{CachedTeX, CachedPDF}, page::Int; scale = 1, render_density = 1)
document = update_handle!(tex)
page2img(document, page, size(tex); scale, render_density)
function page2img(ct::Union{CachedTeX, CachedTypst, CachedPDF}, page::Int; scale = 1, render_density = 1)
document = update_handle!(ct)
page2img(document, page, size(ct); scale, render_density)
end

function page2img(document::Ptr{Cvoid}, page::Int, tex_dims::Tuple; scale = 1, render_density = 1)
Expand Down Expand Up @@ -136,7 +136,7 @@ function page2img(document::Ptr{Cvoid}, page::Int, tex_dims::Tuple; scale = 1, r

end

firstpage2img(tex; kwargs...) = page2img(tex, 0; kwargs...)
firstpage2img(ct; kwargs...) = page2img(ct, 0; kwargs...)

function page2recordsurf(document::Ptr{Cvoid}, page::Int; scale = 1, render_density = 1)
w, h = pdf_get_page_size(document, page)
Expand Down Expand Up @@ -167,16 +167,16 @@ function page2recordsurf(document::Ptr{Cvoid}, page::Int; scale = 1, render_dens

end

firstpage2recordsurf(tex; kwargs...) = page2recordsurf(tex, 0; kwargs...)
firstpage2recordsurf(ct; kwargs...) = page2recordsurf(ct, 0; kwargs...)

function recordsurf2img(tex::CachedTeX, render_density = 1)
function recordsurf2img(ct::Union{CachedTeX, CachedTypst}, render_density = 1)

# We can find the final dimensions (in pixel units) of the Rsvg image.
# Then, it's possible to store the image in a native Julia array,
# which simplifies the process of rendering.
# Cairo does not draw "empty" pixels, so we need to fill here
w = ceil(Int, tex.dims[1] * render_density)
h = ceil(Int, tex.dims[2] * render_density)
w = ceil(Int, ct.dims[1] * render_density)
h = ceil(Int, ct.dims[2] * render_density)

img = fill(Colors.ARGB32(0,0,0,0), w, h)

Expand All @@ -187,7 +187,7 @@ function recordsurf2img(tex::CachedTeX, render_density = 1)
c = Cairo.CairoContext(cs)

# Render the parsed SVG to a Cairo context
render_surface(c, tex.surf)
render_surface(c, ct.surf)

# The image is rendered transposed, so we need to flip it.
return rotr90(permutedims(img))
Expand Down
41 changes: 41 additions & 0 deletions src/rendering/pdf_utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -144,4 +144,45 @@ Crop a PDF file using Ghostscript. This alters the crop box but does not
actually remove elements.
"""
function crop_pdf(path::String; margin = _PDFCROP_DEFAULT_MARGINS[])
# if pdf_num_pages("temp.pdf") > 1
# @warn("The PDF has more than 1 page! Choosing the first page.")
# end

# Generate the cropping margins
bbox = get_pdf_bbox(path)
crop_box = (
bbox[1] - margin[1],
bbox[2] - margin[2],
bbox[3] + margin[3],
bbox[4] + margin[4]
)
crop_cmd = join(crop_box, " ")


out = Pipe()
err = Pipe()
try
redirect_stderr(err) do
redirect_stdout(out) do
Ghostscript_jll.gs() do gs_exe
run(`$gs_exe -o temp_cropped.pdf -sDEVICE=pdfwrite -c "[/CropBox [$crop_cmd]" -c "/PAGES pdfmark" -f $path`)
end
end
end
catch e
finally
close(out.in)
close(err.in)
if !isfile("temp_cropped.pdf")
println("`gs` failed to crop the PDF!")
println("Files in temp directory are:\n" * join(readdir(), ','))
printstyled("Stdout\n", bold=true, color = :blue)
println(read(out, String))
printstyled("Stderr\n", bold=true, color = :red)
println(read(err, String))
error()
end
end

return isfile("temp_cropped.pdf") ? read("temp_cropped.pdf", String) : read(path, String)
end
47 changes: 4 additions & 43 deletions src/rendering/tex.jl
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ function compile_latex(

# First, create the tex file and write the document to it.
touch("temp.tex")
path = "temp.pdf"
file = open("temp.tex", "w")
print(file, document)
close(file)
Expand All @@ -48,8 +49,8 @@ function compile_latex(
suc = success(latex)
close(out.in)
close(err.in)
if !isfile("temp.pdf")
println("Latex did not write temp.pdf! Using the $(tex_engine) engine.")
if !isfile(path)
println("Latex did not write $(path)! Using the $(tex_engine) engine.")
println("Files in temp directory are:\n" * join(readdir(), ','))
printstyled("Stdout\n", bold=true, color = :blue)
println(read(out, String))
Expand All @@ -58,47 +59,7 @@ function compile_latex(
error()
end
finally

# if pdf_num_pages("temp.pdf") > 1
# @warn("The PDF has more than 1 page! Choosing the first page.")
# end

# Generate the cropping margins
bbox = get_pdf_bbox("temp.pdf")
crop_box = (
bbox[1] - _PDFCROP_DEFAULT_MARGINS[][1],
bbox[2] - _PDFCROP_DEFAULT_MARGINS[][2],
bbox[3] + _PDFCROP_DEFAULT_MARGINS[][3],
bbox[4] + _PDFCROP_DEFAULT_MARGINS[][4],
)
crop_cmd = join(crop_box, " ")


out = Pipe()
err = Pipe()
try
redirect_stderr(err) do
redirect_stdout(out) do
Ghostscript_jll.gs() do gs_exe
run(`$gs_exe -o temp_cropped.pdf -sDEVICE=pdfwrite -c "[/CropBox [$crop_cmd]" -c "/PAGES pdfmark" -f temp.pdf`)
end
end
end
catch e
finally
close(out.in)
close(err.in)
if !isfile("temp_cropped.pdf")
println("`gs` failed to crop the PDF!")
println("Files in temp directory are:\n" * join(readdir(), ','))
printstyled("Stdout\n", bold=true, color = :blue)
println(read(out, String))
printstyled("Stderr\n", bold=true, color = :red)
println(read(err, String))
error()
end
end
return isfile("temp_cropped.pdf") ? read("temp_cropped.pdf", String) : read("temp.pdf", String)
return crop_pdf(path)
end
end
end
Expand Down
66 changes: 66 additions & 0 deletions src/rendering/typst.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#=
# Typst rendering
=#

function rasterize(ct::CachedTypst, scale::Int64 = 1)
return page2img(ct, ct.doc.page; scale)
end

# The main compilation method - compiles arbitrary Typst documents
"""
compile_typst(document::AbstractString)

Compile the given document as a String and return the resulting PDF (also as a String).
"""
function compile_typst(document::AbstractString)
#=
Typst_jll v0.11+ supports compiling from `stdin`.
It does not yet support compiling to `stdout`.

See also:
https://github.com/typst/typst/issues/410
https://github.com/typst/typst/pull/3339
=#
return mktempdir() do dir
cd(dir) do

# First, create the typst file and write the document to it.
touch("temp.typ")
file = open("temp.typ", "w")
path = "temp.pdf"
print(file, document)
close(file)

# Now, we run the latexmk command in a pipeline so that we can redirect stdout and stderr to internal containers.
# First we establish these pipelines:
out = Pipe()
err = Pipe()

try
# `pipeline` is not yet supported for `TypstCommand`
redirect_stdio(stdout=out, stderr=err) do
run(ignorestatus(typst`compile temp.typ`))
end

close(out.in)
close(err.in)
if !isfile(path)
println("Typst did not write $(path)! Using Typst_jll.jl.")
println("Files in temp directory are:\n" * join(readdir(), ','))
printstyled("Stdout\n", bold=true, color = :blue)
println(read(out, String))
printstyled("Stderr\n", bold=true, color = :red)
println(read(err, String))
error()
end
finally
return crop_pdf(path)
end
end
end
end


compile_typst(doc::TypstDocument) = compile_typst(String(doc.doc))

typst2pdf(args...) = compile_typst(args...)
Loading