Skip to main content

Get started with jlbind

Source: vignettes/jlbind.qmd

jlbind lets an R package call Julia code without starting a separate Julia session. It compiles Julia code into a shared library using JuliaC.jl, then exposes Julia functions to R through native .Call() routines.

This vignette explains how to add Julia functions to your own R package with jlbind. An R package that uses jlbind is called a client. For how jlbind compares with other ways of calling Julia from R, see Performance.

An R process can load only one compiled Julia library1, so jlbind compiles the Julia code of all installed clients into one shared library, which each user builds with jlbind::install_library(). Your client therefore shares Julia dependency versions with every other client the user has installed. Using packages built on jlbind describes how this works and what your users need to do; your README should tell them to run jlbind::install_library() after installing your package.

For complete, working examples, see the two small clients that jlbind’s tests build: hellojl, which passes plain values, and counterjl, which keeps Julia objects in R (see Julia objects in R).

Set up

You need Julia 1.13 or later. From your R package’s directory, run:

jlbind::use_jlbind()

This creates the files and configuration needed to connect the package to jlbind:

  • inst/julia/, containing a Julia binding package named after the R package with a Bindings suffix (for example, MyPkgBindings), including an example function.
  • src/jlbind.c and src/entrypoints.h, which register the native routines, and a rule in src/Makevars that recompiles jlbind.c whenever entrypoints.h changes.
  • R/jlbind-wrappers.R, with an R function for each Julia function the binding package declares.
  • R/jlbind.R, which calls jlbind::bind_client() from .onLoad() and includes the roxygen tag for useDynLib().
  • The Config/jlbind, Imports: jlbind and SystemRequirements: Julia (>= 1.13) fields in DESCRIPTION.
  • A .jlbind/ directory for development builds, added to .Rbuildignore and .gitignore.

If your package already defines .onLoad(), add jlbind::bind_client(pkgname) to the existing function yourself. Then run devtools::document() to update NAMESPACE.

ImportantGive the Julia binding package a unique name

All clients are compiled into one library, so binding package names must be unique across all jlbind clients. The default, the R package name with a Bindings suffix, ensures this.

Declare Julia functions

Inside the binding package, declare each function that R should be able to call using @rcall. Each declaration specifies the argument types and return type expected by the binding.

For example:

module MyPkgBindings

using JLBind

greet(name::String) = "Hello, " * name * "!"
shift(x::Vector{Float64}, by::Union{Nothing,Float64}) =
    by === nothing ? x : x .+ by

@rcall greet(name::String)::String
@rcall shift(x::Vector{Float64}, by::Union{Nothing,Float64} = nothing)::Vector{Float64}

end

The binding package is an ordinary Julia package. Declare its dependencies in Project.toml and import them as usual.

ImportantWrite @rcall declarations literally at the top level

jlbind finds @rcall declarations by scanning the source files, not by running them, so write them literally at the top level of the module.

Call them from R

Whenever you add or change @rcall declarations, regenerate the native routine list:

This generates an R function for each declaration, in R/jlbind-wrappers.R:

greet <- function(name) .Call(C_greet, name)
shift <- function(x, by = NULL) .Call(C_shift, x, by)

Defaults in the declaration (literal numbers, strings, true, false or nothing) become R defaults.

To export or document a generated function, use a roxygen block on NULL with @name greet in another file. To add R code around a call, declare the Julia function with a jl_ prefix, e.g. @rcall jl_shift(...), and write shift() yourself.

Build and test locally

During development, you usually want to build the library from your package’s source rather than from the installed clients. Run:

jlbind::build_library()
devtools::load_all()

By default, build_library() writes the library to .jlbind/ in your package’s source directory. When you load the package with devtools::load_all(), jlbind uses this development library instead of the one created by install_library().

Once .jlbind/ exists, devtools::load_all() rebuilds it when your binding package’s files or your package’s version have changed since it was built.

NoteThe development library is only used when your package loads first

The .jlbind/ library is used only if your package is the first jlbind client loaded in the R session. A library that has loaded cannot be replaced, so after changing your Julia code, restart R; the next load_all() rebuilds it.

If you need to use a library stored elsewhere, set the JLBIND_LIBRARY environment variable to its path before loading the package.

You can also test against an unreleased version of a Julia dependency by providing a local checkout through develop:

jlbind::build_library(develop = "~/dev/SomePackage.jl")
TipPin the dependency versions

Include a Manifest.toml next to the binding package’s Project.toml with the Julia dependency versions known to compile with your package. CRAN’s policy asks that downloaded libraries be a fixed version rather than the latest.

The build uses the manifest of only the first client that has one (see Dependency versions), so keep your [compat] bounds accurate too.

Tests and CRAN

CRAN’s check machines do not have Julia installed, so your package must install and pass R CMD check without a compiled Julia library.

In tests, skip anything that needs the library:

skip_if_not(jlbind::is_bound("mypkg"))

In examples, wrap calls that require the library in if (jlbind::is_bound("mypkg")).

To check your package on GitHub Actions both ways, run:

jlbind::use_jlbind_ci()

This writes .github/workflows/R-CMD-check.yaml, which runs R CMD check on Linux, Windows and macOS after compiling your package’s Julia code with build_library(), and once on Linux without Julia, as CRAN does.

Types

@rcall converts each argument using from_sexp() and converts the return value using to_sexp(). The built-in conversions are:

Single values

Julia R argument R result
String, Symbol A single string A character vector of length one
Float64 A single number A double
Int64 A single whole number A double
Int32 A single whole number An integer
Int16, Int8, UInt16 A single whole number in range An integer
UInt32 A single whole number in range A double
Float32, Float16 A single number in range A double
Bool TRUE or FALSE A logical
ComplexF64 A single complex number A complex
Date A single Date A Date
DateTime A single POSIXct A POSIXct in UTC
Union{Missing,T} A value of type T, or NA A value of type T, or NA
Union{Nothing,T} NULL, or a value of type T NULL, or a value of type T
Nothing NULL

Vectors and arrays

Julia R argument R result
Vector{String}, Vector{Symbol} A character vector or factor A character vector
Vector{Float64}, Vector{Int64} A numeric vector A double vector
Vector{Int32} A numeric vector of whole numbers An integer vector
Vectors of the smaller types above A numeric vector in range An integer or double vector
Vector{Bool} A logical vector A logical vector
Vector{ComplexF64} A complex or numeric vector A complex vector
Vector{UInt8} A raw vector A raw vector
Vector{Date} A Date vector A Date vector
Vector{DateTime} A POSIXct vector A POSIXct vector in UTC
Vector{Union{Missing,T}} A vector of type T, which can have NA A vector of type T with NA
RVector{T} A double, integer or raw vector The same vector
Matrix{T}, Array{T,N} A matrix or array of type T A matrix or array of type T

Lists, data frames and references

Julia R argument R result
Other Vectors, Tuples A list A list
NamedTuple A named list, such as a data frame A named list
RDataFrame A data frame A data frame
Dict{String,T}, Dict{Symbol,T} A named list A named list, sorted by name
RRef{T} A reference to a Julia object A reference to a Julia object
Vector{RRef{T}} A list of references A list of references

NA is an error unless the type is Union{Missing,T} or Vector{Union{Missing,T}}, which turn NA into missing and back. A number must fit its type, so an Int16 argument rejects 40000, and NaN is accepted only where missing is. Int64 results become doubles, since R’s integers have 32 bits; return Int32 for an R integer.

To use Date or DateTime, add the Dates standard library to the binding package’s dependencies.

A NamedTuple argument must have exactly the names of its fields, in any order, so you can pass a data frame as a named tuple of its columns. To take or return an actual data frame, wrap the columns in an RDataFrame. Its type lists the column names and the type of each column, so a type alias keeps signatures short:

const Sales = RDataFrame{(:group, :amount),Tuple{Vector{String},Vector{Float64}}}
const Summary = RDataFrame{(:group, :mean),Tuple{Vector{String},Vector{Float64}}}

function summarise(df::Sales)
    gs = sort(unique(df.group))
    means = [sum(df.amount[df.group .== g]) / count(==(g), df.group) for g in gs]
    return RDataFrame((group = gs, mean = means))
end

@rcall summarise(df::Sales)::Summary
summarise(data.frame(group = c("a", "b", "a"), amount = c(1, 2, 3)))
#>   group mean
#> 1     a    2
#> 2     b    2

An RDataFrame argument must be a data frame (a tibble works too) with exactly the declared columns, in any order, so select the columns in the R wrapper if the data frame has others: summarise(df[c("group", "amount")]). Its columns are its properties (df.group), and NamedTuple(df) returns them as a named tuple. Each column converts by the usual rules, so a factor column arrives as strings, and a Vector{Union{Missing,Float64}} column allows NA. Returning columns of different lengths is an error.

Errors inside a list name the element, as in element 2: expected a number.

Every other conversion copies, so a function given a large vector it only reads can take an RVector{Float64} (or RVector{Int32}, RVector{UInt8}). That is an AbstractVector viewing R’s memory, without checking for NA. It is valid only until the call returns, so don’t store it.

A binding package can support additional types by defining methods for JLBind.from_sexp(::Type{T}, x::SEXP) and JLBind.to_sexp(x::T). JLBind also provides helpers for building R objects directly, such as JLBind.data_frame() for a data frame whose columns are not known when the library is compiled. See the JLBind source for the existing conversions and helpers.

Julia objects in R

Sometimes an object is easier to keep in Julia than to convert into an R data structure. You can pass such objects by reference using RRef{T}. R receives an external pointer to the Julia object, allowing it to be reused across calls. The object is released when R garbage-collects the reference.

For example, define a mutable Julia object and expose functions that operate on it (the counterjl test client is a complete version of this):

mutable struct Counter
    n::Int64
end

JLBind.r_class(::Type{Counter}) = "counter"

new_counter(start::Int64) = Counter(start)
increment(c::Counter, by::Int64) = (c.n += by; c.n)

@rcall new_counter(start::Int64)::RRef{Counter}
@rcall increment(c::RRef{Counter}, by::Int64)::Int64

From R, you can create a counter and update it across calls:

counter <- new_counter(0)
class(counter)
#> [1] "counter"

increment(counter, 2)
#> [1] 2

r_class() sets the R class of the external pointer, so your package can define methods such as print() for it. An argument declared as RRef{Counter} also checks that the reference points to a Counter, rather than accepting an arbitrary Julia object.

Errors

If a function called through @rcall throws a Julia exception, jlbind turns it into an R error. The messages from ArgumentError, ErrorException, DimensionMismatch, OverflowError, and AssertionError are passed through. Other exceptions are summarized.

Compilation constraints (trimming)

By default, jlbind compiles the shared library with juliac --trim=safe. This keeps the compiled image small by including only code reachable from the functions exposed through @rcall. It also requires calls to be resolvable at compile time.

If your package cannot be compiled with trimming, add the following field to DESCRIPTION:

Config/jlbind/trim: false
Warning

An untrimmed library includes the full Julia system image and is much larger, so avoid it if you can.

Abstract types

With trimming, a common source of compilation errors is a function that accepts several concrete types through an abstract RRef. Suppose your binding package works with shapes:

Shape types and their areas 
abstract type Shape end

struct Circle <: Shape
    radius::Float64
end
struct Square <: Shape
    side::Float64
end
struct Rectangle <: Shape
    width::Float64
    height::Float64
end
struct Triangle <: Shape
    base::Float64
    height::Float64
end

JLBind.r_class(::Type{<:Shape}) = "shape"

area(s::Circle) = pi * s.radius^2
area(s::Square) = s.side^2
area(s::Rectangle) = s.width * s.height
area(s::Triangle) = s.base * s.height / 2

An entrypoint that takes an RRef{Shape} knows only that its argument is some Shape. If it simply calls area(s), the compiler cannot tell which of the four methods that call reaches, and the build fails with an error such as:

Verifier error #1: unresolved call from statement area(s::Shape)::Any

One possible solution is to branch on the concrete types once, with @nospecialize, and to make the call inside each branch, where the type is known:

function with_shape(f::F, @nospecialize(s::Shape)) where {F}
    s isa Circle && return f(s)
    s isa Square && return f(s)
    s isa Rectangle && return f(s)
    s isa Triangle && return f(s)
    throw(ArgumentError("unknown shape"))
end

circle(radius::Float64) = Circle(radius)
square(side::Float64) = Square(side)
rectangle(width::Float64, height::Float64) = Rectangle(width, height)
triangle(base::Float64, height::Float64) = Triangle(base, height)
shape_area(@nospecialize(s::Shape)) = with_shape(area, s)
scaled_area(@nospecialize(s::Shape), factor::Float64) =
    with_shape(x -> factor^2 * area(x), s)

@rcall circle(radius::Float64)::RRef{Shape}
@rcall square(side::Float64)::RRef{Shape}
@rcall rectangle(width::Float64, height::Float64)::RRef{Shape}
@rcall triangle(base::Float64, height::Float64)::RRef{Shape}
@rcall shape_area(shape::RRef{Shape})::Float64
@rcall scaled_area(shape::RRef{Shape}, factor::Float64)::Float64

Every shape can then be passed to the same generated R functions:

shapes <- list(circle(1), square(2), rectangle(2, 3), triangle(2, 3))
vapply(shapes, shape_area, numeric(1))
#> [1] 3.141593 4.000000 6.000000 3.000000

scaled_area(square(2), 3)
#> [1] 36

Without @nospecialize, Julia compiles a separate with_shape for each concrete shape, and the call into it can’t be resolved, for the same reason as area(s). Functions that pass the shape on, such as shape_area, need it too (this small example happens to compile without it, since it is inlined).

If a hierarchy has only a few types, some alternatives to @nospecialize are:

  • Declare a separate entrypoint for each concrete type (RRef{Circle},RRef{Square}, …) and choose between them in R.

  • Use a small Union such as RRef{Union{Circle,Square}}, which Julia resolves by checking each member in turn.

Footnotes

  1. JuliaC can privatize a library, giving it its own copy of the Julia runtime, but jlbind doesn’t use this. Each client would then ship a bundle with its own runtime and compiled copy of Base: about 60 MB per client on Linux and 100 MB on Windows, even for a tiny client, where a shared library adds only each client’s own code. Each runtime would also have its own garbage-collected heap and threads in memory, and Julia objects couldn’t be passed between clients. Privatization also doesn’t work on Windows yet (JuliaC.jl#151).↩︎