Package 'tilework'

Title: Extensible Tile Planning and Windowed Parallel Processing
Description: An extensible framework for tile-based processing of large, spatially indexed data. Tile plans are first-class objects describing a set of bounds, which may be regular grids in pixel or coordinate space, tiles centered on supplied points, or adaptive decompositions such as quadtrees. Bounds may carry padding, supplying the overlapping context that neighborhood operations and objects spanning several tiles require. Extraction dispatches on the class of the input, so one plan can window two data sources of differing type together; methods are provided for 'terra' rasters and vector proxies, and other packages can register methods for their own formats. Tiles are processed in parallel through 'future', or optionally 'BiocParallel', and a per-worker initialization step amortizes expensive setup such as loading a model.
Authors: Jiaji George Chen [aut, cre] (ORCID: <https://orcid.org/0000-0002-4565-832X>), Ruben Dries [aut] (ORCID: <https://orcid.org/0000-0001-7650-7754>)
Maintainer: Jiaji George Chen <[email protected]>
License: MIT + file LICENSE
Version: 1.0.0
Built: 2026-10-08 18:23:18 UTC
Source: https://github.com/drieslab/tilework

Help Index


Tile Pads

Description

Tile padding extends the bounds of the tile beyond the region that they are initially planned for by an equal amount on all 4 sides. This can help with things like spatial contiguity, so that artificial borders are less obvious.

Padding can be added either via $pad or + and - operators. The + and - operators specifically modify the padding value based on the arithmetic ops. This is similar to their usage in terra::Arith-methods

⁠$stride<-⁠ is an alternative way to set padding via the stride between tile starts: pad = (tile_dim - stride) / 2. Requires tile_dims to be set (i.e. not freeTilePlan). ⁠$stride⁠ returns the effective stride given the current pad.

Usage

## S4 method for signature 'tilePlan,numeric'
e1 + e2

## S4 method for signature 'tilePlan,numeric'
e1 - e2

## S4 method for signature 'pointTilePlan,numeric'
e1 + e2

## S4 method for signature 'pixelTilePlan,numeric'
e1 + e2

## S4 method for signature 'tileGroup,numeric'
e1 + e2

## S4 method for signature 'tileGroup,numeric'
e1 - e2

## S4 method for signature 'tileSelection,numeric'
e1 + e2

## S4 method for signature 'tileSelection,numeric'
e1 - e2

Arguments

e1

tilePlan

e2

numeric to add to padding value

Value

tilePlan

spatial and pixel differences

  • Padding for spatial tile plans is added without any further changes.

  • Padding for pixel tile plans are automatically zeroed out against the top left

See Also

Other tile* methods: as.polygons, bracket, centroids, dim, dollar, double_bracket, ext, intersect, plot

Examples

spat <- tilePlan("spatial")
ext(spat) <- c(0, 1000, 0, 1000)
length(spat) <- 9
plot(spat)
spat[1:3]
spat <- spat + 10
plot(spat)
spat[1:3]

px <- tilePlan("pixel")
px$pxdims <- c(1000, 1000)
px$ncols <- 100
px$nrows <- 100
plot(px)
px[1:3]
px <- px + 5
plot(px)
px[1:3]

Coerce a tile plan to polygons

Description

Convert a tilePlan-inheriting object to a SpatVector of rectangle polygons, one per tile, with a tile attribute column holding the tile index. Padding is included in the polygon bounds.

Accelerated vectorized methods are provided for spatialTilePlan, freeTilePlan, and pointTilePlan. All other tilePlan subclasses fall back to extracting bounds via x[].

Usage

## S4 method for signature 'tilePlan'
as.polygons(x, ...)

## S4 method for signature 'freeTilePlan'
as.polygons(x, ...)

## S4 method for signature 'pointTilePlan'
as.polygons(x, ...)

## S4 method for signature 'spatialTilePlan'
as.polygons(x, ...)

Arguments

x

tilePlan-inheriting object

...

additional arguments (ignored)

Value

SpatVector of polygons

See Also

Other tile* methods: arith, bracket, centroids, dim, dollar, double_bracket, ext, intersect, plot


Extract Bounds from Tile Object

Description

Get a set of tile bounds from a ⁠tile*⁠ object. Values are always returned as a list, even when length one to reduce surprises with lapply() usage.

Usage

## S4 method for signature 'tilePlan,numeric,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tilePlan,missing,numeric,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tilePlan,numeric,numeric,missing'

  x[
  i,
  j,
  tile_fun = .spat_tile_bounds,
  fun = function(x) x,
  zero = FALSE,
  expand_grid = TRUE,
  drop
]

## S4 method for signature 'tilePlan,missing,missing,missing'
x[i, j]

## S4 method for signature 'tilePlan,numeric,missing,logical'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tilePlan,missing,numeric,logical'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tilePlan,numeric,numeric,logical'
x[i, j, expand_grid = TRUE, ..., drop]

## S4 method for signature 'tilePlan,missing,missing,logical'
x[i, j, drop]

## S4 method for signature 'freeTilePlan,numeric,numeric,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'pointTilePlan,numeric,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'pixelTilePlan,numeric,numeric,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'spatialTilePlan,numeric,numeric,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileGroup,missing,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileGroup,.index,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileGroup,missing,.index,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileGroup,numeric,.index,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileIterator,missing,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 replacement method for signature 'tileIterator,missing,missing'
x[i, j, ...] <- value

## S4 method for signature 'tileSelection,numeric,missing,missing'
x[i, j, ..., drop = TRUE]

## S4 method for signature 'tileSelection,numeric,missing,logical'
x[i, j, ..., drop = TRUE]

Arguments

x

⁠tile*⁠ object

i

numeric vector index if j is not given. Row index if j is also present. Works like matrix-like indexing.

j

numeric. Column index

...

addtional params to pass (not used).

drop

not used.

tile_fun

(internal) function used to define tile bounds based on the iterator object and ij indices.

fun

(internal) post-processing function used to convert bounds to expected output. Such as ext() for spatialTilePlan

zero

logical. Whether to zero out padding effects. (used by default with pixelTilePlan)

expand_grid

logical (internal) whether to use expand.grid() on ij indices.

value

ANY value to set

Value

list of numeric or SpatExtent depending on underlying tilePlan

See Also

Other tile* methods: arith, as.polygons, centroids, dim, dollar, double_bracket, ext, intersect, plot

Examples

spat <- tilePlan("spatial")
ext(spat) <- c(0, 1000, 0, 1000)
length(spat) <- 9
spat[1]
spat[1, 2:3]
spat[]

px <- tilePlan("pixel")
px$pxdims <- c(1000, 1000)
px$ncols <- 200
px$nrows <- 200
px[1]
px[2, 2:3]
px[]

Combine tileSelections

Description

Combine tileSelection. Should only concatenate tileSelection build off the same plan.

Usage

## S4 method for signature 'tileSelection'
c(x, ...)

Arguments

x

item to concatenate

...

addtional items to concatenate to x


Get Tile Centroids

Description

Get the centroids of the tiles. For spatialTilePlan, these will be returned as SpatVector centroids. For non-spatial plans like pixelTileGrid, this will be returne as a matrix.

Usage

## S4 method for signature 'tilePlan'
centroids(x, fun = function(x) x, offset = c(0, 0), zero = FALSE, ...)

## S4 method for signature 'freeTilePlan'
centroids(x, ...)

## S4 method for signature 'pointTilePlan'
centroids(x, ...)

## S4 method for signature 'pixelTilePlan'
centroids(x, fun = function(x) x, zero = TRUE, ...)

## S4 method for signature 'spatialTilePlan'
centroids(x, zero = FALSE, ...)

Arguments

x

tilePlan

fun

(internal) post-processing function used to convert centroids to expected output. Such as vect() for spatialTilePlan

offset

numeric of length 2. x and y values to offset by. Default is c(0, 0).

zero

logical. Whether to zero out padding effects. (used by default with pixelTilePlan)

...

additional params to pass.

See Also

Other tile* methods: arith, as.polygons, bracket, dim, dollar, double_bracket, ext, intersect, plot

Examples

x <- tilePlan()
ext(x) <- c(0, 100, 0, 100)
length(x) <- 9
centroids(x)

Tile Plan Array Characteristics

Description

Get dimension characteristics of the tilePlan tiling plan. These produce information on how the tiles are arrayed in rows nrow(), cols ncol(), and total number of tiles length().

⁠length<-()⁠ is a special case and currently only used with spatialTilePlan, where it is used to request a minimum number of tiles to generate.

Usage

## S4 method for signature 'tilePlan'
nrow(x)

## S4 method for signature 'tilePlan'
ncol(x)

## S4 method for signature 'tilePlan'
length(x)

## S4 method for signature 'tilePlan'
dim(x)

## S4 replacement method for signature 'spatialTilePlan'
length(x) <- value

## S4 method for signature 'tileGroup'
length(x)

## S4 method for signature 'tileIterator'
length(x)

## S4 method for signature 'tileSelection'
length(x)

Arguments

x

tilePlan object

value

number of tiles to request

Value

numeric or spatialTilePlan object when using ⁠length<-()⁠

See Also

Other tile* methods: arith, as.polygons, bracket, centroids, dollar, double_bracket, ext, intersect, plot

Examples

x <- tilePlan()
ext(x) <- c(0, 100, 0, 100)
length(x)
length(x) <- 9
length(x)
dim(x)
nrow(x)
ncol(x)

Get and Set Tile Metadata and Params

Description

Get and set tile metadata. Some params can also be modified with this operator, for example the ⁠$pad⁠ value or ⁠$pxdims⁠, ⁠$ncols⁠, or ⁠$nrows⁠ for pixelTilePlan.

⁠$tile⁠ is a default created piece of metadata.

Usage

## S4 replacement method for signature 'tilePlan'
x$name <- value

## S4 method for signature 'tilePlan'
x$name

## S4 replacement method for signature 'freeTilePlan'
x$name <- value

## S4 method for signature 'freeTilePlan'
x$name

## S4 replacement method for signature 'pointTilePlan'
x$name <- value

## S4 method for signature 'pointTilePlan'
x$name

## S4 replacement method for signature 'pixelTilePlan'
x$name <- value

## S4 method for signature 'pixelTilePlan'
x$name

## S4 replacement method for signature 'tileGroup'
x$name <- value

## S4 method for signature 'tileGroup'
x$name

## S4 method for signature 'tileSelection'
x$name

## S4 replacement method for signature 'tileSelection'
x$name <- value

Arguments

x

tilePlan

name

name of item to get or set

value

value to set

Value

metadata value when using getter or ⁠tile*⁠ object when using the setter function.

See Also

Other tile* methods: arith, as.polygons, bracket, centroids, dim, double_bracket, ext, intersect, plot

Examples

x <- tilePlan()
ext(x) <- c(0, 100, 0, 100)
length(x) <- 9
x$test <- sprintf("test_value_%d", x$tile)
x$test
tile_ext <- x[5][[1]]
attr(tile_ext, "test")

Get and set metadata

Description

[[ can be used to get the table of metadata for a specific tile.

Usage

## S4 method for signature 'tilePlan,numeric,missing'
x[[i, j, ...]]

## S4 method for signature 'tilePlan,missing,missing'
x[[i, j, ...]]

## S4 method for signature 'tilePlan,missing,character'
x[[i, j, ...]]

## S4 method for signature 'tilePlan,numeric,character'
x[[i, j, ...]]

## S4 replacement method for signature 'tilePlan,numeric,character'
x[[i, j, ...]] <- value

## S4 replacement method for signature 'tilePlan,missing,character'
x[[i, j, ...]] <- value

Arguments

x

tilePlan

i

integer-like. tile vector index

j

character. Name of metadata information to get

...

not used.

value

ANY value to set

Value

data.frame

See Also

dollar

Other tile* methods: arith, as.polygons, bracket, centroids, dim, dollar, ext, intersect, plot

Examples

spat <- tilePlan("spatial")
ext(spat) <- c(0, 1000, 0, 1000)
length(spat) <- 9
spat[[1]]

Get and Set Spatial Extent

Description

Get and set a spatial extent.

Usage

## S4 method for signature 'pointTilePlan'
ext(x, ...)

## S4 replacement method for signature 'pointTilePlan,ANY'
ext(x) <- value

## S4 method for signature 'spatialTilePlan'
ext(x, ...)

## S4 replacement method for signature 'spatialTilePlan,ANY'
ext(x) <- value

Arguments

x

tilePlan

...

addtional params to pass (none implemented)

value

numeric of length 4 or SpatExtent defining spatial extent.

Value

tilePlan if ⁠ext<-()⁠ and SpatExtent if ext()

See Also

Other tile* methods: arith, as.polygons, bracket, centroids, dim, dollar, double_bracket, intersect, plot


Extending {tilework}

Description

{tilework} provides an extensible framework for:

Data flow

The framework accesses and processes data according to the following flow:

  1. tileApply(x, tiles, ...) # operate across tiles

  2. getTile(x, tiles, ..., get_params) # get data for tiles

    1. bounds <- tiles[...] # generate bounds info

    2. getBoundedData(x, bounds, get_params) # data extraction

Expanded roles:

  • tileApply()- handles the function parallelization strategy and logic.

  • getTile() - deals with any interactions between the data type and the tile indexing. Operations that should be performed before extraction of the actual data by tile bounds should happen at this level.

  • getBoundedData() - performs the low-level data extraction from the data source, provided with a set of bounds information (commonly a SpatExtent or numeric of c(xmin, xmax, ymin, ymax))

Adding new data type support

{tilework} functionalities can be extended to work with other data types/backends by creating a getBoundedData() method for the data type. Optional additional steps:

  • Method for tileApply() should also be added if there are pre-run validation/data collection steps needed.

  • Method for getTile() should be added if there are operations that affect data/tile interactions.

See Also

Other tilework extension: redispatch_tileapply,SpatRaster,freeTilePlan-method, token-class

Examples

if (FALSE) {
    # Extending for a custom data type:
    setMethod(
        "getBoundedData", signature("MyDataType", "SpatExtent"),
        function(x, bound) {
            # Your data extraction logic here
            my_extract_function(x, bound)
        }
    )
}

Create a Fluid Tile Plan

Description

Create an empty freeTilePlan. Populate bounds via ⁠$bounds<-⁠ or use quadtreePlan() to generate adaptive bounds automatically.

Usage

freeTilePlan(...)

Arguments

...

additional params passed to new().

See Also

quadtreePlan(), freeTilePlan

Other tile plans: freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilePlan-class, tilework-class


Free Tile Plan

Description

Tile plan defined by explicit per-tile bounds with no required uniformity in size or spacing. The bounds matrix is the canonical representation — there is no center-plus-dims formula. Always returns SpatExtent from ⁠[i]⁠. Primary use case is as the output of adaptive spatial decomposition algorithms such as quadtrees on vector/point data.

Tile plan defined by explicit per-tile bounds with no required uniformity in size or spacing. Unlike grid-based plans, the bounds matrix is the canonical representation — tile positions are not computed from a formula.

The primary use case is adaptive spatial decomposition of vector/point data, where high-density regions require small tiles and sparse regions can use large tiles. See quadtreePlan() for the iterative construction workflow.

Slots

bounds

matrix. n x 4 matrix: xmin, xmax, ymin, ymax (one row per tile).

Setup

  • freeTilePlan() creates an empty instance.

  • ⁠$bounds<-⁠ sets the n x 4 bounds matrix (⁠xmin, xmax, ymin, ymax⁠). Setting bounds triggers reinitialization of ⁠@n⁠, ⁠@dims⁠, and ⁠@metadata⁠.

  • ⁠$pad⁠ / + / - add spatial padding as with other plans.

Notes

  • ⁠@tile_dims⁠ is intentionally not populated. Operations that require uniform tile dimensions (e.g. extend = TRUE in getTile()) are not supported.

  • nrow() returns n, ncol() returns 1, length() returns n.

  • plot() works via the existing spatial extent preview.

Note

⁠@tile_dims⁠ is intentionally not populated. Any method that requires uniform tile dimensions will not work with freeTilePlan.

See Also

Other tile plans: freeTilePlan(), pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilePlan-class, tilework-class

Examples

tp <- freeTilePlan()
tp$bounds <- rbind(
    c(0,  50,  0,  50),
    c(50, 100, 0,  50),
    c(0,  50,  50, 100),
    c(50, 100, 50, 100)
)
length(tp)
tp[2]
plot(tp)

Get Data Within Bounds

Description

Subset or otherwise make available only the data that is within the provided bounds. Accepted bounds types depends on the data.

A related lower-level function is getTile() which handles data/⁠tile*⁠ interactions and bound info collection. getTile() calls this generic for its data selection capabilities.

Usage

## S4 method for signature 'SpatRaster,numeric'
getBoundedData(x, bound, extend = FALSE, fill = NA)

## S4 method for signature 'SpatRaster,SpatExtent'
getBoundedData(x, bound, extend = FALSE, fill = NA)

## S4 method for signature 'SpatVectorProxy,SpatExtent'
getBoundedData(x, bound)

Arguments

x

data

bound

bounds to filter with

extend

logical (default = FALSE) whether to extend tile data to reach expected tile dimensions

fill

numeric. if extend = TRUE, what value to fill with

Bounds types

Depending on the method, different bound types are used. Current expected patterns are:

  • numeric of length 4 (xmin, xmax, ymin, ymax)

  • SpatExtent

SpatRaster snapping

getBoundedData() is implemented for SpatRaster, SpatExtent using terra::window(). This uses terra's default snapping behavior (equivalent to snap = "near" in terra::crop()), with no way to set another strategy.

For more precise boundary control:

  • Use pixel-based indexing via the SpatRaster, numeric method

  • Use pixelTilePlan() for exact pixel-level tiling

  • Add padding with tiles + pad_value to ensure spatial context

  • Use terra::crop() directly if you need snap = "out" or snap = "in"

Boundary Inclusivity

Adjacent tiles share exact boundaries. Since {tilework} does not know the format or representation of the underlying data, it does not enforce whether those boundaries are inclusive or exclusive. It is up to the getBoundedData() implementation to decide how features on shared edges are handled.

The existing {terra} methods do not implement inclusive/exclusive boundary control because raster pixel snapping and padding make exact boundary behavior largely irrelevant for that format. For point or tabular data, features on a shared boundary may appear in multiple tiles unless the implementation applies its own filtering (e.g. >= vs > comparisons). The tile's grid position can be passed via get_params from getTile() to inform which edges are interior.

See Also

getTile()

Other tile processing: getTile(), tileApply(), tileApply-group, tileApply-iterator, tileApply-plan

Examples

f <- system.file("ex/elev.tif", package = "terra")
r <- terra::rast(f)

pixel_selection <- getBoundedData(r, c(10, 50, 30, 40))
plot(pixel_selection)

extent_selection <- getBoundedData(r, ext(6, 6.5, 49.7, 50))
plot(extent_selection)

Get Tile

Description

Get specific tile(s) from the data based on a ⁠tile*⁠ object. Indexing via i and j params is used to select the tile(s) to get, in a manner similar to [ extraction of bounds from ⁠tile*⁠ objects.

All params are usable with any method except when otherwise stated.

This generic handles data/⁠tile*⁠ object interactions. For simple selection of data based on a set of bounds, see the low-level generic getBoundedData().

The character method only works for filetypes readable by {terra}.

New x data types should automatically work with this generic as long as a getBoundedData() method is created for it. A specific getTile() method is only needed if additional params/operations should be performed before the tile selection, (for example layer selection with raster data).

Usage

## S4 method for signature 'token,tilePlan'
getTile(x, tiles, ...)

## S4 method for signature 'ANY,tilePlan'
getTile(x, tiles, i = NULL, j, pad = NULL, sel_params = list(), ...)

## S4 method for signature 'character,tilePlan'
getTile(x, tiles, prefer = NULL, ext = NULL, ...)

## S4 method for signature 'SpatRaster,tilePlan'
getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...)

## S4 method for signature 'SpatRaster,pointTilePlan'
getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...)

## S4 method for signature 'SpatRaster,tileGroup'
getTile(x, tiles, i, j, lyr = NULL, ...)

## S4 method for signature 'ANY,tileGroup'
getTile(x, tiles, i, j, ...)

## S4 method for signature 'ANY,tileIterator'
getTile(x, tiles, advance = TRUE, ...)

Arguments

x

data

tiles

⁠tile*⁠ object

...

additional params to pass to [ tilePlan indexing

i

ANY except tileIterator tile vector index or row index if j is also provided

j

ANY except tileIterator tile col index

pad

(optional) additional padding to apply before tile retrieval. Useful for temporarily increasing padding without affecting ⁠tile*⁠ object.

sel_params

list of named params to pass to the tile [ selection call. Rarely needed; covers internal [ options such as expand_grid.

prefer

character filepath only character. Hint for file reading ("raster" or "vector"). If provided, skips automatic type detection.

ext

SpatRaster or raster filepath (character) only numeric or SpatExtent (optional) Set an extent before extracting tiles from raster.

lyr

SpatRaster only if provided, which layers/channels to include

extend

pixelTilePlan only logical (default = FALSE). Whether to extend tile data to reach expected tile dimensions

fill

pixelTilePlan only numeric. if extend = TRUE, what value to fill with

advance

tileIterator only logical (default = TRUE). Whether to advance the iterator.

Value

list of tile data

See Also

getBoundedData()

Other tile processing: getBoundedData(), tileApply(), tileApply-group, tileApply-iterator, tileApply-plan

Examples

f <- system.file("ex/elev.tif", package = "terra")
r <- terra::rast(f)
tp <- tilePlan("pixel")
tp$pxdims <- dim(r)[1:2]
tp$nrows <- 10
tp$ncols <- 10

# get tiles from specific array grid indices.
tile_list <- getTile(f, tp, i = 3, j = 5)
force(tile_list)
plot(tile_list[[1]])

# get tiles via iteration
iter <- tileIterator(tp, batch_size = 3)
b1 <- getTile(r, iter) # get first batch of 3...
b2 <- getTile(r, iter) # get second batch of 3...

Find tiles intersecting a spatial region

Description

Return a tileSelection of tiles whose padded bounds intersect the query region y. Tile indices are available via ⁠$tile⁠ on the result.

Accepts any object coercible via ext() for an axis-aligned query, or a SpatVector polygon for an exact query (e.g. a back-projected parallelogram from a rotated affine crop).

For SpatVector input, an AABB pre-cull is applied first, then exact polygon-rectangle intersection is tested on candidates via terra::relate().

spatialTilePlan uses an analytic grid formula for the AABB case (O(1) range computation). freeTilePlan uses a vectorized bounds check. All other tilePlan subclasses fall back to building polygons via as.polygons().

Usage

## S4 method for signature 'tilePlan,ANY'
intersect(x, y)

## S4 method for signature 'freeTilePlan,ANY'
intersect(x, y)

## S4 method for signature 'spatialTilePlan,ANY'
intersect(x, y)

Arguments

x

tilePlan-inheriting object

y

spatial query – any object coercible via ext(), or a SpatVector polygon

Value

tileSelection. Use ⁠$tile⁠ to access tile indices.

See Also

Other tile* methods: arith, as.polygons, bracket, centroids, dim, dollar, double_bracket, ext, plot

Examples

# spatialTilePlan -- axis-aligned query
tp <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 16)
e <- ext(20, 60, 20, 60)
sel <- intersect(tp, e)
sel$tile          # indices of intersecting tiles
length(sel)       # number of tiles hit
plot(sel)
plot(e, add = TRUE, border = "cyan")

# with padding -- tiles near the border are included
tp_pad <- tp + 5
sel_pad <- intersect(tp_pad, terra::ext(20, 60, 20, 60))
length(sel_pad) >= length(sel)  # TRUE: padding pulls in border tiles

# freeTilePlan -- quadtree bounds
fp <- freeTilePlan()
fp$bounds <- rbind(
    c(0,  50,  0,  50),
    c(50, 100, 0,  50),
    c(0,  50,  50, 100),
    c(50, 100, 50, 100)
)
intersect(fp, terra::ext(40, 60, 40, 60))$tile  # all 4 tiles touch the centre

# SpatVector polygon query (e.g. back-projected rotated crop)
corners <- rbind(c(30, 10), c(70, 10), c(90, 50), c(10, 50), c(30, 10))
poly <- terra::vect(corners, type = "polygons")
sel_poly <- intersect(tp, poly)
sel_poly$tile
plot(sel_poly)
plot(poly, add = TRUE, border = "cyan")

Create multiple walkers from a single iterator

Description

Utility function to create multiple independent walkers for parallel processing. Each iterator will have the same underlying iterator but independent state.

Usage

## S4 method for signature 'tileIterator'
iterSplit(tiles, n, batch_size = NULL, distribute = TRUE, ...)

Arguments

tiles

tileIterator object

n

integer. Number of iterators to create

batch_size

integer (optional). Assign a batch size for each iterator. If not provided, inherits same batch_size as source tiles.

distribute

logical (default = TRUE). If TRUE, distribute tiles evenly across iterators. Otherwise return multiple true copies.

...

additional params to pass (none implemented)

See Also

Other tile orchestration: tileGroup(), tileGroup-class, tileIterator(), tileIterator-class


Parallel Processing Parameters

Description

tileApply() is parallelized through either future.apply::future_lapply() (default) or BiocParallel::bplapply().

Additional parameters to control parallelization can be passed as a list through the parallel_params argument in tileApply() methods.

Usage

getTileworkParMethod()

setTileworkParMethod(method)

Arguments

method

character. Either "future" (default) or "biocparallel". Which parallelization framework to use.

Value

getTileworkParMethod returns character. The parallelization method being used.

setTileworkParMethod returns NULL invisibly

Parameters

Available parameters for the parallel_params list:

  • method - character. Either "future" (default) or "biocparallel". Which parallelization framework to use. Overrides setTileworkParMethod()

  • future_seed - logical (default = TRUE). Passed to the future.seed parameter in future.apply::future_lapply(). Only used when method = "future".

  • BPPARAM - a {BiocParallel} parallelization parameter object. Only used when method = "biocparallel".

Additional parameters relevant to the underlying functions may also be passed in the list and will be forwarded via ....

Package Options

  • "tilework.warn_sequential" - logical (default = TRUE). Whether to warn when using parallelization defaults that run sequentially (i.e., future::sequential() or BiocParallel::SerialParam()).

  • "tilework.par_method" - character (default = "future"). Either "future" or "biocparallel". Which parallelization framework to use.

  • "tilework.bpparam" - a {BiocParallel} parameter object. Set this to automatically use a specific BPPARAM, analogous to setting a future::plan() for the {future} backend.

See Also

tileApply()

Other parallel settings: tilework_management

Examples

getTileworkParMethod()

# Set to BiocParallel
setTileworkParMethod("biocparallel")
getTileworkParMethod()

## Not run: 
# Example pseudocode for how to use parallelization

# Using future backend with multisession
future::plan(future::multisession, workers = 4)
tileApply(x, tiles, fun)

# Using BiocParallel backend
setTileworkParMethod("biocparallel")
options(tilework.bpparam = BiocParallel::SnowParam(workers = 4))
tileApply(x, tiles, fun)

# Or override per-call
setTileworkParMethod("future")  # global default is future
tileApply(x, tiles, fun, 
    parallel_params = list(method = "biocparallel")
) # but use BiocParallel here

# Suppress sequential warnings
options(tilework.warn_sequential = FALSE)

## End(Not run)

Pixel Tile Plan

Description

Utility class for defining pixel-exact tiles of images.

Usage

pixelTilePlan(pxdims = NULL, ncols = NULL, nrows = NULL, ...)

Arguments

pxdims

numeric of length 2. Pixel dimensions c(nrow, ncol) of the image to iterate across. Equivalent to x$pxdims <- value.

ncols

numeric. Tile width in pixels. Equivalent to x$ncols <- value.

nrows

numeric. Tile height in pixels. Equivalent to x$nrows <- value.

...

additional params passed to new().

Slots

pxdims

pixel dimensions to iterate across

n

numeric. Number of tiles to create.

dims

numeric. Number of rows/cols in the array of tiles

tile_dims

numeric. Row/col dimensions of each tile

pad

numeric. Tile padding

metadata

data.frame. Metadata per tile

Setup

  • pixelTilePlan(pxdims, ncols, nrows) creates an instance. All three params can also be set after construction via ⁠$pxdims<-⁠, ⁠$ncols<-⁠, and ⁠$nrows<-⁠.

  • length(), dim(), nrow(), ncol() report tile counts and layout.

Getting tile pixel indices

⁠[i]⁠ and ⁠[i, j]⁠ indexing can be used to select tiles, similarly to a matrix. ⁠[]⁠ without any indexing will return the entire set of indices as a list. Indices are returned as lists of integer vectors of length 4, with values xmin, xmax, ymin, ymax. Extracted vectors of indices will have metadata attached via attr(). See metadata section below.

padding

+/- can be used to add or subtract padding to each of the tiles. Each bound will be expanded by the pad value. Pad values may only be integer values.

previewing tiles

plot() can be used to check the layout of the tiles.

metadata

The pixelTilePlan object can contain metadata. By default after extent and tiles setup, a column called "tile" will be set up that simply records which tile it is.

  • $ and ⁠$<-⁠ can be used to get and set specific metadata, the "pxdims" which are the pixel dimensions of the image to iterate across, "ncols" and "nrows" which are the px dims of a tile, and the "pad" value.

  • ⁠[[i]]⁠ selection will pull specific metadata rows corresponding to the selected tiles.

See Also

Other tile plans: freeTilePlan(), freeTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilePlan-class, tilework-class

Examples

x <- pixelTilePlan(pxdims = c(100, 100), ncols = 20)

length(x) # check how many tiles there are
dim(x)
nrow(x)
ncol(x)

# previewing
plot(x)

# tile padding
y <- x + 3
plot(y, alpha = 0.3) # red border shows the image space
# this is now larger than the original space.

# negative padding
z <- x - 5
plot(ext(c(0, 100, -100, 0)))
plot(z, add = TRUE)

# tile selection
x[5]
x[2, 2:3]

# metadata
x$tile
x$fname <- sprintf("tile_%03d.tif", x$tile)
x[[5]]
x[[1:3]]$fname

Plot a tilePlan

Description

Plot and preview the tile plan. This is likely to be very slow if there are a lot of tiles (in the neighborhood of >10,000)

Usage

## S4 method for signature 'tilePlan,missing'
plot(x, values = "tile", color_as_factor = FALSE, ...)

## S4 method for signature 'freeTilePlan,missing'
plot(x, y, ...)

## S4 method for signature 'pointTilePlan,missing'
plot(x, y, ...)

## S4 method for signature 'pixelTilePlan,missing'
plot(x, y, ...)

## S4 method for signature 'tileSelection,missing'
plot(x, y, border = "red", ...)

Arguments

x

tilePlan to plot

values

character. Metadata item to color tiles as. Default is "tile" which produces the tile index.

color_as_factor

logical (default = FALSE). Whether to convert value to a factor to plot as categorical.

...

additional params to pass

y

not used.

border

tileSelection only color to outline the selected tiles with

See Also

Other tile* methods: arith, as.polygons, bracket, centroids, dim, dollar, double_bracket, ext, intersect

Examples

x <- tilePlan()
ext(x) <- c(0, 100, 0, 100)
length(x) <- 9
plot(x)

x$new_meta <- rev(head(letters, 9))
plot(x, value = "new_meta")

Point Tile Plan

Description

Tile plan where each tile is centered on a user-supplied (x, y) coordinate with a uniform tile size. The center coordinates are primary — for use cases like survey sampling, object detection patches, or site-centered extractions where the point location is semantically meaningful.

All coordinates, tile dims, and padding live in one consistent reference frame set by ⁠$input⁠ ("spatial" for CRS units, "pixel" for pixel indices). The bound type returned by getTile() is a separate concern controlled by ⁠$output⁠ and resolved at extraction time.

Usage

pointTilePlan(
  input = c("spatial", "pixel"),
  output = input,
  coords = NULL,
  width = NULL,
  height = NULL,
  ...
)

Arguments

input

character. Coordinate space of coords, dims, and padding: "spatial" for CRS units, "pixel" for pixel indices.

output

character. Bound type returned by getTile(). Defaults to input. Cross-mode conversion requires a SpatRaster at getTile() time (or ⁠@rast_dims⁠/⁠@extent⁠ set on the plan for standalone use).

coords

n x 2 matrix or data frame of tile center coordinates (columns: x, y). Data frames are coerced via as.matrix(). Equivalent to x$coords <- value after construction.

width

numeric. Uniform tile width in input coordinate units. Equivalent to x$width <- value after construction.

height

numeric. Uniform tile height in input coordinate units. Equivalent to x$height <- value after construction.

...

additional params to pass to pointTilePlan class constructor

Slots

coords

matrix. n x 2 matrix of tile center coordinates (columns: x, y).

input

character. Coordinate space of coords, dims, and padding: "spatial" for CRS units, "pixel" for pixel indices.

output

character. Bound type returned by getTile(): "spatial" or "pixel". Defaults to input. Evaluated at getTile() time, not ⁠[i]⁠.

rast_dims

numeric. c(nrow, ncol) of the reference raster. Required for plot() when input = "pixel" and for cross-mode conversion without a raster.

extent

numeric. c(xmin, xmax, ymin, ymax) of the reference raster. Required for pixel → CRS conversion without a raster.

n

numeric. Number of tiles (equals number of input points).

dims

integer. Always c(n, 1L).

tile_dims

numeric. Uniform c(height, width) in input coordinate units.

pad

numeric. Tile padding in input coordinate units.

metadata

data.frame. Per-tile metadata; always has "tile", "x", and "y" columns.

Setup

  • pointTilePlan() creates an instance. input defaults to "spatial"; output defaults to input.

  • ⁠$coords<-⁠ sets the n x 2 center coordinate matrix (columns: x, y). Data frames are also accepted and coerced via as.matrix().

  • ⁠$width<-⁠ / ⁠$height<-⁠ set the uniform tile width and height in input coordinate units.

  • ⁠$input<-⁠ / ⁠$output<-⁠ change the coordinate / output mode.

  • ⁠$nrows<-⁠ / ⁠$ncols<-⁠ are aliases for ⁠$height<-⁠ / ⁠$width<-⁠ for consistency with pixelTilePlan when using pixel coordinates.

  • ⁠$rast_dims<-⁠ sets c(nrow, ncol) of the reference raster — required for plot() when input = "pixel" and for cross-mode getTile() calls when no raster is provided.

  • ⁠$extent<-⁠ sets c(xmin, xmax, ymin, ymax) — required for pixel → CRS conversion without a raster.

Coordinate modes and required raster metadata

Scenario ⁠@rast_dims⁠ ⁠@extent⁠
input = "spatial", output = "spatial" not needed not needed
input = "pixel", output = "pixel" for plot() not needed
input = "pixel", output = "spatial" required required
input = "spatial", output = "pixel" required derived

When getTile() is called with a SpatRaster, the raster's own metadata is used for conversion — ⁠@rast_dims⁠ and ⁠@extent⁠ are only needed for standalone operations (plotting, validation) without a raster.

Getting tile bounds

⁠[i]⁠ returns bounds in the input coordinate space: SpatExtent when input = "spatial", integer[4] when input = "pixel". ⁠$output⁠ is not consulted at ⁠[i]⁠ time. Use getTile() for output-mode conversion.

Pixel mode and center ambiguity

When input = "pixel", tile dims should be odd for an unambiguous center pixel. With even dimensions ⁠center ± width/2⁠ produces non-integer bounds that are truncated by as.integer().

metadata

After setting coords, metadata is auto-populated with "tile", "x", and "y" columns. Additional columns can be added via ⁠$<-⁠.

See Also

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilePlan-class, tilework-class

Examples

tp <- pointTilePlan("spatial")
tp$coords <- cbind(x = c(10, 20, 30), y = c(5, 15, 25))
tp$width  <- 5
tp$height <- 5
tp

length(tp)
tp[1]
tp[2:3]
centroids(tp)
plot(tp)

# CRS input, pixel output (raster needed at getTile time)
tp2 <- pointTilePlan(input = "spatial", output = "pixel")
tp2$coords <- cbind(x = c(10, 20), y = c(5, 15))
tp2$width  <- 100
tp2$height <- 100

Adaptive Quadtree Tile Plan

Description

Iteratively subdivides a spatial extent into a freeTilePlan by running FUN on each tile and splitting any tile whose value exceeds threshold. Uses tileApply() internally so each pass can be parallelized via future::plan().

FUN must return a single numeric scalar per tile — typically a point count, density, or variance measure. Tiles are split into four equal quadrants until all tiles are either below threshold or smaller than min_tile_size.

Usage

quadtreePlan(
  x,
  tiles = NULL,
  FUN = nrow,
  threshold,
  min_tile_size = NULL,
  max_depth = 10L,
  verbose = NULL,
  ...
)

Arguments

x

input data (e.g. SpatVector or file path).

tiles

starting tilePlan. Defines the initial coarse grid.

FUN

function. Applied to each tile; must return a single numeric.

threshold

numeric. Tiles with FUN(tile) > threshold are subdivided.

min_tile_size

numeric. Minimum tile side length (in CRS units). Tiles below this size are kept as leaves regardless of density.

max_depth

integer (default 10L). Maximum subdivision depth.

verbose

logical. Whether to be verbose.

...

additional params passed to tileApply().

Value

A freeTilePlan with an n_records metadata column containing the last FUN value for each leaf tile.

See Also

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, spatialTilePlan-class, tilePlan(), tilePlan-class, tilework-class

Examples

# dummy data
pts <- terra::vect(cbind(x = rnorm(1000, 0, 100), y = rnorm(1000, 0, 100)))
pts <- rbind(pts, terra::shift(pts, dx = 1000, dy = 1000))
plot(pts)
# data must exist on disk
f <- tempfile(fileext = "shp")
terra::writeVector(pts, f)
pts <- terra::vect(f, proxy = TRUE)

fp <- quadtreePlan(pts,
    threshold = 500L,
    min_tile_size = 1
)
plot(fp)
# plot with items per tile
plot(fp, values = "n_records")

Developer API Redispatch for tileApply()

Description

Utility generic for modifying tileApply() calls for streamlining extension with new datatypes and forcing datatype-specific handling to be added in the expected order.

Why

  • tileApply is a framework function for parallelized computing across tiled data. It allows dispatch on up to 3 signatures (x, y, and tile), allowing flexibility, but at the cost of normally having to deal with combinatorial explosion of methods when extending with a new x and y datatype. Especially the case when considering compatibility with existing types and specific handling for different tile types.

  • Setting specific methods using the following example dispatch chain is ambiguous, with it being unclear whether 1 or 2 is first and whether both will be reached before 3.

    1. ⁠(ANY, myclass, mytiles)⁠

    2. ⁠(myclass, ANY, mytiles)⁠

    3. ⁠(ANY, ANY, mytiles)⁠

redispatch_tileapply() solves the above problems by modifying tileApply() calls based on the classes of x and y, appropriate to the provided ⁠tile*⁠ type.

This is done by passing x or y as input sig alongside the tiles and dispatching on the combination, and properly routing changes when they are x or y param specific based on the param_xy.

After a sig is processed, it is coerced to the token class signifying that it is ready for processing and the call is passed back to tileApply() which can either proceed with eval or pass back to redispatch_tileapply if y is also provided.

Only when x (for single input tileApply()) or x and y are all of class token, does the tileApply() call fully evaluate.

The default ANY, ANY method performs no modifications to the input data or the call.

Usage

## S4 method for signature 'SpatRaster,freeTilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'ANY,ANY'
redispatch_tileapply(
  sig,
  tiles,
  default_get_params = list(),
  param_xy,
  verbose = NULL,
  ...
)

## S4 method for signature 'character,tileGroup'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatVector,tileGroup'
redispatch_tileapply(sig, tiles, parallel_strategy = c("groups", "tiles"), ...)

## S4 method for signature 'SpatRaster,tileGroup'
redispatch_tileapply(
  sig,
  tiles,
  param_xy,
  parallel_strategy = c("groups", "tiles"),
  ...
)

## S4 method for signature 'ANY,tileGroup'
redispatch_tileapply(
  sig,
  tiles,
  default_get_params = list(),
  default_callback = NULL,
  param_xy,
  parallel_strategy = c("groups", "tiles"),
  verbose = NULL,
  ...
)

## S4 method for signature 'character,tileIterator'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatRaster,tileIterator'
redispatch_tileapply(sig, tiles, param_xy, ...)

## S4 method for signature 'SpatVector,tileIterator'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'ANY,tileIterator'
redispatch_tileapply(
  sig,
  tiles,
  default_get_params = list(),
  default_callback = NULL,
  param_xy,
  verbose = NULL,
  ...
)

## S4 method for signature 'character,tilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatVector,spatialTilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatRaster,spatialTilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatRaster,pixelTilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatRaster,pointTilePlan'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'character,tileSelection'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatVector,tileSelection'
redispatch_tileapply(sig, tiles, ...)

## S4 method for signature 'SpatRaster,tileSelection'
redispatch_tileapply(sig, tiles, ...)

Arguments

sig

data type to modify the tileApply() call based on.

tiles

tile* object

...

additional params to pass

default_get_params

internal use: list of default getTile() parameters to use with this sig, passed from an upstream (likely more specific) redispatch_tileapply() method.

Depending on param_xy, this list of defaults will be merged with get_params_x or get_params_y in ...

Duplicate entries in default_get_params will be ignored in favor of get_params_x or get_params_y contents.

param_xy

character. Either "x" or "y" which param it is to modify dispatch on

verbose

verbosity. TRUE, FALSE or "debug" for more info on stack tracing.

parallel_strategy

character. "groups" to parallelize across groups, or "tiles" to parallelize within groups

default_callback

internal use: Default callback function passed from an upstream (likely more specific) redispatch_tileapply() method to use as callback_x or callback_y if none has been provided.

Logic flow between tileApply() and redispatch_tileapply()

x only case:

  1. tileApply(ANY, missing, ANY, ...)

  2. redispatch_tileapply(mydata, mytiles, param_xy = "x", ...)

  3. redispatch_tileapply(ANY, ANY, param_xy = "x", ...)

  4. tileApply(token, missing, ANY, ...)

  5. evaluation

x and y case:

  1. tileApply(ANY, ANY, ANY)

  2. redispatch_tileapply(mydata1, mytiles, param_xy = "x")

  3. redispatch_tileapply(ANY, ANY, param_xy = "x", ...)

  4. tileApply(token, ANY, ANY)

  5. redispatch_tileapply(mydata2, mytiles, param_xy = "y")

  6. redispatch_tileapply(ANY, ANY, param_xy = "y", ...)

  7. tileApply(token, token, plan)

  8. evaluation

This allows:

  • Data type-specific preprocessing, checks, and additional params to be injected into the tileApply() call without having to interact with the core tileApply() logic for tilewise data selection and parallelization

  • Avoid duplicating data type-specific handling code for each of the following tileApply() signature types:

    • ⁠(mydata, missing, tiles)⁠

    • ⁠(mydata, ANY, tiles)⁠

    • ⁠(token, mydata, tiles)⁠

Instead, extending developers only have to provide code for a single redispatch_tileapply(myclass, tiles) signature, and tileApply() <-> redispatch_tileapply() calls will iteratively apply the appropriate preprocessing and x or y routed params.

The x and y routing and token coercion are also handled by the redispatch_tileapply(ANY, ANY) method.

See Also

Other tilework extension: extending_tilework, token-class


Spatial Tile Plan

Description

Utility class that simplifies the setup of tiles across a spatial extent. Tiles are stored in a lightweight format safe to be passed to child processes. Tile SpatExtent objects can be extracted on-demand.

Usage

spatialTilePlan(ext = NULL, n = NULL, ...)

Arguments

ext

numeric or SpatExtent. Spatial extent to tile across. Equivalent to calling ext(x) <- value after construction.

n

numeric. Desired number of tiles. Equivalent to calling length(x) <- value after construction.

...

additional params passed to new().

Slots

extent

numeric. Spatial extent to tile across.

n

numeric. Number of tiles to create.

dims

numeric. Number of rows/cols in the array of tiles

tile_dims

numeric. Row/col dimensions of each tile

pad

numeric. Tile padding

metadata

data.frame. Metadata per tile

setup and basic characteristics

A spatialTilePlan needs both a spatial extent to tile across and also a request for a certain number of tiles.

  • spatialTilePlan(ext, n) is used to create a spatialTilePlan instance. ext and n can also be supplied after construction via ⁠ext()<-⁠ and ⁠length()<-⁠.

  • ext() is used to check extent.

  • length() can be used to find out how many tiles there are.

  • dim()/nrow()/ncol() basic generics are implemented and return information about how the tiles are arranged.

Note that the number of requested tiles may not be the actual length, since a grid pattern must be followed. However, the number of generated tiles will be AT LEAST the number that is requested. Generated tiles will have as square a shape as possible.

Getting tile extent

⁠[i]⁠ and ⁠[i, j]⁠ indexing can be used to select tiles, similarly to a matrix. ⁠[]⁠ without any indexing will return the entire set of extents as a list. Extracted extents will have metadata attached via attr(). See metadata section below.

padding

+/- can be used to add or subtract padding to each of the tiles. Note that this value does not affect the setting or retrieval of extent info via ext() and ⁠ext()<-⁠. Each bound will be expanded by the padding value. To avoid having to use terra::extend() when padded tiles exceed raster extent, you can decrease the extent by the same size as the padding.

previewing tiles

plot() can be used to check the layout of the tiles.

metadata

The spatialTilePlan object can contain metadata. By default after extent and tiles setup, a column called "tile" will be set up that simply records which tile it is.

  • $ can be used to view a specific type of metadata and the padding value

  • ⁠$<-⁠ can be used to set additional metadata items and the padding value

  • ⁠[[i]]⁠ selection will pull specific metadata rows corresponding to the selected tiles.

See Also

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), tilePlan(), tilePlan-class, tilework-class

Examples

x <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 8)
force(x)

length(x) # how many were actually generated? AT LEAST n
dim(x)
nrow(x)
ncol(x)

# previewing
plot(x)

# tile padding
y <- x + 10
plot(ext(x), border = "red")
plot(y, alpha = 0.3, add = TRUE)
# this is now larger than the original space
ext(y) <- ext(y) - 10
plot(y, alpha = 0.3)
plot(ext(x), add = TRUE, border = "red")
# now this does not exceed the image

# negative padding
z <- x - 5
plot(ext(x), border = "red")
plot(z, add = TRUE)

# tile selection
x[5]
x[1, 2:3]

# metadata
x$tile
x$fname <- sprintf("tile_%03d.tif", x$tile)
x[[5]]
x[[1:3]]$fname

# selected tiles carry metadata as attributes
attr(x[4][[1]], "tile")
attr(x[4][[1]], "fname")

Hierarchical Tile Group Processing

Description

Apply functions across tileGroup objects with control over parallelization strategy. Useful when tiles are organized into logical groups that need different processing or aggregation.

token is a stand-in for any input data class (e.g. SpatRaster, SpatExtent, filepath, etc). See redispatch_tileapply and extending_tilework for further information.

Usage

## S4 method for signature 'token,missing,tileGroup'
tileApply(
  x,
  tiles,
  FUN,
  get_params_x = list(),
  parallel_strategy = c("groups", "tiles"),
  setup_FUN = NULL,
  group_FUN = NULL,
  callback_x = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  simplify = FALSE,
  parallel_params = list(),
  verbose = NULL,
  ...
)

## S4 method for signature 'token,token,tileGroup'
tileApply(
  x,
  y,
  tiles,
  FUN,
  get_params_x = list(),
  get_params_y = list(),
  pad_y = NULL,
  parallel_strategy = c("groups", "tiles"),
  setup_FUN = NULL,
  group_FUN = NULL,
  callback_x = NULL,
  callback_y = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  simplify = FALSE,
  parallel_params = list(),
  verbose = NULL,
  ...
)

Arguments

x

input data 1

tiles

tileGroup object

FUN

function to apply to each tile

get_params_x

named list. Additional params to pass to getTile() for x

parallel_strategy

character. "groups" to parallelize across groups, or "tiles" to parallelize within groups

setup_FUN

function. Optional per-group initialization function when parallel_strategy = "groups". Output is accessible within FUN as .SETUP_OUT

group_FUN

function. Optional function to apply to each group's results

callback_x

function. Optional preprocessing function for x per worker when parallel_strategy = "groups"

log

logical. Whether to log processing steps

logpath

character. Log file path (if log = TRUE)

simplify

logical. Whether to flatten group results into single list. Group names will not be retained.

parallel_params

named param list. See parallel_params

verbose

verbosity. TRUE, FALSE or "debug" for more info on stack tracing.

...

additional params to pass to [

y

input data 2 (optional)

get_params_y

named list. Additional params to pass to getTile() for y

pad_y

numeric. Additional padding applied to y tiling so x has full spatial context of y

callback_y

function. Optional preprocessing function for y per worker when parallel_strategy = "groups"

Parallelization Strategies

  • "groups" - Process groups in parallel, tiles within groups sequentially

  • "tiles" - Process groups sequentially, tiles within groups in parallel

Special Function Parameters

Your FUN can optionally include these special parameters:

  • .I - tile number (integer)

  • .TILE - tile bounds/metadata

  • .R - tile row number

  • .C - tile column number

  • .GROUP - current group name (character)

  • .SETUP_OUT - the output of setup_FUN

Your setup_FUN can optionally include these special parameters:

  • .GROUP - current group name (character)

  • .X - the input x object

  • .Y - the input y object (when provided)

See Also

tileApply, tileGroup(), tileGroup

Other tile processing: getBoundedData(), getTile(), tileApply(), tileApply-iterator, tileApply-plan

Examples

f <- system.file("ex/elev.tif", package = "terra")
r <- terra::rast(f)

# Create tile plan
tp <- tilePlan("spatial")
ext(tp) <- ext(r)
length(tp) <- 16

# Organize into groups (e.g., by geographic region)
tg <- tileGroup(tp, groups = list(
    "north" = 1:8, # northern tiles
    "south" = 9:16, # southern tiles
    "corners" = c(1, 4, 13, 16) # corner tiles
))

# Process groups in parallel, with group-level aggregation
results <- tileApply(r,
    tiles = tg,
    parallel_strategy = "groups",
    FUN = function(tile, .I, .GROUP) {
        # Process individual tile
        list(
            tile_id = .I,
            group = .GROUP,
            stats = terra::global(tile, c("mean", "sd"), na.rm = TRUE)
        )
    },
    group_FUN = function(group_results, .GROUP) {
        # Aggregate results within each group
        means <- sapply(group_results, function(x) x$stats$mean)
        list(
            group = .GROUP,
            n_tiles = length(group_results),
            group_mean = mean(means),
            group_range = range(means)
        )
    }
)

# Results organized by group
str(results)

Streaming Tile Processing with Iterators

Description

Apply functions using tileIterator objects for memory-constrained batch processing. Ideal for very large datasets or when you need fine control over processing workflow.

token is a stand-in for any input data class (e.g. SpatRaster, SpatExtent, filepath, etc). See redispatch_tileapply and extending_tilework for further information.

Usage

## S4 method for signature 'token,missing,tileIterator'
tileApply(
  x,
  tiles,
  FUN,
  get_params_x = list(),
  setup_FUN = NULL,
  callback_x = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  simplify = FALSE,
  parallel_params = list(),
  verbose = NULL,
  ...
)

## S4 method for signature 'token,token,tileIterator'
tileApply(
  x,
  y,
  tiles,
  FUN,
  get_params_x = list(),
  get_params_y = list(),
  pad_y = NULL,
  setup_FUN = NULL,
  callback_x = NULL,
  callback_y = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  simplify = FALSE,
  parallel_params = list(),
  verbose = NULL,
  ...
)

Arguments

x

input data 1

tiles

tileIterator object

FUN

function to apply to each batch of tiles

get_params_x

named list. Additional params to pass to getTile() for x

setup_FUN

function. Optional per-worker initialization function. Output is accessible within FUN as .SETUP_OUT

callback_x

function, (advanced). If provided, programmatic escape hatch for preprocessing x per worker before setup_FUN and batch streamed processing of FUN begins.

log

logical. Whether to log processing steps

logpath

character. Log file path (if log = TRUE)

simplify

logical. Whether to flatten results

parallel_params

named param list. See parallel_params

verbose

verbosity. TRUE, FALSE or "debug" for more info on stack tracing.

...

additional params to pass to [

y

input data 2 (optional)

get_params_y

named list. Additional params to pass to getTile() for y

pad_y

numeric. Additional padding applied to y tiling so x has full spatial context of y

callback_y

function, (advanced). If provided, programmatic escape hatch for preprocessing y per worker before setup_FUN and batch streamed processing of FUN begins.

Worker Distribution

iterSplit() is run with ⁠n =⁠ future::nbrOfWorkers() to distribute the tiles to process across the expected number of workers, with each worker processing its assigned tiles in batches.

Special Function Parameters

Your FUN can optionally include these special parameters:

  • .I - tile number (integer)

  • .TILE - tile bounds/metadata

  • .R - tile row number

  • .C - tile column number

  • .POSITION - batch position range (start, end)

  • .BATCH - batch number within worker

  • .SETUP_OUT - the output of setup_FUN

Your setup_FUN can optionally include these special parameters:

  • .W - worker number

  • .X - the input x object.

  • .Y - the input y object (when provided)

See Also

tileApply, tileIterator(), tileIterator

Other tile processing: getBoundedData(), getTile(), tileApply(), tileApply-group, tileApply-plan

Examples

f <- system.file("ex/elev.tif", package = "terra")
r <- terra::rast(f)

# Create pixel tile plan
tp <- tilePlan("pixel")
tp$pxdims <- dim(r)[1:2]
tp$nrows <- 30
tp$ncols <- 30

# Create iterator for batch processing
iter <- tileIterator(tp, batch_size = 5)

# Process with worker initialization
results <- tileApply(r,
    tiles = iter,
    setup_FUN = function(.W, .X) {
        # Initialize per-worker state
        list(
            worker_id = .W,
            start_time = Sys.time(),
            raster_info = list(nrow = nrow(.X), ncol = ncol(.X))
        )
    },
    FUN = function(batch, .BATCH, .POSITION, .SETUP_OUT) {
        # Process batch of tiles
        batch_stats <- lapply(batch, function(tile) {
            terra::global(tile, "mean", na.rm = TRUE)[[1]]
        })

        list(
            worker = .SETUP_OUT$worker_id,
            batch_num = .BATCH,
            tiles_processed = .POSITION,
            batch_mean = mean(unlist(batch_stats))
        )
    }
)

# Check results structure
str(results)

# Example: Streaming processing for memory management
large_iter <- tileIterator(tp, batch_size = 3)
processed_count <- 0

while (large_iter$has_next) {
    batch <- getTile(r, large_iter)

    # Process batch
    batch_results <- lapply(batch, function(tile) {
        # Your processing here
        terra::global(tile, "mean")
    })

    processed_count <- processed_count + length(batch)
    cat("Processed", processed_count, "of", length(tp), "tiles\n")
}

Basic Tile Processing

Description

Apply functions across tilePlan-inheriting objects.

token is a stand-in for any input data class (e.g. SpatRaster, SpatExtent, filepath, etc). See redispatch_tileapply and extending_tilework for further information.

Usage

## S4 method for signature 'token,missing,tilePlan'
tileApply(
  x,
  tiles,
  FUN,
  get_params_x = list(),
  log = FALSE,
  logpath = getTileworkLogDir(),
  parallel_params = list(),
  verbose = NULL,
  ...
)

## S4 method for signature 'token,token,tilePlan'
tileApply(
  x,
  y,
  tiles,
  FUN,
  get_params_x = list(),
  get_params_y = list(),
  pad_y = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  parallel_params = list(),
  verbose = NULL,
  ...
)

## S4 method for signature 'token,missing,tileSelection'
tileApply(
  x,
  tiles,
  FUN,
  get_params_x = list(),
  log = FALSE,
  logpath = getTileworkLogDir(),
  parallel_params = list(),
  verbose = NULL,
  ...
)

## S4 method for signature 'token,token,tileSelection'
tileApply(
  x,
  y,
  tiles,
  FUN,
  get_params_x = list(),
  get_params_y = list(),
  pad_y = NULL,
  log = FALSE,
  logpath = getTileworkLogDir(),
  parallel_params = list(),
  verbose = NULL,
  ...
)

Arguments

x

input data 1

tiles

tilePlan inheriting object (spatialTilePlan or pixelTilePlan)

FUN

function to apply to each tile

get_params_x

named list. Additional params to pass to getTile() for x

log

logical. Whether to log processing steps

logpath

character. Log file path (if log = TRUE)

parallel_params

named param list. See parallel_params

verbose

verbosity. TRUE, FALSE or "debug" for more info on stack tracing.

...

additional params to pass to [

y

input data 2 (optional)

get_params_y

named list. Additional params to pass to getTile() for y

pad_y

numeric. Additional padding applied to y tiling so x has full spatial context of y

Special Function Parameters

Your FUN can optionally include these special parameters:

  • .I - tile number (integer)

  • .TILE - tile bounds/metadata

  • .R - tile row number

  • .C - tile column number

See Also

tileApply, tilePlan(), spatialTilePlan, pixelTilePlan

Other tile processing: getBoundedData(), getTile(), tileApply(), tileApply-group, tileApply-iterator

Examples

f <- system.file("ex/elev.tif", package = "terra")
r <- terra::rast(f)

# Spatial tiling example
tp_spatial <- tilePlan("spatial")
ext(tp_spatial) <- ext(r)
length(tp_spatial) <- 4

# Apply function with tile metadata
results <- tileApply(r, tiles = tp_spatial, FUN = function(tile, .I, .R, .C) {
    list(
        tile_id = .I,
        position = paste0("row_", .R, "_col_", .C),
        mean_value = terra::global(tile, "mean", na.rm = TRUE)[[1]]
    )
})
force(results)

# Pixel tiling example
tp_pixel <- tilePlan("pixel")
tp_pixel$pxdims <- dim(r)[1:2]
tp_pixel$nrows <- 50
tp_pixel$ncols <- 50

# Save tiles to disk
outdir <- file.path(tempdir(), "tiles")
dir.create(outdir, showWarnings = FALSE)

tileApply(r, tiles = tp_pixel, FUN = function(tile, .I) {
    filename <- file.path(outdir, sprintf("tile_%03d.tif", .I))
    terra::writeRaster(tile, filename, overwrite = TRUE)
    return(filename)
})

list.files(outdir)
unlink(outdir, recursive = TRUE)

Create a Tile Group

Description

Organize tiles from a tilePlan into hierarchical groups for batch processing. Groups can represent spatial regions, processing stages, or any logical organization of tiles.

Usage

tileGroup(tp, groups = list(), metadata = data.frame())

Arguments

tp

A tilePlan object

groups

Named list where each element contains tile indices for that group

metadata

Optional data.frame with group metadata

See Also

Other tile orchestration: iterSplit(), tileGroup-class, tileIterator(), tileIterator-class

Examples

tp <- tilePlan("spatial")
ext(tp) <- c(0, 100, 0, 100)
length(tp) <- 16

# Create groups
tg <- tileGroup(tp, groups = list(
    "g1" = 1:4, # vector indexing
    "g2" = list(2, 1:4), # ij indexing
    "g3" = list(2:4, 1:2) # selection overlaps are allowed
    # (not all tiles need to be selected)
))

# length() returns number of groups
length(tg)

# Get group bounds:
tg[, "g1"] # first 4
tg[, "g2"] # next 4
tg[, "g3"] # specific 6
# not recommended for large groups

tg[c(3, 1), "g3"] # get nth item in group

# Set active group for shorthand indexing
tg$active <- "g1"

# length() is based on group length when active is set
length(tg)

tg[2] # Position 2 from g1

# iterator can be created when active group is set
iter <- tileIterator(tg, batch_size = 2)
iter$next_batch()
iter$next_batch()
iter$next_batch() # no more items

tg$active <- NULL # Clear active group by setting NULL

Tile Group

Description

Class for organizing tiles into hierarchical groups for batch processing. Groups can represent spatial regions, processing stages, or any logical organization of tiles.

Slots

tp

tilePlan. The underlying tilePlan-inheriting object

groups

list. Named list where each element contains tile indices for that group

active

character. Name of a group to set as active for ⁠[i]⁠ shorthand indexing and length().

metadata

data.frame. Metadata about each group

See Also

Other tile orchestration: iterSplit(), tileGroup(), tileIterator(), tileIterator-class


Stateful Tile Iterator

Description

Create a stateful iterator that maintains an internal position and yields batches of tiles on demand. Uses closures for serializable stateful behavior without external dependencies. Useful for streaming processing, memory-constrained environments, or when you want to process tiles in smaller chunks.

This is a structure that works with any underlying tilePlan. tileGroup is also accepted when an ⁠$active⁠ group has already been set, or a group is specified via tileIterator(active_group)

Note that tileIterator contains its own state and an independent copy of it cannot be generated by assigning to a new variable. Instead, use either the ⁠$copy()⁠ method or iterSplit().

Usage

## S4 method for signature 'tileIterator'
x$name

## S4 replacement method for signature 'tileIterator'
x$name <- value

tileIterator(
  tiles = NULL,
  position = 0,
  bound = NULL,
  batch_size = 1,
  active_group = NULL
)

Arguments

x

tileIterator

name

Name of method or attribute to access (See Methods section)

value

value to set for an attribute (See Methods section)

tiles

A tile* object

position

numeric. Set the starting position of the iterator. Default is 0.

bound

numeric, length of 2. Min and max indexing bounds through which to iterate. Default is c(1, length(tiles))

batch_size

Integer. Number of tiles to return per call to next_batch()

active_group

character (optional). Only used when tiles is a tileGroup. Syntactic sugar for setting the active group of the tileGroup before using it.

Value

A tileIterator function with attached methods

Methods

  • ⁠[]⁠/⁠[]<-⁠ or ⁠$tiles⁠/⁠$tiles<-⁠

    • get or set a tilePlan or tileGroup (with ⁠$active⁠ set). ⁠[]⁠ notation is just shorthand.

  • ⁠$position⁠/⁠$position<-⁠

    • get and set current position of iterator

  • ⁠$batch_size⁠/⁠$batch_size<-⁠

    • get and set batch size.

  • ⁠$bound⁠/⁠$bound<-⁠

    • get and set iteration tile index bounds. This is a numeric vector of 2 (min, max)

  • ⁠$next_batch()⁠/⁠$next_indices()⁠

    • get next batch or just their tile indices.

      • Has an advance param (default = TRUE) that advances iterator position.

      • When there are no more tiles, gracefully returns empty list

  • ⁠$peek_batch()⁠/⁠$peek_indices⁠

    • Get next batch or indices without advancing position. When there are no more tiles, gracefully returns empty integer

  • ⁠$has_next⁠

    • Return logical if there are still tiles left to iterate through (if position < max(bound)). Useful for while looping.

  • ⁠$total_tiles⁠

    • Return total number of tiles in bound

  • ⁠$remaining⁠

    • Return number of remaining indices to traverse.

  • ⁠$progress⁠

    • Return current percentage of position traversal across bound

  • ⁠$reset()⁠

    • Reset position to 1 before the lower bound

  • ⁠$copy()⁠

    • Create a new tileIterator with the same settings (deep copy)

Usage Patterns

  • Stream processing large datasets

  • Memory-constrained batch processing

  • Progressive/incremental analysis

  • Checkpoint and resume workflows

  • Parallel processing with foreach

See Also

tileIterator

Other tile orchestration: iterSplit(), tileGroup(), tileGroup-class, tileIterator-class

Examples

# Create a spatial tile iterator
tp <- tilePlan("spatial")
ext(tp) <- c(0, 100, 0, 100)
length(tp) <- 16

# Create a iterator that processes 3 tiles at a time
iter <- tileIterator(tp, batch_size = 3)

# Check status
iter
iter$has_next
iter$remaining
iter$progress

# Get next batch
batch1 <- iter$next_batch()
length(batch1) # 3 tiles

# Check updated status
iter$progress # position has advanced

# Peek at next batch without advancing
peek <- iter$peek_batch()
peek
iter$position # unchanged

# serialize and unserialize, preserving state
temp <- tempfile()
saveRDS(iter, temp)
rm(iter)
iter <- readRDS(temp)

# Next batch of indices
iter$next_indices()

# Peek next batch of indices
iter$peek_indices()

# Process all remaining batches
while (iter$has_next) {
    batch <- iter$next_batch()
    cat("Processing batch of", length(batch), "tiles\n")
}

# Reset and try different batch size
iter$reset()
iter$batch_size <- 5

# Apply function across all batches
while (iter$has_next) {
    batch <- iter$next_batch()
    cat("Processing batch of", length(batch), "tiles\n")
}

# tileIterator with tileGroup
tg <- tileGroup(tp, groups = list(
    "g1" = c(2, 4, 6, 8, 10, 12, 14, 16),
    "g2" = 1:16
))

tg$active <- "g1"
iter <- tileIterator(tg, batch_size = 3)
iter
iter$next_batch()

# iterator splitting
siter <- iterSplit(iter, n = 2)
siter

tileIterator

Description

A stateful iterator that progresses through tiles of an underlying tilePlan (or tileGroup if ⁠$active⁠ is set) object upon every call to ⁠$next_batch()⁠

The closures that power this functionality are stored in ⁠@funs⁠. The stateful position handling is also internalized within.

Slots

funs

list of closure methods

See Also

tileIterator

Other tile orchestration: iterSplit(), tileGroup(), tileGroup-class, tileIterator()


Create a Tiling Plan

Description

Create a Tiling Plan

Usage

tilePlan(type = c("spatial", "pixel", "point", "free"), ...)

Arguments

type

character. One of "spatial", "pixel", "point", "free". Type of plan to create.

...

additional params passed to the specific constructor: spatialTilePlan(), pixelTilePlan(), pointTilePlan(), or freeTilePlan().

See Also

spatialTilePlan, pixelTilePlan, pointTilePlan, freeTilePlan

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan-class, tilework-class

Examples

tilePlan("spatial")
tilePlan("pixel")
tilePlan("point")
tilePlan("free")

Tile Plan

Description

Virtual parent class for tile planning objects. These objects are for planning tiles/patches of data to be operated over. Objects are indexable across rows/columns of tiles and as a vector of tiles (similarly to a matrix).

[ indexing returns a set of bounds information for selection of the data. Tiles and bounds information are generated in a lazy fashion based on plan parameters, avoiding overhead with large amounts of tiles.

Slots

n

numeric. Number of tiles to create.

dims

numeric. Number of rows/cols in the array of tiles

tile_dims

numeric. Row/col dimensions of each tile

pad

numeric. Tile padding

metadata

data.frame. Metadata per tile

See Also

spatialTilePlan and pixelTilePlan for concrete classes dealing with spatial and pixel-exact tiling respectively. tilePlan() for creation of these objects.

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilework-class


tilework management

Description

A set of utilities to control the location and naming of automated tilework outputs. These are mostly to do with logging information right now.

Usage

getTileworkLogDir()

setTileworkLogDir(path)

getTileworkJobID(advance = FALSE)

Arguments

path

directory path to use

advance

logical (default = FALSE). Whether to consume this job ID

Value

getTileworkLogDir the logging directory to use

setTileworkLogDir returns the previous logging directory invisibly

getTileworkJobID returns the next job ID to use. If advance = FALSE this call will be treated as a peek and will not consume the ID.

See Also

Other parallel settings: parallel_params


Virtual Class tilework

Description

The tilework class is a class contained by all tile* classes in the tilework package. It is a "virtual" class.

See Also

Other tile plans: freeTilePlan(), freeTilePlan-class, pixelTilePlan-class, pointTilePlan-class, quadtreePlan(), spatialTilePlan-class, tilePlan(), tilePlan-class


token Class

Description

Utility class for flagging a piece of data as being ready for processing. This is internal machinery that is mainly useful for forcing S4 dispatch to progress in the expected order and should not be interacted with by end users.

This class and related methods are exported so developers are able to write extending methods for tileApply() where this utility class is used.

Usage

## S4 method for signature 'token,missing,missing,missing'
x[i, j, ..., drop = TRUE]

Arguments

x, i, j, ..., drop

not used

Slots

data

ANY. The wrapped data object.

See Also

Other tilework extension: extending_tilework, redispatch_tileapply,SpatRaster,freeTilePlan-method

Examples

# Flag as being ready for processing
x <- as(letters, "token")

# Retrieve wrapped data
x[]