| Title: | Extensible Tile Planning and Windowed Parallel Processing |
|---|---|
| Description: | An extensible framework for tile-based processing of large, spatially indexed data. Tile plans are first-class objects describing a set of bounds, which may be regular grids in pixel or coordinate space, tiles centered on supplied points, or adaptive decompositions such as quadtrees. Bounds may carry padding, supplying the overlapping context that neighborhood operations and objects spanning several tiles require. Extraction dispatches on the class of the input, so one plan can window two data sources of differing type together; methods are provided for 'terra' rasters and vector proxies, and other packages can register methods for their own formats. Tiles are processed in parallel through 'future', or optionally 'BiocParallel', and a per-worker initialization step amortizes expensive setup such as loading a model. |
| Authors: | Jiaji George Chen [aut, cre] (ORCID: <https://orcid.org/0000-0002-4565-832X>), Ruben Dries [aut] (ORCID: <https://orcid.org/0000-0001-7650-7754>) |
| Maintainer: | Jiaji George Chen <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.0.0 |
| Built: | 2026-10-08 18:23:18 UTC |
| Source: | https://github.com/drieslab/tilework |
Tile padding extends the bounds of the tile beyond the region that they are initially planned for by an equal amount on all 4 sides. This can help with things like spatial contiguity, so that artificial borders are less obvious.
Padding can be added either via $pad or + and - operators.
The + and - operators specifically modify the padding value based on the
arithmetic ops. This is similar to their usage in terra::Arith-methods
$stride<- is an alternative way to set padding via the stride between
tile starts: pad = (tile_dim - stride) / 2. Requires tile_dims to be
set (i.e. not freeTilePlan). $stride returns the effective stride given
the current pad.
## S4 method for signature 'tilePlan,numeric' e1 + e2 ## S4 method for signature 'tilePlan,numeric' e1 - e2 ## S4 method for signature 'pointTilePlan,numeric' e1 + e2 ## S4 method for signature 'pixelTilePlan,numeric' e1 + e2 ## S4 method for signature 'tileGroup,numeric' e1 + e2 ## S4 method for signature 'tileGroup,numeric' e1 - e2 ## S4 method for signature 'tileSelection,numeric' e1 + e2 ## S4 method for signature 'tileSelection,numeric' e1 - e2## S4 method for signature 'tilePlan,numeric' e1 + e2 ## S4 method for signature 'tilePlan,numeric' e1 - e2 ## S4 method for signature 'pointTilePlan,numeric' e1 + e2 ## S4 method for signature 'pixelTilePlan,numeric' e1 + e2 ## S4 method for signature 'tileGroup,numeric' e1 + e2 ## S4 method for signature 'tileGroup,numeric' e1 - e2 ## S4 method for signature 'tileSelection,numeric' e1 + e2 ## S4 method for signature 'tileSelection,numeric' e1 - e2
e1 |
|
e2 |
|
tilePlan
Padding for spatial tile plans is added without any further changes.
Padding for pixel tile plans are automatically zeroed out against the top left
Other tile* methods:
as.polygons,
bracket,
centroids,
dim,
dollar,
double_bracket,
ext,
intersect,
plot
spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 plot(spat) spat[1:3] spat <- spat + 10 plot(spat) spat[1:3] px <- tilePlan("pixel") px$pxdims <- c(1000, 1000) px$ncols <- 100 px$nrows <- 100 plot(px) px[1:3] px <- px + 5 plot(px) px[1:3]spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 plot(spat) spat[1:3] spat <- spat + 10 plot(spat) spat[1:3] px <- tilePlan("pixel") px$pxdims <- c(1000, 1000) px$ncols <- 100 px$nrows <- 100 plot(px) px[1:3] px <- px + 5 plot(px) px[1:3]
Convert a tilePlan-inheriting object to a SpatVector of rectangle
polygons, one per tile, with a tile attribute column holding the tile
index. Padding is included in the polygon bounds.
Accelerated vectorized methods are provided for spatialTilePlan,
freeTilePlan, and pointTilePlan. All other tilePlan subclasses fall
back to extracting bounds via x[].
## S4 method for signature 'tilePlan' as.polygons(x, ...) ## S4 method for signature 'freeTilePlan' as.polygons(x, ...) ## S4 method for signature 'pointTilePlan' as.polygons(x, ...) ## S4 method for signature 'spatialTilePlan' as.polygons(x, ...)## S4 method for signature 'tilePlan' as.polygons(x, ...) ## S4 method for signature 'freeTilePlan' as.polygons(x, ...) ## S4 method for signature 'pointTilePlan' as.polygons(x, ...) ## S4 method for signature 'spatialTilePlan' as.polygons(x, ...)
x |
|
... |
additional arguments (ignored) |
SpatVector of polygons
Other tile* methods:
arith,
bracket,
centroids,
dim,
dollar,
double_bracket,
ext,
intersect,
plot
Get a set of tile bounds from a tile* object. Values are always returned as
a list, even when length one to reduce surprises with lapply() usage.
## S4 method for signature 'tilePlan,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,missing,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,numeric,numeric,missing' x[ i, j, tile_fun = .spat_tile_bounds, fun = function(x) x, zero = FALSE, expand_grid = TRUE, drop ] ## S4 method for signature 'tilePlan,missing,missing,missing' x[i, j] ## S4 method for signature 'tilePlan,numeric,missing,logical' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,missing,numeric,logical' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,numeric,numeric,logical' x[i, j, expand_grid = TRUE, ..., drop] ## S4 method for signature 'tilePlan,missing,missing,logical' x[i, j, drop] ## S4 method for signature 'freeTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'pointTilePlan,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'pixelTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'spatialTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,missing,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,.index,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,missing,.index,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,numeric,.index,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileIterator,missing,missing,missing' x[i, j, ..., drop = TRUE] ## S4 replacement method for signature 'tileIterator,missing,missing' x[i, j, ...] <- value ## S4 method for signature 'tileSelection,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileSelection,numeric,missing,logical' x[i, j, ..., drop = TRUE]## S4 method for signature 'tilePlan,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,missing,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,numeric,numeric,missing' x[ i, j, tile_fun = .spat_tile_bounds, fun = function(x) x, zero = FALSE, expand_grid = TRUE, drop ] ## S4 method for signature 'tilePlan,missing,missing,missing' x[i, j] ## S4 method for signature 'tilePlan,numeric,missing,logical' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,missing,numeric,logical' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tilePlan,numeric,numeric,logical' x[i, j, expand_grid = TRUE, ..., drop] ## S4 method for signature 'tilePlan,missing,missing,logical' x[i, j, drop] ## S4 method for signature 'freeTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'pointTilePlan,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'pixelTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'spatialTilePlan,numeric,numeric,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,missing,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,.index,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,missing,.index,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileGroup,numeric,.index,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileIterator,missing,missing,missing' x[i, j, ..., drop = TRUE] ## S4 replacement method for signature 'tileIterator,missing,missing' x[i, j, ...] <- value ## S4 method for signature 'tileSelection,numeric,missing,missing' x[i, j, ..., drop = TRUE] ## S4 method for signature 'tileSelection,numeric,missing,logical' x[i, j, ..., drop = TRUE]
x |
|
i |
numeric vector index if |
j |
numeric. Column index |
... |
addtional params to pass (not used). |
drop |
not used. |
tile_fun |
(internal) function used to define tile bounds based on the iterator object and ij indices. |
fun |
(internal) post-processing function used to convert bounds to
expected output. Such as |
zero |
logical. Whether to zero out padding effects. (used by default with
|
expand_grid |
logical (internal) whether to use |
value |
|
list of numeric or SpatExtent depending on underlying tilePlan
Other tile* methods:
arith,
as.polygons,
centroids,
dim,
dollar,
double_bracket,
ext,
intersect,
plot
spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 spat[1] spat[1, 2:3] spat[] px <- tilePlan("pixel") px$pxdims <- c(1000, 1000) px$ncols <- 200 px$nrows <- 200 px[1] px[2, 2:3] px[]spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 spat[1] spat[1, 2:3] spat[] px <- tilePlan("pixel") px$pxdims <- c(1000, 1000) px$ncols <- 200 px$nrows <- 200 px[1] px[2, 2:3] px[]
Combine tileSelection. Should only concatenate tileSelection
build off the same plan.
## S4 method for signature 'tileSelection' c(x, ...)## S4 method for signature 'tileSelection' c(x, ...)
x |
item to concatenate |
... |
addtional items to concatenate to |
Get the centroids of the tiles. For spatialTilePlan, these will be returned
as SpatVector centroids. For non-spatial plans like pixelTileGrid, this
will be returne as a matrix.
## S4 method for signature 'tilePlan' centroids(x, fun = function(x) x, offset = c(0, 0), zero = FALSE, ...) ## S4 method for signature 'freeTilePlan' centroids(x, ...) ## S4 method for signature 'pointTilePlan' centroids(x, ...) ## S4 method for signature 'pixelTilePlan' centroids(x, fun = function(x) x, zero = TRUE, ...) ## S4 method for signature 'spatialTilePlan' centroids(x, zero = FALSE, ...)## S4 method for signature 'tilePlan' centroids(x, fun = function(x) x, offset = c(0, 0), zero = FALSE, ...) ## S4 method for signature 'freeTilePlan' centroids(x, ...) ## S4 method for signature 'pointTilePlan' centroids(x, ...) ## S4 method for signature 'pixelTilePlan' centroids(x, fun = function(x) x, zero = TRUE, ...) ## S4 method for signature 'spatialTilePlan' centroids(x, zero = FALSE, ...)
x |
|
fun |
(internal) post-processing function used to convert centroids to
expected output. Such as |
offset |
numeric of length 2. x and y values to offset by. Default is c(0, 0). |
zero |
logical. Whether to zero out padding effects. (used by default with
|
... |
additional params to pass. |
Other tile* methods:
arith,
as.polygons,
bracket,
dim,
dollar,
double_bracket,
ext,
intersect,
plot
x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 centroids(x)x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 centroids(x)
Get dimension characteristics of the tilePlan tiling plan. These
produce information on how the tiles are arrayed in rows nrow(), cols ncol(),
and total number of tiles length().
length<-() is a special case and currently only used with spatialTilePlan,
where it is used to request a minimum number of tiles to generate.
## S4 method for signature 'tilePlan' nrow(x) ## S4 method for signature 'tilePlan' ncol(x) ## S4 method for signature 'tilePlan' length(x) ## S4 method for signature 'tilePlan' dim(x) ## S4 replacement method for signature 'spatialTilePlan' length(x) <- value ## S4 method for signature 'tileGroup' length(x) ## S4 method for signature 'tileIterator' length(x) ## S4 method for signature 'tileSelection' length(x)## S4 method for signature 'tilePlan' nrow(x) ## S4 method for signature 'tilePlan' ncol(x) ## S4 method for signature 'tilePlan' length(x) ## S4 method for signature 'tilePlan' dim(x) ## S4 replacement method for signature 'spatialTilePlan' length(x) <- value ## S4 method for signature 'tileGroup' length(x) ## S4 method for signature 'tileIterator' length(x) ## S4 method for signature 'tileSelection' length(x)
x |
|
value |
number of tiles to request |
numeric or spatialTilePlan object when using length<-()
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dollar,
double_bracket,
ext,
intersect,
plot
x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) length(x) <- 9 length(x) dim(x) nrow(x) ncol(x)x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) length(x) <- 9 length(x) dim(x) nrow(x) ncol(x)
Get and set tile metadata. Some params can also be modified with
this operator, for example the $pad value or $pxdims, $ncols, or
$nrows for pixelTilePlan.
$tile is a default created piece of metadata.
## S4 replacement method for signature 'tilePlan' x$name <- value ## S4 method for signature 'tilePlan' x$name ## S4 replacement method for signature 'freeTilePlan' x$name <- value ## S4 method for signature 'freeTilePlan' x$name ## S4 replacement method for signature 'pointTilePlan' x$name <- value ## S4 method for signature 'pointTilePlan' x$name ## S4 replacement method for signature 'pixelTilePlan' x$name <- value ## S4 method for signature 'pixelTilePlan' x$name ## S4 replacement method for signature 'tileGroup' x$name <- value ## S4 method for signature 'tileGroup' x$name ## S4 method for signature 'tileSelection' x$name ## S4 replacement method for signature 'tileSelection' x$name <- value## S4 replacement method for signature 'tilePlan' x$name <- value ## S4 method for signature 'tilePlan' x$name ## S4 replacement method for signature 'freeTilePlan' x$name <- value ## S4 method for signature 'freeTilePlan' x$name ## S4 replacement method for signature 'pointTilePlan' x$name <- value ## S4 method for signature 'pointTilePlan' x$name ## S4 replacement method for signature 'pixelTilePlan' x$name <- value ## S4 method for signature 'pixelTilePlan' x$name ## S4 replacement method for signature 'tileGroup' x$name <- value ## S4 method for signature 'tileGroup' x$name ## S4 method for signature 'tileSelection' x$name ## S4 replacement method for signature 'tileSelection' x$name <- value
x |
|
name |
name of item to get or set |
value |
value to set |
metadata value when using getter or tile* object when using the setter
function.
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dim,
double_bracket,
ext,
intersect,
plot
x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 x$test <- sprintf("test_value_%d", x$tile) x$test tile_ext <- x[5][[1]] attr(tile_ext, "test")x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 x$test <- sprintf("test_value_%d", x$tile) x$test tile_ext <- x[5][[1]] attr(tile_ext, "test")
[[ can be used to get the table of metadata for a specific tile.
## S4 method for signature 'tilePlan,numeric,missing' x[[i, j, ...]] ## S4 method for signature 'tilePlan,missing,missing' x[[i, j, ...]] ## S4 method for signature 'tilePlan,missing,character' x[[i, j, ...]] ## S4 method for signature 'tilePlan,numeric,character' x[[i, j, ...]] ## S4 replacement method for signature 'tilePlan,numeric,character' x[[i, j, ...]] <- value ## S4 replacement method for signature 'tilePlan,missing,character' x[[i, j, ...]] <- value## S4 method for signature 'tilePlan,numeric,missing' x[[i, j, ...]] ## S4 method for signature 'tilePlan,missing,missing' x[[i, j, ...]] ## S4 method for signature 'tilePlan,missing,character' x[[i, j, ...]] ## S4 method for signature 'tilePlan,numeric,character' x[[i, j, ...]] ## S4 replacement method for signature 'tilePlan,numeric,character' x[[i, j, ...]] <- value ## S4 replacement method for signature 'tilePlan,missing,character' x[[i, j, ...]] <- value
x |
|
i |
|
j |
|
... |
not used. |
value |
|
data.frame
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dim,
dollar,
ext,
intersect,
plot
spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 spat[[1]]spat <- tilePlan("spatial") ext(spat) <- c(0, 1000, 0, 1000) length(spat) <- 9 spat[[1]]
Get and set a spatial extent.
## S4 method for signature 'pointTilePlan' ext(x, ...) ## S4 replacement method for signature 'pointTilePlan,ANY' ext(x) <- value ## S4 method for signature 'spatialTilePlan' ext(x, ...) ## S4 replacement method for signature 'spatialTilePlan,ANY' ext(x) <- value## S4 method for signature 'pointTilePlan' ext(x, ...) ## S4 replacement method for signature 'pointTilePlan,ANY' ext(x) <- value ## S4 method for signature 'spatialTilePlan' ext(x, ...) ## S4 replacement method for signature 'spatialTilePlan,ANY' ext(x) <- value
x |
|
... |
addtional params to pass (none implemented) |
value |
|
tilePlan if ext<-() and SpatExtent if ext()
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dim,
dollar,
double_bracket,
intersect,
plot
{tilework} provides an extensible framework for:
The framework accesses and processes data according to the following flow:
tileApply(x, tiles, ...) # operate across tiles
getTile(x, tiles, ..., get_params) # get data for tiles
bounds <- tiles[...] # generate bounds info
getBoundedData(x, bounds, get_params) # data extraction
Expanded roles:
tileApply()- handles the function parallelization strategy and logic.
getTile() - deals with any interactions between the data type and the tile
indexing. Operations that should be performed before extraction of the
actual data by tile bounds should happen at this level.
getBoundedData() - performs the low-level data extraction from the data
source, provided with a set of bounds information (commonly a SpatExtent or
numeric of c(xmin, xmax, ymin, ymax))
{tilework} functionalities can be extended to work with other data
types/backends by creating a getBoundedData() method for the data type.
Optional additional steps:
Method for tileApply() should also be added if there are pre-run
validation/data collection steps needed.
Method for getTile() should be added if there are operations that affect
data/tile interactions.
Other tilework extension:
redispatch_tileapply,SpatRaster,freeTilePlan-method,
token-class
if (FALSE) { # Extending for a custom data type: setMethod( "getBoundedData", signature("MyDataType", "SpatExtent"), function(x, bound) { # Your data extraction logic here my_extract_function(x, bound) } ) }if (FALSE) { # Extending for a custom data type: setMethod( "getBoundedData", signature("MyDataType", "SpatExtent"), function(x, bound) { # Your data extraction logic here my_extract_function(x, bound) } ) }
Create an empty freeTilePlan. Populate bounds via $bounds<- or use
quadtreePlan() to generate adaptive bounds automatically.
freeTilePlan(...)freeTilePlan(...)
... |
additional params passed to |
Other tile plans:
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilePlan-class,
tilework-class
Tile plan defined by explicit per-tile bounds with no required uniformity
in size or spacing. The bounds matrix is the canonical representation —
there is no center-plus-dims formula. Always returns SpatExtent from
[i]. Primary use case is as the output of adaptive spatial decomposition
algorithms such as quadtrees on vector/point data.
Tile plan defined by explicit per-tile bounds with no required uniformity in size or spacing. Unlike grid-based plans, the bounds matrix is the canonical representation — tile positions are not computed from a formula.
The primary use case is adaptive spatial decomposition of vector/point
data, where high-density regions require small tiles and sparse regions
can use large tiles. See quadtreePlan() for the iterative construction
workflow.
boundsmatrix. n x 4 matrix: xmin, xmax, ymin, ymax (one row per tile).
freeTilePlan() creates an empty instance.
$bounds<- sets the n x 4 bounds matrix (xmin, xmax, ymin, ymax).
Setting bounds triggers reinitialization of @n, @dims, and
@metadata.
$pad / + / - add spatial padding as with other plans.
@tile_dims is intentionally not populated. Operations that require
uniform tile dimensions (e.g. extend = TRUE in getTile()) are not
supported.
nrow() returns n, ncol() returns 1, length() returns n.
plot() works via the existing spatial extent preview.
@tile_dims is intentionally not populated. Any method that requires
uniform tile dimensions will not work with freeTilePlan.
Other tile plans:
freeTilePlan(),
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilePlan-class,
tilework-class
tp <- freeTilePlan() tp$bounds <- rbind( c(0, 50, 0, 50), c(50, 100, 0, 50), c(0, 50, 50, 100), c(50, 100, 50, 100) ) length(tp) tp[2] plot(tp)tp <- freeTilePlan() tp$bounds <- rbind( c(0, 50, 0, 50), c(50, 100, 0, 50), c(0, 50, 50, 100), c(50, 100, 50, 100) ) length(tp) tp[2] plot(tp)
Subset or otherwise make available only the data that is within the provided bounds. Accepted bounds types depends on the data.
A related lower-level function is getTile() which handles data/tile*
interactions and bound info collection. getTile() calls this generic
for its data selection capabilities.
## S4 method for signature 'SpatRaster,numeric' getBoundedData(x, bound, extend = FALSE, fill = NA) ## S4 method for signature 'SpatRaster,SpatExtent' getBoundedData(x, bound, extend = FALSE, fill = NA) ## S4 method for signature 'SpatVectorProxy,SpatExtent' getBoundedData(x, bound)## S4 method for signature 'SpatRaster,numeric' getBoundedData(x, bound, extend = FALSE, fill = NA) ## S4 method for signature 'SpatRaster,SpatExtent' getBoundedData(x, bound, extend = FALSE, fill = NA) ## S4 method for signature 'SpatVectorProxy,SpatExtent' getBoundedData(x, bound)
x |
data |
bound |
bounds to filter with |
extend |
logical (default = |
fill |
numeric. if |
Depending on the method, different bound types are used. Current expected patterns are:
numeric of length 4 (xmin, xmax, ymin, ymax)
SpatExtent
SpatRaster snappinggetBoundedData() is implemented for SpatRaster, SpatExtent using
terra::window(). This uses terra's default snapping behavior (equivalent to
snap = "near" in terra::crop()), with no way to set another strategy.
For more precise boundary control:
Use pixel-based indexing via the SpatRaster, numeric method
Use pixelTilePlan() for exact pixel-level tiling
Add padding with tiles + pad_value to ensure spatial context
Use terra::crop() directly if you need snap = "out" or snap = "in"
Adjacent tiles share exact boundaries. Since {tilework} does not know the
format or representation of the underlying data, it does not enforce whether
those boundaries are inclusive or exclusive. It is up to the
getBoundedData() implementation to decide how features on shared edges are
handled.
The existing {terra} methods do not implement inclusive/exclusive boundary
control because raster pixel snapping and padding make exact boundary
behavior largely irrelevant for that format. For point or tabular data,
features on a shared boundary may appear in multiple tiles unless the
implementation applies its own filtering (e.g. >= vs > comparisons).
The tile's grid position can be passed via get_params from getTile() to
inform which edges are interior.
Other tile processing:
getTile(),
tileApply(),
tileApply-group,
tileApply-iterator,
tileApply-plan
f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) pixel_selection <- getBoundedData(r, c(10, 50, 30, 40)) plot(pixel_selection) extent_selection <- getBoundedData(r, ext(6, 6.5, 49.7, 50)) plot(extent_selection)f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) pixel_selection <- getBoundedData(r, c(10, 50, 30, 40)) plot(pixel_selection) extent_selection <- getBoundedData(r, ext(6, 6.5, 49.7, 50)) plot(extent_selection)
Get specific tile(s) from the data based on a tile* object. Indexing via
i and j params is used to select the tile(s) to get, in a manner similar
to [ extraction of bounds from tile* objects.
All params are usable with any method except when otherwise stated.
This generic handles data/tile* object interactions. For simple selection
of data based on a set of bounds, see the low-level generic getBoundedData().
The character method only works for filetypes readable by {terra}.
New x data types should automatically work with this generic as long as a
getBoundedData() method is created for it. A specific getTile() method
is only needed if additional params/operations should be performed before
the tile selection, (for example layer selection with raster data).
## S4 method for signature 'token,tilePlan' getTile(x, tiles, ...) ## S4 method for signature 'ANY,tilePlan' getTile(x, tiles, i = NULL, j, pad = NULL, sel_params = list(), ...) ## S4 method for signature 'character,tilePlan' getTile(x, tiles, prefer = NULL, ext = NULL, ...) ## S4 method for signature 'SpatRaster,tilePlan' getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...) ## S4 method for signature 'SpatRaster,pointTilePlan' getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...) ## S4 method for signature 'SpatRaster,tileGroup' getTile(x, tiles, i, j, lyr = NULL, ...) ## S4 method for signature 'ANY,tileGroup' getTile(x, tiles, i, j, ...) ## S4 method for signature 'ANY,tileIterator' getTile(x, tiles, advance = TRUE, ...)## S4 method for signature 'token,tilePlan' getTile(x, tiles, ...) ## S4 method for signature 'ANY,tilePlan' getTile(x, tiles, i = NULL, j, pad = NULL, sel_params = list(), ...) ## S4 method for signature 'character,tilePlan' getTile(x, tiles, prefer = NULL, ext = NULL, ...) ## S4 method for signature 'SpatRaster,tilePlan' getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...) ## S4 method for signature 'SpatRaster,pointTilePlan' getTile(x, tiles, lyr = NULL, extend = FALSE, fill = NA, ...) ## S4 method for signature 'SpatRaster,tileGroup' getTile(x, tiles, i, j, lyr = NULL, ...) ## S4 method for signature 'ANY,tileGroup' getTile(x, tiles, i, j, ...) ## S4 method for signature 'ANY,tileIterator' getTile(x, tiles, advance = TRUE, ...)
x |
data |
tiles |
|
... |
additional params to pass to |
i |
ANY except tileIterator tile vector index or row index if |
j |
ANY except tileIterator tile col index |
pad |
(optional) additional padding to apply before tile retrieval.
Useful for temporarily increasing padding without affecting |
sel_params |
list of named params to pass to the tile |
prefer |
|
ext |
|
lyr |
|
extend |
|
fill |
|
advance |
|
list of tile data
Other tile processing:
getBoundedData(),
tileApply(),
tileApply-group,
tileApply-iterator,
tileApply-plan
f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) tp <- tilePlan("pixel") tp$pxdims <- dim(r)[1:2] tp$nrows <- 10 tp$ncols <- 10 # get tiles from specific array grid indices. tile_list <- getTile(f, tp, i = 3, j = 5) force(tile_list) plot(tile_list[[1]]) # get tiles via iteration iter <- tileIterator(tp, batch_size = 3) b1 <- getTile(r, iter) # get first batch of 3... b2 <- getTile(r, iter) # get second batch of 3...f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) tp <- tilePlan("pixel") tp$pxdims <- dim(r)[1:2] tp$nrows <- 10 tp$ncols <- 10 # get tiles from specific array grid indices. tile_list <- getTile(f, tp, i = 3, j = 5) force(tile_list) plot(tile_list[[1]]) # get tiles via iteration iter <- tileIterator(tp, batch_size = 3) b1 <- getTile(r, iter) # get first batch of 3... b2 <- getTile(r, iter) # get second batch of 3...
Return a tileSelection of tiles whose padded bounds intersect the query
region y. Tile indices are available via $tile on the result.
Accepts any object coercible via ext() for an axis-aligned query, or a
SpatVector polygon for an exact query (e.g. a back-projected parallelogram
from a rotated affine crop).
For SpatVector input, an AABB pre-cull is applied first, then exact
polygon-rectangle intersection is tested on candidates via terra::relate().
spatialTilePlan uses an analytic grid formula for the AABB case (O(1)
range computation). freeTilePlan uses a vectorized bounds check. All
other tilePlan subclasses fall back to building polygons via as.polygons().
## S4 method for signature 'tilePlan,ANY' intersect(x, y) ## S4 method for signature 'freeTilePlan,ANY' intersect(x, y) ## S4 method for signature 'spatialTilePlan,ANY' intersect(x, y)## S4 method for signature 'tilePlan,ANY' intersect(x, y) ## S4 method for signature 'freeTilePlan,ANY' intersect(x, y) ## S4 method for signature 'spatialTilePlan,ANY' intersect(x, y)
x |
|
y |
spatial query – any object coercible via |
tileSelection. Use $tile to access tile indices.
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dim,
dollar,
double_bracket,
ext,
plot
# spatialTilePlan -- axis-aligned query tp <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 16) e <- ext(20, 60, 20, 60) sel <- intersect(tp, e) sel$tile # indices of intersecting tiles length(sel) # number of tiles hit plot(sel) plot(e, add = TRUE, border = "cyan") # with padding -- tiles near the border are included tp_pad <- tp + 5 sel_pad <- intersect(tp_pad, terra::ext(20, 60, 20, 60)) length(sel_pad) >= length(sel) # TRUE: padding pulls in border tiles # freeTilePlan -- quadtree bounds fp <- freeTilePlan() fp$bounds <- rbind( c(0, 50, 0, 50), c(50, 100, 0, 50), c(0, 50, 50, 100), c(50, 100, 50, 100) ) intersect(fp, terra::ext(40, 60, 40, 60))$tile # all 4 tiles touch the centre # SpatVector polygon query (e.g. back-projected rotated crop) corners <- rbind(c(30, 10), c(70, 10), c(90, 50), c(10, 50), c(30, 10)) poly <- terra::vect(corners, type = "polygons") sel_poly <- intersect(tp, poly) sel_poly$tile plot(sel_poly) plot(poly, add = TRUE, border = "cyan")# spatialTilePlan -- axis-aligned query tp <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 16) e <- ext(20, 60, 20, 60) sel <- intersect(tp, e) sel$tile # indices of intersecting tiles length(sel) # number of tiles hit plot(sel) plot(e, add = TRUE, border = "cyan") # with padding -- tiles near the border are included tp_pad <- tp + 5 sel_pad <- intersect(tp_pad, terra::ext(20, 60, 20, 60)) length(sel_pad) >= length(sel) # TRUE: padding pulls in border tiles # freeTilePlan -- quadtree bounds fp <- freeTilePlan() fp$bounds <- rbind( c(0, 50, 0, 50), c(50, 100, 0, 50), c(0, 50, 50, 100), c(50, 100, 50, 100) ) intersect(fp, terra::ext(40, 60, 40, 60))$tile # all 4 tiles touch the centre # SpatVector polygon query (e.g. back-projected rotated crop) corners <- rbind(c(30, 10), c(70, 10), c(90, 50), c(10, 50), c(30, 10)) poly <- terra::vect(corners, type = "polygons") sel_poly <- intersect(tp, poly) sel_poly$tile plot(sel_poly) plot(poly, add = TRUE, border = "cyan")
Utility function to create multiple independent walkers for parallel processing. Each iterator will have the same underlying iterator but independent state.
## S4 method for signature 'tileIterator' iterSplit(tiles, n, batch_size = NULL, distribute = TRUE, ...)## S4 method for signature 'tileIterator' iterSplit(tiles, n, batch_size = NULL, distribute = TRUE, ...)
tiles |
tileIterator object |
n |
integer. Number of iterators to create |
batch_size |
integer (optional). Assign a batch size for each iterator.
If not provided, inherits same |
distribute |
logical (default = |
... |
additional params to pass (none implemented) |
Other tile orchestration:
tileGroup(),
tileGroup-class,
tileIterator(),
tileIterator-class
tileApply() is parallelized through either future.apply::future_lapply()
(default) or BiocParallel::bplapply().
Additional parameters to control parallelization can be passed as a list
through the parallel_params argument in tileApply() methods.
getTileworkParMethod() setTileworkParMethod(method)getTileworkParMethod() setTileworkParMethod(method)
method |
character. Either |
getTileworkParMethod returns character. The parallelization
method being used.
setTileworkParMethod returns NULL invisibly
Available parameters for the parallel_params list:
method - character. Either "future" (default) or "biocparallel".
Which parallelization framework to use. Overrides setTileworkParMethod()
future_seed - logical (default = TRUE). Passed to the future.seed
parameter in future.apply::future_lapply(). Only used when
method = "future".
BPPARAM - a {BiocParallel} parallelization parameter object. Only
used when method = "biocparallel".
Additional parameters relevant to the underlying functions may also be
passed in the list and will be forwarded via ....
"tilework.warn_sequential" - logical (default = TRUE). Whether to
warn when using parallelization defaults that run sequentially
(i.e., future::sequential() or BiocParallel::SerialParam()).
"tilework.par_method" - character (default = "future"). Either
"future" or "biocparallel". Which parallelization framework to use.
"tilework.bpparam" - a {BiocParallel} parameter object. Set this to
automatically use a specific BPPARAM, analogous to setting a
future::plan() for the {future} backend.
Other parallel settings:
tilework_management
getTileworkParMethod() # Set to BiocParallel setTileworkParMethod("biocparallel") getTileworkParMethod() ## Not run: # Example pseudocode for how to use parallelization # Using future backend with multisession future::plan(future::multisession, workers = 4) tileApply(x, tiles, fun) # Using BiocParallel backend setTileworkParMethod("biocparallel") options(tilework.bpparam = BiocParallel::SnowParam(workers = 4)) tileApply(x, tiles, fun) # Or override per-call setTileworkParMethod("future") # global default is future tileApply(x, tiles, fun, parallel_params = list(method = "biocparallel") ) # but use BiocParallel here # Suppress sequential warnings options(tilework.warn_sequential = FALSE) ## End(Not run)getTileworkParMethod() # Set to BiocParallel setTileworkParMethod("biocparallel") getTileworkParMethod() ## Not run: # Example pseudocode for how to use parallelization # Using future backend with multisession future::plan(future::multisession, workers = 4) tileApply(x, tiles, fun) # Using BiocParallel backend setTileworkParMethod("biocparallel") options(tilework.bpparam = BiocParallel::SnowParam(workers = 4)) tileApply(x, tiles, fun) # Or override per-call setTileworkParMethod("future") # global default is future tileApply(x, tiles, fun, parallel_params = list(method = "biocparallel") ) # but use BiocParallel here # Suppress sequential warnings options(tilework.warn_sequential = FALSE) ## End(Not run)
Utility class for defining pixel-exact tiles of images.
pixelTilePlan(pxdims = NULL, ncols = NULL, nrows = NULL, ...)pixelTilePlan(pxdims = NULL, ncols = NULL, nrows = NULL, ...)
pxdims |
numeric of length 2. Pixel dimensions |
ncols |
numeric. Tile width in pixels. Equivalent to |
nrows |
numeric. Tile height in pixels. Equivalent to |
... |
additional params passed to |
pxdimspixel dimensions to iterate across
nnumeric. Number of tiles to create.
dimsnumeric. Number of rows/cols in the array of tiles
tile_dimsnumeric. Row/col dimensions of each tile
padnumeric. Tile padding
metadatadata.frame. Metadata per tile
pixelTilePlan(pxdims, ncols, nrows) creates an instance.
All three params can also be set after construction via $pxdims<-,
$ncols<-, and $nrows<-.
length(), dim(), nrow(), ncol() report tile counts and layout.
[i] and [i, j] indexing can be used to select tiles, similarly to a
matrix. [] without any indexing will return the entire set of indices
as a list. Indices are returned as lists of integer vectors of length 4,
with values xmin, xmax, ymin, ymax. Extracted vectors of indices will
have metadata attached via attr(). See metadata section below.
+/- can be used to add or subtract padding to each of the tiles.
Each bound will be expanded by the pad value. Pad values may only be
integer values.
plot() can be used to check the layout of the tiles.
The pixelTilePlan object can contain metadata. By default after extent and
tiles setup, a column called "tile" will be set up that simply records
which tile it is.
$ and $<- can be used to get and set specific metadata, the "pxdims"
which are the pixel dimensions of the image to iterate across, "ncols"
and "nrows" which are the px dims of a tile, and the "pad" value.
[[i]] selection will pull specific metadata rows corresponding to the
selected tiles.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilePlan-class,
tilework-class
x <- pixelTilePlan(pxdims = c(100, 100), ncols = 20) length(x) # check how many tiles there are dim(x) nrow(x) ncol(x) # previewing plot(x) # tile padding y <- x + 3 plot(y, alpha = 0.3) # red border shows the image space # this is now larger than the original space. # negative padding z <- x - 5 plot(ext(c(0, 100, -100, 0))) plot(z, add = TRUE) # tile selection x[5] x[2, 2:3] # metadata x$tile x$fname <- sprintf("tile_%03d.tif", x$tile) x[[5]] x[[1:3]]$fnamex <- pixelTilePlan(pxdims = c(100, 100), ncols = 20) length(x) # check how many tiles there are dim(x) nrow(x) ncol(x) # previewing plot(x) # tile padding y <- x + 3 plot(y, alpha = 0.3) # red border shows the image space # this is now larger than the original space. # negative padding z <- x - 5 plot(ext(c(0, 100, -100, 0))) plot(z, add = TRUE) # tile selection x[5] x[2, 2:3] # metadata x$tile x$fname <- sprintf("tile_%03d.tif", x$tile) x[[5]] x[[1:3]]$fname
tilePlan
Plot and preview the tile plan. This is likely to be very slow if there are a lot of tiles (in the neighborhood of >10,000)
## S4 method for signature 'tilePlan,missing' plot(x, values = "tile", color_as_factor = FALSE, ...) ## S4 method for signature 'freeTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'pointTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'pixelTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'tileSelection,missing' plot(x, y, border = "red", ...)## S4 method for signature 'tilePlan,missing' plot(x, values = "tile", color_as_factor = FALSE, ...) ## S4 method for signature 'freeTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'pointTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'pixelTilePlan,missing' plot(x, y, ...) ## S4 method for signature 'tileSelection,missing' plot(x, y, border = "red", ...)
x |
|
values |
character. Metadata item to color tiles as. Default is |
color_as_factor |
logical (default = |
... |
additional params to pass |
y |
not used. |
border |
|
Other tile* methods:
arith,
as.polygons,
bracket,
centroids,
dim,
dollar,
double_bracket,
ext,
intersect
x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 plot(x) x$new_meta <- rev(head(letters, 9)) plot(x, value = "new_meta")x <- tilePlan() ext(x) <- c(0, 100, 0, 100) length(x) <- 9 plot(x) x$new_meta <- rev(head(letters, 9)) plot(x, value = "new_meta")
Tile plan where each tile is centered on a user-supplied (x, y) coordinate with a uniform tile size. The center coordinates are primary — for use cases like survey sampling, object detection patches, or site-centered extractions where the point location is semantically meaningful.
All coordinates, tile dims, and padding live in one consistent reference
frame set by $input ("spatial" for CRS units, "pixel" for pixel
indices). The bound type returned by getTile() is a separate concern
controlled by $output and resolved at extraction time.
pointTilePlan( input = c("spatial", "pixel"), output = input, coords = NULL, width = NULL, height = NULL, ... )pointTilePlan( input = c("spatial", "pixel"), output = input, coords = NULL, width = NULL, height = NULL, ... )
input |
character. Coordinate space of coords, dims, and padding:
|
output |
character. Bound type returned by |
coords |
n x 2 matrix or data frame of tile center coordinates (columns:
x, y). Data frames are coerced via |
width |
numeric. Uniform tile width in input coordinate units. Equivalent
to |
height |
numeric. Uniform tile height in input coordinate units.
Equivalent to |
... |
additional params to pass to |
coordsmatrix. n x 2 matrix of tile center coordinates (columns: x, y).
inputcharacter. Coordinate space of coords, dims, and padding:
"spatial" for CRS units, "pixel" for pixel indices.
outputcharacter. Bound type returned by getTile(): "spatial" or
"pixel". Defaults to input. Evaluated at getTile() time, not [i].
rast_dimsnumeric. c(nrow, ncol) of the reference raster. Required
for plot() when input = "pixel" and for cross-mode conversion without a
raster.
extentnumeric. c(xmin, xmax, ymin, ymax) of the reference raster.
Required for pixel → CRS conversion without a raster.
nnumeric. Number of tiles (equals number of input points).
dimsinteger. Always c(n, 1L).
tile_dimsnumeric. Uniform c(height, width) in input coordinate units.
padnumeric. Tile padding in input coordinate units.
metadatadata.frame. Per-tile metadata; always has "tile", "x",
and "y" columns.
pointTilePlan() creates an instance. input defaults to "spatial";
output defaults to input.
$coords<- sets the n x 2 center coordinate matrix (columns: x, y).
Data frames are also accepted and coerced via as.matrix().
$width<- / $height<- set the uniform tile width and height in input
coordinate units.
$input<- / $output<- change the coordinate / output mode.
$nrows<- / $ncols<- are aliases for $height<- / $width<- for
consistency with pixelTilePlan when using pixel coordinates.
$rast_dims<- sets c(nrow, ncol) of the reference raster — required
for plot() when input = "pixel" and for cross-mode getTile() calls
when no raster is provided.
$extent<- sets c(xmin, xmax, ymin, ymax) — required for
pixel → CRS conversion without a raster.
| Scenario | @rast_dims |
@extent |
input = "spatial", output = "spatial" |
not needed | not needed |
input = "pixel", output = "pixel" |
for plot() |
not needed |
input = "pixel", output = "spatial" |
required | required |
input = "spatial", output = "pixel" |
required | derived |
When getTile() is called with a SpatRaster, the raster's own metadata
is used for conversion — @rast_dims and @extent are only needed for
standalone operations (plotting, validation) without a raster.
[i] returns bounds in the input coordinate space: SpatExtent when
input = "spatial", integer[4] when input = "pixel". $output is not
consulted at [i] time. Use getTile() for output-mode conversion.
When input = "pixel", tile dims should be odd for an unambiguous center
pixel. With even dimensions center ± width/2 produces non-integer bounds
that are truncated by as.integer().
After setting coords, metadata is auto-populated with "tile", "x", and
"y" columns. Additional columns can be added via $<-.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilePlan-class,
tilework-class
tp <- pointTilePlan("spatial") tp$coords <- cbind(x = c(10, 20, 30), y = c(5, 15, 25)) tp$width <- 5 tp$height <- 5 tp length(tp) tp[1] tp[2:3] centroids(tp) plot(tp) # CRS input, pixel output (raster needed at getTile time) tp2 <- pointTilePlan(input = "spatial", output = "pixel") tp2$coords <- cbind(x = c(10, 20), y = c(5, 15)) tp2$width <- 100 tp2$height <- 100tp <- pointTilePlan("spatial") tp$coords <- cbind(x = c(10, 20, 30), y = c(5, 15, 25)) tp$width <- 5 tp$height <- 5 tp length(tp) tp[1] tp[2:3] centroids(tp) plot(tp) # CRS input, pixel output (raster needed at getTile time) tp2 <- pointTilePlan(input = "spatial", output = "pixel") tp2$coords <- cbind(x = c(10, 20), y = c(5, 15)) tp2$width <- 100 tp2$height <- 100
Iteratively subdivides a spatial extent into a freeTilePlan by running
FUN on each tile and splitting any tile whose value exceeds threshold.
Uses tileApply() internally so each pass can be parallelized via
future::plan().
FUN must return a single numeric scalar per tile — typically a point
count, density, or variance measure. Tiles are split into four equal
quadrants until all tiles are either below threshold or smaller than
min_tile_size.
quadtreePlan( x, tiles = NULL, FUN = nrow, threshold, min_tile_size = NULL, max_depth = 10L, verbose = NULL, ... )quadtreePlan( x, tiles = NULL, FUN = nrow, threshold, min_tile_size = NULL, max_depth = 10L, verbose = NULL, ... )
x |
input data (e.g. |
tiles |
starting |
FUN |
function. Applied to each tile; must return a single numeric. |
threshold |
numeric. Tiles with |
min_tile_size |
numeric. Minimum tile side length (in CRS units). Tiles below this size are kept as leaves regardless of density. |
max_depth |
integer (default |
verbose |
logical. Whether to be verbose. |
... |
additional params passed to |
A freeTilePlan with an n_records metadata column containing
the last FUN value for each leaf tile.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
spatialTilePlan-class,
tilePlan(),
tilePlan-class,
tilework-class
# dummy data pts <- terra::vect(cbind(x = rnorm(1000, 0, 100), y = rnorm(1000, 0, 100))) pts <- rbind(pts, terra::shift(pts, dx = 1000, dy = 1000)) plot(pts) # data must exist on disk f <- tempfile(fileext = "shp") terra::writeVector(pts, f) pts <- terra::vect(f, proxy = TRUE) fp <- quadtreePlan(pts, threshold = 500L, min_tile_size = 1 ) plot(fp) # plot with items per tile plot(fp, values = "n_records")# dummy data pts <- terra::vect(cbind(x = rnorm(1000, 0, 100), y = rnorm(1000, 0, 100))) pts <- rbind(pts, terra::shift(pts, dx = 1000, dy = 1000)) plot(pts) # data must exist on disk f <- tempfile(fileext = "shp") terra::writeVector(pts, f) pts <- terra::vect(f, proxy = TRUE) fp <- quadtreePlan(pts, threshold = 500L, min_tile_size = 1 ) plot(fp) # plot with items per tile plot(fp, values = "n_records")
tileApply()
Utility generic for modifying tileApply() calls for streamlining extension
with new datatypes and forcing datatype-specific handling to be added in the
expected order.
Why
tileApply is a framework function for parallelized computing across tiled
data. It allows dispatch on up to 3 signatures (x, y, and tile),
allowing flexibility, but at the cost of normally having to deal with
combinatorial explosion of methods when extending with a new x and y
datatype. Especially the case when considering compatibility with existing
types and specific handling for different tile types.
Setting specific methods using the following example dispatch chain is ambiguous, with it being unclear whether 1 or 2 is first and whether both will be reached before 3.
(ANY, myclass, mytiles)
(myclass, ANY, mytiles)
(ANY, ANY, mytiles)
redispatch_tileapply() solves the above problems by modifying tileApply()
calls based on the classes of x and y, appropriate to the provided
tile* type.
This is done by passing x or y as input sig alongside the tiles and
dispatching on the combination, and properly routing changes when they are
x or y param specific based on the param_xy.
After a sig is processed, it is coerced to the token class signifying
that it is ready for processing and the call is passed back to tileApply()
which can either proceed with eval or pass back to redispatch_tileapply if
y is also provided.
Only when x (for single input tileApply()) or x and y are all of class
token, does the tileApply() call fully evaluate.
The default ANY, ANY method performs no modifications to the input data
or the call.
## S4 method for signature 'SpatRaster,freeTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'ANY,ANY' redispatch_tileapply( sig, tiles, default_get_params = list(), param_xy, verbose = NULL, ... ) ## S4 method for signature 'character,tileGroup' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,tileGroup' redispatch_tileapply(sig, tiles, parallel_strategy = c("groups", "tiles"), ...) ## S4 method for signature 'SpatRaster,tileGroup' redispatch_tileapply( sig, tiles, param_xy, parallel_strategy = c("groups", "tiles"), ... ) ## S4 method for signature 'ANY,tileGroup' redispatch_tileapply( sig, tiles, default_get_params = list(), default_callback = NULL, param_xy, parallel_strategy = c("groups", "tiles"), verbose = NULL, ... ) ## S4 method for signature 'character,tileIterator' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,tileIterator' redispatch_tileapply(sig, tiles, param_xy, ...) ## S4 method for signature 'SpatVector,tileIterator' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'ANY,tileIterator' redispatch_tileapply( sig, tiles, default_get_params = list(), default_callback = NULL, param_xy, verbose = NULL, ... ) ## S4 method for signature 'character,tilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,spatialTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,spatialTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,pixelTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,pointTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'character,tileSelection' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,tileSelection' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,tileSelection' redispatch_tileapply(sig, tiles, ...)## S4 method for signature 'SpatRaster,freeTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'ANY,ANY' redispatch_tileapply( sig, tiles, default_get_params = list(), param_xy, verbose = NULL, ... ) ## S4 method for signature 'character,tileGroup' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,tileGroup' redispatch_tileapply(sig, tiles, parallel_strategy = c("groups", "tiles"), ...) ## S4 method for signature 'SpatRaster,tileGroup' redispatch_tileapply( sig, tiles, param_xy, parallel_strategy = c("groups", "tiles"), ... ) ## S4 method for signature 'ANY,tileGroup' redispatch_tileapply( sig, tiles, default_get_params = list(), default_callback = NULL, param_xy, parallel_strategy = c("groups", "tiles"), verbose = NULL, ... ) ## S4 method for signature 'character,tileIterator' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,tileIterator' redispatch_tileapply(sig, tiles, param_xy, ...) ## S4 method for signature 'SpatVector,tileIterator' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'ANY,tileIterator' redispatch_tileapply( sig, tiles, default_get_params = list(), default_callback = NULL, param_xy, verbose = NULL, ... ) ## S4 method for signature 'character,tilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,spatialTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,spatialTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,pixelTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,pointTilePlan' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'character,tileSelection' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatVector,tileSelection' redispatch_tileapply(sig, tiles, ...) ## S4 method for signature 'SpatRaster,tileSelection' redispatch_tileapply(sig, tiles, ...)
sig |
data type to modify the |
tiles |
tile* object |
... |
additional params to pass |
default_get_params |
internal use: list of default Depending on Duplicate entries in |
param_xy |
character. Either "x" or "y" which param it is to modify dispatch on |
verbose |
verbosity. |
parallel_strategy |
character. |
default_callback |
internal use: Default callback function passed from an
upstream (likely more specific) |
tileApply() and redispatch_tileapply()
x only case:
tileApply(ANY, missing, ANY, ...)
redispatch_tileapply(mydata, mytiles, param_xy = "x", ...)
redispatch_tileapply(ANY, ANY, param_xy = "x", ...)
tileApply(token, missing, ANY, ...)
evaluation
x and y case:
tileApply(ANY, ANY, ANY)
redispatch_tileapply(mydata1, mytiles, param_xy = "x")
redispatch_tileapply(ANY, ANY, param_xy = "x", ...)
tileApply(token, ANY, ANY)
redispatch_tileapply(mydata2, mytiles, param_xy = "y")
redispatch_tileapply(ANY, ANY, param_xy = "y", ...)
tileApply(token, token, plan)
evaluation
This allows:
Data type-specific preprocessing, checks, and additional params to be
injected into the tileApply() call without having to interact with the core
tileApply() logic for tilewise data selection and parallelization
Avoid duplicating data type-specific handling code for each of the following
tileApply() signature types:
(mydata, missing, tiles)
(mydata, ANY, tiles)
(token, mydata, tiles)
Instead, extending developers only have to provide code for a single
redispatch_tileapply(myclass, tiles) signature, and
tileApply() <-> redispatch_tileapply() calls will iteratively apply the
appropriate preprocessing and x or y routed params.
The x and y routing and token coercion are also handled by the
redispatch_tileapply(ANY, ANY) method.
Other tilework extension:
extending_tilework,
token-class
Utility class that simplifies the setup of tiles across a spatial extent.
Tiles are stored in a lightweight format safe to be passed to child
processes. Tile SpatExtent objects can be extracted on-demand.
spatialTilePlan(ext = NULL, n = NULL, ...)spatialTilePlan(ext = NULL, n = NULL, ...)
ext |
numeric or |
n |
numeric. Desired number of tiles. Equivalent to calling
|
... |
additional params passed to |
extentnumeric. Spatial extent to tile across.
nnumeric. Number of tiles to create.
dimsnumeric. Number of rows/cols in the array of tiles
tile_dimsnumeric. Row/col dimensions of each tile
padnumeric. Tile padding
metadatadata.frame. Metadata per tile
A spatialTilePlan needs both a spatial extent to tile across and also a
request for a certain number of tiles.
spatialTilePlan(ext, n) is used to create a spatialTilePlan instance.
ext and n can also be supplied after construction via ext()<- and
length()<-.
ext() is used to check extent.
length() can be used to find out how many tiles there are.
dim()/nrow()/ncol() basic generics are implemented and return
information about how the tiles are arranged.
Note that the number of requested tiles may not be the actual length, since a grid pattern must be followed. However, the number of generated tiles will be AT LEAST the number that is requested. Generated tiles will have as square a shape as possible.
[i] and [i, j] indexing can be used to select tiles, similarly to a
matrix. [] without any indexing will return the entire set of extents
as a list. Extracted extents will have metadata attached via attr().
See metadata section below.
+/- can be used to add or subtract padding to each of the tiles. Note
that this value does not affect the setting or retrieval of extent info via
ext() and ext()<-. Each bound will be expanded by the padding value.
To avoid having to use terra::extend() when padded tiles exceed raster
extent, you can decrease the extent by the same size as the padding.
plot() can be used to check the layout of the tiles.
The spatialTilePlan object can contain metadata. By default after extent and
tiles setup, a column called "tile" will be set up that simply records
which tile it is.
$ can be used to view a specific type of metadata and the padding value
$<- can be used to set additional metadata items and the padding value
[[i]] selection will pull specific metadata rows corresponding to the
selected tiles.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
tilePlan(),
tilePlan-class,
tilework-class
x <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 8) force(x) length(x) # how many were actually generated? AT LEAST n dim(x) nrow(x) ncol(x) # previewing plot(x) # tile padding y <- x + 10 plot(ext(x), border = "red") plot(y, alpha = 0.3, add = TRUE) # this is now larger than the original space ext(y) <- ext(y) - 10 plot(y, alpha = 0.3) plot(ext(x), add = TRUE, border = "red") # now this does not exceed the image # negative padding z <- x - 5 plot(ext(x), border = "red") plot(z, add = TRUE) # tile selection x[5] x[1, 2:3] # metadata x$tile x$fname <- sprintf("tile_%03d.tif", x$tile) x[[5]] x[[1:3]]$fname # selected tiles carry metadata as attributes attr(x[4][[1]], "tile") attr(x[4][[1]], "fname")x <- spatialTilePlan(ext = c(0, 100, 0, 100), n = 8) force(x) length(x) # how many were actually generated? AT LEAST n dim(x) nrow(x) ncol(x) # previewing plot(x) # tile padding y <- x + 10 plot(ext(x), border = "red") plot(y, alpha = 0.3, add = TRUE) # this is now larger than the original space ext(y) <- ext(y) - 10 plot(y, alpha = 0.3) plot(ext(x), add = TRUE, border = "red") # now this does not exceed the image # negative padding z <- x - 5 plot(ext(x), border = "red") plot(z, add = TRUE) # tile selection x[5] x[1, 2:3] # metadata x$tile x$fname <- sprintf("tile_%03d.tif", x$tile) x[[5]] x[[1:3]]$fname # selected tiles carry metadata as attributes attr(x[4][[1]], "tile") attr(x[4][[1]], "fname")
Apply functions across tileGroup objects with control over parallelization strategy. Useful when tiles are organized into logical groups that need different processing or aggregation.
token is a stand-in for any input data class (e.g. SpatRaster,
SpatExtent, filepath, etc). See redispatch_tileapply
and extending_tilework for further information.
## S4 method for signature 'token,missing,tileGroup' tileApply( x, tiles, FUN, get_params_x = list(), parallel_strategy = c("groups", "tiles"), setup_FUN = NULL, group_FUN = NULL, callback_x = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileGroup' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, parallel_strategy = c("groups", "tiles"), setup_FUN = NULL, group_FUN = NULL, callback_x = NULL, callback_y = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... )## S4 method for signature 'token,missing,tileGroup' tileApply( x, tiles, FUN, get_params_x = list(), parallel_strategy = c("groups", "tiles"), setup_FUN = NULL, group_FUN = NULL, callback_x = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileGroup' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, parallel_strategy = c("groups", "tiles"), setup_FUN = NULL, group_FUN = NULL, callback_x = NULL, callback_y = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... )
x |
input data 1 |
tiles |
|
FUN |
function to apply to each tile |
get_params_x |
named list. Additional params to pass to |
parallel_strategy |
character. |
setup_FUN |
function. Optional per-group initialization function when
|
group_FUN |
function. Optional function to apply to each group's results |
callback_x |
function. Optional preprocessing function for |
log |
logical. Whether to log processing steps |
logpath |
character. Log file path (if log = |
simplify |
logical. Whether to flatten group results into single list. Group names will not be retained. |
parallel_params |
named param list. See parallel_params |
verbose |
verbosity. |
... |
additional params to pass to |
y |
input data 2 (optional) |
get_params_y |
named list. Additional params to pass to |
pad_y |
numeric. Additional padding applied to |
callback_y |
function. Optional preprocessing function for |
"groups" - Process groups in parallel, tiles within groups sequentially
"tiles" - Process groups sequentially, tiles within groups in parallel
Your FUN can optionally include these special parameters:
.I - tile number (integer)
.TILE - tile bounds/metadata
.R - tile row number
.C - tile column number
.GROUP - current group name (character)
.SETUP_OUT - the output of setup_FUN
Your setup_FUN can optionally include these special parameters:
.GROUP - current group name (character)
.X - the input x object
.Y - the input y object (when provided)
tileApply, tileGroup(), tileGroup
Other tile processing:
getBoundedData(),
getTile(),
tileApply(),
tileApply-iterator,
tileApply-plan
f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Create tile plan tp <- tilePlan("spatial") ext(tp) <- ext(r) length(tp) <- 16 # Organize into groups (e.g., by geographic region) tg <- tileGroup(tp, groups = list( "north" = 1:8, # northern tiles "south" = 9:16, # southern tiles "corners" = c(1, 4, 13, 16) # corner tiles )) # Process groups in parallel, with group-level aggregation results <- tileApply(r, tiles = tg, parallel_strategy = "groups", FUN = function(tile, .I, .GROUP) { # Process individual tile list( tile_id = .I, group = .GROUP, stats = terra::global(tile, c("mean", "sd"), na.rm = TRUE) ) }, group_FUN = function(group_results, .GROUP) { # Aggregate results within each group means <- sapply(group_results, function(x) x$stats$mean) list( group = .GROUP, n_tiles = length(group_results), group_mean = mean(means), group_range = range(means) ) } ) # Results organized by group str(results)f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Create tile plan tp <- tilePlan("spatial") ext(tp) <- ext(r) length(tp) <- 16 # Organize into groups (e.g., by geographic region) tg <- tileGroup(tp, groups = list( "north" = 1:8, # northern tiles "south" = 9:16, # southern tiles "corners" = c(1, 4, 13, 16) # corner tiles )) # Process groups in parallel, with group-level aggregation results <- tileApply(r, tiles = tg, parallel_strategy = "groups", FUN = function(tile, .I, .GROUP) { # Process individual tile list( tile_id = .I, group = .GROUP, stats = terra::global(tile, c("mean", "sd"), na.rm = TRUE) ) }, group_FUN = function(group_results, .GROUP) { # Aggregate results within each group means <- sapply(group_results, function(x) x$stats$mean) list( group = .GROUP, n_tiles = length(group_results), group_mean = mean(means), group_range = range(means) ) } ) # Results organized by group str(results)
Apply functions using tileIterator objects for memory-constrained batch processing. Ideal for very large datasets or when you need fine control over processing workflow.
token is a stand-in for any input data class (e.g. SpatRaster,
SpatExtent, filepath, etc). See redispatch_tileapply
and extending_tilework for further information.
## S4 method for signature 'token,missing,tileIterator' tileApply( x, tiles, FUN, get_params_x = list(), setup_FUN = NULL, callback_x = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileIterator' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, setup_FUN = NULL, callback_x = NULL, callback_y = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... )## S4 method for signature 'token,missing,tileIterator' tileApply( x, tiles, FUN, get_params_x = list(), setup_FUN = NULL, callback_x = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileIterator' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, setup_FUN = NULL, callback_x = NULL, callback_y = NULL, log = FALSE, logpath = getTileworkLogDir(), simplify = FALSE, parallel_params = list(), verbose = NULL, ... )
x |
input data 1 |
tiles |
|
FUN |
function to apply to each batch of tiles |
get_params_x |
named list. Additional params to pass to |
setup_FUN |
function. Optional per-worker initialization function. Output
is accessible within |
callback_x |
function, (advanced). If provided, programmatic escape hatch for
preprocessing |
log |
logical. Whether to log processing steps |
logpath |
character. Log file path (if log = |
simplify |
logical. Whether to flatten results |
parallel_params |
named param list. See parallel_params |
verbose |
verbosity. |
... |
additional params to pass to |
y |
input data 2 (optional) |
get_params_y |
named list. Additional params to pass to |
pad_y |
numeric. Additional padding applied to |
callback_y |
function, (advanced). If provided, programmatic escape hatch for
preprocessing |
iterSplit() is run with n = future::nbrOfWorkers() to distribute the
tiles to process across the expected number of workers, with each worker
processing its assigned tiles in batches.
Your FUN can optionally include these special parameters:
.I - tile number (integer)
.TILE - tile bounds/metadata
.R - tile row number
.C - tile column number
.POSITION - batch position range (start, end)
.BATCH - batch number within worker
.SETUP_OUT - the output of setup_FUN
Your setup_FUN can optionally include these special parameters:
.W - worker number
.X - the input x object.
.Y - the input y object (when provided)
tileApply, tileIterator(), tileIterator
Other tile processing:
getBoundedData(),
getTile(),
tileApply(),
tileApply-group,
tileApply-plan
f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Create pixel tile plan tp <- tilePlan("pixel") tp$pxdims <- dim(r)[1:2] tp$nrows <- 30 tp$ncols <- 30 # Create iterator for batch processing iter <- tileIterator(tp, batch_size = 5) # Process with worker initialization results <- tileApply(r, tiles = iter, setup_FUN = function(.W, .X) { # Initialize per-worker state list( worker_id = .W, start_time = Sys.time(), raster_info = list(nrow = nrow(.X), ncol = ncol(.X)) ) }, FUN = function(batch, .BATCH, .POSITION, .SETUP_OUT) { # Process batch of tiles batch_stats <- lapply(batch, function(tile) { terra::global(tile, "mean", na.rm = TRUE)[[1]] }) list( worker = .SETUP_OUT$worker_id, batch_num = .BATCH, tiles_processed = .POSITION, batch_mean = mean(unlist(batch_stats)) ) } ) # Check results structure str(results) # Example: Streaming processing for memory management large_iter <- tileIterator(tp, batch_size = 3) processed_count <- 0 while (large_iter$has_next) { batch <- getTile(r, large_iter) # Process batch batch_results <- lapply(batch, function(tile) { # Your processing here terra::global(tile, "mean") }) processed_count <- processed_count + length(batch) cat("Processed", processed_count, "of", length(tp), "tiles\n") }f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Create pixel tile plan tp <- tilePlan("pixel") tp$pxdims <- dim(r)[1:2] tp$nrows <- 30 tp$ncols <- 30 # Create iterator for batch processing iter <- tileIterator(tp, batch_size = 5) # Process with worker initialization results <- tileApply(r, tiles = iter, setup_FUN = function(.W, .X) { # Initialize per-worker state list( worker_id = .W, start_time = Sys.time(), raster_info = list(nrow = nrow(.X), ncol = ncol(.X)) ) }, FUN = function(batch, .BATCH, .POSITION, .SETUP_OUT) { # Process batch of tiles batch_stats <- lapply(batch, function(tile) { terra::global(tile, "mean", na.rm = TRUE)[[1]] }) list( worker = .SETUP_OUT$worker_id, batch_num = .BATCH, tiles_processed = .POSITION, batch_mean = mean(unlist(batch_stats)) ) } ) # Check results structure str(results) # Example: Streaming processing for memory management large_iter <- tileIterator(tp, batch_size = 3) processed_count <- 0 while (large_iter$has_next) { batch <- getTile(r, large_iter) # Process batch batch_results <- lapply(batch, function(tile) { # Your processing here terra::global(tile, "mean") }) processed_count <- processed_count + length(batch) cat("Processed", processed_count, "of", length(tp), "tiles\n") }
Apply functions across tilePlan-inheriting objects.
token is a stand-in for any input data class (e.g. SpatRaster,
SpatExtent, filepath, etc). See redispatch_tileapply
and extending_tilework for further information.
## S4 method for signature 'token,missing,tilePlan' tileApply( x, tiles, FUN, get_params_x = list(), log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tilePlan' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,missing,tileSelection' tileApply( x, tiles, FUN, get_params_x = list(), log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileSelection' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... )## S4 method for signature 'token,missing,tilePlan' tileApply( x, tiles, FUN, get_params_x = list(), log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tilePlan' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,missing,tileSelection' tileApply( x, tiles, FUN, get_params_x = list(), log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... ) ## S4 method for signature 'token,token,tileSelection' tileApply( x, y, tiles, FUN, get_params_x = list(), get_params_y = list(), pad_y = NULL, log = FALSE, logpath = getTileworkLogDir(), parallel_params = list(), verbose = NULL, ... )
x |
input data 1 |
tiles |
|
FUN |
function to apply to each tile |
get_params_x |
named list. Additional params to pass to |
log |
logical. Whether to log processing steps |
logpath |
character. Log file path (if log = |
parallel_params |
named param list. See parallel_params |
verbose |
verbosity. |
... |
additional params to pass to |
y |
input data 2 (optional) |
get_params_y |
named list. Additional params to pass to |
pad_y |
numeric. Additional padding applied to |
Your FUN can optionally include these special parameters:
.I - tile number (integer)
.TILE - tile bounds/metadata
.R - tile row number
.C - tile column number
tileApply, tilePlan(), spatialTilePlan, pixelTilePlan
Other tile processing:
getBoundedData(),
getTile(),
tileApply(),
tileApply-group,
tileApply-iterator
f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Spatial tiling example tp_spatial <- tilePlan("spatial") ext(tp_spatial) <- ext(r) length(tp_spatial) <- 4 # Apply function with tile metadata results <- tileApply(r, tiles = tp_spatial, FUN = function(tile, .I, .R, .C) { list( tile_id = .I, position = paste0("row_", .R, "_col_", .C), mean_value = terra::global(tile, "mean", na.rm = TRUE)[[1]] ) }) force(results) # Pixel tiling example tp_pixel <- tilePlan("pixel") tp_pixel$pxdims <- dim(r)[1:2] tp_pixel$nrows <- 50 tp_pixel$ncols <- 50 # Save tiles to disk outdir <- file.path(tempdir(), "tiles") dir.create(outdir, showWarnings = FALSE) tileApply(r, tiles = tp_pixel, FUN = function(tile, .I) { filename <- file.path(outdir, sprintf("tile_%03d.tif", .I)) terra::writeRaster(tile, filename, overwrite = TRUE) return(filename) }) list.files(outdir) unlink(outdir, recursive = TRUE)f <- system.file("ex/elev.tif", package = "terra") r <- terra::rast(f) # Spatial tiling example tp_spatial <- tilePlan("spatial") ext(tp_spatial) <- ext(r) length(tp_spatial) <- 4 # Apply function with tile metadata results <- tileApply(r, tiles = tp_spatial, FUN = function(tile, .I, .R, .C) { list( tile_id = .I, position = paste0("row_", .R, "_col_", .C), mean_value = terra::global(tile, "mean", na.rm = TRUE)[[1]] ) }) force(results) # Pixel tiling example tp_pixel <- tilePlan("pixel") tp_pixel$pxdims <- dim(r)[1:2] tp_pixel$nrows <- 50 tp_pixel$ncols <- 50 # Save tiles to disk outdir <- file.path(tempdir(), "tiles") dir.create(outdir, showWarnings = FALSE) tileApply(r, tiles = tp_pixel, FUN = function(tile, .I) { filename <- file.path(outdir, sprintf("tile_%03d.tif", .I)) terra::writeRaster(tile, filename, overwrite = TRUE) return(filename) }) list.files(outdir) unlink(outdir, recursive = TRUE)
Organize tiles from a tilePlan into hierarchical groups for batch processing. Groups can represent spatial regions, processing stages, or any logical organization of tiles.
tileGroup(tp, groups = list(), metadata = data.frame())tileGroup(tp, groups = list(), metadata = data.frame())
tp |
A tilePlan object |
groups |
Named list where each element contains tile indices for that group |
metadata |
Optional data.frame with group metadata |
Other tile orchestration:
iterSplit(),
tileGroup-class,
tileIterator(),
tileIterator-class
tp <- tilePlan("spatial") ext(tp) <- c(0, 100, 0, 100) length(tp) <- 16 # Create groups tg <- tileGroup(tp, groups = list( "g1" = 1:4, # vector indexing "g2" = list(2, 1:4), # ij indexing "g3" = list(2:4, 1:2) # selection overlaps are allowed # (not all tiles need to be selected) )) # length() returns number of groups length(tg) # Get group bounds: tg[, "g1"] # first 4 tg[, "g2"] # next 4 tg[, "g3"] # specific 6 # not recommended for large groups tg[c(3, 1), "g3"] # get nth item in group # Set active group for shorthand indexing tg$active <- "g1" # length() is based on group length when active is set length(tg) tg[2] # Position 2 from g1 # iterator can be created when active group is set iter <- tileIterator(tg, batch_size = 2) iter$next_batch() iter$next_batch() iter$next_batch() # no more items tg$active <- NULL # Clear active group by setting NULLtp <- tilePlan("spatial") ext(tp) <- c(0, 100, 0, 100) length(tp) <- 16 # Create groups tg <- tileGroup(tp, groups = list( "g1" = 1:4, # vector indexing "g2" = list(2, 1:4), # ij indexing "g3" = list(2:4, 1:2) # selection overlaps are allowed # (not all tiles need to be selected) )) # length() returns number of groups length(tg) # Get group bounds: tg[, "g1"] # first 4 tg[, "g2"] # next 4 tg[, "g3"] # specific 6 # not recommended for large groups tg[c(3, 1), "g3"] # get nth item in group # Set active group for shorthand indexing tg$active <- "g1" # length() is based on group length when active is set length(tg) tg[2] # Position 2 from g1 # iterator can be created when active group is set iter <- tileIterator(tg, batch_size = 2) iter$next_batch() iter$next_batch() iter$next_batch() # no more items tg$active <- NULL # Clear active group by setting NULL
Class for organizing tiles into hierarchical groups for batch processing. Groups can represent spatial regions, processing stages, or any logical organization of tiles.
tptilePlan. The underlying tilePlan-inheriting object
groupslist. Named list where each element contains tile indices for that group
activecharacter. Name of a group to set as active for [i] shorthand
indexing and length().
metadatadata.frame. Metadata about each group
Other tile orchestration:
iterSplit(),
tileGroup(),
tileIterator(),
tileIterator-class
Create a stateful iterator that maintains an internal position and yields batches of tiles on demand. Uses closures for serializable stateful behavior without external dependencies. Useful for streaming processing, memory-constrained environments, or when you want to process tiles in smaller chunks.
This is a structure that works with any underlying tilePlan. tileGroup
is also accepted when an $active group has already been set, or a group
is specified via tileIterator(active_group)
Note that tileIterator contains its own state and an independent copy of it
cannot be generated by assigning to a new variable. Instead, use either the
$copy() method or iterSplit().
## S4 method for signature 'tileIterator' x$name ## S4 replacement method for signature 'tileIterator' x$name <- value tileIterator( tiles = NULL, position = 0, bound = NULL, batch_size = 1, active_group = NULL )## S4 method for signature 'tileIterator' x$name ## S4 replacement method for signature 'tileIterator' x$name <- value tileIterator( tiles = NULL, position = 0, bound = NULL, batch_size = 1, active_group = NULL )
x |
|
name |
Name of method or attribute to access (See Methods section) |
value |
value to set for an attribute (See Methods section) |
tiles |
A tile* object |
position |
numeric. Set the starting position of the iterator. Default is 0. |
bound |
numeric, length of 2. Min and max indexing bounds through which
to iterate. Default is |
batch_size |
Integer. Number of tiles to return per call to next_batch() |
active_group |
character (optional). Only used when |
A tileIterator function with attached methods
[]/[]<- or $tiles/$tiles<-
get or set a tilePlan or tileGroup (with $active set). [] notation is just shorthand.
$position/$position<-
get and set current position of iterator
$batch_size/$batch_size<-
get and set batch size.
$bound/$bound<-
get and set iteration tile index bounds. This is a numeric vector of 2 (min, max)
$next_batch()/$next_indices()
get next batch or just their tile indices.
Has an advance param (default = TRUE) that advances iterator position.
When there are no more tiles, gracefully returns empty list
$peek_batch()/$peek_indices
Get next batch or indices without advancing position. When there are no more tiles, gracefully returns empty integer
$has_next
Return logical if there are still tiles left to iterate through (if position < max(bound)). Useful for while looping.
$total_tiles
Return total number of tiles in bound
$remaining
Return number of remaining indices to traverse.
$progress
Return current percentage of position traversal across bound
$reset()
Reset position to 1 before the lower bound
$copy()
Create a new tileIterator with the same settings (deep copy)
Stream processing large datasets
Memory-constrained batch processing
Progressive/incremental analysis
Checkpoint and resume workflows
Parallel processing with foreach
Other tile orchestration:
iterSplit(),
tileGroup(),
tileGroup-class,
tileIterator-class
# Create a spatial tile iterator tp <- tilePlan("spatial") ext(tp) <- c(0, 100, 0, 100) length(tp) <- 16 # Create a iterator that processes 3 tiles at a time iter <- tileIterator(tp, batch_size = 3) # Check status iter iter$has_next iter$remaining iter$progress # Get next batch batch1 <- iter$next_batch() length(batch1) # 3 tiles # Check updated status iter$progress # position has advanced # Peek at next batch without advancing peek <- iter$peek_batch() peek iter$position # unchanged # serialize and unserialize, preserving state temp <- tempfile() saveRDS(iter, temp) rm(iter) iter <- readRDS(temp) # Next batch of indices iter$next_indices() # Peek next batch of indices iter$peek_indices() # Process all remaining batches while (iter$has_next) { batch <- iter$next_batch() cat("Processing batch of", length(batch), "tiles\n") } # Reset and try different batch size iter$reset() iter$batch_size <- 5 # Apply function across all batches while (iter$has_next) { batch <- iter$next_batch() cat("Processing batch of", length(batch), "tiles\n") } # tileIterator with tileGroup tg <- tileGroup(tp, groups = list( "g1" = c(2, 4, 6, 8, 10, 12, 14, 16), "g2" = 1:16 )) tg$active <- "g1" iter <- tileIterator(tg, batch_size = 3) iter iter$next_batch() # iterator splitting siter <- iterSplit(iter, n = 2) siter# Create a spatial tile iterator tp <- tilePlan("spatial") ext(tp) <- c(0, 100, 0, 100) length(tp) <- 16 # Create a iterator that processes 3 tiles at a time iter <- tileIterator(tp, batch_size = 3) # Check status iter iter$has_next iter$remaining iter$progress # Get next batch batch1 <- iter$next_batch() length(batch1) # 3 tiles # Check updated status iter$progress # position has advanced # Peek at next batch without advancing peek <- iter$peek_batch() peek iter$position # unchanged # serialize and unserialize, preserving state temp <- tempfile() saveRDS(iter, temp) rm(iter) iter <- readRDS(temp) # Next batch of indices iter$next_indices() # Peek next batch of indices iter$peek_indices() # Process all remaining batches while (iter$has_next) { batch <- iter$next_batch() cat("Processing batch of", length(batch), "tiles\n") } # Reset and try different batch size iter$reset() iter$batch_size <- 5 # Apply function across all batches while (iter$has_next) { batch <- iter$next_batch() cat("Processing batch of", length(batch), "tiles\n") } # tileIterator with tileGroup tg <- tileGroup(tp, groups = list( "g1" = c(2, 4, 6, 8, 10, 12, 14, 16), "g2" = 1:16 )) tg$active <- "g1" iter <- tileIterator(tg, batch_size = 3) iter iter$next_batch() # iterator splitting siter <- iterSplit(iter, n = 2) siter
A stateful iterator that progresses through tiles of an underlying tilePlan
(or tileGroup if $active is set) object upon every call to $next_batch()
The closures that power this functionality are stored in @funs. The stateful
position handling is also internalized within.
funslist of closure methods
Other tile orchestration:
iterSplit(),
tileGroup(),
tileGroup-class,
tileIterator()
Create a Tiling Plan
tilePlan(type = c("spatial", "pixel", "point", "free"), ...)tilePlan(type = c("spatial", "pixel", "point", "free"), ...)
type |
character. One of |
... |
additional params passed to the specific constructor:
|
spatialTilePlan, pixelTilePlan, pointTilePlan, freeTilePlan
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan-class,
tilework-class
tilePlan("spatial") tilePlan("pixel") tilePlan("point") tilePlan("free")tilePlan("spatial") tilePlan("pixel") tilePlan("point") tilePlan("free")
Virtual parent class for tile planning objects. These objects are for planning tiles/patches of data to be operated over. Objects are indexable across rows/columns of tiles and as a vector of tiles (similarly to a matrix).
[ indexing returns a set of bounds information for selection of the
data. Tiles and bounds information are generated in a lazy fashion based on
plan parameters, avoiding overhead with large amounts of tiles.
nnumeric. Number of tiles to create.
dimsnumeric. Number of rows/cols in the array of tiles
tile_dimsnumeric. Row/col dimensions of each tile
padnumeric. Tile padding
metadatadata.frame. Metadata per tile
spatialTilePlan and pixelTilePlan for
concrete classes dealing with spatial and pixel-exact tiling respectively.
tilePlan() for creation of these objects.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilework-class
A set of utilities to control the location and naming of automated tilework outputs. These are mostly to do with logging information right now.
getTileworkLogDir() setTileworkLogDir(path) getTileworkJobID(advance = FALSE)getTileworkLogDir() setTileworkLogDir(path) getTileworkJobID(advance = FALSE)
path |
directory path to use |
advance |
logical (default = FALSE). Whether to consume this job ID |
getTileworkLogDir the logging directory to use
setTileworkLogDir returns the previous logging directory invisibly
getTileworkJobID returns the next job ID to use. If
advance = FALSE this call will be treated as a peek and will not consume
the ID.
Other parallel settings:
parallel_params
tilework
The tilework class is a class contained by all tile* classes in the
tilework package. It is a "virtual" class.
Other tile plans:
freeTilePlan(),
freeTilePlan-class,
pixelTilePlan-class,
pointTilePlan-class,
quadtreePlan(),
spatialTilePlan-class,
tilePlan(),
tilePlan-class
token ClassUtility class for flagging a piece of data as being ready for processing. This is internal machinery that is mainly useful for forcing S4 dispatch to progress in the expected order and should not be interacted with by end users.
This class and related methods are exported so developers are able to write
extending methods for tileApply() where this utility class is used.
## S4 method for signature 'token,missing,missing,missing' x[i, j, ..., drop = TRUE]## S4 method for signature 'token,missing,missing,missing' x[i, j, ..., drop = TRUE]
x, i, j, ..., drop
|
not used |
dataANY. The wrapped data object.
Other tilework extension:
extending_tilework,
redispatch_tileapply,SpatRaster,freeTilePlan-method
# Flag as being ready for processing x <- as(letters, "token") # Retrieve wrapped data x[]# Flag as being ready for processing x <- as(letters, "token") # Retrieve wrapped data x[]