--- title: "Introduction to taxodist" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Introduction to taxodist} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include=FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.align = "center" ) library(taxodist) if (is.null(taxobase$matrix)) { stop( "The installed taxobase object does not contain the reference matrix. ", "Rebuild data/taxobase.rda before building this vignette." ) } ``` ## Overview `taxodist` retrieves ordered taxonomic lineages from The Taxonomicon and uses them to compare named nodes in a classification hierarchy. The package provides functions for: - lineage retrieval; - most-recent-common-ancestor queries; - taxonomic hierarchy distances; - clade membership and filtering; - paths between hierarchy nodes; - auditable batch name resolution; - pairwise distance matrices and distances between two sets of taxa; - classification trees whose internal nodes are named clades; - clustering and principal coordinates analysis; - portable analysis bundles that can be compared over time; - session and persistent cache management. Inputs may represent species, genera, families, orders, named clades, or other nodes available in the source classification. ## What the distance represents Let \(L_A\) and \(L_B\) be the ordered lineages of two hierarchy nodes. Let \(h(A,B)\) be the depth of their most recent common ancestor, defined from their continuous common prefix. The distance is \[ d(A,B) = \begin{cases} 0, & L_A = L_B, \\ 1/h(A,B), & L_A \ne L_B. \end{cases} \] A deeper shared ancestor produces a smaller distance. Within a connected hierarchy, finite distances lie between zero and one. If two lineages have no shared root, the package returns an infinite distance. Missing lineages instead produce missing pairwise values in a distance matrix. The measure is an ultrametric within each connected hierarchy. Its mathematical properties and assumptions are discussed in: ```{r methodology-link, eval=FALSE} vignette("methodological-notes", package = "taxodist") ``` ## Distance and membership are different A descendant belongs to each clade represented among its ancestors. However, the descendant and ancestor remain distinct hierarchy nodes. For example, *Tyrannosaurus* is a member of *Dinosauria*, but the distance between the two nodes is positive: \[ d(\textit{Tyrannosaurus},\textit{Dinosauria}) = \frac{1}{\operatorname{depth}(\textit{Dinosauria})}. \] Use: - `taxo_distance()` to compare hierarchy nodes; - `is_member()` to test containment within a clade; - `taxo_path()` to inspect the complete ancestry path between nodes. The distance represents classification depth. It is not evolutionary time, genetic distance, morphological divergence, or phylogenetic branch length. ## Reproducible packaged examples The package includes `taxobase`, a reference object generated from The Taxonomicon. It allows documentation and examples to run without contacting an external service. ```{r taxobase} names(taxobase) taxobase$metadata ``` The metadata records the source, generation date, package version, and distance definition used to construct the object. ### Stored lineages ```{r stored-lineages} tail(taxobase$lineage_tyrannosaurus) tail(taxobase$lineage_homo) ``` Lineages are ordered from the root of the source classification to the queried node. ### Stored pairwise comparison ```{r stored-pairwise} taxobase$pairwise ``` A `taxodist_result` records: - the distance; - the MRCA and its depth; - the depth of each complete lineage; - the two queried names. ```{r result-structure} class(taxobase$pairwise) names(taxobase$pairwise) ``` ### Reference distance matrix ```{r reference-matrix} reference_matrix <- taxobase$matrix inherits(reference_matrix, "dist") attr(reference_matrix, "Size") head(attr(reference_matrix, "Labels")) round( as.matrix(reference_matrix)[1:6, 1:6], digits = 4 ) ``` A matrix returned by `distance_matrix()` is a standard R `dist` object. It is symmetric, has a zero diagonal, and can be supplied to functions accepting pairwise dissimilarities. ## Live lineage queries The following examples contact The Taxonomicon and are not evaluated while the vignette is built. ### Retrieving a lineage ```{r live-lineage, eval=FALSE} get_lineage("Tyrannosaurus") get_lineage("Drosophila melanogaster") ``` Numeric Taxonomicon identifiers can also be supplied directly: ```{r lineage-id, eval=FALSE} get_lineage("67263") ``` Using an identifier is useful when a name corresponds to more than one valid taxonomic entry. ### Computing a pairwise distance ```{r live-distance, eval=FALSE} result <- taxo_distance( "Tyrannosaurus", "Velociraptor" ) result result$distance result$mrca ``` ### Finding the MRCA ```{r live-mrca, eval=FALSE} mrca("Tyrannosaurus", "Velociraptor") mrca("Tyrannosaurus", "Triceratops") mrca("Tyrannosaurus", "Homo") ``` The returned MRCA depends on the classification currently provided by The Taxonomicon and may change following taxonomic revisions. ### Inspecting the path between nodes ```{r live-path, eval=FALSE} taxo_path( "Tyrannosaurus", "Triceratops" ) taxo_path( "Dinosauria", "Tyrannosaurus" ) ``` The path ascends from the first node to the MRCA and descends toward the second node. For an ancestor-descendant pair, the ancestor itself is the MRCA. ## Working with multiple taxa ### Constructing a distance matrix ```{r live-matrix, eval=FALSE} taxa <- c( "Tyrannosaurus", "Velociraptor", "Spinosaurus", "Allosaurus" ) mat <- distance_matrix( taxa, progress = TRUE ) mat ``` Lineages are retrieved once and cached for subsequent comparisons. The pairwise computation itself is vectorized, so matrices of several thousand taxa are computed in seconds once their lineages are available. ### Distances between two sets of taxa When only the distances from one set of taxa to another are needed, such as from the members of a community to a set of reference taxa, `cross_distances()` returns the rectangular matrix directly. The example below uses user-supplied lineages and therefore runs offline: ```{r cross-distances} community <- taxo_from_lineages(list( Tyrannosaurus = c("Biota", "Dinosauria", "Theropoda", "Tyrannosaurus"), Triceratops = c("Biota", "Dinosauria", "Ornithischia", "Triceratops") )) reference <- taxo_from_lineages(list( Velociraptor = c("Biota", "Dinosauria", "Theropoda", "Velociraptor"), Homo = c("Biota", "Mammalia", "Homo") )) cross_distances(community, reference) ``` ### Finding the closest candidates A stored example is available in `taxobase`: ```{r stored-closest} taxobase$closest ``` For a live query: ```{r live-closest, eval=FALSE} closest_relative( "Carnotaurus", c( "Aucasaurus", "Velociraptor", "Triceratops", "Brachiosaurus" ) ) ``` Candidates are sorted from smallest to largest distance. A missing candidate is retained with a missing distance. ### Focal distances ```{r live-focal, eval=FALSE} focal_distances( focal = "Tyrannosaurus", community = c( "Velociraptor", "Triceratops", "Spinosaurus" ) ) ``` This produces a table containing the candidate name, distance, MRCA, and MRCA depth. ## Clade operations ### Testing membership ```{r live-membership, eval=FALSE} is_member("Tyrannosaurus", "Dinosauria") is_member("Tyrannosaurus", "Theropoda") is_member("Tyrannosaurus", "Ornithischia") ``` Matching is case-insensitive, ignores surrounding whitespace, and requires the complete clade name rather than a partial substring. ### Filtering taxa The package data include a stored filtering example: ```{r stored-filter} taxobase$filter ``` A live operation can be performed with: ```{r live-filter, eval=FALSE} taxa <- c( "Tyrannosaurus", "Carnotaurus", "Triceratops", "Velociraptor", "Homo", "Drosophila" ) filter_clade(taxa, "Dinosauria") filter_clade(taxa, "Theropoda") ``` ### Shared clades and lineage comparison ```{r live-comparison, eval=FALSE} shared_clades( "Tyrannosaurus", "Triceratops" ) compare_lineages( "Carnotaurus", "Tyrannosaurus" ) ``` Shared ancestry is determined from the continuous common lineage prefix. A name repeated after the first divergence is not treated as a shared ancestor. ## Clustering, ordination, and visualization The following examples use a subset of the packaged matrix and therefore require no network access. ```{r analysis-matrix} labels <- attr(reference_matrix, "Labels")[1:8] example_matrix <- stats::as.dist( as.matrix(reference_matrix)[labels, labels] ) example_matrix ``` ### Hierarchical clustering ```{r clustering, fig.width=7, fig.height=5} clustering <- taxo_cluster( example_matrix, method = "average" ) plot( clustering, main = "Taxonomic hierarchy distance clustering", xlab = "", sub = "" ) ``` The result is a clustering dendrogram, not an independently inferred phylogenetic tree. ### Classification trees `taxo_tree()` builds a tree directly from lineages rather than from distances. Its topology is the classification itself, its internal nodes are labeled with the named clades at which the taxa diverge, and its branch lengths are chosen so that patristic distances reproduce `distance_matrix()` exactly. The result is a standard `phylo` object. ```{r taxo-tree, eval=requireNamespace("ape", quietly = TRUE), fig.width=7, fig.height=4} lineages <- taxo_from_lineages(list( Tyrannosaurus = c("Biota", "Dinosauria", "Theropoda", "Tyrannosaurus"), Velociraptor = c("Biota", "Dinosauria", "Theropoda", "Velociraptor"), Triceratops = c("Biota", "Dinosauria", "Ornithischia", "Triceratops"), Homo = c("Biota", "Mammalia", "Homo") )) tree <- taxo_tree(lineages) tree$node.label plot(tree, show.node.label = TRUE, main = "Classification tree") tree_distances <- distance_matrix(lineages, progress = FALSE) all.equal( ape::cophenetic.phylo(tree)[labels(tree_distances), labels(tree_distances)], as.matrix(tree_distances) ) ``` The tree can be written in Newick format with `ape::write.tree(tree)`. ### Principal coordinates analysis ```{r ordination, fig.width=7, fig.height=5} ordination <- taxo_ordinate( example_matrix, k = 2 ) summary(ordination) plot( ordination, main = "Taxonomic hierarchy distance space" ) ``` The goodness-of-fit and eigenvalues should be examined before interpreting a low-dimensional representation. ### Heatmap ```{r heatmap, fig.width=7, fig.height=6} taxo_heatmap( example_matrix, main = "Taxonomic hierarchy distance matrix", col = hcl.colors(12, "YlOrRd", rev = TRUE) ) ``` Graphical arguments such as `main` and `col` are passed to `stats::heatmap()`, while `verbose` and `progress` control lineage retrieval when taxon names are supplied instead of a matrix. Additional statistical examples and interpretation are provided in: ```{r statistics-link, eval=FALSE} vignette("statistical-applications", package = "taxodist") ``` ## Name resolution and coverage Taxonomic names may be missing, ambiguous, redirected, or associated with non-biological homonyms. `taxo_search()` returns candidate Taxonomicon entries: ```{r live-search, eval=FALSE} taxo_search("Panthera") taxo_search("Bacteria") ``` When several valid biological entries exist, the package reports the available identifiers. A numeric identifier should be supplied when the intended taxonomic concept is known. Names are matched after removing surrounding whitespace and collapsing repeated spaces, and the queried name is located in the lineage without regard to case. Consequently, `"tyrannosaurus"` and `" Tyrannosaurus "` resolve to the same node as `"Tyrannosaurus"`. For batches of names, `taxo_resolve()` records the status, candidates, selected identifier, and lineage of every input. Homonyms in different parts of the classification can often be resolved automatically by restricting candidates to a clade with `within`: ```{r live-resolve, eval=FALSE} resolved <- taxo_resolve( c("Tyrannosaurus", "Nereis", "Not_a_real_taxon"), within = "Animalia" ) resolved distance_matrix(resolved) ``` A matrix computed from a resolution is identical to one computed from the same names, but it requires no further network requests and keeps an audit trail. Before calculating a large matrix, coverage can be checked explicitly: ```{r live-coverage, eval=FALSE} taxa <- c( "Tyrannosaurus", "Velociraptor", "Quercus", "Not_a_real_taxon" ) check_coverage(taxa) ``` `check_coverage()` returns `TRUE` for names found, `FALSE` for names absent from The Taxonomicon, and `NA` when the server could not be reached, so a temporary network failure is never mistaken for missing coverage. Unresolved taxa generate missing pairwise distances. These values should be examined before clustering, ordination, or other methods requiring complete finite matrices. ## Cache management Taxonomicon identifiers and lineages are cached in memory during an R session. This avoids repeated requests for the same taxa. ```{r live-cache, eval=FALSE} cache_info() clear_cache() ``` The cache can also be saved and restored: ```{r persistent-cache, eval=FALSE} save_cache("taxodist-cache.rds") clear_cache() load_cache("taxodist-cache.rds") ``` Called without a file, `save_cache()` and `load_cache()` use a persistent per-user location given by `tools::R_user_dir("taxodist", "cache")`: ```{r default-cache, eval=FALSE} save_cache() # In a later session: load_cache() ``` Cache files preserve retrieved lineages across sessions. They should be regenerated when an analysis is intended to incorporate subsequent revisions of the source classification. Requests to The Taxonomicon are retried automatically after transient network or server errors. The number of attempts can be changed with `options(taxodist.max_tries = 5)`. ## Interpretation and reproducibility The numerical distance depends on the classification supplied by The Taxonomicon. Changes in lineage resolution or taxonomic concepts can alter MRCA depths and therefore alter distances. A reproducible analysis should record: - the `taxodist` version; - the retrieval or dataset-generation date; - the source classification; - numeric identifiers used to resolve ambiguous names; - missing or excluded taxa; - the distance definition; - downstream clustering or ordination settings. Saved caches and distance matrices can preserve the exact input used in an analysis. `taxo_bundle()` collects the resolution, lineages, matrix, distance definition, and provenance in a single object that can be written to portable JSON. When an analysis is repeated later, `compare_bundles()` reports which taxa changed identifier, status, or lineage, and which distances changed as a result: ```{r live-bundles, eval=FALSE} before <- read_taxodist_bundle("analysis-2026.json") after <- taxo_bundle(before$resolution$input) compare_bundles(before, after) ``` ## Data source and citation All retrieved lineage data originate from The Taxonomicon, based on *Systema Naturae 2000*. Formatted references for both the package and the data source are available with: ```{r citation, eval=FALSE} citation("taxodist") ``` Both the software and the underlying taxonomic source should be cited in published analyses.