Retrieve ENCODE microRNA-seq quantifications

Introduction

The ENCODE Portal is a public repository of experiment records, biosample descriptions, data files, and metadata from the ENCODE Project. ENCODE provides programmatic access to this information through a REST API. The API returns data in JSON format.

encodeUtils provides an R interface for searching ENCODE metadata, selecting and verifying files, reading supported formats, and recording retrieval provenance. It converts JSON responses from the API into R and Bioconductor objects.

This vignette shows how to retrieve replicate-level raw microRNA counts from male 5xFAD mouse heart tissue.

ENCODE API disclaimer

The following rate-limit disclaimer is reproduced directly from the ENCODE REST API documentation:

Use of the programmatic API is limited to 10 GET requests/sec from any single user, group, company or lab.

Abuse of this will result in denial of access by IP block

See the ENCODE data-use policy for use and citation guidance.

Installation

install.packages("BiocManager")
BiocManager::install("encodeUtils")

Install the development version with:

install.packages("pak")
pak::pak("ZohebKhan1/encodeUtils")

Find the experiment

Begin with the biological criteria. The search argument looks for 5xFAD in ENCODE metadata. The remaining arguments restrict the results to male mouse heart microRNA-seq experiments. To investigate a different question, change organism, assay, organ, sex, and search. limit sets the maximum number of results and does not filter by biology.

library(encodeUtils)

discovery <- encode_search(
    organism = "mouse",
    assay = "microRNA-seq",
    organ = "heart",
    sex = "male",
    search = "5xFAD",
    limit = 5
)
#> Querying ENCODE search (Experiment, limit 5).
#> ENCODE search returned 1 of 1 matching record(s).

experiment_table <- encode_results(discovery)

experiment_table[c(
    "accession", "assay_title", "biosample_summary", "life_stage_age", "sex",
    "file_count", "status"
)]
#> ENCODE experiments
#> - experiments: 1
#> Experiments:
#>   experiment        assay               age  sex files   status
#>  ENCSR523CTA microRNA-seq adult 8-10 months male    14 released

The search identifies a released microRNA-seq experiment from male 5xFAD/CAST mouse heart tissue. The current query returns one experiment. If a search returns several experiments, compare their assay, biosample, age, sex, and status fields, then refine the search arguments before continuing.

List the experiment’s files

Use the accession returned by the search to retrieve metadata for every released file associated with the experiment.

files <- encode_list_files(
    experiment_table$accession[[1L]]
)
#> ENCODE file listing returned 14 file record(s) (1.28 GB with known
#> sizes).

file_table <- as.data.frame(encode_results(files))

file_inventory <- unique(file_table[c("file_format", "output_type", "assembly")])

print(file_inventory, row.names = FALSE)
#>  file_format                         output_type assembly
#>       bigWig minus strand signal of unique reads     mm10
#>       bigWig     plus strand signal of all reads     mm10
#>        fastq                               reads     <NA>
#>          tsv            microRNA quantifications     mm10
#>          bam                          alignments     mm10
#>       bigWig  plus strand signal of unique reads     mm10
#>       bigWig    minus strand signal of all reads     mm10

Select replicate-level files

file_format identifies the storage format, while output_type identifies the data product. To build a count matrix, select TSV files containing microRNA quantifications. These processed files use the mm10 mouse assembly, and replicate_level retains a separate file for each biological replicate. To retrieve another data product, inspect the inventory and change the relevant selection arguments.

selected <- encode_select_files(
    files,
    file_format = "tsv",
    output_type = "microRNA quantifications",
    assembly = "mm10",
    replicate_policy = "replicate_level"
)
#> ENCODE file selection kept 2 of 14 file(s).

selected_table <- encode_results(selected)

selected_display <- data.frame(
    accession = selected_table$file_accession,
    replicate = selected_table$biological_replicates,
    size = selected_table$file_size_pretty
)

print(selected_display, row.names = FALSE)
#>    accession replicate     size
#>  ENCFF838WBE         2 59.57 KB
#>  ENCFF859GWB         1 59.54 KB

The selection narrows the inventory from 14 files to 2 quantification files. The remaining steps use these accessions, and the selection object retains the criteria used to choose them.

selected$criteria[c(
    "file_format", "output_type", "assembly", "status", "replicate_policy"
)]
#> $file_format
#> [1] "tsv"
#> 
#> $output_type
#> [1] "microRNA quantifications"
#> 
#> $assembly
#> [1] "mm10"
#> 
#> $status
#> [1] "released"
#> 
#> $replicate_policy
#> [1] "replicate_level"

Preview the download

Create a download plan before transferring either file. The plan determines the destination paths and compares the file sizes reported by ENCODE with the configured limits, but it does not download the files.

Each selected file is about 60 KB, so the 100 KB per-file and 200 KB total limits provide modest headroom above the expected 120 KB transfer.

The vignette uses a temporary directory so package builds do not leave data in the source tree. Files under tempdir() are temporary. For an analysis, use a persistent, project-relative directory instead.

download_directory <- file.path(tempdir(), "encodeUtils-live-vignette")

plan <- encode_download(
    selected,
    directory = download_directory,
    max_file_size = "100KB",
    max_total_size = "200KB",
    dry_run = TRUE
)
#> Planned ENCODE download: 2 file(s), at least 119.11 KB known total size.
#> ℹ 0 file(s) have unknown size.
#> ℹ Destination: /tmp/RtmpNQGUyh/encodeUtils-live-vignette

plan_table <- encode_results(plan)

plan_display <- data.frame(
    accession = plan_table$file_accession,
    status = plan_table$download_status,
    size = plan_table$file_size_pretty,
    file = basename(plan_table$local_path)
)

print(plan_display, row.names = FALSE)
#>    accession  status     size            file
#>  ENCFF838WBE planned 59.57 KB ENCFF838WBE.tsv
#>  ENCFF859GWB planned 59.54 KB ENCFF859GWB.tsv

Confirm the file accessions, destination paths, and expected total size before downloading.

Download and verify the files

Run the same request without dry_run. encode_download() first writes each transfer to a temporary .part file, then verifies its size and MD5 checksum against ENCODE metadata before moving it to the requested destination.

downloaded <- encode_download(
    selected,
    directory = download_directory,
    max_file_size = "100KB",
    max_total_size = "200KB"
)
#> Planned ENCODE download: 2 file(s), at least 119.11 KB known total size.
#> ℹ 0 file(s) have unknown size.
#> ℹ Destination: /tmp/RtmpNQGUyh/encodeUtils-live-vignette
#> Downloading (1/2) "ENCFF838WBE".
#> Downloading (2/2) "ENCFF859GWB".
#> ENCODE download finished: 2 downloaded, 0 already present, and 0 failed.

The following checks require every local file to have a verified size and checksum.

download_table <- as.data.frame(encode_results(downloaded))

stopifnot(
    all(download_table$download_status %in% c("downloaded", "exists")),
    all(download_table$size_verified %in% TRUE),
    all(download_table$md5_verified %in% TRUE)
)

download_display <- data.frame(
    accession = download_table$file_accession,
    status = download_table$download_status,
    size_verified = download_table$size_verified,
    md5_verified = download_table$md5_verified
)

print(download_display, row.names = FALSE)
#>    accession     status size_verified md5_verified
#>  ENCFF838WBE downloaded          TRUE         TRUE
#>  ENCFF859GWB downloaded          TRUE         TRUE

Read the count tables

The selected microRNA quantification files are four-column tables in HTSeq output format. encodeUtils stores the raw-count column in a matrix named raw_counts. Both tables contain complete, unique gene_id values, which encode_read() uses to align their rows. The matrix columns correspond to the file accessions shown earlier.

count_data <- encode_read(
    downloaded,
    values = "raw_counts",
    row_names = "gene_id"
)

print(count_data)
#> ENCODE loaded files
#> - files: 2
#> - file objects: 2
#> - feature rows: 2202
#> - matrices: 1
#> Metadata:
#>         file  experiment        assay     organism assembly file_size   status
#>  ENCFF838WBE ENCSR523CTA microRNA-seq Mus musculus     mm10  59.57 KB released
#>  ENCFF859GWB ENCSR523CTA microRNA-seq Mus musculus     mm10  59.54 KB released

count_data contains the source tables, file and feature metadata, and the aligned raw-count matrix.

Inspect the first five rows of the count matrix.

print(head(count_data$matrices$raw_counts, 5L))
#>                      ENCFF838WBE ENCFF859GWB
#> ENSMUSG00000093015.1           0           0
#> ENSMUSG00000093970.1           0           0
#> ENSMUSG00000076135.1           0           0
#> ENSMUSG00000098555.1           0           0
#> ENSMUSG00000099183.1          21          12

Write a manifest

Create a JSON manifest for the completed retrieval. The manifest records the file-listing request stored in count_data, filters, selection criteria, downloaded files, checksums, matrix dimensions, and R session information. The earlier experiment search remains in discovery and is not part of this manifest.

manifest_path <- file.path(tempdir(), "encodeUtils-live-manifest.json")

manifest <- encode_manifest(count_data, path = manifest_path)

print(
    data.frame(
        manifest = basename(attr(manifest, "path")),
        files = nrow(manifest$files),
        requests = length(manifest$requests),
        matrices = nrow(manifest$matrices)
    ),
    row.names = FALSE
)
#>                        manifest files requests matrices
#>  encodeUtils-live-manifest.json     2        2        1

This temporary manifest exists only for the package build. For an analysis, write it to a persistent, project-relative path.

Inspect the file records stored in the manifest.

manifest$files[c(
    "file_accession", "md5sum", "observed_md5", "downloaded_at"
)]
#>    file_accession                           md5sum
#> 5     ENCFF838WBE 458272e7f622ad72fae6945958c30a59
#> 12    ENCFF859GWB 1ed90c37b983efdd969fb2ae73ca9bbb
#>                        observed_md5       downloaded_at
#> 5  458272e7f622ad72fae6945958c30a59 2026-08-07 10:29:51
#> 12 1ed90c37b983efdd969fb2ae73ca9bbb 2026-08-07 10:29:51

Optional: create a SummarizedExperiment

To work with a Bioconductor container instead, read the same verified local files into a SummarizedExperiment. The assay stores raw counts, rowData() stores feature identifiers, and colData() stores the ENCODE file metadata.

se <- encode_read(
    downloaded,
    values = "raw_counts",
    row_names = "gene_id",
    as = "SummarizedExperiment"
)
#> Warning: replacing previous import 'S4Arrays::makeNindexFromArrayViewport' by
#> 'DelayedArray::makeNindexFromArrayViewport' when loading 'SummarizedExperiment'

print(se)
#> class: SummarizedExperiment 
#> dim: 2202 2 
#> metadata(1): encodeUtils
#> assays(1): raw_counts
#> rownames(2202): ENSMUSG00000093015.1 ENSMUSG00000093970.1 ...
#>   ENSMUSG00000098868.1 ENSMUSG00000099228.1
#> rowData names(1): gene_id
#> colnames(2): ENCFF838WBE ENCFF859GWB
#> colData names(58): file_accession accession ... md5_verified
#>   failure_reason

The column metadata links each matrix column to its ENCODE file and biological replicate.

column_data <- as.data.frame(SummarizedExperiment::colData(se))

column_data[c(
    "file_accession", "biological_replicates", "output_type", "assembly"
)]
#>             file_accession biological_replicates              output_type
#> ENCFF838WBE    ENCFF838WBE                     2 microRNA quantifications
#> ENCFF859GWB    ENCFF859GWB                     1 microRNA quantifications
#>             assembly
#> ENCFF838WBE     mm10
#> ENCFF859GWB     mm10

References

ENCODE Project Consortium. An integrated encyclopedia of DNA elements in the human genome. Nature. 2012;489:57-74.
Kagda MS et al. Data navigation on the ENCODE portal. Nature Communications. 2025;16:9592.
ENCODE REST API
ENCODE citation guidance

Session information

sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 24.04.4 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
#>  [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
#>  [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
#> [10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   
#> 
#> time zone: UTC
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] encodeUtils_0.99.0 BiocStyle_2.40.0  
#> 
#> loaded via a namespace (and not attached):
#>  [1] Matrix_1.7-5                jsonlite_2.0.0             
#>  [3] compiler_4.6.1              BiocManager_1.30.27        
#>  [5] SummarizedExperiment_1.42.0 Biobase_2.72.0             
#>  [7] GenomicRanges_1.64.0        jquerylib_0.1.4            
#>  [9] systemfonts_1.3.2           IRanges_2.46.0             
#> [11] Seqinfo_1.2.0               textshaping_1.0.5          
#> [13] yaml_2.3.12                 fastmap_1.2.0              
#> [15] lattice_0.22-9              XVector_0.52.0             
#> [17] R6_2.6.1                    S4Arrays_1.12.0            
#> [19] generics_0.1.4              curl_7.1.0                 
#> [21] httr2_1.3.0                 knitr_1.51                 
#> [23] BiocGenerics_0.58.1         DelayedArray_0.38.2        
#> [25] bookdown_0.47               desc_1.4.3                 
#> [27] MatrixGenerics_1.24.0       bslib_0.12.0               
#> [29] rlang_1.3.0                 cachem_1.1.0               
#> [31] xfun_0.60                   fs_2.1.0                   
#> [33] sass_0.4.10                 otel_0.2.0                 
#> [35] SparseArray_1.12.2          cli_3.6.6                  
#> [37] pkgdown_2.2.1               magrittr_2.0.5             
#> [39] grid_4.6.1                  digest_0.6.39              
#> [41] lifecycle_1.0.5             S4Vectors_0.50.1           
#> [43] evaluate_1.0.5              glue_1.8.1                 
#> [45] ragg_1.5.2                  abind_1.4-8                
#> [47] stats4_4.6.1                rmarkdown_2.31             
#> [49] matrixStats_1.5.0           tools_4.6.1                
#> [51] htmltools_0.5.9

Zoheb Khan

2026-08-07