nf-core · mashehu · Oct 20, 2023 · Nov 3, 2023 · Nov 29, 2023 · Dec 4, 2023
diff --git a/.github/workflows/awsfulltest.yml b/.github/workflows/awsfulltest.yml
@@ -45,7 +45,6 @@ jobs:
 
       - name: Launch workflow via Seqera Platform
         uses: seqeralabs/action-tower-launch@v2
-        # TODO nf-core: You can customise AWS full pipeline tests as required
         # Add full size test data (but still relatively small datasets for few samples)
         # on the `test_full.config` test runs with only one set of parameters
         with:

diff --git a/.gitignore b/.gitignore
@@ -7,3 +7,4 @@ testing/
 testing*
 *.pyc
 null/
+.nf-test*
diff --git a/.nf-core.yml b/.nf-core.yml
@@ -15,4 +15,5 @@ template:
   outdir: .
   skip_features:
     - igenomes
+    - fastqc
   version: 1.0.0
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,9 +3,10 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## v1.0.0 - [date]
+## v1.0.0 - [2025-01-17]
 
-Initial release of nf-core/rangeland, created with the [nf-core](https://nf-co.re/) template.
+First release of `nf-core/rangeland`.
+This work is a continuation, and nf-core port, of the [original version of this pipeline](https://github.com/CRC-FONDA/FORCE2NXF-Rangeland).
 
 ### `Added`
 

diff --git a/CITATIONS.md b/CITATIONS.md
@@ -1,5 +1,11 @@
 # nf-core/rangeland: Citations
 
+## [FORCE on Nextflow](https://www.informatik.hu-berlin.de/de/forschung/gebiete/wbi/research/publications/2021/force_nextflow.pdf/@@download/file/force_nextflow.pdf)
+
+This paper describes technical details of deploying the FORCE tool in a Nextflow workflow. The workflow developed in this paper is the foundation for this nf-core pipeline.
+
+> Lehmann, F., Frantz, D., Becker, S., Leser, U., Hostert, P. (2021). FORCE on Nextflow: Scalable Analysis of Earth Observation Data on Commodity Clusters. In CIKM Workshops.
+
 ## [nf-core](https://pubmed.ncbi.nlm.nih.gov/32055031/)
 
 > Ewels PA, Peltzer A, Fillinger S, Patel H, Alneberg J, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Mar;38(3):276-278. doi: 10.1038/s41587-020-0439-x. PubMed PMID: 32055031.
@@ -10,9 +16,11 @@
 
 ## Pipeline tools
 
-- [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)
+- [FORCE](https://www.mdpi.com/2072-4292/11/9/1124)
+
+  > Frantz, D. (2019). FORCE—Landsat + Sentinel-2 Analysis Ready Data and Beyond. Remote Sensing, 11, 1124
 
-> Andrews, S. (2010). FastQC: A Quality Control Tool for High Throughput Sequence Data [Online].- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
+- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
 
 > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
 

diff --git a/README.md b/README.md
@@ -17,47 +17,41 @@
 
 ## Introduction
 
-**nf-core/rangeland** is a bioinformatics pipeline that ...
-
-<!-- TODO nf-core:
-   Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the
-   major pipeline sections and the types of output it produces. You're giving an overview to someone new
-   to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction
--->
-
-<!-- TODO nf-core: Include a figure that guides the user through the major workflow steps. Many nf-core
-     workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples.   -->
-<!-- TODO nf-core: Fill in short bullet-pointed list of the default steps in the pipeline -->1. Read QC ([`FastQC`](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/))2. Present QC for raw reads ([`MultiQC`](http://multiqc.info/))
+**nf-core/rangeland** is a geographical best-practice analysis pipeline for remotely sensed imagery.
+The pipeline processes satellite imagery alongside auxiliary data in multiple steps to arrive at a set of trend files related to land-cover changes. The main pipeline steps are:
+
+1. Read satellite imagery, digital elevation model (dem), endmember definition, water vapor database (wvdb), datacube definition and area of interest definition (aoi)
+2. Generate allow list and analysis mask to determine which pixels from the satellite data can be used
+3. Preprocess data to obtain atmospherically corrected images alongside quality assurance information (aka. level 2 analysis read data)
+4. Merge spatially and temporally overlapping preprocessed data
+5. Classify pixels by applying linear spectral unmixing
+6. Time series analyses to obtain trends in vegetation dynamics to derive level 3 data
+7. Create mosaic and pyramid visualizations of the results
+8. Version reporting with MultiQC ([`MultiQC`](http://multiqc.info/))
+
+<p align="center">
+    <img title="nf-core/rangeland diagram" src="docs/images/rangeland_diagram.png" width=95%>
+</p>
 
 ## Usage
 
 > [!NOTE]
-> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow.Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.
-
-<!-- TODO nf-core: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
-     Explain what rows and columns represent. For instance (please edit as appropriate):
+> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.
 
-First, prepare a samplesheet with your input data that looks as follows:
-
-`samplesheet.csv`:
-
-```csv
-sample,fastq_1,fastq_2
-CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
-```
-
-Each row represents a fastq file (single-end) or a pair of fastq files (paired end).
-
--->
+To run, satellite imagery, water vapor data, a digital elevation model, endmember definitions, a datacube specification, and a area-of-interest specification are required as input data.
+Please refer to the [usage documentation](https://nf-co.re/rangeland/usage) for details on the input structure.
 
 Now, you can run the pipeline using:
 
-<!-- TODO nf-core: update the following command to include all required parameters for a minimal example -->
-
 ```bash
 nextflow run nf-core/rangeland \
    -profile <docker/singularity/.../institute> \
-   --input samplesheet.csv \
+   --input <SATELLITE IMAGES> \
+   --dem <DIGITAL ELEVATION MODEL> \
+   --wvdb <WATER VAPOR DATA> \
+   --data_cube <DATA CUBE> \
+   --aoi <AREA OF INTEREST> \
+   --endmember <ENDMEMBER SPECIFICATION> \
    --outdir <OUTDIR>
 ```
 
@@ -74,11 +68,34 @@ For more details about the output files and reports, please refer to the
 
 ## Credits
 
-nf-core/rangeland was originally written by Fabian Lehmann, David Frantz, Felix Kummer.
+The rangeland workflow was originally written by:
+
+- [Fabian Lehmann](https://github.com/Lehmann-Fabian)
+- [David Frantz](https://github.com/davidfrantz)
+
+The original workflow can be found on [github](https://github.com/CRC-FONDA/FORCE2NXF-Rangeland).
+
+Transformation to nf-core/rangeland was conducted by [Felix Kummer](https://github.com/Felix-Kummer).
+nf-core alignment started on the [nf-core branch of the original repository](https://github.com/CRC-FONDA/FORCE2NXF-Rangeland/tree/nf-core).
 
 We thank the following people for their extensive assistance in the development of this pipeline:
 
-<!-- TODO nf-core: If applicable, make list of people who have also contributed -->
+- [Fabian Lehmann](https://github.com/Lehmann-Fabian)
+- [Katarzyna Ewa Lewinska](https://github.com/kelewinska).
+
+## Acknowledgements
+
+This pipeline was developed and aligned with nf-core as part of the [Foundations of Workflows for Large-Scale Scientific Data Analysis (FONDA)](https://fonda.hu-berlin.de/) initiative.
+
+[![FONDA](docs/images/fonda_logo2_cropped.png)](https://fonda.hu-berlin.de/)
+
+FONDA can be cited as follows:
+
+> **The Collaborative Research Center FONDA.**
+>
+> Ulf Leser, Marcus Hilbrich, Claudia Draxl, Peter Eisert, Lars Grunske, Patrick Hostert, Dagmar Kainmüller, Odej Kao, Birte Kehr, Timo Kehrer, Christoph Koch, Volker Markl, Henning Meyerhenke, Tilmann Rabl, Alexander Reinefeld, Knut Reinert, Kerstin Ritter, Björn Scheuermann, Florian Schintke, Nicole Schweikardt, Matthias Weidlich.
+>
+> _Datenbank Spektrum_ 2021 doi: [10.1007/s13222-021-00397-5](https://doi.org/10.1007/s13222-021-00397-5)
 
 ## Contributions and Support
 
@@ -89,7 +106,7 @@ For further information or help, don't hesitate to get in touch on the [Slack `#
 ## Citations
 
 <!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. -->
-<!-- If you use nf-core/rangeland for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) --><!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
+<!-- If you use nf-core/rangeland for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
 
 An extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.
 
@@ -100,3 +117,10 @@ You can cite the `nf-core` publication as follows:
 > Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
 >
 > _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).
+
+This pipeline is based one the publication listed below.
+The publication can be cited as follows:
+
+> **FORCE on Nextflow: Scalable Analysis of Earth Observation Data on Commodity Clusters**
+>
+> [Lehmann, F., Frantz, D., Becker, S., Leser, U., Hostert, P. (2021). FORCE on Nextflow: Scalable Analysis of Earth Observation Data on Commodity Clusters. In CIKM Workshops.](https://www.informatik.hu-berlin.de/de/forschung/gebiete/wbi/research/publications/2021/force_nextflow.pdf/@@download/file/force_nextflow.pdf)
diff --git a/bin/merge_boa.r b/bin/merge_boa.r
@@ -0,0 +1,46 @@
+#!/usr/bin/env Rscript
+
+## Originally written by Felix Kummer and released under the MIT license.
+## See git repository (https://github.com/nf-core/rangeland) for full license text.
+
+# Script for merging bottom of atmosphere (boa) .tif raster files.
+# This can improve the performance of downstream tasks.
+
+require(terra)
+
+args <- commandArgs(trailingOnly = TRUE)
+
+
+if (length(args) < 3) {
+    stop("\nError: this program needs at least 3 inputs\n1: output filename\n2-*: input files", call.=FALSE)
+}
+
+fout <- args[1]
+finp <- args[2:length(args)]
+
+# Load input rasters
+rasters <- lapply(finp, rast)
+
+# Calculate the sum of non-NA values across all rasters
+sum_rasters <- Reduce("+", lapply(rasters, function(x) {
+    x[is.na(x)] <- 0
+    return(x)
+}))
+
+# Calculate the number of values non-NA values for each cell
+count_rasters <- Reduce("+", lapply(rasters, function(x) {
+    return(!is.na(x))
+}))
+
+# Calculate the mean raster
+mean_raster <- sum_rasters / count_rasters
+
+# Write the mean raster
+writeRaster(mean_raster,
+            filename = fout,
+            datatype = "INT2S",
+            filetype = "GTiff",
+            gdal     = c("COMPRESS=LZW", "PREDICTOR=2",
+                        "NUM_THREADS=ALL_CPUS", "BIGTIFF=YES",
+                        sprintf("BLOCKXSIZE=%s", ncol(mean_raster)),
+                        sprintf("BLOCKYSIZE=%s", nrow(mean_raster))))
diff --git a/bin/merge_qai.r b/bin/merge_qai.r
@@ -0,0 +1,41 @@
+#!/usr/bin/env Rscript
+
+## Originally written by Felix Kummer and released under the MIT license.
+## See git repository (https://github.com/nf-core/rangeland) for full license text.
+
+# Script for merging quality information (qai) .tif raster files.
+# This can improve the performance of downstream tasks.
+
+require(terra)
+
+args <- commandArgs(trailingOnly = TRUE)
+
+
+if (length(args) < 3) {
+    stop("\nError: this program needs at least 3 inputs\n1: output filename\n2-*: input files", call.=FALSE)
+}
+
+fout <- args[1]
+finp <- args[2:length(args)]
+
+# load raster files into single SpatRaster
+rasters <- rast(finp)
+
+# Merge rasters by maintaining the last non-NA value
+merged_raster <- app(rasters, function(x) {
+    non_na_values <- na.omit(x)
+    if (length(non_na_values) == 0) {
+        return(1)
+    }
+    return(tail(non_na_values, 1)[1])
+})
+
+# Write merged raster
+writeRaster(merged_raster,
+            filename = fout,
+            filetype = "GTiff",
+            datatype = "INT2S",
+            gdal     = c("INTERLEAVE=BAND", "COMPRESS=LZW", "PREDICTOR=2",
+                        "NUM_THREADS=ALL_CPUS", "BIGTIFF=YES",
+                        sprintf("BLOCKXSIZE=%s", ncol(merged_raster)),
+                        sprintf("BLOCKYSIZE=%s", nrow(merged_raster))))
-Original file line number
+Diff line change
@@ Expand Up / @@ -7,3 +7,4 @@ testing/ @@
     testing*
     *.pyc
     null/
+    .nf-test*