Usage

Running Off-the-shelf LENS

RAFT supports “off-the-shelf” (OTS) workflows that can be modified by the end user if desired. The OTS workflow offering includes:

Pre-defined, compatible reference files
Pre-defined tools
Pre-defined, default parameters

More information regarding the default references, tools, and parameters for off-the-shelf LENS can be found at the Technical Details page.

Users can run an off-the-shelf LENS with their own manifest with the RAFT’s run-ots (run off-the-shelf) mode. The run-ots mode requires several input paramaters:

Column Parameter	Description
`--project-id`	Unique identifier for your project. Projects are stored in `/path/to/raft/projects/`.
`--workflow`	Species the workflow to run (LENS, in this case, though other workflows are supported).
`--version`	Specifies version of the workflow to run.
`--species`	Indicates the species for which the workflow should run.
`--inputs`	Indicates the input data type.
`--manifest`	The manifest file to use. Full path is not required (as long as the manifest is in `/path/to/raft/metadata`).

Users can run off-the-shelf LENS on their own manifest by running

$ raft.py run-ots --project-id <PROJECT_ID> \
                  --workflow lens \
                  --version 1.7-dev \
                  --species <SPECIES> \
                  --inputs <INPUT_TYPE> \
                  --manifest <MANIFEST>

For example:

$ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                  --workflow lens \
                  --version 1.7-dev \
                  --species human \
                  --inputs fastqs \
                  --manifest my_manifest.tsv

Note that raft.py run-ots assumes human samples and FASTQs inputs, so the the above command is equivilant to the following:

$ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                  --workflow lens \
                  --version 1.7-dev \
                  --manifest my_manifest.tsv

Running Off-the-shelf LENS with Mouse Data

LENS can be run on mouse samples, but users must:

Provide the MHC alleles in the user-provided manifest (see Manifest)
Modify the nextflow.config file to change the starfusion Docker image (see below)

With these requirements in mind, users should run following command:

$ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                  --workflow lens \
                  --version 1.7-dev \
                  --manifest my_manifest.tsv \
                  --species mouse \
                  --setup-only

Users can then run the LENS workflow using:

$ raft.py run-workflow --project-id my-project-lens-01JAN2025

Running Off-the-shelf LENS using cloud service providers (EXPERIMENTAL)

LENS has been run using the Google Cloud Platform (GCP) after setting up an appropriate Nextflow profile. Users must perform the following to support cloud usage. Note that we intend for this feature to be more user-friendly in future versions of LENS.

Requirements:

Upload all required references (those downloaded by raft.py run-ots) to gs://<your_bucket>/references/cloud/.
Upload FASTQs or BAMs to gs://<your_bucket>/fastqs/<your_dataset>/ or gs://<your_bucket>/bams/<your_dataset>/, respectively.
Upload your manifest file to gs://<your_bucket>/metadata/<your_dataset>/.

At this point, users should run:

$ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                  --workflow lens \
                  --version 1.7-dev \
                  --manifest my_manifest.tsv \
                  --cloud gs://<your_bucket \
                  -n="-profile <your_gcp_profile>"

Running Off-the-shelf LENS with User-defined Parameters

Users can run an off-the-shelf LENS instance with their preferred tools and tool parameters using the raft.py -up/--user-params argument. This argument can be specified any number of times. For example, if a user wants to run the LENS workflow with their own manifest, but with trim_galore instead of fastp, then they can run:

$ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                  --workflow lens \
                  --manifest </PATH/TO/MANIFEST>  \
                  --version 1.7-dev \
                  --user-params fq_trim_tool=fastp \
                  --user-params fq_trim_tool_parameters=fastp:'--paired'

Users may specify multiple tools and parameters for each of multiple tools using the following syntax:

$ raft.py run-ots --project-id ... \
                  ... \
                  --user-params fq_trim_tool=tool1,tool2 \
                  --user-params fq_trim_tool_parameters=tool1:'--params --for --tool 1',\
                                                        tool2:'--params --for --tool 2'

Generally speaking, it is only recommended to use multiple tools for the last modifying step of the workflow. For example, the gene-fusions workflow for humans starting from FASTQs allow both fq_trim_tool and fusion_tool to be specified. fusion_tool generates the ultimate outputs from the workflow and thus users can specify multiple fusion_tool tools (e.g. fusion_tool=starfusion,arriba). fq_trim_tool, however, is upstream of fusion calling and users should only specify a single tool for fq_trim_tool.

Running Off-the-shelf LENS with User-specified References

In order to use alternative references files, users should:

Download the new reference file and move or symlink it to RAFT’s references/ directory (/path/to/raft/references).
Users should then load the reference file into their LENS project using raft.py load-reference -p <PROJECT_NAME> -f <FILE_NAME>.
Finally, users should modify their LENS project’s main.nf file to use their new desired reference. For example, the line

params.lens$alignment$manifest_to_dna_alns$aln_ref = "${params.ref_dir}/Homo_sapiens.assembly38.no_ebv.fa"

could be changed to:

params.lens$alignment$manifest_to_dna_alns$aln_ref = "${params.ref_dir}/hg19.fa"

Running Off-the-shelf LENS with User-specified Tools

A variety of alternative tools are supported for each subworkflow within LENS. In order to use alternative tools, users should modify their LENS project’s main.nf file to change the default tools to those they require. For example, if a user wanted to run trimmomatic for DNA and RNA FASTQ trimming (rather than the default fastp) then they would change the following lines:

params.lens$alignment$manifest_to_dna_alns$fq_trim_tool = "fastp"
params.lens$alignment$manifest_to_rna_alns$fq_trim_tool = "fastp"

to

params.lens$alignment$manifest_to_dna_alns$fq_trim_tool = "trimmomatic"
params.lens$alignment$manifest_to_rna_alns$fq_trim_tool = "trimmomatic"

RAFT supported tools
Tool Parameter	Supported Tools
aln_tool	`bwa`, `bwa-mem2`, `bbmap`, `star`
antigen_tool	`mhcflurry`, `netmhcpan`, `netmhcstabpan`, `netctlpan`, `deephlapan`, `antigen_garnish`
base_recalibrator_tool	`gatk`
cna_tool	`cnvkit`
dup_marker_tool	`bamblaster` (`samblaster`), `picard2`
fq_trim_tool	`fastp`, `trimmomatic`, `trim_galore`
fusion_tool	`starfusion`
indel_realignment_tool	`abra2`, `abra2_rna`
mhc_caller_tool	`seq2hla`, `optitype`, `arcashla`, `hlaprofiler`
sample_swap_tool	`somalier`
tumor_purities_tool	`sequenza`
tx_quant_tool	`salmon`, `kallisto`
var_phaser_tool	`whatshap`
vcf_filtering_tool	`bcftools`
vcf_isecing_tool	`bedtools`
vcf_merging_tool	`jacquard_merge`
vcf_norming_tool	`bcftools`

Filesystem Storage Management

LENS can generate between 100 GB and 1 TB of intermediate files as part of the workflow. Most users will find themselves quickly hitting storage limits if LENS is using many samples. The --clean-intermediates option has been added to raft.py’s run-workflow mode to help alleviate excessive storage usage.

The --clean-intermediates option should only be used after LENS ha completed successfully upon a full set of patients for a project. Additional samples or patients can only be added to the project’s manifest by first deleting the contents of the work/ directory. LENS will be run on the project from scratch.

Using the --clean-intermediates option will result in many of the large files generated (trimmed FASTQs and BAM files) to be replaced by zero-sized files with identical file-level characteristics as the files they replace. This allows raft.py run-workflow ... to be run again on your project while allowing Nextflow’s caching system to work as expected. This allows users to effectively clean the project’s work/ directory while maintaining access to quant.sf files, vcf files, HLA typing outputs, and other potentially useful files (see lens_outputs).

More information on the technical aspects of --clean-intermediates can be found on the PIRL blog: https://tinyurl.com/trick-nf-caching.

Users can also modify tool-level parameters and re-run LENS at their own risk, but it is not recommended until we have more information regarding which parameters can be safely modified after intermediate file cleaning.