Usage
=====

Running Off-the-shelf LENS
______________________________

RAFT supports "off-the-shelf" (OTS) workflows that can be modified by the end user if
desired. The OTS workflow offering includes:

- Pre-defined, compatible reference files
- Pre-defined tools
- Pre-defined, default parameters

More information regarding the default references, tools, and parameters for
off-the-shelf LENS can be found at the :doc:`technical` page.

Users can run an off-the-shelf LENS with their own manifest with the RAFT's `run-ots` (run off-the-shelf) mode. The `run-ots` mode requires several input paramaters:

.. list-table::
   :widths: 50 50
   :header-rows: 1

   * - Column Parameter
     - Description
   * - ``--project-id``
     - Unique identifier for your project. Projects are stored in ``/path/to/raft/projects/``.
   * - ``--workflow``
     - Species the workflow to run (LENS, in this case, though other workflows are supported).
   * - ``--version``
     - Specifies version of the workflow to run.
   * - ``--species``
     - Indicates the species for which the workflow should run.
   * - ``--inputs``
     - Indicates the input data type.
   * - ``--manifest``
     - The manifest file to use. Full path is not required (as long as the manifest is in ``/path/to/raft/metadata``).

Users can run off-the-shelf LENS on their own manifest by running

.. code-block:: console

  $ raft.py run-ots --project-id <PROJECT_ID> \
                    --workflow lens \
                    --version 1.8 \
                    --species <SPECIES> \
                    --inputs <INPUT_TYPE> \
                    --manifest <MANIFEST>

For example:

.. code-block:: console

  $ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                    --workflow lens \
                    --version 1.8 \
                    --species human \
                    --inputs fastqs \
                    --manifest my_manifest.tsv

Note that ``raft.py run-ots`` assumes human samples and FASTQs inputs, so the
the above command is equivilant to the following:

.. code-block:: console

  $ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                    --workflow lens \
                    --version 1.8 \
                    --manifest my_manifest.tsv

Running Off-the-shelf LENS with Mouse Data
__________________________________________

LENS can be run on mouse samples, but users must:

- Provide the MHC alleles in the user-provided manifest (see :doc:`manifest`)
- Modify the ``nextflow.config`` file to change the ``starfusion`` Docker image (see below)

With these requirements in mind, users should run following command:

.. code-block:: console

  $ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                    --workflow lens \
                    --version 1.8 \
                    --manifest my_manifest.tsv \
                    --species mouse \
                    --setup-only

Users can then run the LENS workflow using:

.. code-block:: console

  $ raft.py run-workflow --project-id my-project-lens-01JAN2025


Running Off-the-shelf LENS using cloud service providers (EXPERIMENTAL)
_______________________________________________________________________

LENS has been run using the Google Cloud Platform (GCP) after setting up an
appropriate Nextflow profile. Users must perform the following to support cloud
usage. Note that we intend for this feature to be more user-friendly in future
versions of LENS.

Requirements:
 - Upload all required references (those downloaded by ``raft.py run-ots``) to ``gs://<your_bucket>/references/cloud/``.
 - Upload FASTQs or BAMs to ``gs://<your_bucket>/fastqs/<your_dataset>/`` or ``gs://<your_bucket>/bams/<your_dataset>/``, respectively.
 - Upload your manifest file to ``gs://<your_bucket>/metadata/<your_dataset>/``.

At this point, users should run:

.. code-block:: console

  $ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                    --workflow lens \
                    --version 1.8 \
                    --manifest my_manifest.tsv \
                    --cloud gs://<your_bucket \
                    -n="-profile <your_gcp_profile>"


Running Off-the-shelf LENS with User-defined Parameters
_______________________________________________________
.. _userparams:

Users can run an off-the-shelf LENS instance with their preferred tools and tool
parameters using the ``raft.py`` ``-up/--user-params`` argument. This argument
can be specified any number of times. For example, if a user wants to run the
LENS workflow with their own manifest, but with
``trim_galore`` instead of ``fastp``, then they can run:

.. code-block:: console

  $ raft.py run-ots --project-id my-project-lens-01JAN2025 \
                    --workflow lens \
                    --manifest </PATH/TO/MANIFEST>  \
                    --version 1.8 \
                    --user-params fq_trim_tool=fastp \
                    --user-params fq_trim_tool_parameters=fastp:'--paired'

Users may specify multiple tools and parameters for each of multiple tools using the following syntax:

.. code-block:: console

  $ raft.py run-ots --project-id ... \
                    ... \
                    --user-params fq_trim_tool=tool1,tool2 \
                    --user-params fq_trim_tool_parameters=tool1:'--params --for --tool 1',\
                                                          tool2:'--params --for --tool 2'

Generally speaking, it is only recommended to use multiple tools for the `last`
modifying step of the workflow. For example, the ``gene-fusions`` workflow for
humans starting from FASTQs allow both ``fq_trim_tool`` and ``fusion_tool`` to
be specified. ``fusion_tool`` generates the ultimate outputs from the workflow
and thus users can specify multiple ``fusion_tool`` tools (e.g.
``fusion_tool=starfusion,arriba``). ``fq_trim_tool``, however, is upstream of
fusion calling and users should only specify a single tool for
``fq_trim_tool``.

Running Off-the-shelf LENS with User-specified References
_________________________________________________________
In order to use alternative references files, users should:

1. Download the new reference file and move or symlink it to RAFT's ``references/`` directory (``/path/to/raft/references``).
2. Users should then load the reference file into their LENS project using ``raft.py load-reference -p <PROJECT_NAME> -f <FILE_NAME>``.
3. Finally, users should modify their LENS project's ``main.nf`` file to use their new desired reference. For example, the line

.. code-block:: console

  params.lens$alignment$manifest_to_dna_alns$aln_ref = "${params.ref_dir}/Homo_sapiens.assembly38.no_ebv.fa"

could be changed to:

.. code-block:: console

  params.lens$alignment$manifest_to_dna_alns$aln_ref = "${params.ref_dir}/hg19.fa"

.. note:
   Several reference files within LENS are interdependent. Specifically, the
   current reference files utilize hg38-based references and gencode v37
   annotations. Users should be aware of these dependencies when choosing
   alternative references.

Running Off-the-shelf LENS with User-specified Tools
____________________________________________________
A variety of alternative tools are supported for each subworkflow within LENS.
In order to use alternative tools, users should modify their LENS project's ``main.nf`` file to change the default tools to those they require. For example, if a user wanted to run ``trimmomatic`` for DNA and RNA FASTQ trimming (rather than the default ``fastp``) then they would change the following lines:

.. code-block:: console

  params.lens$alignment$manifest_to_dna_alns$fq_trim_tool = "fastp"
  params.lens$alignment$manifest_to_rna_alns$fq_trim_tool = "fastp"

to

.. code-block:: console

  params.lens$alignment$manifest_to_dna_alns$fq_trim_tool = "trimmomatic"
  params.lens$alignment$manifest_to_rna_alns$fq_trim_tool = "trimmomatic"


.. list-table:: RAFT supported tools
   :widths: 50 50
   :header-rows: 1

   * - Tool Parameter
     - Supported Tools
   * - aln_tool
     - ``bwa``, ``bwa-mem2``, ``bbmap``, ``star``
   * - antigen_tool
     - ``mhcflurry``, ``netmhcpan``, ``netmhcstabpan``, ``netctlpan``, ``deephlapan``, ``antigen_garnish``
   * - base_recalibrator_tool
     - ``gatk``
   * - cna_tool
     - ``cnvkit``
   * - dup_marker_tool
     - ``bamblaster`` (``samblaster``), ``picard2``
   * - fq_trim_tool
     - ``fastp``, ``trimmomatic``, ``trim_galore``
   * - fusion_tool
     - ``starfusion``
   * - indel_realignment_tool
     - ``abra2``, ``abra2_rna``
   * - mhc_caller_tool
     - ``seq2hla``, ``optitype``, ``arcashla``, ``hlaprofiler``
   * - sample_swap_tool
     - ``somalier``
   * - tumor_purities_tool
     - ``sequenza``
   * - tx_quant_tool
     - ``salmon``, ``kallisto``
   * - var_phaser_tool
     - ``whatshap``
   * - vcf_filtering_tool
     - ``bcftools``
   * - vcf_isecing_tool
     - ``bedtools``
   * - vcf_merging_tool
     - ``jacquard_merge``
   * - vcf_norming_tool
     - ``bcftools``

Filesystem Storage Management
_____________________________

LENS can generate between 100 GB and 1 TB of intermediate files as part
of the workflow. Most users will find themselves quickly hitting storage limits
if LENS is using many samples. The ``--clean-intermediates`` option has been
added to ``raft.py``'s ``run-workflow`` mode to help alleviate excessive
storage usage.

The ``--clean-intermediates`` option should **only** be used after LENS ha
completed successfully upon a full set of patients for a project. **Additional
samples or patients can only be added to the project's manifest by first
deleting the contents of the** ``work/`` **directory. LENS will be run on the project
from scratch**.

Using the ``--clean-intermediates`` option will result in many of the large
files generated (trimmed FASTQs and BAM files) to be replaced by zero-sized
files with identical file-level characteristics as the files they replace.
**This allows** ``raft.py run-workflow ...`` **to be run again on your project
while allowing Nextflow's caching system to work as expected**. This allows
users to effectively clean the project's ``work/`` directory while maintaining
access to ``quant.sf`` files, ``vcf`` files, HLA typing outputs, and other
potentially useful files (see :doc:`lens_outputs`).

More information on the technical aspects of ``--clean-intermediates`` can be
found on the PIRL blog: https://tinyurl.com/trick-nf-caching.

Users can also modify tool-level parameters and re-run LENS **at their own
risk**, but it is not recommended until we have more information regarding
which parameters can be safely modified after intermediate file cleaning.