Questions tagged [snakemake]
Snakemake is a workflow management system with a Python-style specification language.
1,675
questions
0
votes
0
answers
5
views
Dynamic determination of snakemake wildcard fails
I have a short question regarding dynamic determination of wildcards in snakemake. In the code posted below a RNAseq read with a given id is downloaded (download_reads). The download either yields a &...
1
vote
1
answer
15
views
Run snakemake but with the .snakemake temporary files written elsewhere
Say I have a Snakemake workflow that has a Snakefile and various input files. I want to run this workflow exactly as it is, but with the .snakemake temporary files being written to a scratch ...
1
vote
1
answer
17
views
snakefile can not be found even it is defined in profile
I created a profile like the following in profileRNASeq/config.yaml:
`snakefile: Snakefile-RNASeq
jobscript: ../jobscript.sh
latency-wait: 60
reason: True
show-failed-logs: True
...`
As in the profile,...
0
votes
2
answers
27
views
Snakemake rule not prducing output in slurm mode
I only recently started using snakemake but so far it has been going really well and is taking alot of manual work off my shoulders. But now I ran into a problem which I just cannot figure out. I want ...
0
votes
1
answer
35
views
how to use snakemake parameters in a chain?
I'm creating a snakemake rule, and I want to use one parameter as the input for another snakemake parameter. I want to know how you refer to other parameters in the same params chunk of a snakemake ...
0
votes
0
answers
26
views
Snakemake: use wildcard as pandas data.frame filter value
I want to use a global wildcard (wildcard_constraints) as filter value for a pandas dataframe in order to build a set of inputs.
# I read the table
manifest = pd.read_csv(str(config["manifest&...
0
votes
2
answers
42
views
Snakemake, slurm and memory
I am struggling to understand how snakemake submits jobs to slurm.
When I have a basic slurm sbatch script I usually add a line, such as
#SBATCH --mem=5G
to determine that slurm may use 5 gigabytes (...
0
votes
1
answer
40
views
Missing Output Exception in rule build_veins
I am trying to run snakemake -jall to build the Serpentine scenario.
I get this error:
MissingOutputException in rule build_veins in file /serpentine-env/Snakefile, line 33:
Job 9 completed ...
0
votes
1
answer
30
views
Snakemake: AttributeError: 'Wildcards' object has no attribute 'sample
InputFunctionException in line 226 of /users/troger50/projects/Rogers_SidersViralAnalysis_XXXX_20XX/preprocess_reads.smk:
Error:
AttributeError: 'Wildcards' object has no attribute 'sample'
...
1
vote
0
answers
36
views
No rule to make target in rule build_veins
I am trying to use the serpentine serpentine scenario following this tutorial.
When I run snakemake -jall I get the following error:
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 4
...
1
vote
0
answers
23
views
Snakemake output redirection and output managemnt
I am very new to Snakemake. Currently just learning using online tutorials. My problem is noted below. Any clue would be helpful.
I have multiple files in input_data directory named as '...
2
votes
2
answers
37
views
Executing rule for pair of string and output in snakemake
I'm trying to learn snakemake and implement a simple workflow:
read all the lines from the file
for each line execute a shell command
redirect output to separate files
I'm running the following ...
0
votes
1
answer
27
views
Cannot use asterisk as glob pattern normally in Snakemake
I want to use asterisk as glob pattern to stand for "any string of characters except /" (as stated in glob Wikipedia) in Snakemake rule file in its shell section. But noticed that I cannot ...
0
votes
0
answers
54
views
Change usage text in Snakemake workflow catalog
How would one change the usage text for a workflow in the Snakemake workflow catalog?
For the example I have shown, I have been through all the directories in the repository, but I cannot find a file ...
0
votes
0
answers
40
views
"The system cannot find the path specified." on Windows while running fine on Linux
I cannot get a very simple example to work..
I want to create a file with a python script, which works fine when executed standalone and has the below content:
with open("test.txt", "w&...
1
vote
1
answer
51
views
snakemake rule without an output directive
let's assume I have a snakemake rule like this:
rule test:
input: myfile="myfile.txt",
params: test_out = "test",
shell: "tool {input.myfile} -p ~/desktop/{params....
0
votes
1
answer
30
views
is there a way to provide a function to ruleorder?
Is possible to provide a function to ruleorder, like this?
def determine_ruleorder(wildcards, config):
samples_file = pd.read_table(config['sample_file'], sep='\t')
sample_layout = ...
1
vote
1
answer
25
views
snakemake different wildcards in input vs output
I'm trying to achieve the following: apply my workflow on set of chromosomes, each will have multiple methods. So the input will always be one file with one wildcard chr, while the output will have ...
0
votes
0
answers
47
views
Using global variables in Snakemake modules
I’m trying to use an existing workflow (A, maintained by someone else) as part of a new one (B) by defining it as a module. The existing workflow uses global variables in its rules. When I execute A, ...
1
vote
0
answers
25
views
Why is Snakemake introducing whitespaces in my path to submodules?
I have a workflow with modules that uses conda and singularity.
Problem
When I try to run the workflow with snakemake -c 8 --use-conda --use-singularity (Snakemake 7.32.4), I get the following error:
...
0
votes
1
answer
27
views
snakemake rule with conda
I want to run a shell command via snakemake within a conda environment in a podman-container.
The snakefile looks like this:
rule download:
output:
"out/SRR12346784.fastq"
...
1
vote
0
answers
40
views
Snakemake workflow run with conda and singularity fails to find conda
I am facing an issue when running snakemake with a container and conda.
Let me explain better.
I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env....
1
vote
0
answers
82
views
Why wildcards calling is not workin?
I'm writing this code to use snakemake to operate bioinformatic pipelines, but for some reason the variable "wildcards.sra" is not being defined in the get_fastq and get_fastp functions. ...
1
vote
1
answer
53
views
how to avoid hidden files when defining glob_wildcards (multiple) from input files
I am trying to define the wildcards based on the input directory. Currently, my solution looks like this:
files = glob_wildcards("input/{organism}/{gene}/{sequence}.txt")
However, I noticed ...
0
votes
1
answer
13
views
snakemake: reading input files from directory structure and creating wildcards
I am looking to create a pipeline that can take multiple input files, apply various tools to the output, and produce the desired results. My input/ directory pattern looks like this:
- organism_A
...
0
votes
1
answer
15
views
stacks process_radtags in snakemake 7.32.4
I have been trying to convert my SLURM scripts to snakemake, which I am not too familiar with. I have paired-end reads from 5 pools. Each set of fastqz's is in it's own directory by pool. Here is my ...
0
votes
1
answer
26
views
Switch between sets of input of Snakemake workflow without starting from the head
I am building a Snakemake workflow to analyze exome sequencing data.
The workflow needs an input file set of one sample; sometimes, I need to work on other samples, but how can I do that I can pick up ...
1
vote
1
answer
57
views
Send email after Snakemake workflow finishes successfully on SLURM
Is there way to send an email after a Snakemake workflow finished when using the --slurm option?
Something like this:
onsuccess:
email([email protected], title="Snakemake job xxx finished ...
0
votes
1
answer
20
views
Is there a way to use a glob_wildcard function on newly created files and use the function as a wildcard in rule all in Snakemake?
I am building a Snakemake pipeline which clusters sequences and outputs a file of accession numbers for each cluster. I do not know how many clusters will be produced. I would like to generate a ...
1
vote
1
answer
37
views
How can I incorporate wildcards in Snakemake in an R script?
I am facing the following issue:
I have a rule in Snakemake that looks something like this:
rule somerule:
input:
tables = expand("results/{tables}_table.txt", tables = ["1", &...
1
vote
1
answer
481
views
MultiQC snakemake wrapper: ModuleNotFoundError No module named 'imp'
I am running FastQC and MultiQC in my snakemake pipeline:
rule fastqc:
input:
"reads/{sample}_trimmed.fq.gz"
output:
html="qc/fastqc/{sample}.html",
...
2
votes
2
answers
57
views
How can I use a for loop to programmatically generate snakemake rules?
I would like to use a for loop to generate a set of rules, based on the contents of of the config.yaml file. Here's a minimum working example.
config={'person1': 'Amy', 'person2': 'Bruce'}
...
1
vote
1
answer
30
views
snakemake get number of input of a rule
I want to do something simple: get the number of input that has a snakemake rules, and use this value in my shell commands.
I have a working solution with the trick below, but this is very ugly and I ...
0
votes
1
answer
37
views
Manage the wrappers in Snakemake
I am trying to make a Snakemake pipeline using a configuration file config.yaml.
---
samples:
- Sample_1: reads/raw_reads_1
- Sample_2: reads/raw_reads_2
- Sample_3: reads/raw_reads_3
reference:
- &...
1
vote
0
answers
43
views
snakemake tolerance for missing input
In Snakemake, is there a built-in flag to tolerate missing input?
I have defined a Variable, say CHR = [1,2,3,4,5] and my starting point is one file of which multiple files could be generated (rule 1 -...
1
vote
1
answer
36
views
Snakemake rule to merge files into one
I am working with a large ensemble of climate model outputs with snakemake. Lets say I have the following set of files:
tas_AFR-44_NOAA-GFDL-GFDL-ESM2M_rcp45_r1i1p1_SMHI-RCA4_v1_mon_200601-201012.nc
...
0
votes
1
answer
47
views
snakemake gives all messages twice
Snakemake is giving me a lot of extra warnings and info, many of them twice. How do I turn this behaviour off?
See screen shot below:
Now with reprex
I've been able to track the source of the problem ...
0
votes
0
answers
38
views
snakemake doesn't execute the right rules
I'm using snakemake 7.25.0. Sometimes, some of my rules are not re-executed after a parameter change. Sometimes it detects a parameter change, and sometimes it doesn't. More generally, I'm ...
0
votes
1
answer
36
views
Samples file not found when running snakemake using tibanna
I am trying to run a snakemake workflow on AWS using Tibanna. The sample list is stored in a TSV file:
samples.tsv.
For some reason however, this samples.tsv is not copied to the EC2 instance when ...
0
votes
0
answers
74
views
How do I get gpu assignment working in SLURM through snakemake?
I am trying to make a pipeline with snakemake which involves the use of a gpu and a singularity container for a specific rule. However, while I entered the same resources as I have for performing the ...
1
vote
1
answer
73
views
Snakemake task fails on SLURM without error - how to debug?
I try to run multiple tasks via SLURM using snakemake --slurm -j 200. Some of the SLURM jobs fail but I cannot find any error message. snakemake prints the following:
...
[Thu Sep 28 10:25:04 2023]
...
1
vote
0
answers
74
views
How to restrict snakejobs to a specific node using --slurm?
I am using snakemake to execute my workflow.
Here is my command:
snakemake -s /path/to/Snakefile --slurm --jobs 50 --use-conda
However, this command submits jobs to all nodes. How can I provide names ...
1
vote
2
answers
38
views
Skip snakemake rules that depend on failed rule
This is a section of my snakemake workflow:
The rule bagel2bf fails occasionally with certain data sets, but I do not want this to fail the entire snakemake run.
How can I adapt the workflow so that ...
0
votes
0
answers
40
views
Snakemake: Handling Preempted Jobs on Slurm: Issues with Job Status and Cleanup
This message follows up on the one posted two years ago (Snakemake: Job preemption can interrupt running jobs on clusters, how to make sure that the task is not considered as failed?). I no longer ...
0
votes
0
answers
30
views
Can someone explain to me how the containerization within snakemake works?
I'm curious about the snakemake --use-singularity function works. Is snakemake running the analysis through a container using singularity exec or singularity pull? It seems to me that it is using ...
1
vote
1
answer
25
views
How to print the cluster command in the log before executing it?
I am using snakemake 7.32.3 on a cluster. I have set the cluster parameter to a complex command with wildcards in my configuration file. Now the cluster command fails, and snakemake outputs the error ...
1
vote
3
answers
77
views
How to solve wildcards in input files cannot be determined from output files in snakelike
I am a new Snakemake user and trying to develop a pipeline using some data to be able to implement to to our real data. I have multiple folders (one folder for each patient) and in each folder there ...
2
votes
2
answers
65
views
How to rename samples based on dictionary values
I have some trouble writing a snakemake rule to change the name of my samples. After demultiplexing with Porechop and some basic trimming with Filtlong, I would like to change the names of my samples ...
1
vote
1
answer
30
views
Unique ID of a snakemake run
When running a Nextflow pipeline, a unique ID is associated with the whole run. It can, for example, be obtained by examining the nextflow log. I am looking for the same feature for Snakemake ...
1
vote
1
answer
30
views
Missing Output Exception Error in Snakemake
I am using snakemake version 7.30.1
I am trying to run my snakemake workflow using snakemake --cores 4. Snakemake seems to be able to locate the input files and seems to start to complete the steps of ...