File Info

Filename
.command.err
Full Path
s3://natera-rnd-fsdx-dev-nextflow-scratch-01/work/c6/61d20867302eb2409daf25239b2f71/.command.err
Size
7.6 KB
Attempt
Looking to launch executable "/opt/conda/bin/bwa-mem2.avx512bw", simd = .avx512bw
Launching executable "/opt/conda/bin/bwa-mem2.avx512bw"
-----------------------------
Executing in AVX512 mode!!
-----------------------------
* SA compression enabled with xfactor: 8
* Ref file: hg38.fa
* Entering FMI_search
* Index file found. Loading index from hg38.fa.bwt.2bit.64
* Reference seq len for bi-index = 6418572211
* sentinel-index: 2729492284
* Count:
0,	1
1,	1879238230
2,	3209286106
3,	4539333982
4,	6418572211

* Reading other elements of the index from files hg38.fa
* Index prefix: hg38.fa
* Read 0 ALT contigs
* Done reading Index!!
* Reading reference genome..
* Binary seq file = hg38.fa.0123
* Reference genome size: 6418572210 bp
* Done reading reference genome !!

------------------------------------------
1. Memory pre-allocation for Chaining: 696.7080 MB
2. Memory pre-allocation for BSW: 958.4681 MB
3. Memory pre-allocation for BWT: 309.2567 MB
------------------------------------------
* Threads used (compute): 4
* No. of pipeline threads: 2

[0000] read_chunk: 50000000, work_chunk_size: 50000046, nseq: 380766
	[0000][ M::kt_pipeline] read 380766 sequences (50000046 bp)...
[0000] Reallocating initial memory allocations!!
[0000] Calling mem_process_seqs.., task: 0
[0000] 1. Calling kt_for - worker_bwt
[0000] read_chunk: 50000000, work_chunk_size: 50000129, nseq: 380614
	[0000][ M::kt_pipeline] read 380614 sequences (50000129 bp)...
[0000] 2. Calling kt_for - worker_aln
[0000] Inferring insert size distribution of PE reads from data, l_pac: 3209286105, n: 380766
[0000][PE] # candidate unique pairs for (FF, FR, RF, RR): (0, 189210, 0, 0)
[0000][PE] skip orientation FF as there are not enough pairs
[0000][PE] analyzing insert size distribution for orientation FR...
[0000][PE] (25, 50, 75) percentile: (136, 169, 202)
[0000][PE] low and high boundaries for computing mean and std.dev: (4, 334)
[0000][PE] mean and std.dev: (169.40, 49.13)
[0000][PE] low and high boundaries for proper pairs: (1, 400)
[0000][PE] skip orientation RF as there are not enough pairs
[0000][PE] skip orientation RR as there are not enough pairs
[0000] 3. Calling kt_for - worker_sam
	[0000][ M::mem_process_seqs] Processed 380766 reads in 45.108 CPU sec, 11.145 real sec
[0000] Calling mem_process_seqs.., task: 1
[0000] 1. Calling kt_for - worker_bwt
[W::sam_hrecs_update_hashes] PG line with multiple ID tags. The first encountered was preferred - ID:bwa-mem2
[0000] read_chunk: 50000000, work_chunk_size: 50000090, nseq: 381478
	[0000][ M::kt_pipeline] read 381478 sequences (50000090 bp)...
[0000] 2. Calling kt_for - worker_aln
[0000] Inferring insert size distribution of PE reads from data, l_pac: 3209286105, n: 380614
[0000][PE] # candidate unique pairs for (FF, FR, RF, RR): (0, 189158, 0, 0)
[0000][PE] skip orientation FF as there are not enough pairs
[0000][PE] analyzing insert size distribution for orientation FR...
[0000][PE] (25, 50, 75) percentile: (136, 169, 203)
[0000][PE] low and high boundaries for computing mean and std.dev: (2, 337)
[0000][PE] mean and std.dev: (169.16, 48.87)
[0000][PE] low and high boundaries for proper pairs: (1, 404)
[0000][PE] skip orientation RF as there are not enough pairs
[0000][PE] skip orientation RR as there are not enough pairs
[0000] 3. Calling kt_for - worker_sam
	[0000][ M::mem_process_seqs] Processed 380614 reads in 45.059 CPU sec, 11.123 real sec
[0000] Reallocating initial memory allocations!!
[0000] Calling mem_process_seqs.., task: 2
[0000] 1. Calling kt_for - worker_bwt
[0000] read_chunk: 50000000, work_chunk_size: 50000276, nseq: 381594
	[0000][ M::kt_pipeline] read 381594 sequences (50000276 bp)...
[0000] 2. Calling kt_for - worker_aln
[0000] Inferring insert size distribution of PE reads from data, l_pac: 3209286105, n: 381478
[0000][PE] # candidate unique pairs for (FF, FR, RF, RR): (0, 189649, 0, 0)
[0000][PE] skip orientation FF as there are not enough pairs
[0000][PE] analyzing insert size distribution for orientation FR...
[0000][PE] (25, 50, 75) percentile: (135, 169, 203)
[0000][PE] low and high boundaries for computing mean and std.dev: (1, 339)
[0000][PE] mean and std.dev: (169.35, 49.72)
[0000][PE] low and high boundaries for proper pairs: (1, 407)
[0000][PE] skip orientation RF as there are not enough pairs
[0000][PE] skip orientation RR as there are not enough pairs
[0000] 3. Calling kt_for - worker_sam
	[0000][ M::mem_process_seqs] Processed 381478 reads in 44.656 CPU sec, 11.002 real sec
[0000] Reallocating initial memory allocations!!
[0000] Calling mem_process_seqs.., task: 3
[0000] 1. Calling kt_for - worker_bwt
[0000] read_chunk: 50000000, work_chunk_size: 29809520, nseq: 227414
	[0000][ M::kt_pipeline] read 227414 sequences (29809520 bp)...
[0000] 2. Calling kt_for - worker_aln
[0000] Inferring insert size distribution of PE reads from data, l_pac: 3209286105, n: 381594
[0000][PE] # candidate unique pairs for (FF, FR, RF, RR): (0, 189618, 0, 0)
[0000][PE] skip orientation FF as there are not enough pairs
[0000][PE] analyzing insert size distribution for orientation FR...
[0000][PE] (25, 50, 75) percentile: (135, 169, 202)
[0000][PE] low and high boundaries for computing mean and std.dev: (1, 336)
[0000][PE] mean and std.dev: (168.71, 49.20)
[0000][PE] low and high boundaries for proper pairs: (1, 403)
[0000][PE] skip orientation RF as there are not enough pairs
[0000][PE] skip orientation RR as there are not enough pairs
[0000] 3. Calling kt_for - worker_sam
	[0000][ M::mem_process_seqs] Processed 381594 reads in 45.112 CPU sec, 11.190 real sec
[0000] Calling mem_process_seqs.., task: 4
[0000] 1. Calling kt_for - worker_bwt
[0000] read_chunk: 50000000, work_chunk_size: 0, nseq: 0
[0000] 2. Calling kt_for - worker_aln
[0000] Inferring insert size distribution of PE reads from data, l_pac: 3209286105, n: 227414
[0000][PE] # candidate unique pairs for (FF, FR, RF, RR): (0, 113083, 0, 0)
[0000][PE] skip orientation FF as there are not enough pairs
[0000][PE] analyzing insert size distribution for orientation FR...
[0000][PE] (25, 50, 75) percentile: (135, 168, 202)
[0000][PE] low and high boundaries for computing mean and std.dev: (1, 336)
[0000][PE] mean and std.dev: (168.71, 49.53)
[0000][PE] low and high boundaries for proper pairs: (1, 403)
[0000][PE] skip orientation RF as there are not enough pairs
[0000][PE] skip orientation RR as there are not enough pairs
[0000] 3. Calling kt_for - worker_sam
	[0000][ M::mem_process_seqs] Processed 227414 reads in 26.611 CPU sec, 6.658 real sec
[0000] read_chunk: 50000000, work_chunk_size: 0, nseq: 0
[0000] Computation ends..
No. of OMP threads: 4
Processor is running @3000.208338 MHz
Runtime profile:

	Time taken for main_mem function: 62.82 sec

	IO times (sec) :
	Reading IO time (reads) avg: 3.17, (3.17, 3.17)
	Writing IO time (SAM) avg: 1.09, (1.09, 1.09)
	Reading IO time (Reference Genome) avg: 3.87, (3.87, 3.87)
	Index read time avg: 6.12, (6.12, 6.12)

	Overall time (sec) (Excluding Index reading time):
	PROCESS() (Total compute time + (read + SAM) IO time) : 52.12
	MEM_PROCESS_SEQ() (Total compute time (Kernel + SAM)), avg: 51.12, (51.12, 51.12)

	 SAM Processing time (sec):
	--WORKER_SAM avg: 16.18, (16.18, 16.18)

	Kernels' compute time (sec):
	Total kernel (smem+sal+bsw) time avg: 34.73, (34.73, 34.73)
		SMEM compute avg: 22.09, (22.13, 22.07)
		SAL compute avg: 2.87, (2.88, 2.86)
				MEM_SA avg: 1.73, (1.73, 1.72)

		BSW time, avg: 9.09, (9.11, 9.09)

Important parameter settings: 
	BATCH_SIZE: 512
	MAX_SEQ_LEN_REF: 256
	MAX_SEQ_LEN_QER: 128
	MAX_SEQ_LEN8: 128
	SEEDS_PER_READ: 500
	SIMD_WIDTH8 X: 64
	SIMD_WIDTH16 X: 32
	AVG_SEEDS_PER_READ: 64
[bam_sort_core] merging from 0 files and 4 in-memory blocks...