Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/0e/4b4a07858010a6e7e03c07efc14e73/.command.sh
Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta.fai
Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/RF_Models/germline_rescue_targets.bed
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/0e/4b4a07858010a6e7e03c07efc14e73/.command.run
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/ba/3ff9499941a6df5e9cecb065774346/Sig_18_Blood.recalibrated.bam.bai
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/ba/3ff9499941a6df5e9cecb065774346/Sig_18_Blood.recalibrated.bam
==> STAGING COMPLETE (7 inputs)
2026-05-27 23:56:39.498304: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I0527 23:56:41.765743 140280229476160 run_deepvariant.py:513] Creating a directory for intermediate results in tmp
***** Intermediate results will be written to tmp in docker. ****
***** Running the command:*****
time seq 0 3 | parallel -q --halt 2 --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "Homo_sapiens_assembly38.fasta" --reads "Sig_18_Blood.recalibrated.bam" --examples "tmp/make_examples.tfrecord@4.gz" --channels "insert_size" --gvcf "tmp/gvcf.tfrecord@4.gz" --regions "germline_rescue_targets.bed" --task {}
I0527 23:56:44.726463 140682491152192 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.733731 140682491152192 make_examples_core.py:301] Task 2/4: Preparing inputs
I0527 23:56:44.749131 140682491152192 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.790164 139951608940352 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.796891 139951608940352 make_examples_core.py:301] Task 1/4: Preparing inputs
I0527 23:56:44.812511 139951608940352 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.784696 139741646640960 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.791483 139741646640960 make_examples_core.py:301] Task 0/4: Preparing inputs
I0527 23:56:44.806834 139741646640960 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.804797 140281403303744 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:44.811652 140281403303744 make_examples_core.py:301] Task 3/4: Preparing inputs
I0527 23:56:44.827413 140281403303744 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:45.324694 140682491152192 make_examples_core.py:301] Task 2/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0527 23:56:45.400083 139951608940352 make_examples_core.py:301] Task 1/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0527 23:56:45.403575 139741646640960 make_examples_core.py:301] Task 0/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0527 23:56:45.676210 140682491152192 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0527 23:56:45.414437 140281403303744 make_examples_core.py:301] Task 3/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0527 23:56:45.783831 139951608940352 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0527 23:56:45.779409 139741646640960 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0527 23:56:45.800982 140281403303744 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0527 23:56:46.277071 139951608940352 make_examples_core.py:301] Task 1/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0527 23:56:46.337203 139951608940352 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.447086 139951608940352 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.448158 139951608940352 make_examples_core.py:301] Task 1/4: Writing gvcf records to tmp/gvcf.tfrecord-00001-of-00004.gz
I0527 23:56:46.448759 139951608940352 make_examples_core.py:301] Task 1/4: Writing examples to tmp/make_examples.tfrecord-00001-of-00004.gz
I0527 23:56:46.448839 139951608940352 make_examples_core.py:301] Task 1/4: Overhead for preparing inputs: 1 seconds
I0527 23:56:46.452904 139951608940352 make_examples_core.py:301] Task 1/4: 0 candidates (0 examples) [0.00s elapsed]
I0527 23:56:46.274759 139741646640960 make_examples_core.py:301] Task 0/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0527 23:56:46.337128 139741646640960 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.448241 139741646640960 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.449372 139741646640960 make_examples_core.py:301] Task 0/4: Writing gvcf records to tmp/gvcf.tfrecord-00000-of-00004.gz
I0527 23:56:46.450010 139741646640960 make_examples_core.py:301] Task 0/4: Writing examples to tmp/make_examples.tfrecord-00000-of-00004.gz
I0527 23:56:46.450083 139741646640960 make_examples_core.py:301] Task 0/4: Overhead for preparing inputs: 1 seconds
I0527 23:56:46.455507 139741646640960 make_examples_core.py:301] Task 0/4: 0 candidates (0 examples) [0.01s elapsed]
I0527 23:56:46.157801 140682491152192 make_examples_core.py:301] Task 2/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0527 23:56:46.213751 140682491152192 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.306273 140682491152192 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.307202 140682491152192 make_examples_core.py:301] Task 2/4: Writing gvcf records to tmp/gvcf.tfrecord-00002-of-00004.gz
I0527 23:56:46.307755 140682491152192 make_examples_core.py:301] Task 2/4: Writing examples to tmp/make_examples.tfrecord-00002-of-00004.gz
I0527 23:56:46.307826 140682491152192 make_examples_core.py:301] Task 2/4: Overhead for preparing inputs: 1 seconds
I0527 23:56:46.311867 140682491152192 make_examples_core.py:301] Task 2/4: 0 candidates (0 examples) [0.00s elapsed]
I0527 23:56:46.298453 140281403303744 make_examples_core.py:301] Task 3/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0527 23:56:46.350574 140281403303744 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.459584 140281403303744 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0527 23:56:46.460689 140281403303744 make_examples_core.py:301] Task 3/4: Writing gvcf records to tmp/gvcf.tfrecord-00003-of-00004.gz
I0527 23:56:46.461279 140281403303744 make_examples_core.py:301] Task 3/4: Writing examples to tmp/make_examples.tfrecord-00003-of-00004.gz
I0527 23:56:46.461353 140281403303744 make_examples_core.py:301] Task 3/4: Overhead for preparing inputs: 1 seconds
I0527 23:56:46.465443 140281403303744 make_examples_core.py:301] Task 3/4: 0 candidates (0 examples) [0.00s elapsed]
I0527 23:56:49.412157 139951608940352 make_examples_core.py:301] Task 1/4: Writing example info to tmp/make_examples.tfrecord-00001-of-00004.gz.example_info.json
I0527 23:56:49.412480 139951608940352 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0527 23:56:49.413189 139951608940352 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0527 23:56:49.413932 139951608940352 make_examples_core.py:301] Task 1/4: Found 2 candidate variants
I0527 23:56:49.414014 139951608940352 make_examples_core.py:301] Task 1/4: Created 2 examples
I0527 23:56:49.668920 140281403303744 make_examples_core.py:301] Task 3/4: Writing example info to tmp/make_examples.tfrecord-00003-of-00004.gz.example_info.json
I0527 23:56:49.669234 140281403303744 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0527 23:56:49.669972 140281403303744 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0527 23:56:49.670706 140281403303744 make_examples_core.py:301] Task 3/4: Found 4 candidate variants
I0527 23:56:49.670789 140281403303744 make_examples_core.py:301] Task 3/4: Created 4 examples
I0527 23:56:49.932637 140682491152192 make_examples_core.py:301] Task 2/4: Writing example info to tmp/make_examples.tfrecord-00002-of-00004.gz.example_info.json
I0527 23:56:49.932928 140682491152192 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0527 23:56:49.933622 140682491152192 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0527 23:56:49.934204 140682491152192 make_examples_core.py:301] Task 2/4: Found 4 candidate variants
I0527 23:56:49.934284 140682491152192 make_examples_core.py:301] Task 2/4: Created 4 examples
I0527 23:56:50.507992 139741646640960 make_examples_core.py:301] Task 0/4: Writing example info to tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json
I0527 23:56:50.508328 139741646640960 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0527 23:56:50.509036 139741646640960 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0527 23:56:50.509730 139741646640960 make_examples_core.py:301] Task 0/4: Found 4 candidate variants
I0527 23:56:50.509811 139741646640960 make_examples_core.py:301] Task 0/4: Created 9 examples
real 0m9.196s
user 0m35.640s
sys 0m13.699s
***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "tmp/call_variants_output.tfrecord.gz" --examples "tmp/make_examples.tfrecord@4.gz" --checkpoint "/opt/models/wes"
/usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning:
TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP).
For more information see: https://github.com/tensorflow/addons/issues/2807
warnings.warn(
I0527 23:56:54.022096 139664886847296 call_variants.py:563] Total 1 writing processes started.
I0527 23:56:54.024298 139664886847296 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0527 23:56:54.024386 139664886847296 call_variants.py:588] Shape of input examples: [100, 221, 7]
I0527 23:56:54.024688 139664886847296 call_variants.py:592] Use saved model: True
I0527 23:57:01.090720 139664886847296 dv_utils.py:370] From /opt/models/wes/example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0527 23:57:01.090971 139664886847296 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0527 23:57:01.908634 139664886847296 call_variants.py:716] Predicted 19 examples in 1 batches [3.423 sec per 100].
I0527 23:57:02.001486 139664886847296 call_variants.py:779] Complete: call_variants.
real 0m12.026s
user 0m17.209s
sys 0m9.593s
***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "Homo_sapiens_assembly38.fasta" --infile "tmp/call_variants_output.tfrecord.gz" --outfile "germline_control_2.dv_rescue.vcf.gz" --cpus "4" --gvcf_outfile "germline_control_2.dv_rescue.g.vcf.gz" --nonvariant_site_tfrecord_path "tmp/gvcf.tfrecord@4.gz"
I0527 23:57:05.740002 139683496466240 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: Sig_18_Blood
2026-05-27 23:57:05.746441: I deepvariant/postprocess_variants.cc:94] Read from: tmp/call_variants_output-00000-of-00001.tfrecord.gz
2026-05-27 23:57:05.746635: I deepvariant/postprocess_variants.cc:109] Total #entries in single_site_calls = 19
I0527 23:57:05.748295 139683496466240 postprocess_variants.py:1313] CVO sorting took 0.00012689431508382162 minutes
I0527 23:57:05.748491 139683496466240 postprocess_variants.py:1316] Transforming call_variants_output to variants.
I0527 23:57:05.748539 139683496466240 postprocess_variants.py:1318] Using 4 CPUs for parallelization of variant transformation.
I0527 23:57:05.773677 139683496466240 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: Sig_18_Blood
I0527 23:57:05.780366 139683496466240 postprocess_variants.py:1386] Processing variants (and writing to temporary file) took 3.000497817993164e-05 minutes
I0527 23:57:05.933332 139683496466240 postprocess_variants.py:1407] Finished writing VCF and gVCF in 0.002547323703765869 minutes.
real 0m3.301s
user 0m8.601s
sys 0m8.064s
***** Running the command:*****
time /opt/deepvariant/bin/vcf_stats_report --input_vcf "germline_control_2.dv_rescue.vcf.gz" --outfile_base "germline_control_2.dv_rescue"
I0527 23:57:09.043582 139903902947136 genomics_reader.py:222] Reading germline_control_2.dv_rescue.vcf.gz with NativeVcfReader
real 0m3.262s
user 0m8.432s
sys 0m7.984s