Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/ad/24edc3b8b5c67ed5074786316ec76c/.command.sh
Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Sequence/WholeGenomeFasta/Homo_sapiens_assembly38.fasta.fai
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/53/ceed4bf612d62859d876f95d1d2d6f/HCC1395_BL.recalibrated.bam
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/53/ceed4bf612d62859d876f95d1d2d6f/HCC1395_BL.recalibrated.bam.bai
Downloading: s3://natera-platform-sandbox/pipeline-resources/ngi-igenomes/igenomes/Homo_sapiens/GATK/GRCh38/Annotation/RF_Models/germline_rescue_targets.bed
Downloading: s3://natera-rnd-pltf-dev-nextflow-scratch-01/work/ad/24edc3b8b5c67ed5074786316ec76c/.command.run
==> STAGING COMPLETE (7 inputs)
2026-06-08 22:19:16.256625: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
I0608 22:19:17.890285 140531213670208 run_deepvariant.py:513] Creating a directory for intermediate results in tmp
***** Intermediate results will be written to tmp in docker. ****
***** Running the command:*****
time seq 0 3 | parallel -q --halt 2 --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "Homo_sapiens_assembly38.fasta" --reads "HCC1395_BL.recalibrated.bam" --examples "tmp/make_examples.tfrecord@4.gz" --channels "insert_size" --gvcf "tmp/gvcf.tfrecord@4.gz" --regions "germline_rescue_targets.bed" --task {}
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.212204 140110422464320 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.222187 140110422464320 make_examples_core.py:301] Task 0/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.233068 140110422464320 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.255918 139701177616192 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.264119 139701177616192 make_examples_core.py:301] Task 3/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.275024 139701177616192 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.212165 140007149168448 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.222181 140007149168448 make_examples_core.py:301] Task 2/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.233078 140007149168448 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.582293 140110422464320 make_examples_core.py:301] Task 0/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.410003 139845464147776 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.418490 139845464147776 make_examples_core.py:301] Task 1/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:20.429543 139845464147776 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:20.580345 140007149168448 make_examples_core.py:301] Task 2/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0608 22:19:20.776444 139845464147776 make_examples_core.py:301] Task 1/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0608 22:19:20.614619 139701177616192 make_examples_core.py:301] Task 3/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0608 22:19:20.864050 140110422464320 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0608 22:19:21.059342 139845464147776 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0608 22:19:20.897660 139701177616192 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0608 22:19:21.034774 140007149168448 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0608 22:19:21.181311 140110422464320 make_examples_core.py:301] Task 0/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.225352 140110422464320 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.280097 140110422464320 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:21.280862 140110422464320 make_examples_core.py:301] Task 0/4: Writing gvcf records to tmp/gvcf.tfrecord-00000-of-00004.gz
I0608 22:19:21.281323 140110422464320 make_examples_core.py:301] Task 0/4: Writing examples to tmp/make_examples.tfrecord-00000-of-00004.gz
I0608 22:19:21.281386 140110422464320 make_examples_core.py:301] Task 0/4: Overhead for preparing inputs: 1 seconds
I0608 22:19:21.284943 140110422464320 make_examples_core.py:301] Task 0/4: 0 candidates (0 examples) [0.00s elapsed]
I0608 22:19:21.356593 139845464147776 make_examples_core.py:301] Task 1/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.401231 139845464147776 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:21.214725 139701177616192 make_examples_core.py:301] Task 3/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.261441 139701177616192 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.315366 139701177616192 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:21.316830 139701177616192 make_examples_core.py:301] Task 3/4: Writing gvcf records to tmp/gvcf.tfrecord-00003-of-00004.gz
I0608 22:19:21.317391 139701177616192 make_examples_core.py:301] Task 3/4: Writing examples to tmp/make_examples.tfrecord-00003-of-00004.gz
I0608 22:19:21.317471 139701177616192 make_examples_core.py:301] Task 3/4: Overhead for preparing inputs: 1 seconds
I0608 22:19:21.320290 139701177616192 make_examples_core.py:301] Task 3/4: 0 candidates (0 examples) [0.00s elapsed]
I0608 22:19:21.439994 140007149168448 make_examples_core.py:301] Task 2/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.456374 139845464147776 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:21.457239 139845464147776 make_examples_core.py:301] Task 1/4: Writing gvcf records to tmp/gvcf.tfrecord-00001-of-00004.gz
I0608 22:19:21.457767 139845464147776 make_examples_core.py:301] Task 1/4: Writing examples to tmp/make_examples.tfrecord-00001-of-00004.gz
I0608 22:19:21.457854 139845464147776 make_examples_core.py:301] Task 1/4: Overhead for preparing inputs: 1 seconds
I0608 22:19:21.460678 139845464147776 make_examples_core.py:301] Task 1/4: 0 candidates (0 examples) [0.00s elapsed]
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.487128 140007149168448 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: HCC1395_BL.recalibrated.bam.bai
I0608 22:19:21.542367 140007149168448 genomics_reader.py:222] Reading HCC1395_BL.recalibrated.bam with NativeSamReader
I0608 22:19:21.543323 140007149168448 make_examples_core.py:301] Task 2/4: Writing gvcf records to tmp/gvcf.tfrecord-00002-of-00004.gz
I0608 22:19:21.543936 140007149168448 make_examples_core.py:301] Task 2/4: Writing examples to tmp/make_examples.tfrecord-00002-of-00004.gz
I0608 22:19:21.544011 140007149168448 make_examples_core.py:301] Task 2/4: Overhead for preparing inputs: 1 seconds
I0608 22:19:21.547370 140007149168448 make_examples_core.py:301] Task 2/4: 0 candidates (0 examples) [0.00s elapsed]
I0608 22:19:23.352592 139845464147776 make_examples_core.py:301] Task 1/4: Writing example info to tmp/make_examples.tfrecord-00001-of-00004.gz.example_info.json
I0608 22:19:23.353040 139845464147776 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0608 22:19:23.353749 139845464147776 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0608 22:19:23.354731 139845464147776 make_examples_core.py:301] Task 1/4: Found 1 candidate variants
I0608 22:19:23.354797 139845464147776 make_examples_core.py:301] Task 1/4: Created 1 examples
I0608 22:19:23.283876 139701177616192 make_examples_core.py:301] Task 3/4: Writing example info to tmp/make_examples.tfrecord-00003-of-00004.gz.example_info.json
I0608 22:19:23.284301 139701177616192 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0608 22:19:23.284975 139701177616192 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0608 22:19:23.285971 139701177616192 make_examples_core.py:301] Task 3/4: Found 4 candidate variants
I0608 22:19:23.286046 139701177616192 make_examples_core.py:301] Task 3/4: Created 4 examples
I0608 22:19:23.973885 140007149168448 make_examples_core.py:301] Task 2/4: Writing example info to tmp/make_examples.tfrecord-00002-of-00004.gz.example_info.json
I0608 22:19:23.974378 140007149168448 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0608 22:19:23.975585 140007149168448 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0608 22:19:23.976521 140007149168448 make_examples_core.py:301] Task 2/4: Found 6 candidate variants
I0608 22:19:23.976587 140007149168448 make_examples_core.py:301] Task 2/4: Created 6 examples
I0608 22:19:24.763649 140110422464320 make_examples_core.py:301] Task 0/4: Writing example info to tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json
I0608 22:19:24.763919 140110422464320 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0608 22:19:24.764471 140110422464320 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0608 22:19:24.765043 140110422464320 make_examples_core.py:301] Task 0/4: Found 4 candidate variants
I0608 22:19:24.765081 140110422464320 make_examples_core.py:301] Task 0/4: Created 9 examples
real 0m7.387s
user 0m33.392s
sys 0m40.267s
***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "tmp/call_variants_output.tfrecord.gz" --examples "tmp/make_examples.tfrecord@4.gz" --checkpoint "/opt/models/wes"
/usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning:
TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP).
For more information see: https://github.com/tensorflow/addons/issues/2807
warnings.warn(
I0608 22:19:27.907566 140503967078208 call_variants.py:563] Total 1 writing processes started.
I0608 22:19:27.909468 140503967078208 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0608 22:19:27.909554 140503967078208 call_variants.py:588] Shape of input examples: [100, 221, 7]
I0608 22:19:27.909860 140503967078208 call_variants.py:592] Use saved model: True
I0608 22:19:33.768234 140503967078208 dv_utils.py:370] From /opt/models/wes/example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0608 22:19:33.768447 140503967078208 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0608 22:19:34.841255 140503967078208 call_variants.py:716] Predicted 20 examples in 1 batches [4.656 sec per 100].
I0608 22:19:34.939695 140503967078208 call_variants.py:779] Complete: call_variants.
real 0m10.737s
user 0m13.293s
sys 0m13.918s
***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "Homo_sapiens_assembly38.fasta" --infile "tmp/call_variants_output.tfrecord.gz" --outfile "germline_control_1.dv_rescue.vcf.gz" --cpus "4" --gvcf_outfile "germline_control_1.dv_rescue.g.vcf.gz" --nonvariant_site_tfrecord_path "tmp/gvcf.tfrecord@4.gz"
I0608 22:19:38.264111 140466652923712 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: HCC1395_BL
2026-06-08 22:19:38.269155: I deepvariant/postprocess_variants.cc:94] Read from: tmp/call_variants_output-00000-of-00001.tfrecord.gz
2026-06-08 22:19:38.269332: I deepvariant/postprocess_variants.cc:109] Total #entries in single_site_calls = 20
I0608 22:19:38.270539 140466652923712 postprocess_variants.py:1313] CVO sorting took 9.246269861857096e-05 minutes
I0608 22:19:38.270752 140466652923712 postprocess_variants.py:1316] Transforming call_variants_output to variants.
I0608 22:19:38.270789 140466652923712 postprocess_variants.py:1318] Using 4 CPUs for parallelization of variant transformation.
I0608 22:19:38.292897 140466652923712 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: HCC1395_BL
I0608 22:19:38.298568 140466652923712 postprocess_variants.py:1386] Processing variants (and writing to temporary file) took 4.3503443400065105e-05 minutes
I0608 22:19:38.415243 140466652923712 postprocess_variants.py:1407] Finished writing VCF and gVCF in 0.0019423206647237142 minutes.
real 0m2.698s
user 0m5.635s
sys 0m11.200s
***** Running the command:*****
time /opt/deepvariant/bin/vcf_stats_report --input_vcf "germline_control_1.dv_rescue.vcf.gz" --outfile_base "germline_control_1.dv_rescue"
I0608 22:19:40.724802 139693396100928 genomics_reader.py:222] Reading germline_control_1.dv_rescue.vcf.gz with NativeVcfReader
real 0m2.371s
user 0m5.286s
sys 0m11.206s