***** Intermediate results will be written to tmp in docker. ****
***** Running the command:*****
time seq 0 3 | parallel -q --halt 2 --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "Homo_sapiens_assembly38.fasta" --reads "Sig_18_Blood.recalibrated.bam" --examples "tmp/make_examples.tfrecord@4.gz" --channels "insert_size" --gvcf "tmp/gvcf.tfrecord@4.gz" --regions "germline_rescue_targets.bed" --task {}
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:52.390149 139900468287296 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.421085 139900468287296 make_examples_core.py:301] Task 2/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:52.429660 139900468287296 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.390198 140335321716544 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.420947 140335321716544 make_examples_core.py:301] Task 3/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:52.429664 140335321716544 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.390180 140580657555264 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.421074 140580657555264 make_examples_core.py:301] Task 0/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:52.429807 140580657555264 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.390451 140240387868480 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.420967 140240387868480 make_examples_core.py:301] Task 1/4: Preparing inputs
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:52.429774 140240387868480 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:52.697735 139900468287296 make_examples_core.py:301] Task 2/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0605 15:19:52.696568 140335321716544 make_examples_core.py:301] Task 3/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0605 15:19:52.697174 140580657555264 make_examples_core.py:301] Task 0/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0605 15:19:52.697649 140240387868480 make_examples_core.py:301] Task 1/4: Common contigs are ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chrM']
I0605 15:19:52.935679 139900468287296 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0605 15:19:52.932807 140335321716544 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0605 15:19:52.933611 140580657555264 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
I0605 15:19:53.189898 139900468287296 make_examples_core.py:301] Task 2/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0605 15:19:53.187264 140335321716544 make_examples_core.py:301] Task 3/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0605 15:19:53.189418 140580657555264 make_examples_core.py:301] Task 0/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
I0605 15:19:53.014014 140240387868480 genomics_reader.py:222] Reading germline_rescue_targets.bed with NativeBedReader
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.263300 139900468287296 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.307059 139900468287296 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:53.307743 139900468287296 make_examples_core.py:301] Task 2/4: Writing gvcf records to tmp/gvcf.tfrecord-00002-of-00004.gz
I0605 15:19:53.308138 139900468287296 make_examples_core.py:301] Task 2/4: Writing examples to tmp/make_examples.tfrecord-00002-of-00004.gz
I0605 15:19:53.308201 139900468287296 make_examples_core.py:301] Task 2/4: Overhead for preparing inputs: 0 seconds
I0605 15:19:53.310583 139900468287296 make_examples_core.py:301] Task 2/4: 0 candidates (0 examples) [0.00s elapsed]
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.250270 140335321716544 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.293420 140335321716544 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:53.294113 140335321716544 make_examples_core.py:301] Task 3/4: Writing gvcf records to tmp/gvcf.tfrecord-00003-of-00004.gz
I0605 15:19:53.294582 140335321716544 make_examples_core.py:301] Task 3/4: Writing examples to tmp/make_examples.tfrecord-00003-of-00004.gz
I0605 15:19:53.294644 140335321716544 make_examples_core.py:301] Task 3/4: Overhead for preparing inputs: 0 seconds
I0605 15:19:53.296802 140335321716544 make_examples_core.py:301] Task 3/4: 0 candidates (0 examples) [0.00s elapsed]
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.263204 140580657555264 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.307877 140580657555264 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:53.308561 140580657555264 make_examples_core.py:301] Task 0/4: Writing gvcf records to tmp/gvcf.tfrecord-00000-of-00004.gz
I0605 15:19:53.309030 140580657555264 make_examples_core.py:301] Task 0/4: Writing examples to tmp/make_examples.tfrecord-00000-of-00004.gz
I0605 15:19:53.309090 140580657555264 make_examples_core.py:301] Task 0/4: Overhead for preparing inputs: 0 seconds
I0605 15:19:53.311853 140580657555264 make_examples_core.py:301] Task 0/4: 0 candidates (0 examples) [0.00s elapsed]
I0605 15:19:53.281530 140240387868480 make_examples_core.py:301] Task 1/4: Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.360071 140240387868480 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
[W::hts_idx_load3] The index file is older than the data file: Sig_18_Blood.recalibrated.bam.bai
I0605 15:19:53.404038 140240387868480 genomics_reader.py:222] Reading Sig_18_Blood.recalibrated.bam with NativeSamReader
I0605 15:19:53.404844 140240387868480 make_examples_core.py:301] Task 1/4: Writing gvcf records to tmp/gvcf.tfrecord-00001-of-00004.gz
I0605 15:19:53.405514 140240387868480 make_examples_core.py:301] Task 1/4: Writing examples to tmp/make_examples.tfrecord-00001-of-00004.gz
I0605 15:19:53.405592 140240387868480 make_examples_core.py:301] Task 1/4: Overhead for preparing inputs: 0 seconds
I0605 15:19:53.408124 140240387868480 make_examples_core.py:301] Task 1/4: 0 candidates (0 examples) [0.00s elapsed]
I0605 15:19:55.001671 139900468287296 make_examples_core.py:301] Task 2/4: Writing example info to tmp/make_examples.tfrecord-00002-of-00004.gz.example_info.json
I0605 15:19:55.001892 139900468287296 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0605 15:19:55.002314 139900468287296 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0605 15:19:55.002717 139900468287296 make_examples_core.py:301] Task 2/4: Found 4 candidate variants
I0605 15:19:55.002753 139900468287296 make_examples_core.py:301] Task 2/4: Created 4 examples
I0605 15:19:54.952937 140335321716544 make_examples_core.py:301] Task 3/4: Writing example info to tmp/make_examples.tfrecord-00003-of-00004.gz.example_info.json
I0605 15:19:54.953167 140335321716544 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0605 15:19:54.953612 140335321716544 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0605 15:19:54.954086 140335321716544 make_examples_core.py:301] Task 3/4: Found 1 candidate variants
I0605 15:19:54.954117 140335321716544 make_examples_core.py:301] Task 3/4: Created 1 examples
I0605 15:19:55.286491 140240387868480 make_examples_core.py:301] Task 1/4: Writing example info to tmp/make_examples.tfrecord-00001-of-00004.gz.example_info.json
I0605 15:19:55.286955 140240387868480 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0605 15:19:55.288223 140240387868480 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0605 15:19:55.289130 140240387868480 make_examples_core.py:301] Task 1/4: Found 2 candidate variants
I0605 15:19:55.289185 140240387868480 make_examples_core.py:301] Task 1/4: Created 2 examples
I0605 15:19:55.471640 140580657555264 make_examples_core.py:301] Task 0/4: Writing example info to tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json
I0605 15:19:55.471882 140580657555264 make_examples_core.py:2958] example_shape = [100, 221, 7]
I0605 15:19:55.472327 140580657555264 make_examples_core.py:2959] example_channels = [1, 2, 3, 4, 5, 6, 19]
I0605 15:19:55.472818 140580657555264 make_examples_core.py:301] Task 0/4: Found 4 candidate variants
I0605 15:19:55.472854 140580657555264 make_examples_core.py:301] Task 0/4: Created 9 examples
real 0m5.290s
user 0m25.941s
sys 0m40.348s
***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "tmp/call_variants_output.tfrecord.gz" --examples "tmp/make_examples.tfrecord@4.gz" --checkpoint "/opt/models/wes"
/usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning:
TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP).
For more information see: https://github.com/tensorflow/addons/issues/2807
warnings.warn(
I0605 15:19:57.739166 140019687417664 call_variants.py:563] Total 1 writing processes started.
I0605 15:19:57.740899 140019687417664 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0605 15:19:57.740968 140019687417664 call_variants.py:588] Shape of input examples: [100, 221, 7]
I0605 15:19:57.741261 140019687417664 call_variants.py:592] Use saved model: True
I0605 15:20:01.998737 140019687417664 dv_utils.py:370] From /opt/models/wes/example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0605 15:20:01.999017 140019687417664 dv_utils.py:370] From tmp/make_examples.tfrecord-00000-of-00004.gz.example_info.json: Shape of input examples: [100, 221, 7], Channels of input examples: [1, 2, 3, 4, 5, 6, 19].
I0605 15:20:03.392998 140019687417664 call_variants.py:716] Predicted 16 examples in 1 batches [7.902 sec per 100].
I0605 15:20:03.509997 140019687417664 call_variants.py:779] Complete: call_variants.
real 0m8.869s
user 0m10.731s
sys 0m15.659s
***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "Homo_sapiens_assembly38.fasta" --infile "tmp/call_variants_output.tfrecord.gz" --outfile "Sig_18_Blood.dv_rescue.vcf.gz" --cpus "4" --gvcf_outfile "Sig_18_Blood.dv_rescue.g.vcf.gz" --nonvariant_site_tfrecord_path "tmp/gvcf.tfrecord@4.gz"
I0605 15:20:06.422645 140475859322688 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: Sig_18_Blood
2026-06-05 15:20:06.426248: I deepvariant/postprocess_variants.cc:94] Read from: tmp/call_variants_output-00000-of-00001.tfrecord.gz
2026-06-05 15:20:06.426382: I deepvariant/postprocess_variants.cc:109] Total #entries in single_site_calls = 16
I0605 15:20:06.427341 140475859322688 postprocess_variants.py:1313] CVO sorting took 6.987253824869792e-05 minutes
I0605 15:20:06.427530 140475859322688 postprocess_variants.py:1316] Transforming call_variants_output to variants.
I0605 15:20:06.427561 140475859322688 postprocess_variants.py:1318] Using 4 CPUs for parallelization of variant transformation.
I0605 15:20:06.457605 140475859322688 postprocess_variants.py:1211] Using sample name from call_variants output. Sample name: Sig_18_Blood
I0605 15:20:06.462709 140475859322688 postprocess_variants.py:1386] Processing variants (and writing to temporary file) took 4.415512084960937e-05 minutes
I0605 15:20:06.556853 140475859322688 postprocess_variants.py:1407] Finished writing VCF and gVCF in 0.0015668590863545735 minutes.
real 0m2.218s
user 0m4.524s
sys 0m10.291s
***** Running the command:*****
time /opt/deepvariant/bin/vcf_stats_report --input_vcf "Sig_18_Blood.dv_rescue.vcf.gz" --outfile_base "Sig_18_Blood.dv_rescue"
I0605 15:20:08.507026 139820149634880 genomics_reader.py:222] Reading Sig_18_Blood.dv_rescue.vcf.gz with NativeVcfReader
real 0m1.918s
user 0m4.466s
sys 0m10.018s