This study was supported by the National Natural Science Foundation of China (Grant No.32101188) and the General Project of Science and Technology Department of Sichuan Province (Grant No. 2021YFS0102), China.

This retrospective study was approved by the institutional review board of the West China Hospital, Sichuan University, Sichuan, China, and the requirement to obtain informed consent was waived.
1. Environment setup
<ol>
	<li>Graphic processing unit (GPU) software
	<ol>
		<li>To implement deep learning applications, first configure the GPU-related environment. Download and install GPU-appropriate software and drivers from the GPU&#39;s website. 
		&#8203;NOTE: See the Table of Materials for those used in this study.</li>
	</ol>
	</li>
	<li>Python3.8 installation
	<ol>
		<li>Open a terminal on the machine. Type the following: 
		Command line: sudo apt-get install python3.8 python-dev python-virtualenv</li>
	</ol>
	</li>
	<li>Pytorch1.7 installation
	<ol>
		<li>Follow the steps on the official website to download and install Miniconda.</li>
		<li>Create a conda environment and activate it. 
		Command line: conda create --name SwinFasterRCNN python=3.8 -y 
		Command line: conda activate SwinFasterRCNN</li>
		<li>Install Pytorch. 
		Command line: conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 </li>
	</ol>
	</li>
	<li>MMDetection installation
	<ol>
		<li>Clone from the official Github repository. 
		Command line: git clone https://github.com/open-mmlab/mmdetection.git</li>
		<li>Install MMDetection. 
		Command line: cd mmdetection 
		Command line: pip install -v -e .</li>
	</ol>
	</li>
</ol>
2. Data preparation
<ol>
	<li>Data collection
	<ol>
		<li>Collected the ultrasound images (here, 3,000 cases from a Grade-A tertiary hospital). Ensure that each case has diagnostic records, treatment plans, US reports, and the corresponding US images.</li>
		<li>Place all the US images in a folder named &#34;images.&#34; 
		NOTE: The data used in this study included 3,853 US images from 3,000 cases.</li>
	</ol>
	</li>
	<li>Data cleaning
	<ol>
		<li>Manually check the dataset for images of non-thyroid areas, such as lymph images.</li>
		<li>Manually check the dataset for images containing color Doppler flow.</li>
		<li>Delete the images selected in the previous two steps. 
		NOTE: After data cleaning, 3,000 images were left from 2,680 cases.</li>
	</ol>
	</li>
	<li>Data annotation
	<ol>
		<li>Have a senior doctor locate the nodule area in the US image and outline the nodule boundary. 
		NOTE: The annotation software and process can be found in Supplemental File 1.</li>
		<li>Have another senior doctor review and revise the annotation results.</li>
		<li>Place the annotated data in a separate folder called &#34;Annotations.&#34;</li>
	</ol>
	</li>
	<li>Data split
	<ol>
		<li>Run the python script, and set the path of the image in step 2.1.2 and the paths of the annotations in step 2.3.3. Randomly divide all the images and the corresponding labeled files into training and validation sets at a ratio of 8:2. Save the training set data in the &#34;Train&#34; folder and the validation set data in the &#34;Val&#34; folder. 
		NOTE: Python scripts are provided in Supplemental File 2.</li>
	</ol>
	</li>
	<li>Converting to the CoCo dataset format 
	NOTE: To use MMDetection, process the data into a CoCo dataset format, which includes a json file that holds the annotation information and an image folder containing the US images.
	<ol>
		<li>Run the python script, and input the annotations folder paths (step 2.3.3) to extract the nodule areas outlined by the doctor and convert them into masks. Save all the masks in the &#34;Masks&#34; folder. 
		NOTE: The Python scripts are provided in Supplemental File 3.</li>
		<li>Run the python script, and set the path of the masks folder in step 2.5.1 to make the data into a dataset in CoCo format and generate a json file with the US images. 
		NOTE: Python scripts are provided in Supplemental File 4.</li>
	</ol>
	</li>
</ol>
3. Swin Faster RCNN configuration
<ol>
	<li>Download the Swin Transformer model file (https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer.py), modify it, and place it in the &#8220;mmdetection/mmdet/models/backbones/&#8221; folder. Open the &#8220;swin_transformer.py&#8221; file in a vim text editor, and modify it as the Swin Transformer model file provided in&#160;Supplemental File 5. 
	Command line: vim swin_transformer.py</li>
	<li>Make a copy of the Faster R-CNN configuration file, change the backbone to Swin Transformer, and set up the FPN parameters. 
	Command line: cd mmdetection/configs/faster_rcnn 
	Command line: cp faster_rcnn_r50_fpn_1x_coco.py swin_faster_rcnn_swin.py 
	NOTE: The Swin Faster R-CNN configuration file (swin_faster_rcnn_swin.py) is provided in Supplemental File 6. The Swin Faster R-CNN network structure is shown in Figure 1.</li>
	<li>Set the dataset path to the CoCo format dataset path (step 2.5.2) in the configuration file. Open the &#34;coco_detection.py&#34; file in the vim text editor, and modify the following line: 
	data_root = &#34;dataset path(step 2.5.2)&#34; 
	Command line:vim mmdetection/configs/_base_/datasets/coco_detection.py</li>
</ol>
4. Training the Swin Faster R-CNN
<ol>
	<li>Edit mmdetection/configs/_base_/schedules/schedule_1x.py, and set the default training-related parameters, including the learning rate, optimizer, and epoch. Open the &#34;schedule_1x.py&#34; file in the vim text editor, and modify the following lines: 
	optimizer = dict(type=&#34;AdamW&#34;, lr=0.001, momentum=0.9, weight_decay=0.0001) 
	runner = dict(type=&#39;EpochBasedRunner&#39;, max_epochs=48) 
	Command line:vim mmdetection/configs/_base_/schedules/schedule_1x.py 
	NOTE: In this protocol for this paper, the learning rate was set to 0.001, AdamW optimizer was used, the maximum training epoch was set to 48, and the batch size was set to 16.</li>
	<li>Begin training by typing the following commands. Wait for the network to begin training for 48 epochs and for the resulting trained weights of the Swin Faster R-CNN network to be generated in the output folder. Save the model weights with the highest accuracy on the validation set. 
	Command line: cd mmdetection 
	Command line: python tools/train.py congfigs/faster_rcnn/swin_faster_rcnn_swin.py --work-dir ./work_dirs 
	NOTE: The model was trained on an &#34;NVIDIA GeForce RTX3090 24G&#34; GPU. The central processing unit used was the &#34;AMD Epyc 7742 64-core processor &#215; 128&#34;, and the operating system was Ubuntu 18.06. The overall training time was ~2 h.</li>
</ol>
5. Performing thyroid nodule detection on new images
<ol>
	<li>After training, select the model with the best performance on the validation set for thyroid nodule detection in the new images.
	<ol>
		<li>First, resize the image to 512 pixels x 512 pixels, and normalize it. These operations are performed automatically when the test script is run. 
		Command line: python tools/test.py congfigs/faster_rcnn/swin_faster_rcnn_swin.py --out ./output</li>
		<li>Wait for the script to automatically load the pretrained model parameters to the Swin Faster R-CNN, and feed the preprocessed image into the Swin Faster R-CNN for inference. Wait for the Swin Faster R-CNN to output the prediction box for each image.</li>
		<li>Finally, allow the script to automatically perform NMS postprocessing on each image to remove duplicate detection boxes. 
		NOTE: The detection results are output to the specified folder, which contains the images with the detection boxes and the bounding box coordinates in a packed file.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Grant, E. G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+ultrasound+reporting+lexicon:+White+paper+of+the+ACR+Thyroid+Imaging,+Reporting+and+Data+System+(TIRADS)+committee.">Thyroid ultrasound reporting lexicon: White paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) committee.</a> Journal of the American College of Radiology. 12 (12 Pt A), 1272-1279 (2015).</li><li>Zhao, J., Zheng, W., Zhang, L., Tian, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Segmentation+of+ultrasound+images+of+thyroid+nodule+for+assisting+fine+needle+aspiration+cytology.">Segmentation of ultrasound images of thyroid nodule for assisting fine needle aspiration cytology.</a> Health Information Science and Systems. 1, 5 (2013).</li><li>Haugen, B. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=American+Thyroid+Association+management+guidelines+for+adult+patients+with+thyroid+nodules+and+differentiated+thyroid+cancer:+What+is+new+and+what+has+changed.">American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: What is new and what has changed.</a> Cancer. 123 (3), 372-381 (2017).</li><li>Shin, J. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ultrasonography+diagnosis+and+imaging-based+management+of+thyroid+nodules:+Revised+Korean+Society+of+Thyroid+Radiology+consensus+statement+and+recommendations.">Ultrasonography diagnosis and imaging-based management of thyroid nodules: Revised Korean Society of Thyroid Radiology consensus statement and recommendations.</a> Korean Journal of Radiology. 17 (3), 370-395 (2016).</li><li>Horvath, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+ultrasonogram+reporting+system+for+thyroid+nodules+stratifying+cancer+risk+for+clinical+management.">An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management.</a> The Journal of Clinical Endocrinology &amp; Metabolism. 94 (5), 1748-1751 (2009).</li><li>Park, J. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+proposal+for+a+thyroid+imaging+reporting+and+data+system+for+ultrasound+features+of+thyroid+carcinoma.">A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma.</a> Thyroid. 19 (11), 1257-1264 (2009).</li><li>Moon, W. -. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Benign+and+malignant+thyroid+nodules:+US+differentiation-Multicenter+retrospective+study.">Benign and malignant thyroid nodules: US differentiation-Multicenter retrospective study.</a> Radiology. 247 (3), 762-770 (2008).</li><li>Park, C. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+in+the+sonographic+evaluation+of+thyroid+nodules.">Observer variability in the sonographic evaluation of thyroid nodules.</a> Journal of Clinical Ultrasound. 38 (6), 287-293 (2010).</li><li>Kim, S. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+and+the+performance+between+faculties+and+residents:+US+criteria+for+benign+and+malignant+thyroid+nodules.">Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules.</a> Korean Journal of Radiology. 11 (2), 149-155 (2010).</li><li>Choi, Y. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+computer-aided+diagnosis+system+using+artificial+intelligence+for+the+diagnosis+and+characterization+of+thyroid+nodules+on+ultrasound:+initial+clinical+assessment.">A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment.</a> Thyroid. 27 (4), 546-552 (2017).</li><li>Chang, T. -. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+computer-aided+detection+and+diagnosis+system+in+the+differential+diagnosis+of+thyroid+lesions+in+ultrasonography.">The role of computer-aided detection and diagnosis system in the differential diagnosis of thyroid lesions in ultrasonography.</a> Journal of Medical Ultrasound. 23 (4), 177-184 (2015).</li><li>Li, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fully+convolutional+networks+for+ultrasound+image+segmentation+of+thyroid+nodules.">Fully convolutional networks for ultrasound image segmentation of thyroid nodules.</a> , 886-890 (2018).</li><li>Nguyen, D. T., Choi, J., Park, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+nodule+segmentation+in+ultrasound+image+based+on+information+fusion+of+suggestion+and+enhancement+networks.">Thyroid nodule segmentation in ultrasound image based on information fusion of suggestion and enhancement networks.</a> Mathematics. 10 (19), 3484 (2022).</li><li>Ma, J., Wu, F., Jiang, T. A., Zhu, J., Kong, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cascade+convolutional+neural+networks+for+automatic+detection+of+thyroid+nodules+in+ultrasound+images.">Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images.</a> Medical Physics. 44 (5), 1678-1691 (2017).</li><li>Song, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multitask+cascade+convolution+neural+networks+for+automatic+thyroid+nodule+detection+and+recognition.">Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition.</a> IEEE Journal of Biomedical and Health Informatics. 23 (3), 1215-1224 (2018).</li><li>Wang, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Learning+from+weakly-labeled+clinical+data+for+automatic+thyroid+nodule+classification+in+ultrasound+images.">Learning from weakly-labeled clinical data for automatic thyroid nodule classification in ultrasound images.</a> , 3114-3118 (2018).</li><li>Wang, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-scale+densely+connected+convolutional+neural+network+for+automated+thyroid+nodule+classification.">A multi-scale densely connected convolutional neural network for automated thyroid nodule classification.</a> Frontiers in Neuroscience. 16, 878718 (2022).</li><li>Krizhevsky, A., Sutskever, I., Hinton, G. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Imagenet+classification+with+deep+convolutional+neural+networks.">Imagenet classification with deep convolutional neural networks.</a> Communications of the ACM. 60 (6), 84-90 (2017).</li><li>He, K., Zhang, X., Ren, S., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+residual+learning+for+image+recognition.">Deep residual learning for image recognition.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 770-778 (2016).</li><li>Hu, H., Gu, J., Zhang, Z., Dai, J., Wei, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Relation+networks+for+object+detection.">Relation networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 3588-3597 (2018).</li><li>Szegedy, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Going+deeper+with+convolutions.">Going deeper with convolutions.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 1-9 (2015).</li><li>Dosovitskiy, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+image+is+worth+16x16+words:+Transformers+for+image+recognition+at+scale.">An image is worth 16x16 words: Transformers for image recognition at scale.</a> arXiv preprint arXiv:2010.11929. , (2020).</li><li>Touvron, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Training+data-efficient+image+transformers+&amp;+distillation+through+attention.">Training data-efficient image transformers &amp; distillation through attention.</a> arXiv:2012.12877. , (2021).</li><li>Liu, Z., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Swin+Transformer:+Hierarchical+vision+transformer+using+shifted+windows.">Swin Transformer: Hierarchical vision transformer using shifted windows.</a> 2021 IEEE/CVF International Conference on Computer Vision (ICCV). , 9992-10002 (2021).</li><li>Vaswani, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Attention+is+all+you+need.">Attention is all you need.</a> Advances in Neural Information Processing Systems. 30, (2017).</li><li>Chen, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=TransUNet:+Transformers+make+strong+encoders+for+medical+image+segmentation.+arXiv.">TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.</a> arXiv:2102.04306. , (2021).</li><li>Ren, S., He, K., Girshick, R., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Faster+r-cnn:+Towards+real-time+object+detection+with+region+proposal+networks.">Faster r-cnn: Towards real-time object detection with region proposal networks.</a> Advances in Neural Information Processing Systems. 28, 91-99 (2015).</li><li>Li, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+improved+deep+learning+approach+for+detection+of+thyroid+papillary+cancer+in+ultrasound+images.">An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images.</a> Scientific Reports. 8, 6600 (2018).</li><li>Lin, T. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feature+pyramid+networks+for+object+detection.">Feature pyramid networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 2117-2125 (2017).</li><li>Ouahabi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+of+wavelet+denoising+in+medical+imaging.">A review of wavelet denoising in medical imaging.</a> 2013 8th International Workshop on Systems, Signal Processing and their Applications. , 19-26 (2013).</li><li>Mahdaoui, A. E., Ouahabi, A., Moulay, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+denoising+using+a+compressive+sensing+approach+based+on+regularization+constraints.">Image denoising using a compressive sensing approach based on regularization constraints.</a> Sensors. 22 (6), 2199 (2022).</li><li>Castleman, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Digital Image Processing. , (1996).</li><li>Liu, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ssd:+Single+shot+multibox+detector.">Ssd: Single shot multibox detector.</a> European Conference on Computer Vision. , 21-37 (2016).</li><li>Redmon, J., Farhadi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Yolov3:+An+incremental+improvement.">Yolov3: An incremental improvement.</a> arXiv. arXiv:1804.02767. , (2018).</li><li>Lin, T. -. Y., Goyal, P., Girshick, R., He, K., Doll&#225;r, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Focalloss+for+dense+object+detection.">Focalloss for dense object detection.</a> arXiv. arXiv:1708.02002. , (2017).</li><li>Carion, N., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=End-to-end+object+detection+with+transformers.">End-to-end object detection with transformers.</a> Computer Vision-ECCV 2020: 16th European Conference. , 23-28 (2020).</li></ol>

The authors declare no conflicts of interest.

This paper describes in detail how to perform the environment setup, data preparation, model configuration, and network training. In the environment setup phase, one needs to pay attention to ensure that the dependent libraries are compatible and matched. Data processing is a very important step; time and effort must be spent to ensure the accuracy of the annotations. When training the model, a &#34;ModuleNotFoundError&#34; may be encountered. In this case, it is necessary to use the &#34;pip install&#34; command to install the missing library. If the loss of the validation set does not decrease or oscillates greatly, one should check the annotation file and try to adjust the learning rate and batch size to make the loss converge.
Thyroid nodule detection is very important for the treatment of thyroid cancer. The CAD system can assist doctors in the detection of nodules, avoid differences in diagnosis results caused by subjective factors, and reduce the missed detection of nodules. Compared with existing CNN-based CAD systems, the network proposed in this paper introduces the Swin Transformer to extract ultrasound image features. By capturing long-distance dependencies, Swin Faster R-CNN can extract the nodule features from ultrasound images more efficiently. The experimental results show that Swin Faster R-CNN improves the sensitivity of nodule detection by ~3% compared to CNN backbone-based Faster R-CNN. The application of this technology can greatly reduce the burden on doctors, as it can detect thyroid nodules in early ultrasound examination and guide doctors to further treatment. However, due to the large number of parameters of the Swin Transformer, the inference time of Swin Faster R-CNN is ~100 ms per image (tested on NVIDIA TITAN 24G GPU and AMD Epyc 7742 CPU). It can be challenging to meet the requirements of real-time diagnosis with Swin Faster R-CNN. In the future, we will continue to collect cases to verify the effectiveness of this method and conduct further studies on dynamic ultrasound image analysis.

The incidence of thyroid cancer has increased rapidly since 1970, especially among middle-aged women1. Thyroid nodules may predict the emergence of thyroid cancer, and most thyroid nodules are asymptomatic2. The early detection of thyroid nodules is very helpful in curing thyroid cancer. Therefore, according to current practice guidelines, all patients with suspected nodular goiter on physical examination or with abnormal imaging findings should undergo further examination3,4.
Thyroid ultrasound (US) is a common method used to detect and characterize thyroid lesions5,6. US is a convenient, inexpensive, and radiation-free technology. However, the application of US is easily affected by the operator7,8. Features such as the shape, size, echogenicity, and texture of thyroid nodules are easily distinguishable on US images. Although certain US features-calcifications, echogenicity, and irregular borders-are often considered criteria for identifying thyroid nodules, the presence of interobserver variability is unavoidable8,9. The diagnosis results of radiologists with different levels of experience are different. Inexperienced radiologists are more likely to misdiagnose than experienced radiologists. Some characteristics of US such as reflections, shadows, and echoes can degrade the image quality. This degradation in image quality caused by the nature of US imaging makes it difficult for even experienced physicians to locate nodules accurately.
Computer-aided diagnosis (CAD) for thyroid nodules has developed rapidly in recent years and can effectively reduce errors caused by different physicians and help radiologists diagnose nodules quickly and accurately10,11. Various CNN-based CAD systems have been proposed for thyroid US nodule analysis, including segmentation12,13, detection14,15, and classification16,17. CNN is a multilayer, supervised learning model18, and the core modules of CNN are the convolution and pooling layers. The convolution layers are used for feature extraction, and the pooling layers are used for downsampling. The shadow convolutional layers can extract primary features such as the texture, edges, and contours, while deep convolutional layers learn high-level semantic features.
CNNs have had great success in computer vision19,20,21. However, CNNs fail to capture long-range contextual dependencies due to the limited valid receptive field of the convolutional layers. In the past, backbone architectures for image classification mostly used CNNs. With the advent of Vision Transformer (ViT)22,23, this trend has changed, and now many state-of-the-art models use transformers as backbones. Based on non-overlapping image patches, ViT uses a standard transformer encoder25 to globally model spatial relationships. The Swin Transformer24 further introduces shift windows to learn features. The shift windows not only bring greater efficiency but also greatly reduce the length of the sequence because self-attention is calculated in the window. At the same time, the interaction between two adjacent windows can be made through the operation of shifting (movement). The successful application of the Swin Transformer in computer vision has led to the investigation of transformer-based architectures for ultrasound image analysis26.
Recently, Li et al. proposed a deep learning approach28 for thyroid papillary cancer detection inspired by Faster R-CNN27. Faster R-CNN is a classic CNN-based object detection architecture. The original Faster R-CNN has four modules-the CNN backbone, the region proposal network (RPN), the ROI pooling layer, and the detection head. The CNN backbone uses a set of basic conv+bn+relu+pooling layers to extract feature maps from the input image. Then, the feature maps are fed into the RPN and the ROI pooling layer. The role of the RPN network is to generate region proposals. This module uses softmax to determine whether anchors are positive and generates accurate anchors by bounding box regression. The ROI pooling layer extracts the proposal feature maps by collecting the input feature maps and proposals and feeds the proposal feature maps into the subsequent detection head. The detection head uses the proposal feature maps to classify objects and obtain accurate positions of the detection boxes by bounding box regression.
This paper presents a new thyroid nodule detection network called Swin Faster R-CNN formed by replacing the CNN backbone in Faster R-CNN with the Swin Transformer, which results in the better extraction of features for nodule detection from ultrasound images. In addition, the feature pyramid network (FPN)29 is used to improve the detection performance of the model for nodules of different sizes by aggregating features of different scales.

<table><tbody><tr><td>GPU RTX3090</td><td>Nvidia</td><td>1</td><td>24G GPU</td></tr><tr><td>mmdetection2.11.0</td><td>SenseTime</td><td>4</td><td>https://github.com/open-mmlab/mmdetection.git</td></tr><tr><td>python3.8</td><td>&amp;mdash;</td><td>2</td><td>https://www.python.org</td></tr><tr><td>pytorch1.7.1</td><td>Facebook</td><td>3</td><td>https://pytorch.org</td></tr></tbody></table>

a swin transformer-based model for thyroid nodule detection in ultrasound images

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid cancer. Convolutional neural networks (CNNs) have achieved good results in thyroid ultrasound image analysis tasks. However, due to the limited valid receptive field of convolutional layers, CNNs fail to capture long-range contextual dependencies, which are important for identifying thyroid nodules in ultrasound images. Transformer networks are effective in capturing long-range contextual information. Inspired by this, we propose a novel thyroid nodule detection method that combines the Swin Transformer backbone and Faster R-CNN. Specifically, an ultrasound image is first projected into a 1D sequence of embeddings, which are then fed into a hierarchical Swin Transformer. 
The Swin Transformer backbone extracts features at five different scales by utilizing shifted windows for the computation of self-attention. Subsequently, a feature pyramid network (FPN) is used to fuse the features from different scales. Finally, a detection head is used to predict bounding boxes and the corresponding confidence scores. Data collected from 2,680 patients were used to conduct the experiments, and the results showed that this method achieved the best mAP score of 44.8%, outperforming CNN-based baselines. In addition, we gained better sensitivity (90.5%) than the competitors. This indicates that context modeling in this model is effective for thyroid nodule detection.

The thyroid US images were collected from two hospitals in China from September 2008 to February 2018. The eligibility criteria for including the US images in this study were conventional US examination before biopsy and surgical treatment, diagnosis with biopsy or postsurgical pathology, and age &#8805; 18 years. The exclusion criteria were images without thyroid tissues.
The 3,000 ultrasound images included 1,384 malignant and 1,616 benign nodules. The majority (90%) of the malignant nodules were papillary carcinoma, and 66% of the benign nodules were nodular goiter. Here, 25% of the nodules were smaller than 5 mm, 38% were between 5 mm and 10 mm, and 37% were larger than 10 mm.
All the US images were collected using Philips IU22 and DC-80, and their default thyroid examination mode was used. Both instruments were equipped with 5-13 MHz linear probes. For good exposure of the lower thyroid margins, all the patients were examined in the supine position with their backs extended. Both thyroid lobes and the isthmus were scanned in the longitudinal and transverse planes according to the American College of Radiology accreditation standards. All the examinations were carried out by two senior thyroid radiologists with &#8805;10 years of clinical experience. The thyroid diagnosis was based on the histopathological findings from fine needle aspiration biopsy or thyroid surgery.
In real life, as US images are corrupted by noise, it is important to conduct proper preprocessing of the US images, such as image denoising based on wavelet transform30, compressive sensing31, and histogram equalization32. In this work, we used histogram equalization to preprocess the US images, enhance image quality, and alleviate image quality degradation caused by noise.
In what follows, true positive, false positive, true negative, and false negative are referred to as TP, FP, TN, and FN, respectively. We used mAP, sensitivity, and specificity to evaluate the model&#39;s nodule detection performance. mAP is a common metric in object detection. Sensitivity and specificity were calculated using equation (1) and equation (2):
<img alt="Equation 1" src="/files/ftp_upload/64480/64480eq01v2.jpg" style="margin: auto;" /> (1)
<img alt="Equation 2" src="/files/ftp_upload/64480/64480eq02v2.jpg" style="margin: auto;" /> (2)
In this paper, TP is defined as the number of correctly detected nodules, which have an intersection over union (IoU) between the prediction box and the ground truth box of &#62;0.3 and a confidence score &#62;0.6. IoU is the intersection over union, which is computed by using equation (3):
<img alt="Equation 3" src="/files/ftp_upload/64480/64480eq03.jpg" style="margin: auto;" /> (3)
We compared several classic object detection networks, including SSD33, YOLO-v334, CNN backbone-based Faster R-CNN27, RetinaNet35, and DETR36. YOLO-v3 and SSD are single-stage detection networks, DETR is a transformer-based object-detection network, and Faster R-CNN and RetinaNet are two-stage detection networks. Table 1 shows that the performance of Swin Faster R-CNN is superior to the other methods, reaching 0.448&#160;mAP, which is 0.028 higher than CNN backbone&#39;s Faster R-CNN and 0.037 higher than YOLO-v3. By using Swin Faster R-CNN, 90.5% of thyroid nodules can be detected automatically, which is ~3% higher than CNN backbone-based Faster R-CNN (87.1%). As shown in Figure 2, using Swin Transformer as the backbone makes boundary positioning more accurate.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/64480/64480fig01.jpg" /> 
Figure 1: Diagram of the Swin Faster R-CNN network architecture. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/64480fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/64480/64480fig02.jpg" /> 
Figure 2: Detection results. The detection results for the same image are in a given row. The columns are the detection results, from left to right, for Swin Faster R-CNN, Faster R-CNN, YOLO-v3, SSD, RetinaNet, and DETR, respectively. The ground truths of the regions are marked with green rectangular boxes. The detection results are framed by the red rectangular boxes. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/64480fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Method</td>
			<td>Backbone</td>
			<td>mAP</td>
			<td>Sensitivity</td>
			<td>Specificity</td>
		</tr>
		<tr>
			<td>YOLO-v3</td>
			<td>DarkNet</td>
			<td>0.411</td>
			<td>0.869</td>
			<td>0.877</td>
		</tr>
		<tr>
			<td>SSD</td>
			<td>VGG16</td>
			<td>0.425</td>
			<td>0.841</td>
			<td>0.849</td>
		</tr>
		<tr>
			<td>RetinaNet</td>
			<td>ResNet50</td>
			<td>0.382</td>
			<td>0.845</td>
			<td>0.841</td>
		</tr>
		<tr>
			<td>Faster R-CNN</td>
			<td>ResNet50</td>
			<td>0.42</td>
			<td>0.871</td>
			<td>0.864</td>
		</tr>
		<tr>
			<td>DETR</td>
			<td>ResNet50</td>
			<td>0.416</td>
			<td>0.882</td>
			<td>0.86</td>
		</tr>
		<tr>
			<td>Swin Faster R-CNN without FPN</td>
			<td rowspan="2">Swin Transformer</td>
			<td>0.431</td>
			<td>0.897</td>
			<td>0.905</td>
		</tr>
		<tr>
			<td>Swin Faster R-CNN with FPN</td>
			<td>0.448</td>
			<td>0.905</td>
			<td>0.909</td>
		</tr>
	</tbody>
</table>
Table 1: Performance comparison with state-of-the-art object detection methods.
Supplemental File 1:&#160;Operating instructions for the data annotation and the software used. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/64480_Supp File 1_data-annotation.zip" target="_blank">Please click here to download this File.</a>
Supplemental File 2: Python script used to divide the dataset into the training set and validation set, as mentioned in step 2.4.1. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/Supp File 2_train_val_split.zip" target="_blank">Please click here to download this File.</a>
Supplemental File 3: Python script used to convert the annotations file into masks, as mentioned in step 2.5.1. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/Supp File 3_marks2masks.zip" target="_blank">Please click here to download this File.</a>
Supplemental File 4: Python script used to make the data into a dataset in CoCo format, as mentioned in step 2.5.2. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/Supp File 4_masks2coco.zip" target="_blank">Please click here to download this File.</a>
Supplemental File 5:&#160;The modified Swin Transformer model file mentioned in step 3.1. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/Supp File 5_swin_transformer.zip" target="_blank">Please click here to download this File.</a>
Supplemental File 6: The Swin Faster R-CNN configuration file mentioned in step 3.2. <a href="https://www-jove-com.bdigitaluss.remotexs.co/files/ftp_upload/64480/Supp File 6_swin_faster_rcnn_swin.zip" target="_blank">Please click here to download this File.</a>

Watch this Scientific Journal Video about A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images at JoVE.com

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid ...

a-swin-transformer-based-model-for-thyroid-nodule-detection

Universidad San Sebastian

Research

JoVE Journal

Medicine

1.7K Views.  West China Hospital of Sichuan University. Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

Video: A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Multi-modal Imaging of Angiogenesis in a Nude Rat Model of Breast Cancer Bone Metastasis Using Magnetic Resonance Imaging, Volumetric Computed Tomography and Ultrasound

Angiogenesis is an essential feature of cancer growth and metastasis formation. In bone metastasis, angiogenic factors are pivotal for tumor cell proliferation in the bone marrow cavity as well as for interaction of tumor and bone cells resulting in local bone destruction. Our aim was to develop a model of experimental bone metastasis that allows in vivo assessment of angiogenesis in skeletal lesions using non-invasive imaging techniques.
For this purpose, we injected 105 MDA-MB-231 human breast cancer cells into the superficial epigastric artery, which precludes the growth of metastases in body areas other than the respective hind leg1. Following 25-30 days after tumor cell inoculation, site-specific bone metastases develop, restricted to the distal femur, proximal tibia and proximal fibula1. Morphological and functional aspects of angiogenesis can be investigated longitudinally in bone metastases using magnetic resonance imaging (MRI), volumetric computed tomography (VCT) and ultrasound (US).
MRI displays morphologic information on the soft tissue part of bone metastases that is initially confined to the bone marrow cavity and subsequently exceeds cortical bone while progressing. Using dynamic contrast-enhanced MRI (DCE-MRI) functional data including regional blood volume, perfusion and vessel permeability can be obtained and quantified2-4. Bone destruction is captured in high resolution using morphological VCT imaging. Complementary to MRI findings, osteolytic lesions can be located adjacent to sites of intramedullary tumor growth. After contrast agent application, VCT angiography reveals the macrovessel architecture in bone metastases in high resolution, and DCE-VCT enables insight in the microcirculation of these lesions5,6. US is applicable to assess morphological and functional features from skeletal lesions due to local osteolysis of cortical bone. Using B-mode and Doppler techniques, structure and perfusion of the soft tissue metastases can be evaluated, respectively. DCE-US allows for real-time imaging of vascularization in bone metastases after injection of microbubbles7.
In conclusion, in a model of site-specific breast cancer bone metastases multi-modal imaging techniques including MRI, VCT and US offer complementary information on morphology and functional parameters of angiogenesis in these skeletal lesions.

In the pathogenesis of bone metastasis, angiogenesis is a crucial process and therefore represents a target for imaging and therapy. Here, we present a rat model of site-specific breast cancer bone metastasis and describe strategies to non-invasively image angiogenesis in vivo using magnetic resonance imaging, volumetric computed tomography and ultrasound.

Angiogenesis is an essential feature of cancer growth and metastasis formation. In bone metastasis, angiogenic factors are pivotal for tumor cell ...

Cerenkov Luminescence Imaging (CLI) for Cancer Therapy Monitoring

In molecular imaging, positron emission tomography (PET) and optical imaging (OI) are two of the most important and thus most widely used modalities1-3. PET is characterized by its excellent sensitivity and quantification ability while OI is notable for non-radiation, relative low cost, short scanning time, high throughput, and wide availability to basic researchers. However, both modalities have their shortcomings as well. PET suffers from poor spatial resolution and high cost, while OI is mostly limited to preclinical applications because of its limited tissue penetration along with prominent scattering optical signals through the thickness of living tissues.
 Recently a bridge between PET and OI has emerged with the discovery of Cerenkov Luminescence Imaging (CLI)4-6. CLI is a new imaging modality that harnesses Cerenkov Radiation (CR) to image radionuclides with OI instruments. Russian Nobel laureate Alekseyevich Cerenkov and his colleagues originally discovered CR in 1934. It is a form of electromagnetic radiation emitted when a charged particle travels at a superluminal speed in a dielectric medium7,8. The charged particle, whether positron or electron, perturbs the electromagnetic field of the medium by displacing the electrons in its atoms. After passing of the disruption photons are emitted as the displaced electrons return to the ground state. For instance, one 18F decay was estimated to produce an average of 3 photons in water5. 
 Since its emergence, CLI has been investigated for its use in a variety of preclinical applications including in vivo tumor imaging, reporter gene imaging, radiotracer development, multimodality imaging, among others4,5,9,10,11. The most important reason why CLI has enjoyed much success so far is that this new technology takes advantage of the low cost and wide availability of OI to image radionuclides, which used to be imaged only by more expensive and less available nuclear imaging modalities such as PET.
 Here, we present the method of using CLI to monitor cancer drug therapy. Our group has recently investigated this new application and validated its feasibility by a proof-of-concept study12. We demonstrated that CLI and PET exhibited excellent correlations across different tumor xenografts and imaging probes. This is consistent with the overarching principle of CR that CLI essentially visualizes the same radionuclides as PET. We selected Bevacizumab (Avastin; Genentech/Roche) as our therapeutic agent because it is a well-known angiogenesis inhibitor13,14. Maturation of this technology in the near future can be envisioned to have a significant impact on preclinical drug development, screening, as well as therapy monitoring of patients receiving treatments.

Cerenkov Luminescence Imaging CLI for Cancer Therapy Monitoring

Use of Cerenkov Luminescence Imaging (CLI) for monitoring preclinical cancer treatment is described here. This method takes advantage of Cerenkov Radiation (CR) and optical imaging (OI) to visualize radiolabeled probes and thus provides an alternative to PET in preclinical therapeutic monitoring and drug screening.

In  molecular imaging, positron emission tomography (PET) and optical imaging (OI)  are two of the most important and thus most widely used modalities1-3. ...

An Orthotopic Mouse Model of Anaplastic Thyroid Carcinoma

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer cells into immunodeficient mice. Subcutaneous xenograft models have been valuable for preclinical screening and evaluation of new therapeutic treatments. There are a number of advantages to using a subcutaneous model; 1) rapid, 2) reproducible, and 3) tumor establishment, growth, and response to therapeutic agents may be monitored by visual inspection. However, substantial evidence has shed light on the short-comings of subcutaneous xenograft models1-3. For instance, medicinal treatments demonstrating curative properties in subcutaneous xenograft models often have no notable impact on the human disease. The microenvironment of the site of xenographic transplantation or injection lies at the heart of this dissimilarity.
Orthotopic tumor xenograft models provide a more biologically relevant context in which to study the disease. The advantages of implanting diseased cells or tissue into their anatomical origin equivalent within a host animal includes a suitable site for tumor-host interactions, development of disease-related metastases and the ability to examine site-specific influence on investigational therapeutic remedies. Therefore, orthotopic xenograft models harbor far more clinical value because they closely reproduce human disease. For these reasons, a number of groups have taken advantage of an orthotopic thyroid cancer model as a research tool4-7.
Here, we describe an approach that establishes an orthotopic model for the study of anaplastic thyroid carcinoma (ATC), which is highly invasive, resists treatment, and is virtually fatal in all diagnosed patients. Cultured ATC cells are prepared as a dissociated cellular suspension in a solution containing a basement membrane matrix. A small volume is slowly injected into the right thyroid gland. Overall appearance and health of the mice are monitored to ensure minimal post-operative complications and to gauge pathological penetrance of the cancer. Mice are sacrificed at 4 weeks, and tissue is collected for histological analysis. Animals may be taken at later time-points to examine more advance progression of the disease. Production of this orthotopic mouse model establishes a platform that accomplishes two objectives: 1) further our understanding of ATC pathology, and 2) screen current and future therapeutic agents for efficacy in combating ATC.

Generation of an orthotopic mouse model of anaplastic thyroid carcinoma is described here. This technique employs surgical placement of human anaplastic thyroid cancer cells into the thyroid of immunodeficient mice, thus creating a more clinically relevant setting to study disease progression as well as screen innovative therapeutic interventions.

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer ...

High-Resolution Ultrasonography for the Analysis of Orthotopic ATC Tumors in a Genetically Engineered Mouse Model

Anaplastic thyroid carcinoma (ATC) is associated with a poor prognosis and short median survival time, but no effective treatment improves the outcomes significantly. Genetically engineered murine models that mimic ATC&#39;s progression may help researchers to study treatments for this disease. Crossing three different genotypes of mice, a TPO-cre/ERT2; BrafCA/wt; Trp53&#916;ex2-10/&#916;ex2-10&#160;transgenic ATC model was developed. The ATC murine model was induced by an intraperitoneal injection of tamoxifen with overexpression of BrafV600E and deletion of Trp53, and the tumors were generated within about 1 month. High-resolution ultrasound was applied to investigate the tumor initiation and progression, and the dynamic growth curve was obtained by measuring the tumor sizes. Compared to magnetic resonance imaging (MRI) and computed tomography scanning, ultrasound has advantages in observing the ATC murine model, such as being noninvasive, portable, in real-time, and without radiation exposure. High-resolution ultrasound is suitable for dynamic and multiple measurements. However, ultrasonographic examination of the thyroid in mice requires relevant anatomical knowledge and experience. This article provides a detailed procedure for utilizing high-resolution ultrasound to scan tumors in the transgenic ATC model. Meanwhile, ultrasonic parameter adjustment, ultrasound scanning skills, anesthesia and recovery of the animals, and other elements that need attention during the process are listed.

The present protocol describes high-frequency ultrasonography for visualizing the entire mouse thyroid gland and monitoring the growth of anaplastic thyroid carcinoma.

Anaplastic thyroid carcinoma (ATC) is associated with a poor prognosis and short median survival time, but no effective treatment improves the outcomes ...

Cancer Research

Swin-PSAxialNet: A Streamlined Approach for Segmenting Multiple Organs

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Abdominal multi-organ segmentation is one of the most important topics in the field of medical image analysis, and it plays an important role in supporting clinical workflows such as disease diagnosis and treatment planning. In this study, an efficient multi-organ segmentation method called Swin-PSAxialNet based on the nnU-Net architecture is proposed. It was designed specifically for the precise segmentation of 11 abdominal organs in CT images. &#65279;The proposed network has made the following improvements compared to nnU-Net. Firstly, Space-to-depth (SPD) modules and parameter-shared axial attention (PSAA) feature extraction blocks were introduced, enhancing the capability of 3D image feature extraction. Secondly, a multi-scale image fusion approach was employed to capture detailed information and spatial features, improving the capability of extracting subtle features and edge features. Lastly, a parameter-sharing method was introduced to reduce the model's computational cost and training speed. &#65279;The proposed network achieves an average Dice coefficient of 0.93342 for the segmentation task involving 11 organs. Experimental results indicate the notable superiority of Swin-PSAxialNet over previous mainstream segmentation methods. The method shows excellent accuracy and low computational costs in segmenting major abdominal organs.

The present protocol describes an efficient multi-organ segmentation method called Swin-PSAxialNet, which has achieved excellent accuracy compared to previous segmentation methods. The key steps of this procedure include dataset collection, environment configuration, data preprocessing, model training and comparison, and ablation experiments.

Abdominal multi-organ segmentation is one of the most important topics in the field of medical image analysis, and it plays an important role in ...

Engineering

A Personalized 3D-Printed Model for Preoperative Evaluation in Thyroid Surgery

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor location and its relation with the capsule, trachea, esophagus, nerves, and blood vessels before operation. This paper introduces an innovative 3D-printed model establishment method based on computerized tomography (CT) DICOM images. We established a personalized 3D-printed model of the cervical thyroid surgery field for each patient who needed thyroid surgery to help clinicians evaluate the key points and difficulties of the surgery and select the operation methods of key parts as a basis. The results showed that this model is conducive to preoperative discussion and the formulation of operation strategies. In particular, as a result of the clear display of the recurrent laryngeal nerve and parathyroid gland locations in the thyroid operation field, injury to them can be avoided during surgery, the difficulty of thyroid surgery reduced, and the incidence of postoperative hypoparathyroidism and complications related to recurrent laryngeal nerve injury reduced too. Moreover, this 3D-printed model is intuitive and aids communication for the signing of informed consent by patients before surgery.

Here, a new method of establishing a personalized 3D-printed model for preoperative evaluation of thyroid surgery is proposed. It is conducive to preoperative discussion, reducing the difficulty of thyroid surgery.

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor ...

3D Reconstruction for the Diagnosis and Treatment of Pulmonary Nodules

A 3D Digital Model for the Diagnosis and Treatment of Pulmonary Nodules

The three-dimensional (3D) reconstruction of pulmonary nodules using medical images has introduced new technical approaches for diagnosing and treating pulmonary nodules, and these approaches are progressively being acknowledged and adopted by physicians and patients. Nonetheless, constructing a relatively universal 3D digital model of pulmonary nodules for diagnosis and treatment is challenging due to device differences, shooting times, and nodule types. The objective of this study is to propose a new 3D digital model of pulmonary nodules that serves as a bridge between physicians and patients and is also a cutting-edge tool for pre-diagnosis and prognostic evaluation. Many AI-driven pulmonary nodule detection and recognition methods employ deep learning techniques to capture the radiological features of pulmonary nodules, and these methods can achieve a good area under-the-curve (AUC) performance. However, false positives and false negatives remain a challenge for radiologists and clinicians. The interpretation and expression of features from the perspective of pulmonary nodule classification and examination are still unsatisfactory. In this study, a method of continuous 3D reconstruction of the whole lung in horizontal and coronal positions is proposed by combining existing medical image processing technologies. Compared with other applicable methods, this method allows users to rapidly locate pulmonary nodules and identify their fundamental properties while also observing pulmonary nodules from multiple perspectives, thereby providing a more effective clinical tool for diagnosing and treating pulmonary nodules.

Author Spotlight: A 3D Digital Model for the Diagnosis and Treatment of Pulmonary Nodules  

The objective of this study is to develop a novel 3D digital model of pulmonary nodules that serves as a communication bridge between physicians and patients and is also a cutting-edge tool for pre-diagnosis and prognostic evaluation.

The three-dimensional (3D) reconstruction of pulmonary nodules using medical images has introduced new technical approaches for diagnosing and treating ...

Computer-Aided Three-Dimensional Visualization in the Treatment of Locally Advanced Thyroid Cancer

The diagnosis and treatment of locally advanced thyroid carcinoma are challenging. The challenge lies in the evaluation of the tumor scope and the formulation of an individualized treatment plan. Three-dimensional (3D) visualization has a wide range of applications in the field of medicine, although there are limited applications in thyroid cancer. We previously applied 3D visualization for the diagnosis and treatment of thyroid cancer. Through data collection, 3D modeling, and preoperative evaluation, we can obtain 3D information regarding the tumor outline, determine the extent of tumor invasion, and conduct adequate preoperative preparation and surgical risk assessment. This study aimed to demonstrate the feasibility of 3D visualization in locally advanced thyroid cancer. Computer-aided 3D visualization can be an effective method for accurate preoperative evaluation, the development of surgical methods, shortening the surgical time, and reducing the surgical risks. Furthermore, it can contribute to medical education and doctor-patient communication. We believe that the application of 3D visualization technology can improve outcomes and quality of life in patients with locally advanced thyroid cancer.

In diagnosing and treating locally advanced thyroid cancer, the application of computer-aided three-dimensional reconstruction can provide additional information regarding the tumor scope and anatomic characteristics, thereby assisting in risk assessment and surgical planning.

The diagnosis and treatment of locally advanced thyroid carcinoma are challenging. The challenge lies in the evaluation of the tumor scope and the ...

Enhancing Pulmonary Nodule Diagnosis Using Whole-Lung 3D Reconstruction

Three-Dimensional Reconstruction for the Whole Lung with Early Multiple Pulmonary Nodules

For patients with early multiple pulmonary nodules, it is essential, from a diagnostic perspective, to determine the spatial distribution, size, location, and relationship with surrounding lung tissue of these nodules throughout the entire lung. This is crucial for identifying the primary lesion and developing more scientifically grounded treatment plans for doctors. However, pattern recognition methods based on machine vision are susceptible to false positives and false negatives and, therefore, cannot fully meet clinical demands in this regard. Visualization methods based on maximum intensity projection (MIP) can better illustrate local and individual pulmonary nodules but lack a macroscopic and holistic description of the distribution and spatial features of multiple pulmonary nodules.
Therefore, this study proposes a whole-lung 3D reconstruction method. It extracts the 3D contour of the lung using medical image processing technology against the background of the entire lung and performs 3D reconstruction of the lung, pulmonary artery, and multiple pulmonary nodules in 3D space. This method can comprehensively depict the spatial distribution and radiological features of multiple nodules throughout the entire lung, providing a simple and convenient means of evaluating the diagnosis and prognosis of multiple pulmonary nodules.

Author Spotlight: Advancing 3D Modeling for Enhanced Diagnosis and Treatment of Pulmonary Nodules in Early-Stage Lung Cancer 

This study introduces a three-dimensional (3D) reconstruction method for the entire lung in patients with early multiple pulmonary nodules. It offers a comprehensive visualization of nodule distribution and their interplay with lung tissue, simplifying the assessment of diagnosis and prognosis for these patients.

For patients with early multiple pulmonary nodules, it is essential, from a diagnostic perspective, to determine the spatial distribution, size, location, ...

Synchronous Visualization of Thyroid Structural and Functional Information

Synchronous Triplanar Reconstruction Integrated with Color Doppler Mapping for Precise and Rapid Localization of Thyroid Lesions

This paper proposes a novel thyroid examination technique based on five-dimensional (5D) synchronous reconstruction of ultrasound data. The raw temporal sequences are reconstructed into 3D volumetric data reflecting anatomical structure. Triplanar visualization from three orthogonal planes is realized to provide a systematic inspection of the entire gland. Color Doppler imaging is integrated into each triplanar slice to map vascularity changes. This multi-modal fusion enables synchronous display of structural, functional, and blood flow information in the reconstructed 5D space. Compared to conventional scanning, this technique offers the benefits of flexible offline diagnosis, reduced dependency on scanning, enhanced intuitive interpretation, and comprehensive multi-aspect evaluation. By minimizing oversight errors, it could improve diagnostic accuracy, especially for novice practitioners. The proposed 5D fusion method allows rapid and precise localization of lesions for early detection. Future work will explore integration with biochemical markers to further improve diagnostic precision. The technique has considerable clinical value for advancing thyroid examination.

Author Spotlight: Integrating Ultrasound Imaging with Biochemical Markers for Thyroid Disease Diagnosis

Here we present a 5D ultrasound technique combining multi-planar 3D reconstruction and color Doppler fusion, which enables synchronous visualization of thyroid structural and functional information. By minimizing blind spots, this method allows rapid, precise localization of lesions to improve diagnostic accuracy, especially benefiting novice practitioners.

This paper proposes a novel thyroid examination technique based on five-dimensional (5D) synchronous reconstruction of ultrasound data. The raw temporal ...