Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images

Jayaram K. Udupa, Dewey Odhner, Liming Zhao, Yubing Tong, Monica M S Matsumoto, Krzysztof C. Ciesielski, Alexandre X. Falcao, Pavithra Vaideeswaran, Victoria Ciesielski, Babak Saboury, Syedmehrdad Mohammadianrasanani, Sanghun Sin, Raanan Arens, Drew A. Torigian

Research output: Contribution to journalArticle

49 Citations (Scopus)

Abstract

To make Quantitative Radiology (QR) a reality in radiological practice, computerized body-wide Automatic Anatomy Recognition (AAR) becomes essential. With the goal of building a general AAR system that is not tied to any specific organ system, body region, or image modality, this paper presents an AAR methodology for localizing and delineating all major organs in different body regions based on fuzzy modeling ideas and a tight integration of fuzzy models with an Iterative Relative Fuzzy Connectedness (IRFC) delineation algorithm. The methodology consists of five main steps: (a) gathering image data for both building models and testing the AAR algorithms from patient image sets existing in our health system; (b) formulating precise definitions of each body region and organ and delineating them following these definitions; (c) building hierarchical fuzzy anatomy models of organs for each body region; (d) recognizing and locating organs in given images by employing the hierarchical models; and (e) delineating the organs following the hierarchy. In Step (c), we explicitly encode object size and positional relationships into the hierarchy and subsequently exploit this information in object recognition in Step (d) and delineation in Step (e). Modality-independent and dependent aspects are carefully separated in model encoding. At the model building stage, a learning process is carried out for rehearsing an optimal threshold-based object recognition method. The recognition process in Step (d) starts from large, well-defined objects and proceeds down the hierarchy in a global to local manner. A fuzzy model-based version of the IRFC algorithm is created by naturally integrating the fuzzy model constraints into the delineation algorithm. The AAR system is tested on three body regions - thorax (on CT), abdomen (on CT and MRI), and neck (on MRI and CT) - involving a total of over 35 organs and 130 data sets (the total used for model building and testing). The training and testing data sets are divided into equal size in all cases except for the neck. Overall the AAR method achieves a mean accuracy of about 2 voxels in localizing non-sparse blob-like objects and most sparse tubular objects. The delineation accuracy in terms of mean false positive and negative volume fractions is 2% and 8%, respectively, for non-sparse objects, and 5% and 15%, respectively, for sparse objects. The two object groups achieve mean boundary distance relative to ground truth of 0.9 and 1.5 voxels, respectively. Some sparse objects - venous system (in the thorax on CT), inferior vena cava (in the abdomen on CT), and mandible and naso-pharynx (in neck on MRI, but not on CT) - pose challenges at all levels, leading to poor recognition and/or delineation results. The AAR method fares quite favorably when compared with methods from the recent literature for liver, kidneys, and spleen on CT images. We conclude that separation of modality-independent from dependent aspects, organization of objects in a hierarchy, encoding of object relationship information explicitly into the hierarchy, optimal threshold-based recognition learning, and fuzzy model-based IRFC are effective concepts which allowed us to demonstrate the feasibility of a general AAR system that works in different body regions on a variety of organs and on different modalities.

Original languageEnglish (US)
Pages (from-to)752-771
Number of pages20
JournalMedical Image Analysis
Volume18
Issue number5
DOIs
StatePublished - 2014

Fingerprint

Anatomy
Body Regions
Magnetic resonance imaging
Object recognition
Neck
Testing
Recognition (Psychology)
Abdomen
Thorax
Radiology
Learning
Liver
Volume fraction
Inferior Vena Cava
Pharynx
Mandible
Health
Spleen
Kidney

Keywords

  • Anatomy modeling
  • Fuzzy connectedness
  • Fuzzy models
  • Image segmentation
  • Object recognition

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Radiology Nuclear Medicine and imaging
  • Health Informatics
  • Radiological and Ultrasound Technology
  • Medicine(all)

Cite this

Udupa, J. K., Odhner, D., Zhao, L., Tong, Y., Matsumoto, M. M. S., Ciesielski, K. C., ... Torigian, D. A. (2014). Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images. Medical Image Analysis, 18(5), 752-771. https://doi.org/10.1016/j.media.2014.04.003

Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images. / Udupa, Jayaram K.; Odhner, Dewey; Zhao, Liming; Tong, Yubing; Matsumoto, Monica M S; Ciesielski, Krzysztof C.; Falcao, Alexandre X.; Vaideeswaran, Pavithra; Ciesielski, Victoria; Saboury, Babak; Mohammadianrasanani, Syedmehrdad; Sin, Sanghun; Arens, Raanan; Torigian, Drew A.

In: Medical Image Analysis, Vol. 18, No. 5, 2014, p. 752-771.

Research output: Contribution to journalArticle

Udupa, JK, Odhner, D, Zhao, L, Tong, Y, Matsumoto, MMS, Ciesielski, KC, Falcao, AX, Vaideeswaran, P, Ciesielski, V, Saboury, B, Mohammadianrasanani, S, Sin, S, Arens, R & Torigian, DA 2014, 'Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images', Medical Image Analysis, vol. 18, no. 5, pp. 752-771. https://doi.org/10.1016/j.media.2014.04.003
Udupa, Jayaram K. ; Odhner, Dewey ; Zhao, Liming ; Tong, Yubing ; Matsumoto, Monica M S ; Ciesielski, Krzysztof C. ; Falcao, Alexandre X. ; Vaideeswaran, Pavithra ; Ciesielski, Victoria ; Saboury, Babak ; Mohammadianrasanani, Syedmehrdad ; Sin, Sanghun ; Arens, Raanan ; Torigian, Drew A. / Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images. In: Medical Image Analysis. 2014 ; Vol. 18, No. 5. pp. 752-771.
@article{34f77a895c7a44c897578624749c54f9,
title = "Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images",
abstract = "To make Quantitative Radiology (QR) a reality in radiological practice, computerized body-wide Automatic Anatomy Recognition (AAR) becomes essential. With the goal of building a general AAR system that is not tied to any specific organ system, body region, or image modality, this paper presents an AAR methodology for localizing and delineating all major organs in different body regions based on fuzzy modeling ideas and a tight integration of fuzzy models with an Iterative Relative Fuzzy Connectedness (IRFC) delineation algorithm. The methodology consists of five main steps: (a) gathering image data for both building models and testing the AAR algorithms from patient image sets existing in our health system; (b) formulating precise definitions of each body region and organ and delineating them following these definitions; (c) building hierarchical fuzzy anatomy models of organs for each body region; (d) recognizing and locating organs in given images by employing the hierarchical models; and (e) delineating the organs following the hierarchy. In Step (c), we explicitly encode object size and positional relationships into the hierarchy and subsequently exploit this information in object recognition in Step (d) and delineation in Step (e). Modality-independent and dependent aspects are carefully separated in model encoding. At the model building stage, a learning process is carried out for rehearsing an optimal threshold-based object recognition method. The recognition process in Step (d) starts from large, well-defined objects and proceeds down the hierarchy in a global to local manner. A fuzzy model-based version of the IRFC algorithm is created by naturally integrating the fuzzy model constraints into the delineation algorithm. The AAR system is tested on three body regions - thorax (on CT), abdomen (on CT and MRI), and neck (on MRI and CT) - involving a total of over 35 organs and 130 data sets (the total used for model building and testing). The training and testing data sets are divided into equal size in all cases except for the neck. Overall the AAR method achieves a mean accuracy of about 2 voxels in localizing non-sparse blob-like objects and most sparse tubular objects. The delineation accuracy in terms of mean false positive and negative volume fractions is 2{\%} and 8{\%}, respectively, for non-sparse objects, and 5{\%} and 15{\%}, respectively, for sparse objects. The two object groups achieve mean boundary distance relative to ground truth of 0.9 and 1.5 voxels, respectively. Some sparse objects - venous system (in the thorax on CT), inferior vena cava (in the abdomen on CT), and mandible and naso-pharynx (in neck on MRI, but not on CT) - pose challenges at all levels, leading to poor recognition and/or delineation results. The AAR method fares quite favorably when compared with methods from the recent literature for liver, kidneys, and spleen on CT images. We conclude that separation of modality-independent from dependent aspects, organization of objects in a hierarchy, encoding of object relationship information explicitly into the hierarchy, optimal threshold-based recognition learning, and fuzzy model-based IRFC are effective concepts which allowed us to demonstrate the feasibility of a general AAR system that works in different body regions on a variety of organs and on different modalities.",
keywords = "Anatomy modeling, Fuzzy connectedness, Fuzzy models, Image segmentation, Object recognition",
author = "Udupa, {Jayaram K.} and Dewey Odhner and Liming Zhao and Yubing Tong and Matsumoto, {Monica M S} and Ciesielski, {Krzysztof C.} and Falcao, {Alexandre X.} and Pavithra Vaideeswaran and Victoria Ciesielski and Babak Saboury and Syedmehrdad Mohammadianrasanani and Sanghun Sin and Raanan Arens and Torigian, {Drew A.}",
year = "2014",
doi = "10.1016/j.media.2014.04.003",
language = "English (US)",
volume = "18",
pages = "752--771",
journal = "Medical Image Analysis",
issn = "1361-8415",
publisher = "Elsevier",
number = "5",

}

TY - JOUR

T1 - Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images

AU - Udupa, Jayaram K.

AU - Odhner, Dewey

AU - Zhao, Liming

AU - Tong, Yubing

AU - Matsumoto, Monica M S

AU - Ciesielski, Krzysztof C.

AU - Falcao, Alexandre X.

AU - Vaideeswaran, Pavithra

AU - Ciesielski, Victoria

AU - Saboury, Babak

AU - Mohammadianrasanani, Syedmehrdad

AU - Sin, Sanghun

AU - Arens, Raanan

AU - Torigian, Drew A.

PY - 2014

Y1 - 2014

N2 - To make Quantitative Radiology (QR) a reality in radiological practice, computerized body-wide Automatic Anatomy Recognition (AAR) becomes essential. With the goal of building a general AAR system that is not tied to any specific organ system, body region, or image modality, this paper presents an AAR methodology for localizing and delineating all major organs in different body regions based on fuzzy modeling ideas and a tight integration of fuzzy models with an Iterative Relative Fuzzy Connectedness (IRFC) delineation algorithm. The methodology consists of five main steps: (a) gathering image data for both building models and testing the AAR algorithms from patient image sets existing in our health system; (b) formulating precise definitions of each body region and organ and delineating them following these definitions; (c) building hierarchical fuzzy anatomy models of organs for each body region; (d) recognizing and locating organs in given images by employing the hierarchical models; and (e) delineating the organs following the hierarchy. In Step (c), we explicitly encode object size and positional relationships into the hierarchy and subsequently exploit this information in object recognition in Step (d) and delineation in Step (e). Modality-independent and dependent aspects are carefully separated in model encoding. At the model building stage, a learning process is carried out for rehearsing an optimal threshold-based object recognition method. The recognition process in Step (d) starts from large, well-defined objects and proceeds down the hierarchy in a global to local manner. A fuzzy model-based version of the IRFC algorithm is created by naturally integrating the fuzzy model constraints into the delineation algorithm. The AAR system is tested on three body regions - thorax (on CT), abdomen (on CT and MRI), and neck (on MRI and CT) - involving a total of over 35 organs and 130 data sets (the total used for model building and testing). The training and testing data sets are divided into equal size in all cases except for the neck. Overall the AAR method achieves a mean accuracy of about 2 voxels in localizing non-sparse blob-like objects and most sparse tubular objects. The delineation accuracy in terms of mean false positive and negative volume fractions is 2% and 8%, respectively, for non-sparse objects, and 5% and 15%, respectively, for sparse objects. The two object groups achieve mean boundary distance relative to ground truth of 0.9 and 1.5 voxels, respectively. Some sparse objects - venous system (in the thorax on CT), inferior vena cava (in the abdomen on CT), and mandible and naso-pharynx (in neck on MRI, but not on CT) - pose challenges at all levels, leading to poor recognition and/or delineation results. The AAR method fares quite favorably when compared with methods from the recent literature for liver, kidneys, and spleen on CT images. We conclude that separation of modality-independent from dependent aspects, organization of objects in a hierarchy, encoding of object relationship information explicitly into the hierarchy, optimal threshold-based recognition learning, and fuzzy model-based IRFC are effective concepts which allowed us to demonstrate the feasibility of a general AAR system that works in different body regions on a variety of organs and on different modalities.

AB - To make Quantitative Radiology (QR) a reality in radiological practice, computerized body-wide Automatic Anatomy Recognition (AAR) becomes essential. With the goal of building a general AAR system that is not tied to any specific organ system, body region, or image modality, this paper presents an AAR methodology for localizing and delineating all major organs in different body regions based on fuzzy modeling ideas and a tight integration of fuzzy models with an Iterative Relative Fuzzy Connectedness (IRFC) delineation algorithm. The methodology consists of five main steps: (a) gathering image data for both building models and testing the AAR algorithms from patient image sets existing in our health system; (b) formulating precise definitions of each body region and organ and delineating them following these definitions; (c) building hierarchical fuzzy anatomy models of organs for each body region; (d) recognizing and locating organs in given images by employing the hierarchical models; and (e) delineating the organs following the hierarchy. In Step (c), we explicitly encode object size and positional relationships into the hierarchy and subsequently exploit this information in object recognition in Step (d) and delineation in Step (e). Modality-independent and dependent aspects are carefully separated in model encoding. At the model building stage, a learning process is carried out for rehearsing an optimal threshold-based object recognition method. The recognition process in Step (d) starts from large, well-defined objects and proceeds down the hierarchy in a global to local manner. A fuzzy model-based version of the IRFC algorithm is created by naturally integrating the fuzzy model constraints into the delineation algorithm. The AAR system is tested on three body regions - thorax (on CT), abdomen (on CT and MRI), and neck (on MRI and CT) - involving a total of over 35 organs and 130 data sets (the total used for model building and testing). The training and testing data sets are divided into equal size in all cases except for the neck. Overall the AAR method achieves a mean accuracy of about 2 voxels in localizing non-sparse blob-like objects and most sparse tubular objects. The delineation accuracy in terms of mean false positive and negative volume fractions is 2% and 8%, respectively, for non-sparse objects, and 5% and 15%, respectively, for sparse objects. The two object groups achieve mean boundary distance relative to ground truth of 0.9 and 1.5 voxels, respectively. Some sparse objects - venous system (in the thorax on CT), inferior vena cava (in the abdomen on CT), and mandible and naso-pharynx (in neck on MRI, but not on CT) - pose challenges at all levels, leading to poor recognition and/or delineation results. The AAR method fares quite favorably when compared with methods from the recent literature for liver, kidneys, and spleen on CT images. We conclude that separation of modality-independent from dependent aspects, organization of objects in a hierarchy, encoding of object relationship information explicitly into the hierarchy, optimal threshold-based recognition learning, and fuzzy model-based IRFC are effective concepts which allowed us to demonstrate the feasibility of a general AAR system that works in different body regions on a variety of organs and on different modalities.

KW - Anatomy modeling

KW - Fuzzy connectedness

KW - Fuzzy models

KW - Image segmentation

KW - Object recognition

UR - http://www.scopus.com/inward/record.url?scp=84900401170&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84900401170&partnerID=8YFLogxK

U2 - 10.1016/j.media.2014.04.003

DO - 10.1016/j.media.2014.04.003

M3 - Article

VL - 18

SP - 752

EP - 771

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

IS - 5

ER -