TeachMe: a web-based teaching system for annotating abdominal lymph nodes


Colorectal cancer (CRC) is one of the most common causes of cancer death in China and many other countries around the world1,2,3. To prevent recurrence of CRC and obtain the best prognosis, detection of abdominal lymph nodes (LN) is of great importance.4,5,6. Generally, contrast-enhanced computed tomography (CE-CT) of the entire abdomen is a crucial way to identify whether abdominal LNs are metastasizing and to diagnose CRC. However, there are many limitations when interpreting CE-CTs manually. First, misdiagnoses and missed diagnoses often occur, which significantly delay the golden time for treatment. Also, it will require considerable labor cost if all the interpretations have to be done by professional experienced doctors. In addition, qualified doctors are limited in some rural areas, which is why many patients find it difficult to consult a doctor in the early stage of the disease. To solve the problem presented above, medical schools should strengthen staff training to improve the professional quality of young doctors. With regard to medical education, radiology education is an essential part of undergraduate medical education programs. These are the essential skills for diagnosing CRC and other illnesses. Worse still, radiology training for junior doctors is also currently limited. Since the skills of interpreting medical images are largely based on experience, young doctors need a lot of training to accumulate knowledge and reach an expert level. But they are often short on time as they have to balance clinical practice with learning medical theory and other responsibilities. Many of them therefore cannot train enough at school. Yet their learning outcomes are sometimes not assessed by their tutors in time. Therefore, to promote the teaching of radiology in medical school, it is necessary to improve the traditional classroom teaching mode and create a new teaching method. In addition, to solve the limitations of the basic diagnosis of CRC, the creation of a support to help doctors interpret medical images is also requested. With the rapid development of artificial intelligence and internet technology, a computer-aided system can provide an outlet for these problems. On the one hand, previous studies7,8,9 have proven that deep neural networks have had enormous success in solving medical problems in recent years. This suggests that testing a precision neural network model using a large dataset with precise labels is an effective way to reduce the work intensity of physicians and improve efficiency. . On the other hand, previous studiesten demonstrated the effectiveness of radiology-themed online learning platforms for senior medical students. These online learning platforms offer a more flexible learning style for young doctors. Motivated by the potential, we intend to develop a computer-aided system to collect an expert data set of abdominal lymph nodes supporting the search for automated lymph node detection models and helping young physicians to practice their interpretation skills. For this purpose, we carry out the following analysis and investigation.

Figure 1

TeachMe’s browser/server architecture.

Figure 2
Figure 2

The three functional modules and their TeachMe workflow. Module 1 is the construction of the expert database for abdominal lymph nodes, which involves primary annotator users and higher annotator users. Module 2 is the expert database based on MongoDB. Module 3 is the feedback mechanism, which is the process of teaching in TeachMe by comparing the “answers” ​​of junior doctor users with knowledge from the expert database.

picture 3
picture 3

The overview of the three user interfaces. TeachMe provides different features for different types of users. Primary annotating users and senior annotating users participate in the construction of the expert database. Lead annotator users are responsible for giving lead annotations. Senior annotating users perform the final review and confirmation. The annotations audited by senior annotator users are the expert knowledge stored in the expert database. Junior physician users are the service object of the constructed expert database. TeachMe may correct their submitted “answers” through TeachMe’s feedback mechanism.

To begin with, what matters first for building an abdominal lymph node dataset is a convenient annotation tool for physicians to cooperate with the collection process. Nevertheless, the existing annotation tools are not able to satisfy the need. Generally, the annotation tools used in hospitals are the custom system. Most of them are desktop applications, which must be part of the hospital network to have direct access to the picture archiving and communication systems (PACS). The networking requirement reduces efficiency and limits flexibility. Although there are other informal annotation tools, such as LabelMe11.12ITK-SNAP13 and 3D-Slicer14, they are still not appropriate for annotating abdominal lymph nodes. Label Me11.12 is a web-based annotation tool that provides polygon labeling and other functions. But it aims to annotate natural images and is unsuitable for medical images. ITK-SNAP13 and 3D-Slicer14 can browse and annotate DICOM (Digital Imaging and Communications in Medicine) images. They also offer many functionalities, such as visualization of medical images, manual segmentation and semi-automatic segmentation. However, they do not provide a unified standard of annotations and cannot centrally store annotation information, which is inconvenient for multiple annotators to collaborate with each other. On the other hand, we investigated the existing online learning platforms of radiology education and also did not find any interactive platform specializing in interpretive skills training for abdominal CE-CT. . When it comes to a great teaching platform, one of the most critical elements is the platform content provided to students. Since our goal is to build a platform for young doctors to practice specialized skills in detecting abdominal lymph nodes through CE-CT, the content must be very reliable and advanced. In addition, to achieve a better educational effect, the platform should be interactive. However, the following six online learning platforms are not perfect. First, MyPACS.net15 and COMPARE Radiology16 are web authoring tools for educators to create educational content using medical images. But both focus only on providing passive theoretical information, not providing features for students to practice their skills. Although ELERA17 and Radiology ExamWeb18 can provide the functionality to create tests or give learners guided instructions and feedback, they always focus on basic concepts, not expert diagnostic skills. Similarly, RadStax19 and RadEd20 can create labels on regions of interest in the medical image and learn to detect them. But the first only deals with introducing the information related to these labels, and the educational content of the second is relatively simple. Overall, these existing e-learning platforms are essentially online classrooms, but not clinical practice tutorials. These are not the appropriate platforms to train abdominal node detection skills, which are complicated and tricky.

Based on the points discussed above, this article introduces TeachMe. TeachMe is not only an annotation tool to annotate abdominal lymph nodes, but also a teaching platform providing instruction on practicing abdominal medical image interpretation skills. Junior physicians can conduct specific training in interpretive skills in this system. They can advance their interpreting skills by repeatedly practicing through this system with error information provided. Thus, the objectives of TeachMe can be summarized as follows:

  1. (1)

    Provide annotation functionality to make accurate annotations on abdominal CE-CTs;

  2. (2)

    Archive precise locations and labels of annotations;

  3. (3)

    Interact with young doctors and help them improve their interpretation skills to detect abdominal lymph nodes.

To achieve these goals, TeachMe is developed as a web-based teaching system for abdominal lymph node annotation with three main modules: an expert database building process for abdominal lymph nodes, a expert data for abdominal lymph nodes and a feedback mechanism providing the corrections for young physician users. To build the expert database, TeachMe adopts a three-level annotation review workflow to collect the annotated data with the gold standard and ensure the accuracy of the expert database. The abdominal lymph node annotations stored in the database and the stored information can be easily exported as a training dataset. TeachMe can teach young doctors through the feedback mechanism using the stored expert content, which is the most significant difference between TeachMe and other platforms and stands out as a new teaching system. To validate the functionality of TeachMe, several gastroenterologists from our medical team used the system, from which a dataset of abdominal CE-CTs with precise lymph node annotations called PLNDataset was created. With the dataset, an efficient neural network model to detect abdominal lymph nodes was proposed in21. In addition, thanks to TeachMe’s feedback mechanism, the young doctors of our medical team have improved their interpreting skills through three cycles of practice. In this work, the proposed TeachMe provides functionality to collect an expert dataset of abdominal lymph nodes with the gold standard and mentor junior doctors based on the expert knowledge built. The development of TeachMe is the critical first step in advancing research in the automatic interpretation of CE-CTs and solving the limitations of clinical work. More importantly, it offers a new way to advance the radiology education of young physicians.


Comments are closed.