validation of aida digital microscopy system for automated...

Validation of AiDA Digital Microscopy System for Automated Use with HEAD AIBy L Auguste & S Malav (NYU Tandon Future Labs) and A Barrios Pérez, T Félix Cañedo, C Maya-Rendon & G Velásquez Rodríguez (II-UNAM)

10-Dec-2018 BMGF OPP1173370, Project 13D (Duke PO 4550610923) Auguste

Prepared for: Duke Internal Subaward #: PO4550610923Name: Dr. Brian StonerOrganization: Duke University Center for WaSH-AIDEmail: [email protected]

Primary Author Contact Information: Date Submitted: Dec. 10, 2018Name: Lou AugustOrganization: AlexapathEmail: [email protected]

BMGF OPP: OPP1173370 (Sanitation Technology Cluster) Project #: 13D________________________________________________________________________________________________

This work is, in part [or whole], supported by a grant, OPP1173370, from the Bill & Melinda Gates Foundation through Duke University’s Center for WaSH-AID. All opinions, findings, and conclusions or recommendations expressed in this work are those of the author(s) and do not necessarily reflect the views of the Foundation, Duke, or the Center.

Statement of ConfidentialityThis report and supporting materials contain confidential and proprietary information. These materials may be printed or photocopied for use in evaluating the project, but are not to be shared with other parties without permission.

2

Abstract

While guidelines exist to limit the density of Helminth ova in various environmental matrices, there are not enough qualified technicians for the task of visual identification using conventional microscopy across all regions with need. Thanks to the support and funding of the Bill & Melinda Gates Foundation, the Treatment and Reuse Group of the Institute of Engineering (II-UNAM) developed the HEAD AI to automatically identify and quantify different species of Helminth using digital images. After three unsuccessful digital imaging prototypes, the team from II-UNAM was introduced to the team from Alexapath (NYU Tandon Future Labs) experts in the field of digital microscopy. The team from Alexapath, with funding from the Bill & Melinda Gates Foundation and Duke University’s Center for WaSH-AID, donated an AiDA microscope (Artificial Intelligence Diagnostic Assistant) for testing in the lab of Dr. Tony Barrios at II-UNAM. AiDA was used to acquiremobile Whole Slide Images (mWSIs) of various concentrations of Helminth ova specimens contained in Sedgewick rafters. The images were then analyzed by the HEAD preclassification program in order to verify that illuminance, contrast and blurriness met the established threshold values. The testing validated that the AiDA system can produce digital images that meet the benchmarks needed for successful analysis by HEAD. Leading to an opportunity to being field testing a system made up of theAiDA microscope with the HEAD Ai as a solution for automatic analysis of water specimens to determine content of Helminth ova.


3

Introduction

Globally there are over 1.5 billion people affected by Helminths (parasitic worms). Helminthiasis (parasitic worm infection) occurs mostly in low income communities where conditions for transmission are rife due to poor sanitation, use of contaminated water for irrigation, and improper disposal of excreta or waste sludge.1 Children infected with Ascariasis may suffer from intestinal blockage resulting in impaired growth and weakness significantly contributing to the poverty of these communities. Preventive chemotherapy (de-worming medications) is very important but alone cannot break the cycle of infection and reinfection. Populations living in contaminated environments continue to be at risk of infection.2 In World Health Assembly resolution WHA66.12, and the resulting ‘Roadmap for Implementation’ over 200 member nations agreed to and adopted provisions that adequate water, sanitation, and hygiene services are fundamental to break the cycle of infection and reinfection.3

While criteria and regulations to limit the density of Helminth ova in environmental matrices exist, successful application of these criteria relies on capacity to correctly identify the ovum of multiple Helminth species by direct observation through a microscope. This step is crucial and a main source of error as highly qualified technicians are needed for analysis. As a result, compliance with such regulations frequently does not take place and transmission of Helminth parasites continues.

In an effort to automate Helminth detection, the Treatment and Reuse Group of the Institute of Engineering UNAM (II-UNAM), funded by the Bill & Melinda Gates Foundation, developed a software to identify and quantify pathogenic Helminth eggs using conventional microscopy images. Using specimens taken from wastewater, soil, sludge, biosolids, and excreta samples. The Helminth Egg Automatic Detector (HEAD) is an image processing software developed to automatically identify and quantify different species of Helminth eggs of medical importance. It aims to aid in analysis and enumeration of Helminth ova without the need for highly trainedmicrobiologists. By using different image processing algorithms, thesoftware is capable of identifying and quantifying 11 species of Helmintheggs with a mean sensibility (capacity to correctly classify and identifythe different species of Helminth eggs) and specificity (capacity todiscriminate between species of Helminth eggs and other objects) of 97%and 95%, respectively.4

For the HEAD AI to successfully analyze a specimen, it requiresdigitized images that meet the following quality thresholds: illumination,contrast and blurriness. Before Alexapath and UNAM-II launched thisexperiment three devices were built as optical system prototypes. Aftertesting all versions, the images obtained did not meet the requiredthresholds and were thus incompatible with the HEAD software. Manualphotography of specimens is a slow and time consuming process whichwas similarly unsatisfactory. As a solution, a collaboration between thelaboratory of Dr. Jimenez and Dr. Barrios at II- UNAM and Mr. Augusteand Mr. Malav of Alexapath (NYU Tandon Future Labs) was established.

Funded by the Bill & Melinda Gates Foundation and DukeUniversity’s Center for WaSH-AID, Alexapath manufactured a customversion of the AiDA-1 Microscope. AiDA is the Artificial Intelligence


Illustration 1: AiDA unit shipped to II-UNAM.

4

Diagnostic Assistant; a microscope system for robotic imaging, viewing and computer assisted diagnosis / analysis of various types of microscopic specimens. Through the collaboration, Alexapath donated the custom AiDA-1 microscope system, complete with a redesigned robotic slide holder, trinocular camera and the Alexapath mWSI software (mobile Whole Slide Imaging) to the team at II-UNAM for a proof of concept validation study. The primary hypothesis is the AiDA-1 microscope is capable of acquiring digital images of microscopic specimens that meet the quality thresholds of the HEAD software. A secondary hypothesis tested during this experiment is that the AiDA-1 microscope can be repaired locally.

The AiDA system is expected to acquire a sequence of images from the specimen sample using a CMOS camera, while the robotic slide holder automatically manipulates the movement of the specimen slide to acquire all regions, and the z-axis focus control allows focus on multiple planes. The task of imaging the slide is carried out without manual manipulation by any technician, eliminating possible missed data due to technician fatigue or failure.

Methods

The team from NYU Tandon Future Labs consisted of Lou Auguste, and Shishir Malav, with translation support from Kevin Munar, while from the lab of Dr. Blanca Elena Jimenez Cisneros and Dr. Jose Antonio Barrios Perez from II-UNAM consisted of Catalina May Rendon. Gustavo Adolfo Rodrigo Velasquez Rodríguez, and Thania Eloina Félix Cañedo.

Before shipping the AiDA-1 from their office at NYU Tandon Future Labs, the Alexapath team modified the robotic slide holder to accommodate a Sedgewick rafter. II-UNAM provided a Visual Studio Project in C# with code to calculate images properties such as blurriness, illumination and contrast. Alexapath integrated this code to their mWSI viewer application.

The accessories were fitted to an Olympus CX33 microscope at the Alexapath office. Once assembled to scan the slide the user manually focuses the microscope then presses a single button in theWindows mWSI application on Microsoft Surface to acquire a mWSI. Coupled with HEAD AI software developed by UNAM, the goal is to build a system able to identify and enumerate any parasites that are present in water, sludge or soil samples.

Thanks to the generosity of the Gates Foundation and Duke University’s Center for WaSH-AID,Alexapath’s Founders Louis Auguste and Shishir Malav donated the following equipment to the research lab at II-UNAM as one AiDA system (Illustration 1):

1- Alexapath Robotic Slide Holder 2- Alexapath Trinocular Mounted HD Camera3- Alexapath Z-Axis Control 4- Alexapath mWSI Viewer Software5- Alexapath mWSI Acquisition Software6- Olympus CX33 Microscope7- Microsoft Surface Go with Windows 10


5

Before shipping the AiDA system, a thorough test conducted at the Alexapath office yielding the following results. A Sedgewick rafter slide was acquired in 3 minutes and 20 seconds at 10x objective with a 10x optical zoom. There was a total of 759 images and the average image size was 790 KB. The validation period of AiDA was scheduled for six months beginning July 1st. Our teams developed an experiment that was carried out by Dr. Barrios’ team at II-UNAM. The lab created various concentrations of different parasite species, then placed each sample on a Sedgewick rafter. The user sets the focal plane using the Z axis controller and then presses the ‘SCAN’ button on the mWSI acquisition app to begin capturing images. (Illustration 2) Once the images are acquired the user then opens the digital slide using the mWSI viewer app. To start the pre-classification AI, the user enters the AI menu in the mWSI viewer application. User is then prompted to enter parameters for preclassification or use the default parameters. (Illustration 3)

Each image is then analyzed by the preclassification program and assigned a value corresponding to the parameters of contrast, illumination and blurriness. Each of those values is then compared to a threshold value, the blurriness and illumination variables represent a maximum threshold, while the contrast variable is a minimum threshold. Any images that meet the threshold values are then passed to the HEAD AI. Images that do not meet threshold values will not be analyzed by the AI. Images that do not meet the threshold value are to be expected especially in images with no data present or oversampled images that capture image data from outside the well of the rafter, i.e. the metal border of the specimen well.


Illustration 2: One button scanning using mWSI acquisition application.

Illustration 3: Preclassificaiton settings on mWSI viewer app.

6

Results

Once the paperwork required for donation was managed, on November 20, 2018 the AiDA system was received in the laboratory of II-UNAM. Unfortunately, the microscope was damaged due to rough handling by the shippingcompany (DHL) somewhere between NYU Tandon Future Labs (Brooklyn, NY) and II-UNAM (Mexico City). Once the equipment was received at the Institute of Engineering andtaken from the hard-shell pelican case, some damage was identified to the robotic slide holder. UNAM informed Alexapath of the damage and forwarded pictures detailing the damage. Alexapath made the following observations (see Illustration 4-5) -

1- CMOS camera system was removed from thetrinocular and carelessly returned to the flight case2- Robotic slide holder was broken due to forcefulremoval without proper disassembly3- The wiring was damaged during forcefulremoval rendering the motors unable to function4- Even packing material was removed to facilitatereturning AiDA-1 to the flight case

The initial arrival of the AiDA-1 causedconcern, but our teams used this as an opportunityto explore a secondary hypothesis: the AiDAsystem can be repaired locally.

Over the course of a two hour Skypeconversation on November 20, 2018, Alexapathwas able to guide an electrician from the Instituteof Engineering on how to repair the system. Thebroken part was repaired using epoxy and thewiring was adjusted to fix faulty connections. Thecall was conducted in Spanish thanks toAlexapath’s web developer Kevin Munar, a nativespeaker.

Due to the forceful disassembly there wasa slight bend to the linear guide rail on the Y axis,


Illustration 6: 1) Ascaris lumbricoides, 2) Trichuris trichiura, 3) Hymenolepis diminuta, 4) Hymenolepis nana, 5) Taenia solium, 6) Toxocara canis, 7) Hookworm, 8) Enterobius vermicularis, 9) Ascaris suum

Illustration 4: Broken robotic slide holder

Illustration 5: AiDA-1 received by II-UNAM

7

this caused the slide holder to lift off the stage by about 3-5 millimeters. The damage was enough for us to rule out deploying this device into the field without proper inspection, however, the AiDA-1 was returned to a proper working condition by UNAM’s electrician and the proof of concept trial continued with only a slight delay.

The electrician in Mexico City was able to take a non-functional AiDA-1 System and bring it back to a working condition with only audio guidance from the Alexapath team. Proving the system

can be repaired locally. In order to acquire the first images,

the team at II-UNAM prepared multiple concentrations of different species of Helminth ova, such as Ascariasis lumbricoides, Hymenolepis nana, and Toxocara canis. Some examples of these species can be seen in Illustration 6.The first test acquired 3,366 images divided into 6 folders. The preclassification software was run on

the images contained in each folder. A full scan produced 759 images, which are divided into 23 rows and 33 columns. The testers at II-UNAM experienced interruptions from pop-up windows at first, but once advised to keep the Microsoft Surface attached to a power source while scanning, the interruptions ceased.


Illustration 8: Test 1 Image Viewer Result with oversample present on left hand side.

Illustration 7: First test prequalification results

8

The first test of the preclassification software yielded the results in Illustration 7. The software analyzed each image and gave each one a value for illumination, contrast, and blurriness.

Illumination is the observable property of light, contrast is the difference in color and light between parts of an image, and blurriness is the quality of being unclear or indistinct. Threshold valuesare set using the UI of mWSI viewer app. For this study, threshold values were set to 23 for illumination, 0.02 for contrast and 0.95 for blurriness.

In the first tests some images were rejected because of over illumination. This is why the standard deviation is so large in folders 1-5, a quick look at the data shows that the outliers were due to proximity to the metal edge of the Sedgewick rafter. This oversampling can be seen in illustration 8. Alexapath provided this oversampling to verify there was no discrepancy between the size of the well on the Sedgewick rafter used at the Alexapath office and that of the Sedgewick rafter used at UNAM. In order to verify no data would be missing, the mWSI acquisition app acquired additional data.

The results of the first test were positive. The variable of most concern to Alexapath was blurriness, which turned out to be the most consistent. The threshold value was 0.95 and the highest mean in the first test was 0.50, with a standard deviation of 0.004. The blurriness parameter was considered accurate.

The next variable measured was contrast. The contrast values were consistently above the threshold with low standard deviation. Since the threshold value for contrast is a minimum, the system was declared valid from the contrast perspective.

Finally, in the first test illumination was a slight issue. Means were high when compared to the threshold, and though the mean never crossed the threshold, standard deviation was also high making the result less than desirable. This is because the oversampled regions did not receive consistent lighting. As we can see at

the edges when the images cross from the well to the metal boundary in illustration 8, the camera begins to auto expose in order to compensate for the change in luminance. Alexapath decided to correct the oversampling error and provided an update of the acquisition software with the oversampling area removed. When the team from II-UNAM ran the updated version of the acquisition application and all variables improved. The illumination mean as well as the standard deviation were now considered acceptable.

In the end, the AiDA system performed well at the task of producing mobile whole slide imagesof the Sedgewick rafter. All parameters were considered valid and illustration 9 demonstrates the abilities of AiDA with three different Helminth highlighted.


Illustration 9: Results after update to software to remove oversampling.

9

Discussion

During the trial two hypotheses were proven. First, due to damage during shipment, the AiDA system had to be repaired by a local technician. This was most physical damage an AiDA unit had sustained to this date. The AiDA was completely non-responsive when it arrived at II-UNAM, but a local electrician at the university was able to repair the wiring and the system was able to generate data.The only damage that could be repaired was the lift resulting from the bending of the metal guide rail on the X axis. This bend caused a lift of 2-3 mm on the slide holder. Such a part could easily be

replaced as well if there were spare parts available in the field. Second, the AiDA system was able to generate whole slide images that were acceptable for analysis by HEAD. Previously three separate attempts were made by II-UNAM to build optical systems that produced whole slide images. Each previous attempt ended up with images that were unsatisfactory for use with HEAD. The AiDA succeeded wherethese other systems failed producing images of satisfactory illuminance, contrast and blurriness. Alexapath and II-UNAM now have the means to automatically detect Helminth eggs in samples taken from

wastewater, soil, sludge, biosolids, and excreta. Together AiDA and HEAD compose an exciting new system that can be used to improve sanitation and strengthen monitoring without the need for an expert in parasitology or microbiology at the point of analysis (Illustration 10).

Future

In the next phase of this project, our two teams wish to continue our research by launching a field trial. II-UNAM have already chosen two locations for the field trial. A field trial will help us identify needs, preferences and limitations to implementation; determine the implications of HEAD diagnostic based decisions; further identify stakeholders; assess feasibility of roll out at scale; account for cost-effectiveness; design the monitoring system; define a regulatory pathway and the product launch plan.

The team from II-UNAM have asked Alexapath to update the mWSI software in order to allow the sharing of data with the HEAD server as well as provide simple to understand actions to the users. This piece of software will be central to communicating with users and remotely monitoring their testing.


Illustration 10: Sample image from AiDA system with samples of Helminth eggs highlighted: Ascaris (red), Hymenolepis nana(black), and Toxocara (blue).

10

Bibliography

1- World Health Organization. “Soil-transmitted Helminth infections.” Fact sheet. WHO. Geneva. 20 Feb 2018. Web.2- Jia, Tie-Wu, et al. “Soil-Transmitted Helminth Reinfection after Drug Treatment: A Systematic Review and Meta-Analysis.” PLoS Neglected Tropical Diseases, vol. 6, no. 5, 2012, doi:10.1371/journal.pntd.0001621.3- Crompton, D.W.T et al. World Health Organization. “Accelerating Work to Over the Global Impact of Neglected Tropical Diseases. A Roadmap for Implementation.” 2012 April. Executive Summary. Web. http://apps.who.int/gb/ebwha/pdf_files/WHA66-REC1/WHA66_2013_REC1_complete.pdf4- Jiménez, B., et al. “Identification and Quantification of Pathogenic Helminth Eggs Using a Digital Image System.” Experimental Parasitology, vol. 166, 2016, pp. 164–172., doi:10.1016/j.exppara.2016.04.016.


validation of aida digital microscopy system for automated...

Documents