Opportunity

Traditional staining methods are crucial for diagnosing a wide range of diseases and usually involve immersing tissue in chemical or antibody solutions prior to microscopy. However, these methods can be time-consuming, costly, and contributors of workplace and environmental pollution. Thus, virtual tissue staining, an emerging alternative approach, provides an opportunity to reduce pollutants, costs, and hospital overcrowding. 

Furthermore, multi-modal image synthesis for virtual staining improves image generation accuracy, compared with uni-modal approaches, by allowing multiple input images to generate the output image. However, a major challenge with multi-modal methods is that the input and output images used for training are typically from different tissue sections, which can lead to image blurring, possibly rendering the output image unsuitable for clinical use. This invention addresses this limitation.

Technology

This invention is a novel multi-modal image synthesis approach for virtual tissue staining. Tissue samples are prepared for sectioning using formalin-fixed/paraffin-embedded or frozen methods, and unstained or stained with fast, cheap chemical stains. The stained sections are then imaged using standard digital light microscopes. This invention reduces the risk of image blurring by estimating what the output section would look like if stained using the input stain. Multiple estimated input images are merged using pixel-wise max procedure, and the most important information from each image is retrieved. The latent representations of each image are extracted and combined to form a synthesized image that contains full information from each image type. The network is then trained with ³100 image sets of similar tissues, each set comprising the synthesized images along with the output image from a tissue sample. Trained networks are prospectively evaluated using  ³10 additional image sets acquired and processed as for training.

Advantages

• Virtual staining is considerably faster and has less hardware requirements than existing staining methods based on chemical and antibody solutions.
• Virtual staining supports digital medicine and reduces hospital overcrowding.
• Multi-modal image synthesis produces more accurate virtual tissue staining images compared with uni-modal methods, by enabling multiple input images to generate the output image.
• In multi-modal image synthesis, tissues can be imaged unstained or stained with fast and cheap chemical stains, and imaging can be performed with brightfield and fluorescence filters.

Applications

• Virtual tissue staining will become a standard method in pathology laboratories worldwide in 10–20 years and even sooner in veterinary pathology laboratories (5–10 years).
• Virtual tissue staining will be considered an in-vitro diagnostic method in most jurisdictions; as there is presently no commercial virtual staining software available, the entire market is potentially available.
• As existing staining methods are already the gold standard, virtual staining should target applications where existing methods are time-consuming or expensive to perform (e.g., immunostaining, low-resourced hospitals).
• Multi-model image synthesis reduces the time and cost of disease diagnosis and enables antibody staining in time-sensitive clinical scenarios such as surgery.