Automated Grain Boundary Detection for Bright-Field Transmission Electron Microscopy Images via U-Net
Researchers led by principle investigator Professor Katayun Barmak (Department of Applied Physics and Applied Mathematics, Columbia University) and graduate student Matthew Patrick (Department of Applied Physics and Applied Mathematics, Columbia University), have developed the first generalizable and automated approach to identifying grain boundaries in bright-field transmission electron microscope (BF-TEM) images using a machine learning model based on a U-Net architecture and a tailored post-processing scheme [MJ Patrick et al., Microsc. Microanal. ozad115 (2023)].
The majority of technologically important materials are polycrystalline, comprised of a myriad of small crystals of different orientations, called grains, which are divided by grain boundaries. The size and shape of the grains, as well as the properties of the network of their boundaries, have important impacts on the properties of the material, from mechanical strength and failure to the electronic properties of the material.
Polycrystalline metallic thin films are particularly important in modern electronics, but characterizing the use of TEM imaging is required to resolve the details of their fine grained structures at the length scale of 10-100 nm. Unfortunately, image formation in BF-TEM micrographs is governed by complex electron scattering and thus the images have many artifacts and complicated contrast between grains. Since studies have begun on these materials, the only viable technique to collect reliable and reproducible data from these images has been tedious hand labelling of grain boundaries by expert tracers, usually begrudging graduate and undergraduate students. This approach severely limits the throughput of experiments and dashes any dreams of analyzing many large datasets like those collected during in situ heating experiments, necessary for the next generation of microstructural studies of nanocrystalline thin films.
The researchers trained the machine learning model, based on a U-Net convolutional neural network, using ground truth data constructed from aluminum images and tracings generated by hand in the early 2000s. The results of the model applied to unseen images from the same era were shown to match their hand tracings through a variety of statistical metrics. Furthermore, the model was also applied to generate new results on a recently prepared aluminum film and has shown great promise in platinum and palladium thin films, even when images are captured during in situ heating experiments, as visualized in Figure 1. This is the first time an automated segmentation approach for this kind of image has been able to be used on images dissimilar to those for which was initially designed!
Full understanding of the evolution of thin film microstructures requires a lot of data, and this advance opens the door to high throughput analysis of thin film microstructures at a scale never before possible, bringing the rapid, quantitative study of the dynamics of grain growth within reach.
Figure 1: Bright field TEM images of (a) Al with its (b) hand tracing and (c) post-processed automated tracing overlaid (c). (d-g) BF-TEM images of Pt and Pd, respectively, with their post-processed automated tracings overlaid.
