Automated Feature Extraction with Machine Learning and Image Processing

PD Stefan Bosse

University of Siegen - Dept. Maschinenbau
University of Bremen - Dept. Mathematics and Computer Science

1 / 50

PD Stefan Bosse - AFEML - Module X:

X-ray Imagaing and Data

Physical principles behind X-ray RAdiography and Computer Tomography

Metrics of Data

Noise and Distortions in Measuring Processes; Data Noise

Finally: Data-driven Feature Extraction in X-ray Images (ML). Models, Algorithms, Issues

2 / 50

PD Stefan Bosse - AFEML - Module X: Further Readings

Further Readings

  1. M. Berger, Q. Yang, and A. Maier, “X-ray Imaging,” in Medical Imaging Systems, Springer, 2018.
  2. S. Bosse, Automated Damage and Defect Detection with Low-Cost X-Ray Radiography using Data-driven Predictor Models and Data Augmentation by X-Ray Simulation, The 10-th International Electronic Conference on Sensors and Applications (ECSA), Session Sensors and Artificial Intelligence, MDPI https://doi.org/10.3390/ecsa-10-16126 2023
  3. Schiebold, Zerstörungsfreie Werkstoffprüfung – Durchstrahlungsprüfung, 2015
3 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Radiation

X-ray Radiation

  • X-rays belong to the group of electromagnetic rays, hence, they follow the rules of electromagnetic radiation.

Berger, Yang, Maier, Medical Imaging Systems, 2018 Wavelengths and frequencies of the different groups of electromagnetic radiation. X-rays lie in the range of 0.01nm up to 10nm.

4 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Radiation

X-ray Radiation

  • In industry, X-rays are often the method of choice, for example to test for very small cracks in metal parts in the field of non-destructive testing.

  • The photon energy E is proportional to its frequency f and inverse proportional to its wavelength λ, that means the higher its frequency, the higher its energy:

Ep=hc0λp=fph

  • Typical energies in NDT: 30-120 keV ⇔ 0.04nm - 0.01nm
5 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Generation

X-ray Generation

  • An X-ray tube is commonly used for X-ray generation
  • An X-ray tube is an evacuated tube made of glass (rarely metal or ceramics) with a cathode and a solid metal anode in it.
  • Thermionic emission occurs by the heated filament at the cathode.
  • Heat induced electrons e are produced because the thermal energy applied to the filament material is larger than its binding energy.
  • Then, the electrons are accelerated by the tube’s acceleration voltage between the negative cathode and the positive anode.
  • When those fast electrons hit the anode, they are decelerated and deflected by the electric field of the atoms of the anode material.
  • Any acceleration of loaded particles results in electromagnetic waves.
  • The slowing down, i. e., the negative acceleration, of the electrons in the metal anode, generates X-rays.
6 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Generation

X-ray Generation

  • The anode is tilted by a certain angle to direct the emerging X-rays in the right direction. The angle determines the Focal Spot Size (FSD), and hence the imaging resolution, discussed later!

Berger, Yang, Maier, Medical Imaging Systems, 2018 Vacuum X-ray tube: The image on the left shows a schematic how electrons are accelerated from the cathode to the anode to generate X-ray photons. The image on the right shows a historic vacuum X-ray tube.

7 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Generation

X-ray Generation

The production of X-rays is caused by two different processes:

  1. Electron Interaction (Discrete)
  2. Nucleus interaction (Continuous)

The firs process: The electron interacts with an inner-shell electron of the target, characteristic X-radiation can be produced. This kind of X-rays results from a sufficiently strong interaction that ionizes the target atom by a total removal of the inner-shell electron.

  • The transition of an orbital electron from an outer-shell to an inner-shell is accompanied by the emission of an X-ray photon, with an energy equal to the difference in the binding energies of the orbital electrons involved.
8 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Generation

X-ray Generation

Another type of interaction in which the electron can lose its kinetic energy delivers the second process of X-ray production:

  • Caused by the interaction of the electron with the nucleus of a target atom.
  • As the colliding electron passes by the nucleus of an anode atom, it is slowed down and deviated, leaving with reduced kinetic energy in a different direction.
  • This loss in kinetic energy reappears as an X-ray photon. This type of X-rays is called Bremsstrahlung,
  • The amount of kinetic energy that is lost in this way can vary from zero to the total incident energy ⇒ Continuous energy distribution!
9 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Generation

X-ray Generation

Berger, Yang, Maier, Medical Imaging Systems, 2018

X-ray spectrum of a tungsten tube. The peaks correspond to the characteristic radiation; the continuous part of the spectrum represents the Bremsstrahlung. Maximum ≡ UHV keV

10 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Material Interaction

X-ray Material Interaction

Berger, Yang, Maier, Medical Imaging Systems, 2018 Principles of photon-matter interactions

11 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Material Interaction

X-ray Material Interaction

The X-ray photons either experience a complete absorption, elastic scattering or inelastic scattering

  • The reduction of radiation intensity is a reduction of the number of photons that arrive at the detector.
  • That process is usually referred to as attenuation.
  • There are several different physical effects contributing to attenuation, including a change of the photon count, photon direction, or photon energy.
12 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Material Interaction: Absorption

X-ray Material Interaction: Absorption

  • Absorption is the main effect that contributes to X-ray imaging:

I=I0eµx

with x as the material thickness and μ as the absorption coefficient that is dependent from the photon energy!

Assuming ray optics (not photons), the absorption and attenuation along a path from the source to a detector is an accumulated attenuation of all materials (composites, e.g.,with different μi values) withing the path!

13 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Material Interaction: Absorption

X-ray Material Interaction: Absorption

Hahn, Mary, DGZfP-Proceedings BB 84-CD , "A general approach to flaw simulation in..", CT-IP 2003 Absorption coefficient µ for aluminum against X-ray energy

14 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Measuring Technologies

X-ray Measuring Technologies

We get intensity images, but want to have a material density images (Scanogram or Tomogram)!

X-ray image processing and output

15 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Measuring Technologies

X-ray Measuring Technologies

Material density can only be "reconstructed" from X-ray intensity correctly if monochromatic (discrete energy) radiation is used.

  • Besides attenuation, there is scattering and diffraction related to very fine structures, lowering the density computation accuracy.

X-ray image quality is influenced by multiple factors

16 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Measuring Technologies

X-ray Measuring Technologies

The Focal Spot Size Diameter (FSD) determines the maximal resolution of an X-ray imaging system!

https://umsystem.pressbooks.pub/digitalradiographicexposure/chapter/focal-spot-size/ The FSD depends on the filament size of the tube and the anode angle

17 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Measuring Technologies

X-ray Measuring Technologies

https://umsystem.pressbooks.pub/digitalradiographicexposure/chapter/focal-spot-size/ Larger focal spot sizes increase penumbral unsharpness (but depends on magnification, too)

18 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Measuring Technologies

X-ray Measuring Technologies

The Source-Object (SOD) and Source-Detector (SDD) distances determine the image magnification M and the unsharpness U!

M=SDDSODU=FSD(M1)

Schiebold, Zerstörungsfreie Werkstoffprüfung – Durchstrahlungsprüfung, 2015

19 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Radiography

X-ray Radiography

  • Single projection measurement (central view)
  • No in-depth information on structure
  • Interpretation of images from multi-density composite materials is difficult

Zeiss Basic principle of radiography. One (ray orthogonal) projection is captured providing an intensity image with intensities inverse proportional to the material density (and thickness)

20 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Radiography

X-ray Radiography

Effective maximal (theoretical) resolution R without unsharpness U is determined by the detectors pixel size pD and the magnification M

Rlimit=pDM

21 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Tomography

X-ray Tomography

Balasubramanian, Krishnamurthi, "X-ray scintillator lens-coupled with CMOS camera ..", PLOS Online, 2022 Basic principle of Tomography capturing multiple projections (rotation) from the same specimen under different angles. Finally, a depth slice image stack is reconstructed from the rotation image stack.

22 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detection

X-ray Detection

At the end of the day we need a digital projection image, which is baed on an analog signal, which is based on electrons, which is based on photons!

There are basically three classes of X-ray detectors:

  1. Semiconductors, direct electron generation by X-ray photons; CMOS or CCD camera;
  2. Semiconductors, indirect electron generation by light photons and scintillator materials (X-ray → light); CMOS or CCD camera; no optics (except light guides)
  3. Semiconductors, indirect electron generation by light photons and scintillator materials (X-ray → light); CMOS or CCD camera; with imaging or microscopy optics
  4. X-ray Image Intensifiers (XRII), optics, camera
23 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Image Intensifier

X-ray Image Intensifier

Berger, Yang, Maier, Medical Imaging Systems, 2018 Schematic principle of an image intensifier detector. The X-rays are first converted to light, which is converted to electrons. An optic accelerates the electrons towards a fluorescent screen which converts the electrons to light, which eventually results in an image.

24 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Image Intensifier

X-ray Image Intensifier

Berger, Yang, Maier, Medical Imaging Systems, 2018 Detailed principle of an image intensifier detector. The X-rays are first converted to light, which is converted to electrons. An optic focuses the electron beam to a fluorescent screen or film material which converts the electrons to light

25 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Image Intensifier

X-ray Image Intensifier

The XRII itself and the following camera optics introduce geometrical distortions of the recorded images!

(Left) Vignetting artifact, i. e., luminescence drops at image periphery (Right) Distortion artifacts due to external fields, field inhomogeneities and optical systems

26 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Image Intensifier

X-ray Image Intensifier

Example image (USAF 1951 test target, steel foil 100μm) from a cheap OV2940 CMOS camera with mini lens optics using a Thales XRII (8"), 55kV, Tungsten X-ray tube (FSD 0.8mm)

27 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Image Intensifier

X-ray Image Intensifier

Advantages:

  • High sensitivity
  • Medium area (about 10000 mm2)
  • Medium resolution, about 4 LP/mm

Disadvantages:

  • Expensive
  • High voltage generators required (up to 30 kV)
  • Geometrical distortions (can depend on environmental conditions, external fields, ...)
  • Sensitivity degrades during radiation exposition irreversible!
28 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Images: Geometrical Errors

X-ray Images: Geometrical Errors

Besides common limitations that all imaging systems share, e. g., spatial resolution and contrast ratio, image intensifier systems are most known for vignetting and distortion artifacts. Vignetting,

Errors by:

  1. Rotation
  2. Translation
  3. Skewing
  4. Barrel distortion
  5. Pincushion distortion
  6. Any non-linear, an-isotropic and inhomogeneous distortions
29 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Images: Geometrical Errors

X-ray Images: Geometrical Errors

http://m43photo.blogspot.com/2012/05/geometric-distortion-correction.html Barrel vs. pincushion distortions

30 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Images: Geometrical Errors

X-ray Images: Geometrical Errors

  • Corrections by various algorithms and methods

  • Commonly using a reference set of points (Ground truth points) and affine or non-linear polynomial transformation

  • Fisheye Distortion Correction

31 / 50

PD Stefan Bosse - AFEML - Module X: Affine Transformations

Affine Transformations

https://neutrium.net/mathematics/basics-of-affine-transformation/ Affine transformations are typically applied through the use of a transformation matrix M and its inverse M-1

32 / 50

PD Stefan Bosse - AFEML - Module X: Affine Transformations

Affine Transformations

For example to apply an affine transformation to a three dimensional point, P to transform it to point Q we have the following equation:

Q=MPP=M1Q

  • When dealing with affine transformation points are represented as P = (Px, Py, Pz, 1) while vectors are represented as v = (vx, vy, vz, 0) = v = (vx, vy, vz).
  • Where multiple transformations are to be performed a single compound transformation matrix can be computed.

M=M2M1M1=M12M11

33 / 50

PD Stefan Bosse - AFEML - Module X: Affine Transformations

Affine Transformations

Translation and Scaling

Mtrans=⎜ ⎜ ⎜100mx010my001mz0001⎟ ⎟ ⎟Mscale=⎜ ⎜ ⎜sx0000sy0000sz00001⎟ ⎟ ⎟

34 / 50

PD Stefan Bosse - AFEML - Module X: Camera Calibration

Camera Calibration

https://euratom-software.github.io/calcam/html/intro_theory.html

  • Using:
    • Rectilinear Lens Distortion (RLD) Model
    • Fisheye Lens Distortion (FLD) Model

The RLD model takes in to account radial (barrel or pincushion) distortion, and tangential (wedge-prism like, usually due to de-centring of optical components) distortions.

The fisheye distortion model only includes radial fisheye distortion.

Problem: Point coordinate transformations can leave holes (uncovered target pixels). Interpolation is required!

35 / 50

PD Stefan Bosse - AFEML - Module X: Camera Calibration

Camera Calibration

RLD Model

(xdyd)=[1+k1r2+k2r4+k3r6](xnyn)+(2p1xnyn+p2(r2+2x2n)p1(r2+2y2)+2p2xnyn)

where r = √(xn2+yn2), and kn and pn are radial and tangential distortion coefficients, respectively. The polynomial r2 in in the first term describes the radial distortion while the second term represents tangential distortion.

36 / 50

PD Stefan Bosse - AFEML - Module X: Camera Calibration

Camera Calibration

FLD Model

(xdyd)=θr[1+k1θ2+k2θ4+k3θ6+k4θ8](xnyn)

where r = √(xn2+yn2), and θ=tan-1(r).

37 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Images: Geometrical Errors

X-ray Images: Geometrical Errors

Geometric corrections using polynomial functions Abdul Basith, 2011, DOI: 10.13140/RG.2.2.35707.00806

38 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Flat Panel

X-ray Detector: Flat Panel

Wotirz, Digital X-ray Sensors, Circuir Cellar, 2012 Principle construction of a flat panel X-ray detector with a scintillator material, a light guide (fibres), and a CMOS/CCD image sensor

39 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Flat Panel

X-ray Detector: Flat Panel

Advantages:

  • Nearly no geometrical distortions
  • Moderate sensitivity
  • Large area (about 250000 mm2)
  • High or medium resolution, real pixel size about 50-200 μm (theoretical resolution limit), up to 30 LP/mm

Disadvantages:

  • Expensive
  • Practical and theoretical resolution differs (especially in lower cost systems, far away from film-based imaging!)
  • Noise (including X-ray induced noise)
40 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

Self-made low-cost X-ray screen imaging detector (Bosse, 2023)

41 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

Self-made low-cost X-ray screen imaging detector (Bosse, 2023)

42 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

  • Scintillator Screen: OrthoFine 100 foil (for medical radiography using films), green light emission

  • CMOS image sensor: Sony IMX290 1920x1080 pixel (monochrome), exposure time 100-5000 ms

  • Resolution limit: 3x3μm × Mopt=13 ⇒ 40μm

43 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

Some recorded samples, contrast, and spectral light emission of scintillator versa camera sensitivity

44 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

Advantages:

  • Inexpensive
  • Using only widely available components
  • Medium area (about 4000 mm2)
  • Medium resolution, real pixel size about 10-50 μm, up to 10 LP/mm

Disadvantages:

  • Geometrical distortions (but known and independent from environmental parameters)
  • Noise (including X-ray induced noise)
  • Low sensitivity, long exposure times
  • Lead glass placed before detector introduces additional geometric distortions
45 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Detector: Optical Screen Imaging

X-ray Detector: Optical Screen Imaging

The optical imaging system is sensitive to "popcorn" noise induced by incident radiation (even a mirror do not eliminate such kind of noise due to scattering). Algorithmic noise cancellation is required by using multiple images.

46 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Noise

X-ray Noise

There are two types of undesirable effects in imaging systems: probabilistic noise and artifacts (+ distortions).

  • Similar to noise, artifacts are image degradations that also find their source in physical effects during the scan.

However, the difference to noise is that when a scan is repeated using the exact same object and scan parameters, artifacts are reproduced exactly whereas noise effects will change based on a probabilistic scheme (and can be reduced/removed).

47 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Noise

X-ray Noise

Berger, Yang, Maier, Medical Imaging Systems, 2018 Overview of noise related processes in X-ray imaging

48 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Noise

A common quality measure for imaging is the signal-to-noise ratio (SNR).

In X-ray imaging it makes sense to use the definition based on statistics, i. e.,

SNR(x)=¯¯¯xσ=E(x)E((x¯¯¯x)2)

For random variables x following a normal distribution (Gaussian), x is the mean value and σ represents the standard deviation.

49 / 50

PD Stefan Bosse - AFEML - Module X: X-ray Noise: Popcorn Cancellation

X-ray Noise: Popcorn Cancellation

σ0 := Σ[1]
∀ (x,y) ∈ coord(σ0) do
if σ0[x,y] > γ then
∀ σ ∈ { Σ / σ0 } do
if σ[x,y] < γ then
σ0[x,y] := σ[x,y]
break
endif
done
endif
∀ σ ∈ { Σ / σ0 } do
if σ[x,y] < γ then
σ0[x,y] := σ0[x,y] + σ[x,y]
else
σ0[x,y] := σ0[x,y] + σ0[x,y]
endif
done
σ0[x,y] := σ0[x,y] / |Σ|
done

Popcorn noise cancellation using multiple images (γ is a threshold, Σ a set of images, σ a pixel value)

50 / 50