**
Processing of 2D and 3D Echo Image Sequences
**

CIRAM – Dept. of Mathematics,
University of Bologna, Italy

**Karol Mikula**: Department of Mathematics,
Slovak University of Technology, Slovakia

**
Alessandro Sarti**:
DEIS, University of Bologna,Italy

**
TecnoBiomedica**,
Roma; http://www.tecnobiomedica.it

**
ESAOTE **
Spa,
Genova;
http://www.esaote.com/

In many applications computers analyse images or image sequences which are often contaminated by noise, and their quality can be poor (e.g. in medical imaging). We have studied mathematical models based on partial differential equations (PDEs) approach to automatically produce images of much higher quality, with enhanced sharpness, filtered out the noise, extracted shapes, closed missing boundaries, etc. We have designed and analysed the robust numerical methods and their efficient implementations in processing of 2D, 3D images, 2D+time and 3D+time image sequences given in rectangular, cylindrical and other geometries related to recent acquizition devices and techniques.

*
Denoised 3D images in cylindrical geometry*

Recent image processing operations involving PDEs are nonlinear image filtration, edge detection, deblurring and image enhancement, restoration, image inpainting, shape extraction and analysis, image segmentation, motion analysis, motion based filtering etc.. Typical 2D examples are given by a large variety of medical images, satellite or camera system images, old archive documents, texts pre-processed for automatic reading, old corrupted photographs or any other digital images of poor quality. 3D examples arise in bioengineering, medicine or in material quality control, where 3D volumetric aquisition methods are widely used nowadays. The processing of image sequences can be found in the restoration of movies, video sequence analysis, visual recording of growth (of human organs, leaves of plants, etc.) or in improvement of the quality of medical image sequences. For example, the ultrasound acquisition of a beating heart in 3D echocardiography gives one such important application; the principal investigators of this project have long time experience in this field and they have made an important contributions to the topic (see [1]).

*
*

*
The multiscale analysis of 1st, 5th and 9th
time step of the echo-cardiographic sequence.
The shape of the left ventricle is extracted in those
moments of cardiac cycle.*

From the
mathematical point of view, the input processed image can be modelled by a real
function u^{0}(x), u^{0} : Ω
→
R, where Ω, a subset of R^{d}, represents a spatial domain. In the case
of an input image sequence u^{0}(x,θ), u^{0} : Ω x [0,T_{A}]→
R, it depends on the additional parameter θ representing a point in a real time
interval of acquisition [0,T_{A}].

The first step to
use PDEs for image processing was done in the beginning of eighties. By the
simple observation that the Gauss function G_{σ}(x)=1/(4πσ)^{d/2
}e^{-|x|^2/ (4σ)} is a fundamental solution of the linear heat
(diffusion) equation, it has been possible to replace the classical image
processing operation - convolution of an image with G_{σ} with a given
variance v=√(2σ) (Gaussian smoothing) - by solving the linear heat equation for
a corresponding time t=σ with initial condition given by the processed image. It
is well known that Gaussian smoothing (linear diffusion) blurs edges in the
images and moves their positions. Although such a phenomenon can cause no
problems in some examples of data analysis, in image processing, where the
visual impression is important and a precise localization of edges is also
necessary (e.g. to compute volumes of segmented objects), the linear image
smoothing is generally not the best choice. A way has been found to overcome
these shortcomings, namely to switch to nonlinear diffusion models. Due to the
evolutionary character of the process which controls the processing using
diffusion equations, application of any
PDE
to an initially given image is understood as its embedding in the so-called
scale space. In the case of nonlinear PDEs one speaks about nonlinear scale
space. The axioms and fundamental properties of such embeddings have been given
and studied in [AGLM], where the notion of image multiscale analysis has been
introduced. The image multiscale analysis associates with a given image u^{0}(x)
a family u(t,x) of smoothed-simplified images depending on an abstract parameter
t in [0,T], the scale. As has been proved in [AGLM], if such a family fulfils
certain basic assumptions - pyramidal structure, regularity and local comparison
principle - then u(t,x) can be represented as the unique viscosity solution of a
general second order (degenerate) parabolic partial differential equation. This
theoretical result has also an important practical counterpart. The equations of
(degenerate) parabolic type have a smoothing property, so they are a natural
tool for filtering (image simplification) by removing spurious structures, e.g.
noise. Moreover, the simplification should be image oriented, e.g. it should
respect edges and not blur them. Or, it should recognize motion of a structure
in an image sequence, and consequently the smoothing (diffusion) should respect
the motion coherence in consecutive frames. Such requirements, or even more
sophisticated ones related to the geometrical characteristics of the image,
bring strong nonlinearity into the parabolic PDEs, and make this field
interesting not only because of the applications but also from a mathematical
and numerical point of view. Moreover, since image processing operations add
further dimension to the problem (the scale space), e.g. in case of 3D+time
sequences we have to solve computationally 5 dimensional problems, it is a
chalenge for high scale and parallel computing facilities.

Since images are given on a discrete grid (pixel/voxel structure), we discretized the PDEs to get a numerical scheme implemented on the computer. The construction of fast and robust numerical schemes represents an important field of research, since the arising equations are higly nonlinear or even degenerate. Moreover one has to handle huge amount of data in fast and unconditionaly stable way. Semi-implicit schemes studied e.g. in [4,6,10], where the nonlinear terms of the equation are treated from the previous discrete scale step, and linear terms are considered at the current scale level, have favorable stability and efficiency properties. For space discretization either finite element method [4], finite/complementary volume method [6,10] or finite difference methods [CLMC,WRV] can be used. The finite element and finite/complementary volume methods are based on integral (weak, variational) formulations of the initial-boundary value problems for PDEs. Variational methods have a strong physical background since they are based on principles like minimization of energy (finite element method) or conservation laws (finite and complemetary volume methods). They allow a clear and physically meaningful derivation of difference equations which are local and easy to implement.

*
Original echo
left ventricle (middle) and denoised (right) after 10 scale steps. Semi-implicit
finite volume*

During this project we have developed new computational methods for the problems of nonlinear image filtering and robust image segmentation for 2D, 3D images, 2D+time and 3D+time image sequences given in rectangular, cylindrical and other nontrivial geometries related to recent acquizition devices and techniques. The new numerical schemes will be studied regarding stability and efficiency of computations as well as regarding the convergence to the weak solutions of the corresponding PDEs. The discretized mathematical models have been based on nontrivial generalizations of the nonlinear image selective smoothing equation of the Perona-Malik type and on the morphologically invariant geometrical equations of mean curvature flow type. The image sequences have been processed and segmented using space and time coherence of moving structures. Preferably, the robust semi-implicit time-scale discretizations, variational spatial discretizations (adaptive finite element, finite/complementary volume methods) and fast and robust linear algebra solvers have been used in order to achieve fast and stable solutions.

**
**

**
**

**
**

**
**

**
**

**
**

**
****
Left ventricle boundaries.**

**
General
References**

[AGLM] L.Alvarez, F.Guichard, P.L.Lions, and J.-M.Morel, Axioms and fundamental equations of image processing, Arch. Rational Mech. Anal. 123(1993), 200--257.

[CLMC] F.Catté, P.L.Lions, J.-M.Morel, and T.Coll, Image selective smoothing and edge detection by nonlinear diffusion, SIAM J. Numer. Anal. 29 (1992), 182--193.

[WRV] J.Weickert, B.M.t.H.Romeny, M.A.Viergever, Efficient and reliable schemes for nonlinear diffusion filtering, IEEE Trans. Image Processing 7 (1998), 398--410.

**
Publications**

**
**

[1] A. SARTI, K.MIKULA, F.
SGALLARI** Nonlinear multiscale analysis of three-dimensional echocardiographic
sequences**, *IEEE Trans. Medical Imaging *, Vol.18, No. 6, pp.453-466,
1999.** **

[2] A. SARTI, K.MIKULA, F.
SGALLARI, C. LAMBERTI **
Nonlinear multiscale analysis models for filtering of 3D + time biomedical
images**,** **in**
**"*Geometric Methods in Bio-Medical image processing*", R. Malladi, Ed.,
Lectures Notes Comput. Science and Eng., pp.107-127, Springer Verlag, 2002.

[3] A. SARTI, K.MIKULA, F.
SGALLARI, C. LAMBERTI **
Evolutionary Partial Differential Equations for Medical Image Processing**,**
***Journal of Biomedical Informatics*, pp. 77-91, Vol. 35, No. 2, April
2002.

[4]
A. HANDLOVICOVA, K. MIKULA** **and F.SGALLARI,**
Variational numerical methods for solving nonlinear diffusion equations arising
in image processing***, Jour. Visual Communication and Image Representation*,
Vol.13, n1/2, pp.217-237, 2002. ** **

[5] K. MIKULA, T. PREUSSER, M.
RUMPF, F. SGALLARI **On Anisotropic Diffusion in 3D image processing and image
sequence analysis**, in “*Trends in Nonlinear Analysis*”, (M.Kirkilionis,
S. Kromker, R.Rannacher, F. Tomi (eds.), Springer Verlag, 2003, pp. 307-321.

[6]
A. HANDLOVICOVA,
K.MIKULA, F. SGALLARI, **
Semi-implicit complementary volume scheme for solving level set like equations
in image processing and curvature evolution**,
*Numerische. Mathematik.*, Vol.93, pp.675-695, 2003.

**
**

[7]
K.MIKULA, F. SGALLARI, **
Semi-implicit finite volume
scheme for image processing in 3D cylindrical geometry**,
*Journ. Comp. Applied Mathem*., Vol.161, pp.119-1325,2003.** **

**
**

[8] A. SARTI, K.MIKULA, F.
SGALLARI, **Co-volume
level set method in subjective surface based medical image segmentation**,
in: Hand Book of Medical Image Analysis, Vol. I: Segmentation Models Part A(ed.
by J.S. Suri, D.L.Wilson, S.Laxminarayan), KluwerAcademic/ Plenum publishers**,
**New York, pp. 583-626,** **2005.** **

**
**

[9] A. SARTI, K.MIKULA, F.
SGALLARI, **Co**-**volume
method for Riemannian mean curvature flow in subjective surfaces multiscale
segmentation**, Computing and Visualization in Science, Vol. 9, pp. 23-31,
2006.

**
**

[10] S. CORSARO, A. SARTI, K.MIKULA, F.
SGALLARI, **Semi-implicit co-volume method in 3D image segmentation,** SIAM
Jour. Scient. Comput,
Vol.28, No.6, pp. 2248-2265, 2006. With

Funded Projects

**C.N.R.
2000-02 Strategic Project **

"*Numerical
Methods for image and image sequences processing"*

Coordinator Research Group University of Bologna: Prof. Fiorella Sgallari

** **

**NATO Project 2004
Reference : PST.CLG.979123 **

“*Efficient and robust
computational methods for biomedical image analysis”.*

*
Scientific Coordinator: *
Prof. Fiorella Sgallari,
University of Bologna.

**
Industrial Research Project:
**“

(Esaote -Genova, TecnoBiomedica-Roma)

Segmentation and vector flow of left ventricle boundaries.