Over the last 15 years, significant efforts have been made to build numerical patient models from multimodal images for surgical planning and image-guided surgery. The components on which the VIGOR++ project will make S&T contributions in relation to the state of the art are described below.

Image analysis

There is a wealth of techniques available in the literature on medical image analysis. Much research was conducted in the context of CT colonography (CTC). This relatively new technique concerns CT imaging of the colon for the detection of colorectal cancer/polyps. Image analysis algorithms were created to register the scans made of a patient in prone and supine positions. However, research has particularly focussed on the so-called electronic cleansing of CT images, i.e. automated detection of faecal residue in the intestines. Electronic cleansing relies on the patient taking a contrast agent that “highlights” the residue, which may subsequently be detected by means of thresholding. More elaborate techniques were also proposed to cope with noise and contrast non-homogeneity. All this colon segmentation work mainly focused on the inner surface of the colon wall (the lumen-mucosal boundary). Outer wall segmentation from CT images remains an unsolved problem, due to the low contrast between attenuation values of the colon wall and the surrounding fat tissue.

VIGOR++ will employ MRI to produce robust tissue discrimination and to quantify the features of Crohn’s disease. Automated measurement of such features in MRI images has not been investigated, to the best of our knowledge. Signal fluctuations in MRI emanate both from global (bias field) as well as local effects (non-homogeneities in bowel content) and this variation in signal value may preclude simple approaches (e.g. thresholding) for MRI data. Sophisticated methods are available to cope with those variations and will be used in the project .

Radiologists assessing MRI data of patients suffering from Crohn’s disease typically look for signs of local bowel wall thickening. The MRI findings of bowel wall thickening are analysed and classified into several types based on the mural signal patterns. Moreover, the dynamic response of the tissue to the inflow of blood is studied since a marked increase in signal intensity of actively diseased bowel results from enhanced perfusion and vascular permeability in inflammatory tissue. All these aspects will be contained in properties measured on the candidate lesions for Crohn’s disease.

We intend to extract morphological (shape), textural (stratification) and structural (vascularisation) features and use a combination of these features for the detection of abnormalities. In particular the following features will be examined:

Morphology of lumen (diameter, curvature), morphology of wall (average thickness, curvature), granularity of wall surface.
Layer detection, presence of fat, oedema, haemorrhage in wall and around wall.
Absolute wall enhancement and enhancement pattern in dynamic contrast enhanced MRI, lymph node enhancements.


Automatic detection and classification of abnormalities from medical images by machine learning has received much attention. However, regarding the gastrointestinal tract such computer aided detection is limited to recognition of polyps/colorectal cancer in CT colonography. The most successful techniques are based on local shape features, particularly the local shape index and curvedness, which are computed after segmentation of the colon surface. Although very good results were obtained, these techniques still often fail to differentiate between polyps and residual material. Moreover, they rely on accurate segmentation of the colon surface, which is not always feasible, especially if there is much faecal remains (particularly in patients that did not take laxatives, which is preferred for patient compliance).

The use of dynamic contrast enhanced (DCE) MRI has been advocated by an increasing number of investigators studying physiological processes in the human body. Quantification of DCE MRI data by means of pharmacokinetic models aims at calculating absolute measures that are directly related to the tissue physiology such as vessel permeability, blood flow, blood transit time through a tissue and extracellular volumes. Unfortunately, such compartmental analysis suffers from wide variability in output, which is a consequence of the large variety of models used. An alternative and potentially more robust method is to study the uptake curve shape and to relate this to pathological findings. This approach is based on the utility of qualitative observations of the time–intensity curves (TIC) generated from ROI’s chosen in lesions by radiologists.

In VIGOR++ techniques from statistical pattern recognition will be used in order to define an objective score of Crohn’s disease severity. Initially, a statistical model will be created that mathematically describes the normal variation in the features emanating from patients without active disease. Subsequently, we intend to ‘train’ a curve shape classifier on manually annotated data to remove the dependency on arbitrary MRI parameter settings. Alternatively, unsupervised techniques, i.e. both principal and independent component analysis will be employed to analyse the prevailing curves shapes. This exploration will enable objective measurement e.g. the prevalence of certain curves shapes in different disease states, that will add to the features described initially (such as wall thickness and stratification pattern).

The combination of multiple imaging features within a classification framework is expected to sustain classification accuracy levels similar to those achieved by trained professionals. A further novel direction in VIGOR++ will be to investigate combining MR data and laboratory data for more reliable and precise diagnostic performance. The combination of imaging and clinical biomarkers of disease activity will draw on the strengths of both methodologies and it is anticipated will add to the robustness of any predictive model.  The design of a classifier for quantifying Crohn’s disease severity will be treated as a regression problem rather than a traditional classification task. In other words, a weighted probability density function is fit to the feature data in which the weights derive from clinical indices of Crohn’s disease activity (CDEIS, CDAI).

Interactive visualisation

A major challenge will be to develop new methods, which allow the examination of GI wall tissue. Conventional volume rendering techniques are applied in CT colonography to visualise the colon’s inner surface. The current proposal requires the depiction of multiscale, spatially diverse data. Consequently, novel ways of mapping the data to create an intuitive visual representation are needed. The difficulty to do so particularly lies in depicting clinically important features while mapping onto a two-dimensional display.

A potential solution to this problem is the concept of importance-driven visualisation, which has been successfully employed for the combined visualisation of multi-modal data. Such an approach involves an ordering of features according to a measure of ‘importance’. Automatically, a region containing less important features is depicted by means of a less detailed visual representation and/or displaced in order to provide an unobstructed view of a more important region. The ordering to importance will be derived from the classifier to be developed. The importance information will also be used to guide so-called unfolding techniques to minimize distortions in important areas while allowing for more relaxed conditions in ‘contextual’ regions.

Clinicians must accurately interpret and integrate findings in order to achieve diagnostic certainty. It is important that uncertainties in the data are made visible. For this recently developed methods will be adapted and extended. Clearly, the goal is to create an integrated visualisation framework which allows intuitive study of the all available information in a unified manner. Techniques for visualising complex anatomies and physiological data in complex anatomies have been studied previously, but a comprehensive solution for the domain addressed here that meets the current clinical demand is certainly not available yet.

Importantly, the consortium regards patient education an obligation that may not be ignored. It is expected that patient education will be supported by the developed visualisation techniques. Patient education is becoming an increasingly important aspect of medical care. Illustrations can be a valuable aid in explaining diagnoses and treatments. Recent progress in the area of illustrative visualisation seeks to reproduce the expressiveness and comprehensibility of traditional illustrations using advanced visualisation techniques. Perceptually effective visualisation techniques as well as methods to evaluate their effectiveness and will support this development. Such an approach makes it feasible to generate patient-specific depictions of particular pathologies which are easy to understand by the general public.