Overview

Achieving a proper segmentation of the cortical sheet is a complex task which involves a substantial amount of work if done manually. In this chapter, we describe how proper segmentation of the inner and outer boundary of the cortical sheet can be achieved fully automatically in many cases. Achieving a robust, automatic segmentation procedure is a difficult task and involves in BrainVoyager a series of tools as described in subsequent sections.

The prerequisites for using the automatic segmentation tool are:

  • The original anatomical 3D data set containing the brain must have an appropriate spatial resolution, i.e. close to 1mm x 1mm x 1mm. If the voxels do not have a 1mm resolution, use the iso-voxel tool to interpolate the data set.

  • The contrast between grey matter and white matter tissue must be high enough. On a Siemens Vision scanner, the 3D T1 FLASH sequence provides excellent results. On other scanners, you might consider (optimized) MPRage sequences using 2 or 3 acquisitions.

  • The intensities across space of the different tissue types should be homogenous. You can check this by using the intensity histogramm tool. If the histogramm does not show two clearly separated peaks, use the inhomogeneity correction tool.

  • The anatomical data set should be in stereotaxic space which is used to exploit anatomical knowledge for initial brain segmentation (Talairach masking), filling of ventricles, removing subcortial structures and disconnection of cortical hemispheres. Use the spatial transformation tools to transform your original anaatomical data set in stereotactic space.

The automatic segmentation routines attempt to provide high-quality results. Still it is recommended that you check the segmentation result and to improve it if necessary.The automatic segmentation routines also reconstruct the segmented hemispheres and produce surface meshes of the inner, middle and outer boundary of the cortex.


Invoking the Automatic Segmentation dialog

The automatic segmentation option can be accessed as follows:

  • Load the anatomical 3D data set you want to segment which must be in Talairach space.

  • Select the Segmentation tab in the 3D Volume Tools dialog

  • Click the Options button

  • In the appearing Segmentation Options dialog, click the Automatic segmentation.. button

After applying these steps, you will see the Automatic Segmentation of the Cortical Sheet dialog (see figure below). You can simply click the Start button to launch the automatic segmentation procedures. The next sections describe the resulting data sets and the individual steps performed.

Automatic Segmentation dialog

The dialog lists a number of steps which are performed in succession to produce a segmentation of the white/grey matter boundary and reconstructions of the cortical sheet. All options except the removal of "bridges" in the resulting segmentation are checked as default. If you intend to inflate and flatten the reconstructed cortical sheet, you should also check the Remove bridges option.


Resulting data sets

The following files will be produced by the automatic segmentation tool. The created file names are based on the loaded data set to which strings are appended (shown in bold below) which briefly describe what's in the respective file.

  • NN_TAL.vmr The loaded 3D data set

  • NN_TAL_WM_LH.vmr the segmented white/grey matter boundary (WM) for the left hemisphere (LH)

  • NN_TAL_WM_RH.vmr the segmented white/grey matter boundary (WM) for the right hemisphere (RH)

  • NN_TAL_WM_LH_BL2.vmr the segmented white/grey matter boundary (WM) for the left hemisphere (LH) after bridge removal

  • NN_TAL_WM_RH_BL2.vmr the segmented white/grey matter boundary (WM) for the right hemisphere (RH) after bridge removal

  • NN_TAL_LH_RECO.srf the reconstructed surface mesh (RECO) of the left hemisphere without any further processing like smoothing.

  • NN_TAL_RH_RECO.srf the reconstructed surface mesh (RECO) of the right hemisphere

  • NN_TAL_LH_RECOSM.srf the reconstructed mesh after smoothing (RECOSM), use as reference mesh for curvature coloring.

  • NN_TAL_RH_RECOSM.srf the reconstructed mesh after smoothing (RECOSM), ref mesh for curv. coloring for inflated/flattened mesh.

  • NN_TAL_LH_WM.srf an optimized and smoothed representation of the white/grey matter boundary (WM) for the left hemisphere

  • NN_TAL_RH_WM.srf an optimized and smoothed representation of the white/grey matter boundary (WM) for the right hemisphere

The files "_WM_LH_BL2.vmr" and "_WM_RH_BL2.vmr" are only generated if the Remove bridges option has been checked in the dialog. This option should be used if you intend to further process the surfaces, i.e. running inflation and flattening procedures. Bridges can be removed by manual editing (see xx) or automatically. If you want to remove bridges automatically but you did not include automatic bridge removal during the segmentaion process, you can run it afterwards by clicking the Rmv bridges button in the Segmentation tab of the 3D Volume Tools dialog. Note, that you must then reconstruct new surfaces from the resulting .VMR files. For more details, see xx

For inflation and flattening you can use any mesh file produced for a particular hemisphere, i.e. "_LH_RECO.srf", "_LH_RECOSM.srf" or "_LH_WM.srf". We recommend to use the "_RECOSM.srf" files because they will produce the nicest curvature coding results on the inflated and flattened meshes.


Sigma filtering

The sigma filter is used to increase the contrast between white and grey matter which improves subsequent region growing operations. It operates like a standard gaussian smoothing filter but excludes neighbor voxels with intensities too far away from the intensity of the currently considered voxel. This critical intensity distance is defined in the Range field. The default of 5 normally produces good results.
Extensive testing has shown that repeating the sigma filter several times with a small Range value produces better results than using it once with a larger Range value. You can adjust the number of repetitions (default "10") in the x times field. The figure below shows a slice before (left) and after (right) the application of the sigma filter with the default values.


Talairach-based Segmentation of the Brain and Subcortical Structures

The Find and fill ventricles step and the Apply BrainTalMask step use a priori positional knowledge based on Talairach space. The figure below shows a slice after these steps have been applied.

In the first step, voxels with low intensity values are searched around positions which belong with high likelihood to ventricles. Beginning with these voxels, a region growing process is started to fill the ventricles up to its boundaries. The dark green regions in the slice shown above depict part of the filled ventricles.
In the second step, an initial Talairach-based segmentation of the brain and labelling of subcortical structues is performed based on a VMR mask file called "BrainTalMask.vmr". This file is located in your BrainVoyager folder. A sample slice corresponding to the same z-level as the one shown above is depicted below. The green regions in that file are used to automatically label subcortical structures as "white matter" and the red/orange regions are used to produce an initial brain segmentation including removal of the cerebellum. These initial segmentation will be refined by subsequent steps.

The Talairach mask file has been created based on the segmentation of an individual brain (red color) and it was then expanded to fit many additional brains (orange color). This resulted in a "conservative" mask file which can be applied to any new brain without removing relevant brain tissue. Since the BrainTalMask.vmr file is a standard VMR file, it is possible to change it easliy yourself. BrainVoyager uses the red color (color index: 226) and orange color (color index:229) to segment the brain and the green color (color index: 245) to label subcortical structures.


Intensity Histogram Analysis

For an automatic labelling of white matter, the intensity histogram across several axial slices is computed (z = 54 + n*10, 0<n<10). The 10 different histograms are drawn in different colors changing from blue (most superior slice) to red (most inferior slice). Intensity is on the X-axis and the number of voxels per intensity is shown on the Y-axis. The curves should show two peaks, the first peak corresponds to the intensity of grey matter whereas the second peak (on the right side) corresponds to the intensity of white matter.

Why not drawing just one intensity histogram pooled across all slices? The reason for drawing multiple slice-based histograms is to give information about the intensity homogeneity across the slices. Axial slices were chosen because many sequences (i.e. MPRage) have homogeneity problems in this direction. The white matter peaks of the different slices should be all ligned up as shown in the graph above. If this is not the case, you should consider performing inhomogeneity correction. The second graph also presents homogeneity information by tracing the white matter (blue) and grey matter (red) peaks across axial slices (54<z<171). If the blue line is horizontal within the x values 20 and 90, the data is homogeneous enough for the subsequent segmentation steps.

After inspection of the two diagrams, you can simply close the respective window by clicking the "x" button on the right side of the title bar.


Region Growing and Morphological Operations

Based on the estimates of the white and grey matter peaks, the Segment white matter step performs a region growing process to label "white matter". The outer boundary of this labelling represents the cortical sheet of both hemispheres. The region growing process needs a minimum and a maximum intensity value to determine a range for the white matter labelling. Determination of the upper white matter range value is not critical and can be set to the maximum intensity (225) but it is more difficult to determine an appropriate lower range value, the white matter intensity threshold. There are two methods implemented in BrainVoyager to determine this value, the WM width estimation method and the Peak-to-peak method. The peak-to-peak method specifies the white matter intensity threshold between the grey and white matter peaks. The exact point is defined by the Relative cut point value. If this value would be set to "0.5", the white matter threshold would be set exactly at the intensity value in the middle of the two peaks. If this value would be set to "1.0", the threshold would be set directly at the white matter peak, if it would be set to "0.0", the threshold would be set at the grey matter peak. A value of "0.7" is used as default providing best results in many cases.
[NOTE: This method is not available in the current program version (4.2.5) but will be made available again in the next release]

The white matter width estimation method has been introduced as a second method because the peak-to-peak method did not always produce robust results across different scanners and MR sequences. The main problem is that it is only dependent on the distance between the grey and white matter peaks but does not take into account the width of the distribution around the peaks. This may lead to results where the segmentation is too much biased towards white or grey matter intensities. The white matter estimation method starts with the white matter peak and determines the intensity value on the right side of the peak where the histogram value drops below 10 percent as compared to the peak histogram value. This is used as a simple white matter distribution "width" estimation. It is only performed on the right side of the white matter peak because on the left side the white matter and grey matter distributions are merged to a certain degree. The intensity distance of the 10 percent value from the white matter peak is then subtracted from the white matter peak to finally find the white matter threshold value (cut point, see figure below).

In the rare cases where the automatic white matter threshold estimation methods do not produce good results or when you want to bias a segmentation towards or away from the white matter peak intensity, you can also overwrite the methods by checking the Force value option and by entering a White matter threshold intensity value. You can get experience about an appropriate threshold value by using the Grow region tool in the Segmentation tab of the 3D Volume Tools dialog by changing the minimum (and maximum) value in the Inclusion range field.

When the white matter region growing has been performed, additional steps are done to improve the quality of the segmented boundary. If a surface would be reconstructed on the basis of the "raw" region growing result, it would contain an very high number of "errors" like holes or handles. Under the a priori knowledge that the cortical sheet is a smoothly varying surface, two morphological operations are, thus, performed to improve the segmentation. The first is a dilation method and the second a smoothing method. The Dilate white matter step expands the segmentation into the grey matter for one voxel. This expansion is carried out without introducing connections between the banks of sulci. The boundary of the segmented volume now runs roughly through the middle of the cortical sheet. A more precise method to estimate the middle of the cortical sheet taking into account the thickness of grey matter at each region will be introduced in the next minor update of BrainVoyager (4.4.x).
The Smooth white matter step finally smoothes the obtained boundary. You can control the number of smoothing steps by entering a value in the Smooth white matter repetitions entry. A value of "5" normally produces good results.


Disconnection of Cortical Hemispheres

After having segmented "white matter" in the previous step, the two hemispheres are segmented by the Disconnect hemispheres step. In this step, a cut is performed within the midsagittal slice (x = 128) separating the brain in the region of the corpus callosum and subcortical structures (90 < y < 175, 90 < z < 150). This cut is not performed along the whole midsagittal slice because this could introduce wrong results due to asymmetries of the two hemispheres. Those asymmetries are often present in the occipital lobes where one hemisphere extends across the midsgittal plane into the region of the other hemisphere. Therefore, it is intended in this step to label the hemispheres based on the initial cut described above and a subsequent region processing step. Only if this attempt fails, the more simple method of cutting through the whole midsagittal plane is used and the user is informed about this with a message dialog. During this step, the two segmented hemispheres are shown in two colors as depicted in the figure below. The segmentation is first performed on the non-dilated version of the brain and is then extended also to the dilated version (see previous step) . The two colors allow to save each hemisphere finally under the names "nn_TAL_WM_LH.vmr" and "nn_TAL_WM_RH.vmr". If you open these files, you will see that the colors are changed to blue (color index: 240) with a yellow boundary (color index: 235). This is because before saving, the "clear for reconstruction" command has been performed which allows to use these files directly for surface reconstruction of the cortical sheet.


Removing Bridges

The files "nn_TAL_WM_LH.vmr" and "nn_TAL_WM_RH.vmr" produced in the previous step can be directly used to reconstruct its boundaries producing a mesh representation of the cortical sheet of each hemisphere. The steps performed produce high-quality polygon-mesh representations which are useful for 3D visualization of the folded cortical sheet in the surface module. They are normally, however, not appropriate for further morphing procedures like inflation and flattening because they are not completely free of topological errors called "handles" or "bridges". These topological errors are often the result of connections across opposite banks of sulci. These bridges can be removed by manual editiing as described in section xx. A more attractive alternative (Kriegeskorte & Goebel, submitted) is to check the Remove bridges step which will detect and remove all existing topological errors automatically for both segmented hemispheres. The resulting data sets, free of any bridges, will be saved as "nn_TAL_WM_LH_BL2.vmr" and "nn_TAL_WM_RH_BL2.vmr" and are used for surface reconstruction of the cortical sheet.

The figure above shows part of a topological error in the calcarine sulcus and the same region after the automatic bridge removal algorithm has been applied. The figure below shows the same region in the corresponding polygon mesh representations (the shown parts appear left-right flipped; this happens because in the surface left is mirrored with respect to the.


Surface Reconstruction of the Cortical Sheet

This step creates polygon meshes from the boundaries of the segmented hemispheres. During this process, three files are saved to disk for each hemisphere. The first file with the string "_RECO.vmr" (RECO = reconstructed) is obtained in a few seconds and it is the result of tessellation of the boundary of the segmented hemisphere. This file is important because all further surfaces can be derived from this "root" or "parent" polygon-mesh. It looks rather "blocky" because it shows the voxelated nature of the underlying VMR data set as shown below.

The second file created is a smoothed version of the "RECO" file as the result of 80 smoothing iterations with a smoothing force of "0.1". It is saved as "_RECOSM.vmr" (SM = smoothed). This file should be used as a reference when the curvature of the folded mesh representation is computed and visualized for inflated and flattened surfaces.

The third file produced for each hemisphere is "_WM.vmr" which contains a smoothed polygon mesh which follows precisely the white/grey matter boundary. The "_RECOSM.vmr" mesh on the other hand follows the dilated version of the white/grey matter boundary. This mesh is obtained by morphing the "_RECOSM.vmr" mesh against the voxel representation of the white/grey matter boundary followed by 80 smoothing iterations with a smoothing force value of "0.07". This mesh can be used to visualize functional data because it allows to look deeply into sulci.

NOTE: An additional option "Create surfaces of pial boundary" is currently tested constructing a morphed representation of the outer boundary of grey matter. This pial surface mesh tool will be provided in the next program release.


Checking the automatic segmentation result

Although the automatic segmentation tools work robustly in many cases, it is important to evaluate the obtained segmentation quality and to improve the result by manual editing, if necessary. There are several tools which allow to inspect the segmentation result. The most important one is the superposition tool which allows to visualize the VMR containing the white matter segmentation within the VMR containing the original (or any other) intensity data. To use this tool, open first the segmented VMR, i.e. "nn_TAL_WM_LH.vmr". In the Spatial Transf tab of its 3D Volume Tools dialog, load the VMR containing the original intensity data by clicking the Load button in the 3D-3D Alignment field. The original intensity data is now loaded as a second VMR. In the title of the VMR window, you will now see the file names of both lodaded VMR data sets as shown in the figures below. Between the two file names an arrow indicates what is currently visible within the VMR viewing window. Initially, the arrow points to the left side indicating that the VMR loaded first, the segmented data set, is shown.

If you click the 'F8' button, the arrow will point to the right side indicating that now the VMR loaded second, the original intensity data, is shown. Repeatedly pressing the 'F8' key will toggle back and forth between the two loaded data sets.

The superposition tool is activated when you click the 'F9' key. Now the arrow will point both to the left and right side indicating that now both data sets are visible (see figure below). This visualization is computed by making transparent all black voxels in the first data set. If you now click the 'F8' toggle, it will switch between the first loaded data set and this superimposed view. In addition the 'F7' key will switch between the second loaded data set and the superimposed view.

This "merged" view of the segmented and original data set is very helpful to evaluate the quality of the segmentation. You can navigate within the data as usual allowing to inspect any region. A second type of image superposition can be invoked by clicking the 'F9' key again. The arrow in the title bar between the two file names will again point to both sides but will have now two horizontal lines. In this mode, only the boundary of the first file is shown, any other voxel is made transparent. This visualization option is very helpful because it allows to see the intensity values on both sides of the segmented cortical sheet (see figure below). On one side of the yellow line you should see only grey matter voxels whereas on the other side you should only see white matter voxels.

An alternative method to visualize the reconstructed cortical sheet is to project one of the surfaces into the voxel space of a VMR. In the figure below the "_WM.srf" polygon mesh has been projected into the original intensity data set using the Project SRF into VMR button in the Meshes->Surface manipulation options.. . Note, that the projected yellow line is directly painted into the VMR data set. To undo this step, click the Reload all button in the Segmentation tab of the 3D Volume Tools dialog.

The surface-into-VMR projection method is useful for quality assurance but the dual VMR method described above is better suited in most cases, especially for manual editing as described in xx.