Performance evaluation of the Biograph mCT Flow PET/CT system according to the NEMA NU2-2012 standard

Our data indicate a similar performance of the PET components of the mCT Flow and
its predecessor system, the mCT 6] with the residual differences being discussed below. Image quality, as assessed by
the standard IQ phantom 19] and sample oncology patients, was also similar for the CTM and SS acquisitions with
the exception of slightly increased noise levels for the SS image planes at the edge
of the axial FOV (Fig. 6).

Of note, there are no published performance measurements for the mCT system series
based on the most recent NEMA NU2-2012 standard. The data published for the mCT 6] are based on the older NEMA NU-2007 standard 18]. The most significant difference between both standards is the change of the positions
of the point source for spatial resolution measurements. NEMA NU2-2012 requires a
measurement at 1-, 10-, and 20-cm radial offset position from the isocenter of the
PET FOV while 2007 version requires a single measurement at 1 cm and two measurements
(in x and y direction) at 10-cm offset. Furthermore, the axial placement of the off-center axial
point sources was changed from one fourth of the axial FOV from the center to three
eighth of the axial FOV from the center, respectively. Of note, the 2012 standard
permits for a range of image reconstruction methods. Thus, users may opt for applying
PSF and other geometric corrections to the data, which inevitable makes inter-system
comparisons more difficult.

For the measurements for the sensitivity, the count rate performance, and the associated
relative count rate error, a provision was made in NEMA NU2-2012 for elevating the
phantoms, if the vertical range of the patient table is insufficient for the required
placement of the phantoms. Furthermore, the filling tolerance was expanded and a method
for correcting the measured activity concentration for axially extended in-tube activity
was introduced. Nonetheless, here, the patient handling system had a sufficient vertical
table range and the line sources were filled to meet exactly the 700-mm in-tube filling,
and, therefore, these changes were not applicable to our measurements.

The NEMA NU2-2012 standard requires a correction for the duration of the acquisition
for the sensitivity test. With the used duration (300 s) and
¹⁸
F as isotope, this corresponds to a change of 1.6 % in outcome, which is negligible.

The count rate error evaluation was also slightly modified in the 2012 standard. The
expected value of the true count rate is gained by a fit through all data below the
peak NECR (2012 standard) as opposed to the extrapolated values from the last three
acquisitions (2007 standard). While the difference in error may be small, the exclusion
of the end slices as permitted in the NEMA NU2-2012 protocol should be followed anyway,
as these end slices often suffer from high noise levels due to the low sensitivity
of the scanner at the end of the axial FOV (Fig. 2).

Despite the modifications from the 2007 to the 2012 standard, the results for the
system sensitivity, the count rates, NECR, scatter fraction, corrections for count
losses and random measurements as well as the count rate error are in accordance with
published values for the mCT system 6] following the NEMA NU2-2007 standard.

Regarding the IQ test, the only difference between the 2007 and 2012 standard is a
reduction of emission scan time by a factor of 2. The contrast recoveries for the
“OSEM” reconstructed image were in accordance with results gained in similar systems
6], 7]. The use of PSF reconstruction with TOF information resulted in an overall improvement
in contrast recovery (CRC) (Table 3). Nevertheless, contrast recoveries for both reconstructions were substantially lower
than the values published for the mCT system 6].

This difference warrants further discussion. A possible explanation could be the differences
in reconstruction algorithms and post-filtering used in 6] and here. Previous studies have shown that by incorporating PSF modeling into the
image reconstruction, an overestimation of the CRC can be observed, in particular,
when no post-filtering is applied 25], 28], 29]. This can be attributed in part to the generation of Gibbs artifacts at the edge
of the reconstructed object, which are particularly noticeable in smaller objects
25], 29]. These artifacts can be reduced by applying post-filtering with small to modest amounts
of blurring but at the cost of lower image contrast. When imaging a slightly modified
NEMA IQ phantom on a Biograph mCT system, Armstrong and colleagues demonstrated an
overcorrection of the SUV
_max
of up to 151 % for the 13-mm sphere in case no post-filtering was applied 25]. After applying a Gaussian filter (FWHM?=?2.9 mm), this overestimation was reduced
to 138 %.

In a study by Tong et al. using the same system, the authors demonstrated a reduction
of 5–8 % of the mean CRC for the 10- to 22-mm spheres at a TBR of 4:1 after a 4-mm
Gaussian post-filter was applied to data reconstructed with 3D OSEM?+?PSF but without
TOF 30]. Based on the results of Armstrong 25], this difference would increase for the smaller spheres when TOF was included into
the reconstruction algorithm. In comparison, Marti-Climent et al. 24] obtained CRC similar to the data here, with a standard IQ phantom and a TBR of 4:1
following 3D-OSEM reconstruction with PSF and TOF and a 2-mm Gaussian post-filter.
These studies clearly demonstrate the dependence of CRC on post-filtering and provide
one explanation to any observed differences in CRC across centers using the same PET/CT
system.

Furthermore, the 50 % reduction in acquisition time, as suggested in the 2012 standards,
resulted in a higher inter-scan variance of the mCT Flow quality results compared
to the data quoted in 6]. The spread of the CRC from the three sequential measurements was rather high, especially
for the 10-mm sphere with a standard deviation of ±5.6 %.

Lesion detectability depends on the acquisition time 31]. Using a whole-body phantom with simulated hot lesions of 8 to 16 mm in diameter,
Kadrmas and colleagues demonstrated that lesion detectability is a function of both
scan time and post-filtering. This relationship was modeled from an empirical logarithmic
relationship. Using a model numerical observer method to calculate localized receiver
operator curves (LROC), the authors measured a reduction of approximately 16 % in
the area under the ROC (AROC) when the acquisition time was reduced from 240 to 120 s
per PET FOV from OSEM?+?PSF?+?TOF reconstructed data with a subsequently applied Gaussian
post-filter of about 1.3 voxels, or less. Based on this relationship, a considerable
decrease in detectability is expected when the acquisition time is reduced by a factor
of 2, which is inevitably related to image contrast. In our measurements, the acquisition
duration was 240 s instead of 600 s as used by Jakoby et al. 6]. Furthermore, the reconstruction parameter differed from the ones in 6] (3D OSEM (3i, 24 s) and 3D OSEM (2i, 21 s?+?PSF?+?TOF), both with 3-mm FWHM Gaussian
post-filtering in this study vs 2D OSEM (3i, 8 s) with 5-mm Gaussian post-filtering
and 3D OSEM (2i, 21 s?+?PSF?+?TOF) with no post-filtering in 6]). When the effects of post-filtering and acquisition duration are taken into account,
the values measured in this study show a closer conformance to those previously published
6].

Background variability values were slightly lower for the PSF?+?TOF reconstruction
compared to the published values from the mCT system, which may be related to using
a 3-mm Gaussian post-filtering. For the OSEM reconstruction, the results were slightly
worse, which can be explained by the difference in applied post-filtering (3-mm FWHM
vs 5-mm FWHM) and the difference in iterations and subsets; the larger number of subsets
in our study will lead to an enhancement of the noise in the image. Nevertheless,
the choice of the reconstruction parameters in our study for the evaluation of the
image quality was based on the NEMA NU2 standard that asks to use clinically relevant
settings. For the purpose of comparison between systems, this may not be sufficiently
accurate. The clinically used settings of each system strongly depend on local on-site
preferences. Therefore, it is suggested to define a set of parameters (iterations,
subsets, post-filtering, and algorithm) for each system for a standardized evaluation
of the image quality. This set of parameters should be defined by the vendor and be
delivered with the system, e.g., within the manual.

Finally, no significant differences in the measured IQ was observed for sequential
and for CTM acquisition modes (Table 4). The CRCs were similar in both acquisition modes. This is most probably a result
of the bed overlap of 46 % used in SS mode. Dahlbom et al. 32] stated that a similar noise reduction as with CTM is achieved with overlapping bed
positions. Nevertheless, Brasse et al. 15] reported improvements in contrast of 16 to 45 % with CTM when placing an IQ phantom
in the bed overlap region. However, this study was performed on a different system
with an axial FOV of 15.2 cm with a 4-cm bed overlap and, thus, rendering a comparison
a challenge. The only difference between the CTM and SS mode was a slightly elevated
background variability in CTM mode which was statistically significant for the IQ
phantom measurements. A similar increase was observed in the CVs (which equate to
the background variability) in the bladder of the two patients. The bladder was chosen
to test for this issue as it contains a uniform activity concentration and therefore
similar to the background compartment in the IQ phantom.

The elevated background variability is assumed to be caused by the choice of the table
speed and the data processing techniques specific to CTM acquisitions. The table speed
was estimated to cover a standard FOV in approximately the same time in CTM and SS
mode. For example, for a five-bed position acquisition, this is total axial FOV?=?axial
FOV?+?4?×?axial FOV?×?0.54?=?69.8 cm were the 0.54 is accounting for the bed overlap.
With a total scan time of 5?×?4 min, this corresponds to a table speed of 0.58 mm/s.
For an axial FOV excluding the low sensitivity regions at the axial endpoints of the
scan range (a hypothetical scan with an unlimited number of bed positions), a 4-min
acquisition per bed position corresponds to a table speed of 0.5 mm/s (this table
speed is also recommended in 12] for a similar image quality). In this study, the IQ phantom images were acquired
with a table speed of 0.6 mm/s in CTM mode. Therefore, the CTM acquisitions resulted
in a lower number of collected true events than in SS mode, which in turn resulted
in an elevated background variability 33]. Nevertheless, based on the findings of Molina-Duran and colleges, 33] not the entire increase in BG variability can be explained from the decrease in count
density. This is further supported by the findings of Braun and colleges 34] who showed increased noise in CTM acquisitions in comparison to SS acquisitions with
equivalent numbers of true events. Therefore, we suspect at least a part of the increase
is caused by noise added during the subtraction of delayed sinograms due to difficulties
in creating a mean random sinograms in CTM mode, the normalization technique 35] and/or the multiple interpolation steps implemented in the “on the fly rebinning”
of the projection data 22].

There were no visual differences in image quality of the patient scans (Fig. 6). Furthermore, no relevant difference of the SUV
_mean
of the liver as reference organ could be observed, and the comparison of the CVs of
the liver showed no clinical relevant differences. Moreover, increases in CV using
the CTM mode could not be reproduced within these VOIs and is most probably attributable
to physiological changes. This and the IQ phantom tests indicate the equivalence of
the SS and CTM acquisition mode for clinical purposes. Nevertheless, enhanced noise
at the axial edges in the SS mode acquisition was observed. This was further confirmed
by the quantitative evaluation of the CV from a corresponding region which showed
an average increase in the CV of 30 % for the SS acquisitions. This is the result
of the low sensitivity in this part of the axial FOV (Fig. 2) and is avoided using the CTM technology 32].

Nevertheless, CTM enables the use of varying table speeds for different axial regions.
This can be used to optimize the protocols, e.g., a faster feeding over the legs to
lower overall scan time or to adjust noise properties in regions of interest, e.g.,
slower feeding in the abdominal region. Also, advanced acquisition protocols like
described by Karakatsanis et al. 36] could benefit from CTM.