Performance evaluation of the Biograph mCT Flow PET/CT system according to the NEMA NU2-2012 standard

Since the first PET/CT prototype became operational in 1998 [1], PET technology has advanced significantly, through, for example, the use of fully 3D acquisition mode and reconstruction [2], the adoption of fast lutetium oxyorthosilicate (LSO) crystals [3], which permit the use of shorter coincidence timing windows, the introduction of time of flight (TOF) [4–6], and the expansion of the axial field-of-view (FOV) for increased volume sensitivity [7]. The TOF measurement of two coincidence photons allows the localization of the positron annihilation point with a spatial accuracy that depends on the system timing resolution. This information can be used to significantly improve the image signal-to-noise ratio (SNR) of the reconstructed PET images, thus, allowing for better image quality and shorter acquisition times [8, 9]. The PET component of the Biograph mCT PET/CT system (Siemens Medical Solutions USA, Inc.) is able to incorporate TOF information into the reconstruction process [6].

Traditionally, whole-body PET examinations are performed in step-and-shoot (SS) mode, introducing the patient table motion between consecutive acquisitions of adjacent bed positions [10, 11]. For this acquisition protocol, planning and scanning are restricted by the fixed size of the detector array. An alternative to the SS acquisition mode is the continuous table motion (CTM) acquisition mode, recently implemented in the Siemens Biograph mCT Flow system (Siemens Medical Solutions USA, Inc.) [12]. Using this acquisition mode, the patient is moved through the gantry while PET emission data is continuously acquired. This technique was proposed by Dahlbom and colleges in 1992 [13] and integrated in a first PET-only system in 2001 [14]. A further implementation in a LSO-based PET/CT system was tested a few years later by several groups [15–17]. The advantage of the continuous bed motion is the increase of uniformity in the sensitivity profile across the axial FOV, due to the fact that the density of lines of response (LOR) does not depend of their axial position (as in the case of SS acquisitions) [13]. Figure 1 further illustrates SS and CTM acquisitions modes.

The evaluation of positron emission tomographs requires reproducible and reliable methods to allow the comparison of different systems using accepted measurement standards. The National Electrical Manufactures Association (NEMA) has published a series of procedures to evaluate the physical performance of PET systems [18, 19]. This NEMA standard is revised periodically, and the latest update of this publication resulted in the NEMA NU2-2012 standard [19].

The purpose of this work was to evaluate the physical performance of the Biograph mCT Flow 64-4R PET/CT system according to the NEMA protocol NU 2-2012 [19]. Furthermore, the clinical PET image quality was compared for standard SS and CTM acquisitions. Finally, the results of these measurements were compared with published values from its predecessor, the Biograph mCT system [6].

Our data indicate a similar performance of the PET components of the mCT Flow and its predecessor system, the mCT [6] with the residual differences being discussed below. Image quality, as assessed by the standard IQ phantom [19] and sample oncology patients, was also similar for the CTM and SS acquisitions with the exception of slightly increased noise levels for the SS image planes at the edge of the axial FOV (Fig. 6).

Of note, there are no published performance measurements for the mCT system series based on the most recent NEMA NU2-2012 standard. The data published for the mCT [6] are based on the older NEMA NU-2007 standard [18]. The most significant difference between both standards is the change of the positions of the point source for spatial resolution measurements. NEMA NU2-2012 requires a measurement at 1-, 10-, and 20-cm radial offset position from the isocenter of the PET FOV while 2007 version requires a single measurement at 1 cm and two measurements (in x and y direction) at 10-cm offset. Furthermore, the axial placement of the off-center axial point sources was changed from one fourth of the axial FOV from the center to three eighth of the axial FOV from the center, respectively. Of note, the 2012 standard permits for a range of image reconstruction methods. Thus, users may opt for applying PSF and other geometric corrections to the data, which inevitable makes inter-system comparisons more difficult.

For the measurements for the sensitivity, the count rate performance, and the associated relative count rate error, a provision was made in NEMA NU2-2012 for elevating the phantoms, if the vertical range of the patient table is insufficient for the required placement of the phantoms. Furthermore, the filling tolerance was expanded and a method for correcting the measured activity concentration for axially extended in-tube activity was introduced. Nonetheless, here, the patient handling system had a sufficient vertical table range and the line sources were filled to meet exactly the 700-mm in-tube filling, and, therefore, these changes were not applicable to our measurements.

The NEMA NU2-2012 standard requires a correction for the duration of the acquisition for the sensitivity test. With the used duration (300 s) and ¹⁸F as isotope, this corresponds to a change of 1.6 % in outcome, which is negligible.

The count rate error evaluation was also slightly modified in the 2012 standard. The expected value of the true count rate is gained by a fit through all data below the peak NECR (2012 standard) as opposed to the extrapolated values from the last three acquisitions (2007 standard). While the difference in error may be small, the exclusion of the end slices as permitted in the NEMA NU2-2012 protocol should be followed anyway, as these end slices often suffer from high noise levels due to the low sensitivity of the scanner at the end of the axial FOV (Fig. 2).

Despite the modifications from the 2007 to the 2012 standard, the results for the system sensitivity, the count rates, NECR, scatter fraction, corrections for count losses and random measurements as well as the count rate error are in accordance with published values for the mCT system [6] following the NEMA NU2-2007 standard.

Regarding the IQ test, the only difference between the 2007 and 2012 standard is a reduction of emission scan time by a factor of 2. The contrast recoveries for the “OSEM” reconstructed image were in accordance with results gained in similar systems [6, 7]. The use of PSF reconstruction with TOF information resulted in an overall improvement in contrast recovery (CRC) (Table 3). Nevertheless, contrast recoveries for both reconstructions were substantially lower than the values published for the mCT system [6].

This difference warrants further discussion. A possible explanation could be the differences in reconstruction algorithms and post-filtering used in [6] and here. Previous studies have shown that by incorporating PSF modeling into the image reconstruction, an overestimation of the CRC can be observed, in particular, when no post-filtering is applied [25, 28, 29]. This can be attributed in part to the generation of Gibbs artifacts at the edge of the reconstructed object, which are particularly noticeable in smaller objects [25, 29]. These artifacts can be reduced by applying post-filtering with small to modest amounts of blurring but at the cost of lower image contrast. When imaging a slightly modified NEMA IQ phantom on a Biograph mCT system, Armstrong and colleagues demonstrated an overcorrection of the SUV_max of up to 151 % for the 13-mm sphere in case no post-filtering was applied [25]. After applying a Gaussian filter (FWHM = 2.9 mm), this overestimation was reduced to 138 %.

In a study by Tong et al. using the same system, the authors demonstrated a reduction of 5–8 % of the mean CRC for the 10- to 22-mm spheres at a TBR of 4:1 after a 4-mm Gaussian post-filter was applied to data reconstructed with 3D OSEM + PSF but without TOF [30]. Based on the results of Armstrong [25], this difference would increase for the smaller spheres when TOF was included into the reconstruction algorithm. In comparison, Marti-Climent et al. [24] obtained CRC similar to the data here, with a standard IQ phantom and a TBR of 4:1 following 3D-OSEM reconstruction with PSF and TOF and a 2-mm Gaussian post-filter. These studies clearly demonstrate the dependence of CRC on post-filtering and provide one explanation to any observed differences in CRC across centers using the same PET/CT system.

Furthermore, the 50 % reduction in acquisition time, as suggested in the 2012 standards, resulted in a higher inter-scan variance of the mCT Flow quality results compared to the data quoted in [6]. The spread of the CRC from the three sequential measurements was rather high, especially for the 10-mm sphere with a standard deviation of ±5.6 %.

Lesion detectability depends on the acquisition time [31]. Using a whole-body phantom with simulated hot lesions of 8 to 16 mm in diameter, Kadrmas and colleagues demonstrated that lesion detectability is a function of both scan time and post-filtering. This relationship was modeled from an empirical logarithmic relationship. Using a model numerical observer method to calculate localized receiver operator curves (LROC), the authors measured a reduction of approximately 16 % in the area under the ROC (AROC) when the acquisition time was reduced from 240 to 120 s per PET FOV from OSEM + PSF + TOF reconstructed data with a subsequently applied Gaussian post-filter of about 1.3 voxels, or less. Based on this relationship, a considerable decrease in detectability is expected when the acquisition time is reduced by a factor of 2, which is inevitably related to image contrast. In our measurements, the acquisition duration was 240 s instead of 600 s as used by Jakoby et al. [6]. Furthermore, the reconstruction parameter differed from the ones in [6] (3D OSEM (3i, 24 s) and 3D OSEM (2i, 21 s + PSF + TOF), both with 3-mm FWHM Gaussian post-filtering in this study vs 2D OSEM (3i, 8 s) with 5-mm Gaussian post-filtering and 3D OSEM (2i, 21 s + PSF + TOF) with no post-filtering in [6]). When the effects of post-filtering and acquisition duration are taken into account, the values measured in this study show a closer conformance to those previously published [6].

Background variability values were slightly lower for the PSF + TOF reconstruction compared to the published values from the mCT system, which may be related to using a 3-mm Gaussian post-filtering. For the OSEM reconstruction, the results were slightly worse, which can be explained by the difference in applied post-filtering (3-mm FWHM vs 5-mm FWHM) and the difference in iterations and subsets; the larger number of subsets in our study will lead to an enhancement of the noise in the image. Nevertheless, the choice of the reconstruction parameters in our study for the evaluation of the image quality was based on the NEMA NU2 standard that asks to use clinically relevant settings. For the purpose of comparison between systems, this may not be sufficiently accurate. The clinically used settings of each system strongly depend on local on-site preferences. Therefore, it is suggested to define a set of parameters (iterations, subsets, post-filtering, and algorithm) for each system for a standardized evaluation of the image quality. This set of parameters should be defined by the vendor and be delivered with the system, e.g., within the manual.

Finally, no significant differences in the measured IQ was observed for sequential and for CTM acquisition modes (Table 4). The CRCs were similar in both acquisition modes. This is most probably a result of the bed overlap of 46 % used in SS mode. Dahlbom et al. [32] stated that a similar noise reduction as with CTM is achieved with overlapping bed positions. Nevertheless, Brasse et al. [15] reported improvements in contrast of 16 to 45 % with CTM when placing an IQ phantom in the bed overlap region. However, this study was performed on a different system with an axial FOV of 15.2 cm with a 4-cm bed overlap and, thus, rendering a comparison a challenge. The only difference between the CTM and SS mode was a slightly elevated background variability in CTM mode which was statistically significant for the IQ phantom measurements. A similar increase was observed in the CVs (which equate to the background variability) in the bladder of the two patients. The bladder was chosen to test for this issue as it contains a uniform activity concentration and therefore similar to the background compartment in the IQ phantom.

The elevated background variability is assumed to be caused by the choice of the table speed and the data processing techniques specific to CTM acquisitions. The table speed was estimated to cover a standard FOV in approximately the same time in CTM and SS mode. For example, for a five-bed position acquisition, this is total axial FOV = axial FOV + 4 × axial FOV × 0.54 = 69.8 cm were the 0.54 is accounting for the bed overlap. With a total scan time of 5 × 4 min, this corresponds to a table speed of 0.58 mm/s. For an axial FOV excluding the low sensitivity regions at the axial endpoints of the scan range (a hypothetical scan with an unlimited number of bed positions), a 4-min acquisition per bed position corresponds to a table speed of 0.5 mm/s (this table speed is also recommended in [12] for a similar image quality). In this study, the IQ phantom images were acquired with a table speed of 0.6 mm/s in CTM mode. Therefore, the CTM acquisitions resulted in a lower number of collected true events than in SS mode, which in turn resulted in an elevated background variability [33]. Nevertheless, based on the findings of Molina-Duran and colleges, [33] not the entire increase in BG variability can be explained from the decrease in count density. This is further supported by the findings of Braun and colleges [34] who showed increased noise in CTM acquisitions in comparison to SS acquisitions with equivalent numbers of true events. Therefore, we suspect at least a part of the increase is caused by noise added during the subtraction of delayed sinograms due to difficulties in creating a mean random sinograms in CTM mode, the normalization technique [35] and/or the multiple interpolation steps implemented in the “on the fly rebinning” of the projection data [22].

There were no visual differences in image quality of the patient scans (Fig. 6). Furthermore, no relevant difference of the SUV_mean of the liver as reference organ could be observed, and the comparison of the CVs of the liver showed no clinical relevant differences. Moreover, increases in CV using the CTM mode could not be reproduced within these VOIs and is most probably attributable to physiological changes. This and the IQ phantom tests indicate the equivalence of the SS and CTM acquisition mode for clinical purposes. Nevertheless, enhanced noise at the axial edges in the SS mode acquisition was observed. This was further confirmed by the quantitative evaluation of the CV from a corresponding region which showed an average increase in the CV of 30 % for the SS acquisitions. This is the result of the low sensitivity in this part of the axial FOV (Fig. 2) and is avoided using the CTM technology [32].

Nevertheless, CTM enables the use of varying table speeds for different axial regions. This can be used to optimize the protocols, e.g., a faster feeding over the legs to lower overall scan time or to adjust noise properties in regions of interest, e.g., slower feeding in the abdominal region. Also, advanced acquisition protocols like described by Karakatsanis et al. [36] could benefit from CTM.

The NEMA performance parameters obtained for the mCT Flow system are in concordance with the published values for the predecessor system. No noticeable differences were seen in the contrast recovery as well as in the image quality obtained with sequential and CTM acquisition modes, although the background variability in the phantom measurements was slightly higher when using CTM acquisitions.

All procedures performed in studies involving human participants were approved by the regional ethics committee and were in accordance with the 1964 Helsinki declaration.