 Original research
 Open access
 Published:
Kinetic modelinformed deep learning for multiplexed PET image separation
EJNMMI Physics volume 11, Article number: 56 (2024)
Abstract
Background
Multiplexed positron emission tomography (mPET) imaging can measure physiological and pathological information from different tracers simultaneously in a single scan. Separation of the multiplexed PET signals within a single PET scan is challenging due to the fact that each tracer gives rise to indistinguishable 511 keV photon pairs, and thus no unique energy information for differentiating the source of each photon pair.
Methods
Recently, many applications of deep learning for mPET image separation have been concentrated on pure datadriven methods, e.g., training a neural network to separate mPET images into singletracer dynamic/static images. These methods use overparameterized networks with only a very weak inductive prior. In this work, we improve the inductive prior of the deep network by incorporating a general kinetic model based on spectral analysis. The model is incorporated, along with deep networks, into an unrolled imagespace version of an iterative fully 4D PET reconstruction algorithm.
Results
The performance of the proposed method was evaluated on a simulated brain image dataset for dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET PET image separation. The results demonstrate that the proposed method can achieve separation performance comparable to that obtained with singletracer imaging. In addition, the proposed method outperformed the modelbased separation methods (the conventional voxelwise multitracer compartment modeling method (vMTCM) and the imagespace dualtracer version of the fully 4D PET image reconstruction algorithm (ISF4D)), as well as a pure datadriven separation [using a convolutional encoderdecoder (CED)], with fewer training examples.
Conclusions
This work proposes a kinetic modelinformed unrolled deep learning method for mPET image separation. In simulation studies, the method proved able to outperform both the conventional vMTCM method and a pure datadriven CED with less training data.
Introduction
Positron emission tomography (PET) is a medical imaging technique that enables direct and quantitative observations of tissue radioactivity over time in vivo. The use of different PET radiotracers facilitates the measurement of various aspects of tumour metabolism for diagnosis, characterization, and monitoring of the response to therapy, as well as the measurement of neurotransmitter release and receptor densities in brain studies [1, 2]. Due to the radioactive and biological halflives involved, it is usually not possible to perform more than one tracer acquisition in a single patient visit. To obtain information on, for example, glucose metabolism with [\(^{18}\)F]FDG (radioactive halflife=109.8 mins) and protein synthesis with [\(^{11}\)C]MET (radioactive halflife=20.4 mins), two scans may be conducted separately [3, 4] (scanning [\(^{11}\)C]MET first and then allowing a long time delay before scanning [\(^{18}\)F]FDG). This extends the scanning time for the patient and only works for comparable simple combinations of radiotracers. To fully exploit the range of tracers now available, it would be useful to scan more than one tracer simultaneously.
Multiplexed PET (mPET) imaging allows for the synchronization of observing physiological and pathological information from multiple tracers in a single scan, reducing the total examination time and providing complementary information for the characterization of disease. In mPET imaging, multiple tracers are injected with a time offset anywhere from zero to several minutes, and the dynamic/static imaging measurements of each individual tracer are then recovered in a single scan. However, the separation of the mPET signals within a single PET scan is challenging due to the fact that each tracer gives rise to indistinguishable 511 keV photon pairs, and thus no unique energy information for differentiating the source of each photon pair.
Research on mPET imaging has been ongoing for the past two decades. The separation of the mPET signals was initially proposed based on the significant differences in the radioactive decay of each tracer [5], which was further investigated by Verharghe et al. [6] and Figueiras et al. [7]. Another widely studied method is based on multitracer compartment modeling (MTCM) which was first proposed by Koeppe et al. [8] for estimating the kinetic parameters of \(^{11}\)Clabelled tracers. Ikoma et al. [9] then applied the MTCM method for dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]flumazenil separation. The feasibility of the MTCM method in mPET imaging, exploring various dualtracer combinations through the analysis of simulated PET data and preclinical PET data, were investigated in [1, 2, 10,11,12,13,14]. Black et al. further extended the MTCM method from dualtracer imaging to tripletracer imaging [15]. The MTCM method is highly sensitive to noise and is prone to fall into local minima, even when the noise level is low, due to the nonlinearity of the fitting problem. In order to improve the separation performance of the MTCM method, Zhang et al. [16] proposed a reformulation of the conventional multitracer compartment model using fewer parameters by separating the linear part from the nonlinear part [16]. On this basis, the separable parameter space technique was incorporated with PET image reconstruction to reduce the influence of the noise in the fitting process for dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET PET image separation [17]. However, the aforementioned MTCMbased methods assume that the arterial input function (AIF) of each tracer is known. This implies that, in practice, separating the measured (dualtracer) AIFs is necessary prior to MTCM. Kudomi et al. [18] and Taheri et al. [19] introduced noninvasive modelbased methods to separate dualtracer AIFs. However, these methods are highly dependent on the shape of the curves. Thus, there remains a lack of an accurate noninvasive method for AIF separation. Verhaeghe and Reader [20] proposed using a general kinetic model based on spectral analysis [21] to separate [\(^{18}\)F]FDG and multiple [\(^{15}\)O] water injections without the need for any AIFs to be known. Despite its advantages of not requiring any AIFs for separation, the solution to the fitting problem in this method is nonunique.
Other modelbased methods not limited to the compartment model have been investigated for mPET signal separation, such as principal component analysis [1], generalised factor analysis [22], the referenceregion model [23] and basis pursuit [24]. However, these methods assume a long timedelayed injection protocol and that the AIF of each tracer is known. A machine learningbased method based on the recurrent extreme gradient boosting algorithm was proposed to separate both the dualtracer AIF and the dualtracer timeactivity curves (TACs) in a region of interest (ROI), allowing a shorter delay between the injection of two tracers than that of the MTCM method [25]. The mPET signal can also be separated based on the case where an additional highenergy \(\gamma\) photon is emitted along with a positron for one of the two tracers, and thus the different isotopes can be discriminated in the measured data [26,27,28,29]. However, these methods are only valid for some tracer combinations: a purely positron emitting isotope and a positron\(\gamma\) emitting isotope, e.g., \(^{18}\)F and \(^{60}\)Cu, limiting the selection of tracers.
The huge recent success of using deep learning (DL) for PET image processing and reconstruction [30, 31] has seen an increasing interest in applying similar strategies to mPET image separation. Many applications of DL for mPET image separation have been concentrated on pure datadriven methods, e.g., training a neural network on separating mPET images into singletracer dynamic/static images from either reconstructed dynamic mPET images (postseparation) [32,33,34,35,36,37,38] or mPET sinogram sequences (direct separation) [39,40,41]. These methods use overparameterized networks with only a very weak inductive prior and require large quantities of training data.
In this paper, we strengthen the inductive prior of the deep separation network by embedding a general kinetic model for dynamic dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET PET imaging without explicitly knowing the AIFs, ensuring network outputs are consistent with known kinetic models. In particular, we incorporate a general kinetic model based on spectral analysis into a neural network by unrolling a regularised implementation of an imagespace version of an iterative fully 4D PET image reconstruction algorithm [42].
Methods
Spectral analysis kinetic model
The modeling of the tracer activity concentrations (i.e., the TACs) at a voxel or within an ROI in an image involves the convolution of the measured AIF with a modeldependent tissue unit impulse response, where the number of compartments is predefined. In our work, we proposed use of a more general kinetic model that makes no a priori assumption regarding to the number of compartments necessary to model the data. Based on spectral analysis [21], the singletracer activity concentrations (i.e., singletracer TACs) for all voxels in an image can be described using a linear model [20, 42]
where \({\textbf{c}} \in {\mathbb {R}}^{JN\times 1}_{\ge 0}\) contains the spectral coefficients for each of the N predetermined exponential temporal basis functions for all J voxels, \({\textbf{B}} \in {\mathbb {R}}^{JT\times JN}\) contains the N exponential temporal basis functions sampled at T time points (as a repeated \(T\times N\) submatrix placed at diagonally consecutive positions along \({\textbf{B}}\), the contents of which are shown in Fig. 1(a) for FDG and (b) for MET, respectively), and \({\textbf{H}}\in {\mathbb {R}}^{JT\times JT}_{\ge 0}\) convolves each of the time functions resulting from \(\textbf{Bc}\) with a global generating function \({\textbf{h}}\in {\mathbb {R}}^{T\times 1}_{\ge 0}\) (shared across all voxels). Note that model (1) allows for the shapes of the generating function \({\textbf{h}}\) to encompass an infinite number of possibilities, with the AIF being a special case of \({\textbf{h}}\). The linear model (1) can be extended to the dualtracer ([\(^{18}\)F]FDG+[\(^{11}\)C]MET) TACs for all voxels
where \({\textbf{B}}_{\rm{FDG}}\in {\mathbb{R}}^{JT\times JE}\) contains the E exponential temporal basis function for FDG, \({\textbf{B}}_{\rm{MET}}\in {\mathbb{R}}^{JT\times JF}\) contains the F exponential temporal basis functions for MET, \({\textbf{B}}_{\rm{D}}\) is a block diagonal matrix with \({\textbf{B}}_{\rm{FDG}}\) and \({\textbf{B}}_{\rm{MET}}\) on the diagonal, \({\mathbf{H}}_{\rm{D}} = \begin{bmatrix} {\textbf{H}}_{\rm{FDG}} {\text{ }} {\textbf{H}}_{\rm{MET}} \end{bmatrix}\), and \({\mathbf{c}}_{\rm{D}} = \begin{bmatrix} {\textbf{c}}_{\rm{FDG}} {\text{ }}{\textbf{c}}_{\rm{MET}} \end{bmatrix}^T\).
Fully 4D inspired dualtracer PET image separation
Integrating the linear model (4) for dualtracer voxelwise TACs with the fully 4D algorithm proposed for PET image reconstruction [20, 42] and considering regularisation on the coefficients \({\textbf{c}}_{\rm{D}}\), leads to finding a solution to the following variational minimization problem
where \({\textbf{m}}\in {{\mathbb {R}}^{JT\times 1}}\) is the measured dualtracer activity images (e.g., voxelwise dualtracer TACs), \({\mathcal {R}}: {\mathbb {R}}^{J(E+F)\times 1}\rightarrow {\mathbb {R}}\) is the regulariser and \(\lambda >0\) is the regularisation hyperparameter. Note that in \({\mathbf{H}}_{\rm{D}} = \begin{bmatrix} {\textbf{H}}_{\rm{FDG}} {\text{ }} {\textbf{H}}_{\rm{MET}} \end{bmatrix}\), matrices \({\textbf{H}}_{\rm{FDG}}\) and \({\textbf{H}}_{\rm{MET}}\) contain the tracerspecific global generating functions \({\textbf{h}}_{\rm{FDG}}\) and \({\textbf{h}}_{\rm{MET}}\), respectively. This means that \(\mathbf{H}_{\rm{D}}\) depends on \(\mathbf{h}_{\rm{D}} = \begin{bmatrix} {\textbf{h}}_{\rm{FDG}} {\text{ }} {\textbf{h}}_{\rm{MET}} \end{bmatrix}\), and thus, we denote \(\mathbf{h}_{\rm{D}}\) as one of the optimization variables in (5). The minimization problem (5) can be solved by alternating minimization for each variable while keeping the other fixed as suggested in [20, 42], resulting in the following iterative scheme (index k)
If \({\mathcal {R}}\) are closed proper convex, the cupdate can be solved using the proximal gradient descent algorithm [43], resulting in the following iteration (index i)
where \(\mu ^{i} > 0\) is a step size in the ith iteration, \(\textbf{prox}(\cdot )\) is the proximal operator w.r.t. \({\mathcal {R}}\), and the subscript “\(+\)” indicates the nonnegativity constraint on the spectral coefficients \({\textbf{c}}_{\rm{D}}\). By choosing \(\mu ^i = {\textbf{c}}_{\rm{D}}^i/({({\textbf{H}}_{\rm{D}}^k\mathbf{B}_{\rm{D}})^T {\textbf{1}}})\), the gradient descent step (6) can be simplified as an MLEM update [44]
The hupdate can be solved using the conventional MLEM algorithm w.r.t. the generating function \(\mathbf{h}_{\rm{D}}\) (index j)
where the nonnegativity constraint on the generating functions \({\textbf{h}}_{\rm{D}}\) was implicitly bundled with the MLEM update. The tracerspecific spectral coefficients and generating functions were then extracted from the final estimated \({\hat{{\textbf{c}}}}_{\rm{D}}\) and \({\hat{{\textbf{h}}}}_{\rm{D}}\), respectively, and further used to recover the singletracer TACs of each voxel in an image via the singletracer linear model (1). We note that the proposed alternating update scheme for solving (5) without regularisation, i.e., \({\mathcal {R}}=0\), is equivalent to an imagespace dualtracer version of the fully 4D PET image reconstruction algorithm (ISF4D), described in Appendix A. In this case, the cupdate only contains the MLEM step w.r.t. the spectral coefficients \({\textbf{c}}_{\rm{D}}\) [42].
We note further that model (5) leads to a nonsmooth, biconvex optimization problem in \({\textbf{c}}_{\rm{D}}\) and \({\textbf{h}}_{\rm{D}}\) (assuming \({\mathcal {R}}\) are closed proper convex), i.e., it is convex in each of the variables \({\textbf{c}}_{\rm{D}}\) and \({\textbf{h}}_{\rm{D}}\) when the other is considered fixed, but nonconvex as a function of both variables. Therefore, the global convergence of the proposed alternating update scheme is not guaranteed, and the solution of the minimization problem is nonunique (even without regularisation [20]), due to their use in \({\hat{{\textbf{h}}}_{\rm{D}}\textbf{B}_{\rm{D}}{\hat{\textbf{c}}}_{\rm{D}}}\) minimizing the objective function in (5).
Kinetic modelinformed deep network for dualtracer PET image separation
The vast majority of the DLbased methods for mPET image separation are purely datadriven with a very weak inductive prior. With the recent success of unrolled deep networks for PET reconstruction [30, 31, 45,46,47,48,49], we improved the inductive prior of the deep network by unrolling the alternating update scheme presented in Section "Fully 4D inspired dual‑tracer PET image separation", and replaced the proximal step in each iteration with a trained blockdependent convolutional neural network (CNN) to form the proposed kinetic modelinformed deep network for dualtracer PET image separation. To sufficiently accelerate the separation process, we use one update with Eqs. (9) and (8), and one update with Eq. (10), leading to the following update scheme
where \(\Gamma _{\theta _k}: {\mathbb {R}}^{J(E+F)\times 1}_{\ge 0}\rightarrow {\mathbb {R}}^{J(E+F)\times 1}_{\ge 0}\) corresponds to a trained CNN in the kth iteration (or the kth block in the kinetic modelinformed unrolled deep network) with blockdependent trainable parameters \(\theta _k\).
The architecture of the proposed kinetic modelinformed unrolled deep network based on the update scheme (11) with K iteration blocks is demonstrated in Fig. 2. In the kth iteration block, the spectral coefficients \({\textbf{c}}^k_{\rm{D}}\) were first updated via an MLEM step. The updated coefficients were then fed into a trained blockdependent CNN, i.e., \(\Gamma _{\theta _k}\), to perform the proximal update (or regularisation). The network \(\Gamma _{\theta _k}\) consists of two branches, one for each tracer. Each branch consists of the repeated application of several multichannel \(3\times 3\) 2D convolutional layers, each followed by a batch normalization (BN) and a parametric rectified linear unit (PReLU), as well as a \(1\times 1\) 2D convolutional layer at the end. In addition, we activated the output layer using a ReLU to enforce the nonnegativity constraint on the spectral coefficients. The generating functions \({\textbf{h}}^k_{\rm{D}}\) were then updated via a onestep MLEM with use of the output from \(\Gamma _{\theta _k}\), i.e., \({\textbf{c}}^{k+1}_{\rm{D}}\). After the Kth iteration block, the singletracer linear model (1) recovers the separated singletracer activity images from the final estimated decay coefficients \({\textbf{c}}^{K}_{\rm{D}}\) and generating functions \({\textbf{h}}^{K}_{\rm{D}}\).
Simulation and validation
Simulation setup
We simulated the tracer activity images of [\(^{18}\)F]FDG and [\(^{11}\)C]MET using a 2D BrainWeb phantom dataset. 5 noncontiguous slices were selected from each of the 20 3D BrainWeb phantoms^{Footnote 1} [50]. 100 slices were obtained in total to form the 2D BrainWeb phantom dataset. Each of the 2D brain phantom contains white matter (WM) and grey matter (GM), and is of resolution \(128\times 128\) with a voxel size of \(2.602\times 2.602\) mm\(^2\). A tumour (TM) with diameter ranging between 12 and 18 mm and randomly located was added to each 2D phantom. To generate the groundtruth parametric maps of each tracer, the kinetic parameters for a given region were sampled from a Gaussian distribution with mean values based on those from the literature [17], as listed in Table 1, and with the coefficient of variation equal to 0.1 (absolute values were taken after sampling). Randomised structures were then introduced to the white matter and grey matter regions to simulate heterogeneous variation within the whole brain using the BrainWeb library [51]. An example of the simulated groundtruth parametric maps \(K_1\) for FDG and MET are shown in Fig. 3. The AIFs of each tracer were generated along with the shape of the AIFs from the literature [17] using Feng’s input function model [52]. Feng’s input function parameters were also modeled as Gaussian variables with the coefficient of variation equal to 0.1 (the absolute values were taken after sampling), to simulate the population variation in the dataset [37, 40].
The singletracer voxelwise TACs (activity images) were generated from the simulated groundtruth parametric maps using the irreversible twotissue compartment model for FDG, and reversible twotissue compartment model for MET, respectively, following a similar approach to that described in [37]. Note that fractional blood volume \(V_B\) was also included in the simulation. Radioactive decay of each tracer was also modeled in the simulation, which requires the integration of decay constants \(\lambda _{[{}^{18}\text {F}]} = \log (2)/109.8\) min\(^{1}\) and \(\lambda _{[{}^{11}\text {C}]} = \log (2)/20.4\) min\(^{1}\). The singletracer activity images then were summed up together to form the groundtruth dualtracer activity images. We followed a dynamic dualtracer PET scanning protocol proposed in [17, 37] for [\(^{18}\)F]FDG+[\(^{11}\)C]MET, which was conducted for 50 mins, with [\(^{11}\)C]MET injected 5 mins after [\(^{18}\)F]FDG injection. The dynamic PET scan (dualtracer or singletracer) was divided into 27 time frames: \(4\times 0.25\) mins, \(2\times 0.5\) mins, \(3\times 1\) mins, \(4\times 0.25\) mins, \(2\times 0.5\) mins, \(3\times 1\) mins, \(2\times 2\) mins, \(2\times 3\) mins, \(4\times 5\) mins, \(1\times 10\) mins. An example of the simulated AIFs and dualtracer (DT) TACs without decay correction within each ROIs is shown in Fig. 4(a).
For reconstructions of the simulated data, we modeled a GE Discovery ST PETCT scanner with system sensitivity \({\sim }\)2 cps/kBq in 2D mode [53]. Groundtruth dualtracer and singletracer activity images were first forward projected to generate noisefree sinogram data based on the aforementioned dynamic scan protocol using a precalculated system matrix. An attenuation map was simulated with a constant linear attenuation coefficient assgined in the whole brain. We follow a similar setup as described in [37, 40], where a \(20\%\) uniform background was included to account for scatter and random events. A scaling factor was applied to this projected data in order to generate the meancount sinogram, prior to the introduction of Poisson noise into each sinogram bin. This scaling factor was chosen so as to obtain datasets containing a predetermined expectation of the total counts in each sinogram, where the expectation of total counts was modelled based on the system sensitivity of the scanner (see Fig. 4(b) for example). Dynamic images for both dualtracer and singletracer acquisitions were then reconstructed frame by frame using the MLEM algorithm (initialised by uniform images) with 128 iterations without postsmoothing. Reconstructed images were framelength corrected reconstructed tracer activity images.^{Footnote 2}
Implementation details and reference methods
Four separation methods were compared in this study: (1) the conventional voxelwise MTCM (vMTCM [11]) method, (2) the imagespace dualtracer version of the fully 4D method (ISF4D; see earlier Section "Fully 4D inspired dual‑tracer PET image separation" and Appendix A), (3) the puredatadriven convolutional encoderdecoder (CED) [37], and (4) the proposed kineticinformed unrolled deep network.
The vMTCM method estimates the singletracer activity images by fitting the dualtracer kinetic model to the measured dualtracer activity images (dynamic dualtracer noisy MLEM reconstruction) using the voxelwise weighted least squares (VWLS) with a known AIF of each tracer. In our study, we focused on the validation of the algorithm and thus assumed the AIFs of each tracer to be known in vMTCM. The weighting factors used were the time frame durations in order to compensate for nonuniform temporal sampling [12]. Note that decay correction can be accounted for after separation because the proportion of each tracer (and hence its decay correction factor) is unknown beforehand [13]. Therefore, radioactive decay is not taken into account in the weighting factors. The vMTCM method first estimates the parametric images of each tracer in the separation process. The estimated singletracer parametric images (with the tracerspecific AIFs) were then used for recovering the singletracer activity images. The trustregionreflective algorithm was used to perform the VWLS fitting. Stopping criteria were set such that the optimization procedure terminates when the relative error of the estimate was less than \(1\times 10^{8}\) or the maximum iteration number (1600 iterations) was achieved. The initial values of the kinetic parameters were set to be 0.01 for all voxels. The lower bounds of each parameter were set to be \(1\times 10^{11}\) and the upper bounds of \(V_B\), \(K_1\) and \(k_2k_4\) were set to be [1, 5, 2, 1, 1, 1], respectively.
The ISF4D method separates the dualtracer activity images (dynamic dualtracer noisy MLEM reconstruction) without the use of AIFs. This is achieved by estimating the spectral coefficients and the generating functions in an alternating manner. A set of predetermined exponential temporal basis functions for [\(^{18}\)F]FDG and [\(^{11}\)C]MET are shown in Fig. 1. The number of the exponential temporal basis function were chosen to be \(E = F = 25\), and the decay constants were logarithnmically spaced between 0.001 and 5 min\(^{1}\) as suggested in [20, 42], including the special case of the decay constant equal to zero (more details on the impact of the number of exponentials for representing TACs in spectral analysis can be found in [42]). Both the spectral coefficients and the generating functions were initialised to vectors containing ones [20]. An initial MLEM 16 updates (indexed by i) was used to solve the cupdate (with \({\mathcal {R}} = 0\)), followed by a repeated cycle of: 4 MLEM updates for the cupdate (\({\mathcal {R}} = 0\), indexed by i) and 1 MLEM update for the hupdate (indexed by j) as suggested in [42]. A total 1600 iterations were used in the outer iteration (indexed by k). Note that, in each MLEM update, we resampled the measured dualtracer activity images (dualtracer voxelwise TACs) and the estimated generating functions of each tracer into finer samples to accommodate the convolution with each of the time functions resulting from \(\textbf{Bc}\).
Both the pure datadriven CED and the proposed kinetic modelinformed unrolled deep network can separate the dualtracer dynamic images without the use of AIFs. For network training, dynamic dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET noisy MLEM reconstruction was used as the network input and dynamic singletracer noisy MLEM reconstructions were used as training labels. The mean square error (MSE) loss is applied to the activity images of each tracer, and their sum is used as the loss function for network training, which is given as
where \(\mathring{C}^{(n)}\) denotes the separated singletracer activity images of the nth tracer in the network output, \({\hat{C}}^{(n)}\) is the label singletracer activity images, and \(N_S\) is the total number of training pairs. The proposed deep network contains \(K=10\) iteration blocks (due to GPU memory limitations) and each blockdependent CNN contains 3 convolution layers (\(3\times 3\) 2D convolution). The same exponential temporal basis functions as ISF4D were used in the proposed deep network. Both the spectral coefficients and the generating functions were again initialised to vectors containing ones. For a fair comparison, we adjusted the number of channels in the CED presented in [37] to match the number of trainable parameters in the proposed deep network. The number of trainable parameters for both networks is around 1.2 million. All network training was performed in the same manner. We note that the proposed deep network was trained endtoend. For network training, we used 80 simulated data examples generated from 16 BrainWeb phantoms. An additional 10 examples, generated from two other BrainWeb phantoms, were allocated for validation. The 10 examples used for testing were generated from the remaining 2 BrainWeb phantoms. The network parameters were initialised using the Xavier initialisation. The Adam algorithm was used with a learning rate \(1\times 10^{4}\) and a batch size equal to 8 for network training. All networks were trained for a maximum of 1200 epochs with early stopping when there were no improvements in the validation metrics. The training and evaluation steps were implemented in PyTorch, on an PC with a NVIDIA GeForce RTX 3090 GPU.
Evaluation metrics
The quality of the separated singletracer activity images was evaluated over \(R=20\) different noise realisations using the voxellevel normalized root mean square error (NRMSE)
with the bias and the standard deviation (SD) given by [54]
where \(\Omega\) is the whole brain region, \(\bar{x}_l = \frac{1}{R}\sum _{r=1}^R x^r_l\) is the mean value for voxel l in the separated image \({\varvec{x}}\), found by taking the average of the R noise realisations, and \({\varvec{x}}^{\rm{Ref}}\) is a reference image for error calculation. In our study, singletracer noisefree (NF) MLEM reconstructions (initialised by uniform images, with 128 iterations) was used as the reference image in all cases.
TACs extracted from the tumour ROIs in the separated singletracer activity images were also used to validate the separation performance at the ROI level. The NRMSE of the TACs was calculated to evaluate the tumour ROITAC quantification
with the bias and SD given by
where \(c^{\rm{Ref}}\) is the singletracer ROI TACs extracted from the tumour regions in the dynamic singletracer noisefree MLEM and \(\bar{c} = \frac{1}{R}\sum _{r=1}^R c^r\) denotes the mean of R noise realisations, and \(c^r\) is the tumour ROI TACs with the mean ROI uptake in each time frame in the rth realisation.
Results
Separated image quality
Figure 5 shows the reference activity images (see column 1) and the separated images by using different methods for frame 14 (an early 30s frame at 1.25 min after the [\(^{11}\)C]MET injection) in a test example. The singletracer noisy MLEM suffers from visuallyevident high noise. The modelbased methods, vMTCM and ISF4D, enable the suppression of noise compared to the results obtained from the singletracer noisy MLEM. By introducing a 3 mm fullwidth at half maximum (FWHM) Gaussian kernel on the separated images obtained from vMTCM, denoted as vMTCM+FWHM (3 mm), we obtained images with reduced noise but excessive smoothing. The DLbased methods (the CED and the proposed kinetic modelinformed unrolled deep network) achieved better image quality than the modelbased methods. By incorporating the general kinetic model into the unrolled deep network, the proposed method enhances the recovery of the detailed structures in the grey matter and white matter regions compared to the pure datadriven CED (see the zoomedin patches shown in Fig. 5 for comparison).
Figure 6 shows the tradeoff between the bias and SD of different methods for frame 14, frame 24 (a later 5min frame at 27.5 min after [\(^{11}\)C]MET injection), and the separated singletracer activity image sequences (all time frames were considered), over the entire test dataset (10 test examples). For the early time frame (frame 14), on average, the singletracer noisy MLEM achieved \({\sim }25\%\) bias and \({\sim }110\%\) SD for FDG, and \({\sim }25\%\) bias and \({\sim }100\%\) SD for MET, respectively (see Fig. 6(row 1)). While fitting the singletracer compartment model (with the simulated tracerspecific AIFs) to the singletracer noisy MLEM voxel by voxel, denoted as vSTCM (with frame durations as weighting factors), the bias and SD were reduced to \({\sim }5\%\) and \({\sim }20\%\) for FDG, and \({\sim }11\%\) bias and \({\sim }30\%\) for MET, which were highlighted in yellow (also see the black clusters for a comparison). This observation shows that fitting the singletracer compartment model to the singletracer noisy MLEM helps to improve the image quality in the early time frame. However, the situation worsens when dualtracer separation is considered. By fitting the combined compartment model to the dualtracer noisy MLEM, vMTCM failed to achieve similar bias and SD levels as those of vSTCM. The main reason is that vMTCM requires a long delayed time interval between two tracer injections, and it is susceptible to noise and prone to fall into local minima. Postsmoothing with a 3 mm FWHM Gaussian kernel can effectively reduce the SD levels, albeit at the cost of an increase in bias compared to the results obtained from vMTCM. Another modelbased method, ISF4D, achieved a similar SD level but exhibited higher bias compared to vMTCM due to the nonunique solution of the optimization problem. For the DLbased methods, the pure datadriven CED reduced the SD values by \({\sim }40\%\) in the early time frame compared to the modelbased methods, and the SD values are also lower than those of vSTCM. The CED uses spatiotemporal information for separation while the modelbased methods only use temporal information for separation. In addition, the CED with MSE loss learns to output the mean of all plausible noisy explanations when it is trained using noisy labels, and thus the CED implicitly learns to denoise the separated images [37]. However, no significant improvement in bias was found using the CED for FDG, and only a small improvement was observed for MET, compared to vMTCM. The proposed kinetic modelinformed unrolled deep network outperformed the aforementioned methods with additional debiasing compared to the CED. It delivered approximately a \(2\%\) higher bias for FDG and a \(4\%\) lower bias for MET in the early time frame, compared to the results obtained from singletracer noisy data using vSTCM.
Similar results were found in the later time frame, where vMTCM+FWHM (3 mm) and the DLbased methods achieved much lower SD compared to vMTCM and ISF4D (see Fig. 6(row 2)). The bias level of the CED is similar for MET and improved for FDG, compared to those of the vMTCM method. Although the proposed method further reduced the bias compared to the CED, it is still higher than that of vSTCM. This indicates that the separation task remains challenging for our proposed method in the later time frames (relatively highcount compared to the early time frame), especially given that only 80 data examples were used for network training.
The overall separation performance of the singletracer activity image sequences is shown in Fig. 6(row 3), where it again shows that the DLbased method achieved much lower SD compared to the modelbased methods. The proposed deep network outperformed the CED by further reducing the bias and SD values. On average, the proposed method achieved a similar bias level and lower SD for both FDG and MET, compared to those that would be obtained with singletracer data using vSTCM.
Impact of the number of training examples
We retrained the pure datadriven CED and the proposed kineticmodel informed unrolled deep network using different sample sizes to assess the impact of the number of training examples for dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET PET image separation. Figure 7 shows that the NRMSE values of the separated singletracer activity image sequences decrease as the number of training examples increases (from 8 to 80) for both the CED and the proposed kinetic modelinformed deep network. On average, the CED dualtracer separation using 80 training examples achieved \({\sim }18\%\) and \({\sim }17\%\) NRMSE for FDG and MET, respectively. Compared to the CED, the proposed deep network reduced the NRMSE values by \({\sim }8\%\) and \({\sim }7\%\) using 80 training examples. When fewer training examples are considered, the proposed deep network still achieves lower NRMSE compared to the pure datadriven CED. This indicates that incorporating the kinetic model into the unrolled deep network forms a stronger inductive prior, leading to a better separation performance with less training data.
Parametric map estimation
Parametric imaging was also performed for the separated singletracer activity images. We note that vMTCM, by definition, directly estimates the parametric images of each tracer in the separation process, eliminating the need for postestimation. To have a fair comparison, the vSTCM method was used to estimate the parametric maps from the separated singletracer activity images. For vMTCM+FWHM (3 mm), postsmoothing was applied to the separated images after vMTCM. Therefore, the parametric images for each tracer were reestimated by applying vSTCM to the separated postsmooth images. The parametric maps recovered from the singletracer noisefree MLEM were used as reference images.
Figure 8 shows a test example of the separated parametric images \(K_1\) for [\(^{18}\)F]FDG and [\(^{11}\)C]MET (also see Fig. 3 for the groundtruth \(K_1\) images). Compared to the singletracer noisy MLEM, the separated parametric image \(K_1\) for both FDG and MET obtained by vMTCM and ISF4D are much noisier. The \(K_1\) images estimated using vMTCM with a 3 mm FWHM postsmoothing were much less noisy but excessively smoothed. The CED substantially reduced the noise but also led to the loss of some detailed structures within the gray and white matters. In comparison, the \(K_1\) images obtained by the proposed kinetic modelinformed unrolled deep network retrieved more details in the whole brain regions for both FDG and MET (see the zoomedin patches shown in Fig. 8 for comparison).
Figure 9 further shows a quantitative comparison of different methods for the separated FDG and MET \(K_1\) images. On average, \({\sim }90\%\) and \({\sim }100\%\) NRMSE were found in the estimated \(K_1\) images using vSTCM. Compared to vSTCM, vMTCM and ISF4D achieved higher NRMSE for both FDG and MET, which is consistent with the visual observation in Fig. 8. The vMTCM+FWHM (3 mm) method effectively reduced the NRMSE values, mainly due to the denoising effect of postsmoothing. Notably, the CED outperformed the two modelbased methods and ST MLEM with \({\sim }20\%\) NRMSE for both FDG and MET. Again, the CED with MSE loss learns to output the mean when trained using noisy labels, implicitly denoising the separated images. As a consequence, we obtained better estimates of \(K_1\). The proposed method further reduced the NRMSE values by \({\sim {4}}\%\) (FDG) and \({\sim {10}}\%\) (MET). This is because embedding the kinetic model into the unrolled deep network offers a stronger inductive prior to the deep learning framework compared to the pure datadriven CED, leading to further debiasing and resulting in a superior image separation performance (see Subsection "Separated image quality" for more details). As a consequence, the proposed deep network achieved an even better estimate of \(K_1\).
Next, a test example of the separated parametric images \(K_i=K_1k_3/(k_2+k_3)\) for [\(^{18}\)F]FDG and [\(^{11}\)C]MET is shown in Fig. 10. In comparison, the \(K_i\) images obtained by the proposed kinetic modelinformed unrolled deep network retrieved more details in the whole brain regions for both FDG and MET compared to other methods (see the zoomedin patches shown in Fig. 10 for comparison). Figure 11 further shows the NRMSE values of different methods for the separated FDG and MET \(K_i\) images over 10 test examples. Similar to the results for \(K_1\), the proposed deep network achieved lower NRMSE values compared to the modelbased methods and the pure datadriven CED.
Separation of tumour ROI TACs
A test example of the tumour ROI TACs (averaged over 20 different noise realisations) extracted from the separated singletracer activity images using different separation methods is demonstrated in Fig. 12. Tumour ROI TACs extracted from singletracer noisefree MLEM were used as references (see the dashed lines in Fig. 12). When considering the results obtained from the singletracer noisy MLEM, the tumour ROI TACs closely matched the reference TACs. However, in dualtracer separation, the tumour ROI TACs obtained from the vMTCM separation failed to align with the references accurately due to the noise in the images, the short timedelayed interval between two tracer injections, and the local minimum solution [17, 37]. While the introduction of a 3 mm FWHM Gaussian kernel on the separated images obtained from vMTCM can reduce the noise level at the voxel level (see earlier Subsection "Separated image quality"), the performance of the separated tumour ROI TACs was notably worse compared to those obtained using vMTCM. This is primarily because postsmoothing introduces bias on the separated images. The performance of the separated tumour ROI TACs using ISF4D was even more compromised due to the nonunique solution of the optimization problem. While the pure datadriven CED method successfully reduced noise in the separated activity images at the voxel level, it leads to nonsmooth separated tumour ROI TACs. The separated tumour ROI TACs obtained with the proposed deep network are closer to the reference TACs compared to those obtained by the other methods. In addition, the sum of the separated tumour ROI TACs with the proposed method (highlighted by a solid black line) aligns with the reference dualtracer tumour ROI TAC (highlighted in a dashed black line), indicating that the proposed method effectively separates the dualtracer signals while preserving data consistency.
Figure 13 shows, on average, the singletracer noisy MLEM achieved \({\sim }12\%\) and \({\sim }7\%\) \(\text {NRMSE}_\text {TAC}^\text {TM}\) for FDG and MET, respectively. The \(\text {NRMSE}_\text {TAC}^\text {TM}\) values of vMTCM were slightly higher than those of the singletracer noisy MLEM, resulting in \({\sim }14\%\) for FDG and \({\sim }10\%\) for MET. Higher \(\text {NRMSE}_\text {TAC}^\text {TM}\) values were found with vMTCM+FWHM (3 mm), ISF4D and the CED when compared to the vMTCM method. The proposed kinetic modelinformed unrolled deep network achieved lower \(\text {NRMSE}_\text {TAC}^\text {TM}\) in comparison to the two modelbased methods and the CED. However, it is noteworthy that the singletracer noisy MLEM still exhibited better performance with lower \(\text {NRMSE}_\text {TAC}^\text {TM}\) (by comparing the median) for both FDG (\(\sim\)0.7%) and MET (\(\sim\)2%), compared to the proposed method.
Discussion
This work proposes a kinetic modelinformed DLbased method that incorporates a general kinetic model based on spectral analysis with the unrolled imagespace version of the fully 4D method to separate dualtracer [\(^{18}\)F]FDG+[\(^{11}\)C]MET activity images into singletracer activity images without the need for any AIFs to be known or supplied to the method. Compared to the modelbased methods (vMTCM and ISF4D), the proposed kinetic modelinformed method can substantially reduce both the bias and SD of the separated activity images, as shown in Fig. 6. Compared to the pure datadriven CED method [37], which trains an overparameterized neural network with a weak inductive prior for separation, the proposed method embeds a stronger inductive prior, i.e., the kinetic model, into the unrolled deep network for separation, resulting in even lower bias and SD with less training data. In addition, the bias level of the proposed method is comparable to the results obtained from singletracer data using vSTCM in the separated activity image sequences for both FDG and MET.
To further evaluate the separation performance, the parametric maps \(K_1\) and \(K_i\) estimated from the separated singletracer activity images using different methods was investigated. On average, vMTCM and ISF4D achieved over \(100\%\) voxellevel NRMSE on the separated \(K_1\) and \(K_i\) images. The vMTCM+FWHM (3 mm) method can reduce NRMSE but leads to oversmooth parametric images. Much lower NRMSE values were found for the DLbased methods (\({<}30\%\) for both FDG and MET in \(K_1\) and \(K_i\)), as shown in Fig. 9 and Fig. 11. Compared to the pure datadriven CED, the proposed method further reduced the NRMSE of the separated \(K_1\) images by \({\sim }4\%\) for FDG and \({\sim }10\%\) for MET, (and by \({\sim }10\%\) for both FDG and MET in \(K_i\)), respectively.
ROIlevel analysis was also performed by examining the separated singletracer tumour ROI TACs to further assess the performance of the proposed method. On average, the vMTCM method achieved lower \(\text {NRMSE}_\text {TAC}^\text {TM}\) compared to those obtained by using vMTCM+FWHM (3 mm), ISF4D and the CED. The proposed method reduced the \(\text {NRMSE}_\text {TAC}^\text {TM}\) by \({\sim }2\%\) for FDG, and \({\sim }4\%\) for MET, compared to the vMTCM method. However, the \(\text {NRMSE}_\text {TAC}^\text {TM}\) of the proposed method failed to get to a similar level to that of the singletracer noisy MLEM, indicating that the separation capability of the proposed approach for tumour ROI TACs is still in need of further improvement.
Although the proposed method achieved a better separation performance compared to the modelbased methods and the pure datadriven CED for mPET image separation at both the voxel and ROI levels, five main limitations should still be borne in mind. (1) The impacts of (i) the relative and absolute injected dose between the two tracers, (ii) the order of tracer injection, and (iii) the scanning protocol, were not investigated in this present study. (2) The current simulation study only focuses on separation for 2D PET (with a simple simulation of data acquisition) while conventional PET imaging is usually conducted in 3D. The proposed model would need to be investigated and validated rigorously on datasets obtained from physical phantoms with real data acquisition or synthetic data generated from real patient data to assess feasibility for practical application. (3) Even when 3D PET imaging is considered, lowcount levels encountered in short time frames can lead to extremely noisy MLEM reconstructions, which makes the mPET image separation task even more challenging. With new techniques for enhancing the quality of reconstructed PET images and the arrival of new scanners, e.g., totalbody PET, the higherquality image data that includes different kinetics in various organs (since all organs would be in the field of view) could provide distinct features to enhance mPET image separation. (4) Feng’s input function model was employed merely to simulate the AIFs for each tracer in our current study. However, many AIFs cannot be fitted to this model, and alternative models have been proposed in [55,56,57]. The impact of different types of AIF would need to be further investigated. (5) The current study exclusively focuses on the dualtracer separation of brain images using the tracer combination of [\(^{18}\)F]FDG and [\(^{11}\)C]MET. Additional investigations are required to explore the application of the proposed framework to other mPET separation tasks, such as the separation of mPET myocardial images using different tracer combinations [38].
It is worth noting that combining reconstruction with separation has been shown to enhance the separation performance in mPET imaging, as discussed in [17, 40]. While, in the present study, the separation process was conducted in the image domain, extending the proposed method to a direct mPET reconstructionseparation framework is achievable. This can be accomplished by integrating the system matrix into the proposed unrolled deep network, building upon the regularised version of the original fully 4D PET reconstruction algorithm [42].
The proposed method encounters a notable challenge in generalisation, particularly when applied to different tracer combinations or unseen images that are outside the training distribution. Another direction of potential future work could focus on performing finetuning on the pretrained model using the unseen dataset in a selfsupervised manner [58] to enhance its generalisation capability.
Conclusion
We have developed a kinetic modelinformed unrolled DLbased method for mPET image separation. Distinct from the pure datadriven deep learning methods, the proposed method improves the inductive prior by embedding the general kinetic model into an unrolled neural network based on a regularised imagespace version of a fully 4D PET reconstruction algorithm. The proposed method has been developed to improve the quality of the separated singletracer activity images and validated using simulated brain imaging data. In comparison to the pure datadriven CED, our proposed method can further reduce the bias and SD while requiring less data for training. It also achieved separation performance comparable to that obtained using singletracer data, highlighting its potential for mPET image separation.
Availability of data and materials
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Notes
While we do model decay, we do not seek to include decay correction on the separated images in this present work.
Abbreviations
 mPET:

Multiplexed positron emission tomography
 TAC:

Timeactivity curve
 AIF:

Arterial input function
 ROI:

Region of interest
 DL:

Deep learning
 CED:

Convolutional encoder–decoder
 CNN:

Convolutional neural network
 vMTMC:

Voxelwise multitracer compartment modeling method
 vSTMC:

Voxelwise singletracer compartment modeling method
 ISF4D:

Imagespace dualtracer version of the fully 4D PET image reconstruction algorithm
 BN:

Batch normalization
 WM:

White matter
 GM:

Grey matter
 TM:

Tumour
 NF:

Noisefree
 ST:

Singletracer
 DT:

Dualtracer
 MSE:

Mean square error
 VWLS:

Voxelwise weighted least squares
 FWHM:

Fullwidth half maximum
 NRMSE:

Normalised root mean square error
 SD:

Standard deviation
References
Kadrmas DJ, Rust TC. Feasibility of rapid multitracer PET tumor imaging. IEEE Trans Nucl Sci. 2005;52(5):1341–7.
Black NF, McJames S, Rust TC, Kadrmas DJ. Evaluation of rapid dualtracer ^{62}CuPTSM+^{62}CuATSM PET in dogs with spontaneously occurring tumors. Phys Med Biol. 2007;53(1):217.
Cook GJ. Normal variants, artifacts and interpretative pitfalls in PET imaging with 18fluoro2deoxyglucose and carbon11 methionine. Eur J Nucl Med. 1999;26:1363–79.
Chung JK, Kim Y, Kim SK, Lee Y, Paek S, Yeo J, Jeong J, Lee D, Jung H, Lee M. Usefulness of ^{11}Cmethionine PET in the evaluation of brain lesions that are hypo or isometabolic on ^{18}FFDG PET. Eur J Nucl Med Mol Imaging. 2002;29:176–82.
Huang SC, Carson RE, Hoffman EJ, Kuhl DE, Phelps ME. An investigation of a doubletracer technique for positron computerized tomography. J Nucl Med. 1982;23(9):816–22.
Verhaeghe J, D’Asseler Y, Staelens S, Lemahieu I. Noise properties of simultaneous dual tracer PET imaging. IEEE Nuclear Sci Symp Conf Record. 2005;5:2611–4.
Figueiras FP, Jiménez X, Pareto D, Gómez V, Llop J, Herance R, Rojas S, Gispert JD. Simultaneous dualtracer PET imaging of the rat brain and its application in the study of cerebral ischemia. Mol Imag Biol. 2011;13:500–10.
Koeppe RA, Raffel DM, Snyder SE, Ficaro EP, Kilbourn MR, Kuhl DE. Dual[^{11}C] tracer singleacquisition positron emission tomography studies. J Cereb Blood Flow Metab. 2001;21(12):1480–92.
Ikoma Y, Toyama H, Suhara T. Simultaneous quantification of two brain functions with dual tracer injection in PET dynamic study. Int Congr Ser. 2004;1265:74–8.
Nishizawa S, Kuwabara H, Ueno M, Shimono T, Toyoda H, Konishi J. Doubleinjection FDG method to measure cerebral glucose metabolism twice in a single procedure. Ann Nucl Med. 2001;15:203–7.
Rust TC, Kadrmas DJ. Rapid dualtracer PTSM+ATSM PET imaging of tumour blood flow and hypoxia: a simulation study. Phys Med Biol. 2005;51(1):61.
Rust TC, DiBella EVR, McGann CJ, Christian PE, Hoffman JM, Kadrmas DJ. Rapid dualinjection singlescan ^{13}Nammonia PET for quantification of rest and stress myocardial blood flows. Phys Med Biol. 2006;51(20):5347.
Kadrmas DJ, Rust TC, Hoffman JM. Singlescan dualtracer FLT+FDG PET tumor characterization. Phys Med Biol. 2013;58(3):429.
Guo J, Guo N, Lang L, Kiesewetter DO, Xie Q, Li Q, Eden HS, Niu G. Chen: ^{18}Falfatide II and ^{18}FFDG dualtracer dynamic PET for parametric, early prediction of tumor response to therapy. J Nucl Med. 2014;55(1):154–60.
Black NF, McJames S, Kadrmas DJ. Rapid multitracer PET tumor imaging with ^{18}FFDG and secondary shorterlived tracers. IEEE Trans Nucl Sci. 2009;56(5):2750–8.
Zhang JL, Morey AM, Kadrmas DJ. Application of separable parameter space techniques to multitracer PET compartment modeling. Phys Med Biol. 2016;61(3):1238.
Cheng X, Li Z, Liu Z, Navab N, Huang SC, Keller U, Ziegler SI, Shi K. Direct parametric image reconstruction in reduced parameter space for rapid multitracer PET imaging. IEEE Trans Med Imaging. 2015;34(7):1498–512.
Kudomi N, Watabe H, Hayashi T, Iida H. Separation of input function for rapid measurement of quantitative CMRO_{2} and CBF in a single PET scan with a dual tracer administration method. Phys Med Biol. 2007;52(7):1893.
Taheri N, Le Crom B, Bouillot C, Chérel M, Costes N, Gouard S, MarionneauLambot S, Merlin T, Visvikis D, Stute S, Carlier T. Design of a generic method for single dualtracer PET imaging acquisition in clinical routine. Phys Med Biol. 2023;68(8): 085016.
Verhaeghe J, Reader AJ. Simultaneous water activation and glucose metabolic rate imaging with PET. Phys Med Biol. 2013;58(3):393.
Cunningham VJ, Jones T. Spectral analysis of dynamic PET studies. J Cereb Blood Flow Metab. 1993;13(1):15–23.
El Fakhri G, Trott CM, Sitek A, Bonab A, Alpert NM. Dualtracer PET using generalized factor analysis of dynamic sequences. Mol Imag Biol. 2013;15:666–74.
Joshi AD, Koeppe RA, Fessier JA, Kilbourn MR. Signal separation and parameter estimation in noninvasive dualtracer PET scans using referenceregion approaches. J Cereb Blood Flow Metab. 2009;29(7):1346–57.
Bell C, Puttick S, Rose S, Smith J, Thomas P, Dowson N. Design and utilisation of protocols to characterise dynamic PET uptake of two tracers using basis pursuit. Phys Med Biol. 2017;62(12):4897.
Ding W, Yu J, Zheng C, Fu P, Huang Q, Feng DD, Yang Z, Wahl RL, Zhou Y. Machine learningbased noninvasive quantification of singleimaging session dualtracer ^{18}FFDG and ^{68}GaDOTATATE dynamic PETCT in oncology. IEEE Trans Med Imaging. 2021;41(2):347–59.
Andreyev A, Celler A. Dualisotope PET using positrongamma emitters. Phys Med Biol. 2017;56(14):4539.
Fukuchi T, Okauchi T, Shigeta M, Yamamoto S, Watanabe Y, Enomoto S. Positron emission tomography with additional γray detectors for multipletracer imaging. Med Phys. 2017;44(6):2257–66.
Fukuchi T, Shigeta M, Haba H, Mori D, Yokokita T, Komori Y, Yamamoto S, Watanabe Y. Image reconstruction method for dualisotope positron emission tomography. J Instrum. 2021;16(1):01035.
Pratt EC, LopezMontes A, Volpe A, Crowley MJ, Carter LM, Mittal V, Pillarsetty N, Ponomarev V, Udías JM, Grimm J, Herraiz JL. Simultaneous quantitative imaging of two PET radiotracers via the detection of positronelectron annihilation and prompt gamma emissions. Nat Biomed Eng. 2023;7(8):1028–39.
Reader AJ, Corda G, Mehranian A, CostaLuis C, Ellis S, Schnabel JA. Deep learning for PET image reconstruction. IEEE Trans Radiat Plasma Med Sci. 2020;5(1):1–25.
Reader AJ, Pan B. AI for PET image reconstruction. Br J Radiol. 2023;96(1150):20230292.
Ruan D, Liu H. Separation of a mixture of simultaneous dualtracer PET signals: a datadriven approach. IEEE Trans Nucl Sci. 2017;64(9):2588–97.
Xu J, Liu H. Deep learningbased separation of a mixture of dualtracer singleacquisition PET signals with equal halflives: a simulation study. IEEE Tran Radiat Med Sci. 2019;3(6):649–59.
Qing M, Wan Y, Huang W, Xu Y, Liu H. Separation of dualtracer PET signals using a deep stacking network. Nucl Instrum Methods Phys Res, Sect A. 2021;1013: 165681.
Tong WJ, Liu H. Temporal informationguided dynamic dualtracer PET signal separation network. Med Phys. 2022;49(7):4585–98.
Lian D, Li Y, Liu H. Spatiotemporal attention constrained deep learning framework for dualtracer PET imaging. In: Annual conference on medical image understanding and analysis, 2022;87–100.
Pan B, Marsden PK, Reader AJ. Dualtracer PET image separation by deep learning: a simulation study. Appl Sci. 2023;13(7):4089.
Pan B, Marsden PK, Reader AJ. Deep learned tripletracer multiplexed PET myocardial image separation. Front Nuclear Med. 2024;4:1379647.
Xu J, Liu H. Threedimensional convolutional neural networks for simultaneous dualtracer PET imaging. Phys Med Biol. 2019;64(18): 185016.
Zeng F, Fang J, Muhashi A, Liu H. Direct reconstruction for simultaneous dualtracer PET imaging based on multitask learning. EJNMMI Res. 2023;13(1):7.
Wang C, Fang J, Liu H, Gong K. Direct reconstruction and separation for tripletracer PET imaging based on threedimensional encoderdecoder network. Med Imag 2023 Phys Med Imag. 2023;2463:585–93.
Reader AJ, Matthews JC, Sureau FC, Comtat C, Trébossen R, Buvat I. Fully 4D image reconstruction by estimation of an input function and spectral coefficients. 2007 IEEE nuclear science symposium conference record. 2007;5:3260–7.
Parikh N, Boyd S. Proximal algorithms. Found Trends Optim. 2014;1(3):127–239.
Shepp LA, Vardi Y. Maximum likelihood reconstruction for emission tomography. IEEE Trans Med Imaging. 1982;1(2):113–22.
Gong K, Wu D, Kim K, Yang J, El Fakhri G, Seo Y, Li Q. EMnet: an unrolled deep neural network for PET image reconstruction. Med Imag 2019 Phys Med Imag. 2019;10948:1203–8.
Gong K, Wu D, Kim K, Yang J, Sun T, El Fakhri YG, Seo, Li Q. MAPEMNet: an unrolled neural network for fully 3D PET image reconstruction. 15th International meeting on fully threedimensional image reconstruction in radiology and nuclear medicine 2019;11072, 109–113.
Mehranian A, Reader AJ. Modelbased deep learning PET image reconstruction using forwardbackward splitting expectationmaximization. IEEE Trans Radiat Plasma Med Sci. 2020;5(1):54–64.
Lim H, Chun IY, Dewaraja YK, Fessler JA. Improved lowcount quantitative PET reconstruction with an iterative neural network. IEEE Trans Med Imaging. 2020;39(11):3512–22.
Guazzo A, ColarietiTosti M. Learned primal dual reconstruction for PET. J Imaging. 2021;7(12):248.
Cocosco CA. Brainweb: Online interface to a 3D MRI simulated brain database. 1997.
CostaLuis C. Brainwebbased multimodal models of 20 normal brains, 2020. https://zenodo.org/records/4032893
Feng D, Wong KP, Wu CM, Siu WC. A technique for extracting physiological parameters and the required input function simultaneously from PET image measurements: theory and simulation study. IEEE Trans Inf Technol Biomed. 1997;1(4):243–54.
Mawlawi O, Podoloff DA, Kohlmyer S, Williams JJ, Stearns CW, Culp RF, Macapinlac H. Performance characteristics of a newly developed PET/CT scanner using nema standards in 2D and 3D modes. IEEE Trans Inf Technol Biomed. 2004;45(10):1734–01742.
Reader AJ, Ellis S. Bootstrapoptimised regularised image reconstruction for emission tomography. IEEE Trans Med Imaging. 2020;39(6):2163–75.
Tofts PS, Kermode AG. Measurement of the bloodbrain barrier permeability and leakage space using dynamic MR imaging. 1. fundamental concepts. Magn Reson Med. 1991;17(2):357–67.
Parker GJ, Roberts C, Macdonald A, Buonaccorsi GA, Cheung S, Buckley DL, Jackson A, Watson Y, Davies K, Jayson GC. Experimentallyderived functional form for a populationaveraged hightemporalresolution arterial input function for dynamic contrastenhanced MRI. Magn Reson Med Offic J Int Soc Magn Reson Med. 2006;56(5):993–1000.
Vriens D, GeusOei LF, Oyen WJ, Visser EP. A curvefitting approach to estimate the arterial plasma input function for the assessment of glucose metabolic rate and response to treatment. J Nucl Med. 2009;50(12):1933–9.
Reader AJ. Selfsupervised and supervised deep learning for PET image reconstruction. AIP Conference Proceedings, 2024;3061(1).
Acknowledgements
This work was supported by the EPSRC program for Next Generation Molecular Imaging and Therapy with Radionuclides [EP/S032789/1, ‘MITHRAS’] and by core funding from the Wellcome/EPSRC Centre for Medical Engineering [WT 203148/Z/16/Z]. For the purpose of Open Access, the Author has applied a Creative Commons Attribution (CC BY) public copyright licence to any Accepted Manuscript version arising from the submission, in accordance with King’s College London’s Rights Retention policy. The data that supports the findings of this study are available within the article with simulation data from https://brainweb.bic.mni.mcgill.ca/brainweb/.
Funding
This work was supported by the EPSRC program for Next Generation Molecular Imaging and Therapy with Radionuclides [EP/S032789/1, ‘MITHRAS’] and by core funding from the Wellcome/EPSRC Centre for Medical Engineering [WT 203148/Z/16/Z].
Author information
Authors and Affiliations
Contributions
BP and AJR conceived the idea of the study and developed the algorithm. BP implemented the algorithm, contributed to the study design, performed the data analysis, statistical analysis, and drafted the original manuscript. AJR and PKM supervised the conduct of the study and mainly revised the manuscript critically on intellectual content. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors have no relevant Conflict of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A imagespace version of the fully 4D algorithm for dualtracer PET image separation (ISF4D)
Appendix A imagespace version of the fully 4D algorithm for dualtracer PET image separation (ISF4D)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Pan, B., Marsden, P.K. & Reader, A.J. Kinetic modelinformed deep learning for multiplexed PET image separation. EJNMMI Phys 11, 56 (2024). https://doi.org/10.1186/s40658024006600
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40658024006600