Past Issue

Over the past years, several forensic tools have been introduced to detect fake images such as noise addition, compression, filtering, and deepfake images. Wu et al. [2] proposed an end-to-end deep learning-based method for noise addition detection in online images. Hussain et al. [3] extracted discrete cosine transform (DCT) features using the DCT layer before convolution layers to provide end-to-end JPEG double compression detection. Dong et al. [4] integrated a magnifying layer to enlarge small-size images. Then, they applied a high-pass filter to transform them into multi-directional residuals for median filtering fingerprint detection.

Compared to noise addition, compression, and filtering, deepfake detection is a more practical and up-to-date technique for image forensics. Several methods have been proposed to detect deepfake images. Mara et al. [5] studied prior image forgery detectors against GAN-based image-to-image translation. Nataraj et al. [6] extracted a co-occurrence matrix on three color channels and used deep learning to detect image-toimage translations and facial expression swapping. Barni et al. [7] extended the co-occurrence-based approach to a cross-band co-occurrence matrix to explore the inconsistency between natural and deepfake images. Wang et al. [8] introduced a universal deepfake detection method in which they trained a deep learning-based classifier on a specific deepfake dataset and applied it to other deepfake datasets.

Forensic methods [2]-[8] targeted detecting natural and fake images without considering any AF attacks. AF attacks on fake images alter the fingerprints and deceive the forensic methods. Unfortunately, forensic methods detect fake images as natural images. Several AF methods, particularly GAN based AF attacks [9]-[13] have been applied in numerous fields including compression, filtering, and deepfake domains. Kim et al. [9] designed a GAN model for median filtering restoration in which they applied high-pass filtering before the discriminator to capture high-frequency information. Uddin et al. [10] applied a GAN-based AF attack on double JPEG images to deceive double JPEG detectors. Zhao et al. [11]-[12] introduced a transferable GAN-based AF attack and applied it to median filtering and deepfake images to deceive median filtering and deepfake detectors. Zhang et al. [13] proposed a similar approach as a transferable GAN-based AF attack with local perturbation generation using regularization loss.

Forensic methods focus on detecting natural and fake images while AF attacks try to conceal or alter manipulation fingerprints from fake images to deceive forensic methods. If a transferable GAN-based AF is applied to deceive state-of-the-art forensic methods, forensic methods can again be updated based on the AF datasets. This process continues in an infinite loop. In this case, forensic methods updated at time t can detect only AF attacks at time t-1. Similarly, an AF attack updated at time t+1 can deceive only forensic methods updated at time t. Considering this limitation, we propose an improved AF attack that can deceive forensic methods updated at any time.

GAN-based AF attacks can be grouped into conventional [9] and transferable [11]. Conventional GAN-based AF attack learns the underlying properties of real images to restore the fake images to make them similar to real images. On the other hand, a transferable AF attack learns the underlying properties of state-of-the-art forensic methods and generates new images close to fake images but deceives the forensic methods. Fig. 1 illustrates the working principle of two different GAN-based AF attacks. Conventional GAN-based AF attack is accomplished by training a GAN model with perceptual (P_Loss) and adversarial (A_Loss) losses defined as follows:

Fig. 1. 
GAN-based AF attacks on digital images

where H and W indicate the height and width of the real and generated images, respectively. I and I’' indicate real and fake images, respectively. G is the generator network in the GAN model.

For transferable GAN-based AF attacks, we add a loss called surrogate loss (S_Loss), defined as follows:

where S and L indicate the number of surrogate CNNs and the total number of samples, respectively. C_i is the i−th classification model of S.

Let us consider I₀ and I0' to be real and fake images captured by a camera and generated by a GAN model. Then forensic methods are trained on I₀ and I0' to detect them. Forensic methods can effectively detect real and fake images. However, if a GAN-based AF attack, particularly a transferable GAN-based AF attack, is applied on I0', a new image (I1') is generated as follows:

where GTAF is the generator model used to accomplish transferable AF attack with parameter PTAF.

Forensic methods trained on I₀ and I0' can not detect I1' as fake image. Unfortunately, it is detected as a real image. If the forensic methods are updated again using I1', then they can detect I1' as a fake image. Similarly, the AF attack is updated again using immediately updated forensic methods as an ensemble of surrogates and successfully deceives them. This process continues an infinite loop, as shown in Fig. 2.

Fig. 2. 
Updating of forensic and AF methods in an infinite loop

If forensic methods and AF attacks are updated N times to compete against each other, then AF attacks (AF_t) updated at time t can deceive only forensic methods (FR_t−1) updated at time t-1. Sometimes, AF_t fails to deceive the rest of the forensic methods.

We build an improved AF attack to deceive all forensic methods updated at any time. First, we update forensic methods N times with AF datasets, defined as follows:

where FRN is updated based on AF dataset AF_N−1 with parameters, PFR. For the proposed method, we updated forensic methods five times (N = 5). Update() function indicates training forensic methods at the time N with AF dataset from time N - 1.

Now, we have N times updated forensic methods. To build an improved AF attack (AF_Improved) as shown in Fig. 3, we randomly select one of N times updated forensic methods at each iteration and updated the AF attack accordingly defined as follows:

Fig. 3. 
Proposed improved AF attack

where i is any random integer and i = 1, 2, ..., 5. Update() function indicates training AF_Improved by randomly selecting one of N times updated forensic methods.

The proposed AF_Improved attack can successfully deceive all the forensic methods. To train AF_Improved, we use perceptual, adversarial, and surrogate losses, respectively. The perceptual and adversarial losses are the same as defined in (1) and (2). We modified the surrogate loss as follows:

where S_N is the randomly selected surrogate forensic method.

We performed the whole experiment on Linux-20.4 with an NVIDIA GeForce GTX 24576MiB GPU. We used the PyTorch framework to implement forensic and AF methods. We adopted five deep learning models, such as InceptionV3 (C1) [14], Xception (C2) [15], DenseNet121 (C3) [16], MISLNet (C4) [17], and SRNet (C5) [18] to evaluate and compare forensic and AF methods. We trained the forensic methods until 50 epochs with a learning rate of 0.001 and batch size of 32. We used stochastic gradient descent (SGD) to optimize the forensic methods. The learning rate of forensic methods dropped by 10% after every 20 epochs. We trained the AF model until 15 epochs with a learning rate of 0.0001 for both generator and discriminator networks. We set a batch size of 32. The learning rate of the AF model drops by 10% after every five epochs. We used Adam and SGD optimizers to optimize generator and discriminator models.

We conducted more practical forgery detection scenarios, such as deepfake detection to evaluate the proposed method. We used the Flickr-faces high quality (FFHQ) [19] and largescale scene understanding (LSUN) [20] datasets. The FFHQ and LSUN dataset contain human faces and different objects. We collected a total of 90,000 images for each dataset in which half of the data is real and half of the data is fake. We split the dataset into 60,000 images for training and 30,000 images for testing. We unified the spatial resolution of each dataset as 256×256.

First, we computed the baseline forensic (FR_Baseline) method to show the effectiveness of the deep learning model to detect fake images in the FFHQ dataset. We reported average detection result for C1, C2, C3, C4, and C5 in Table 1. The average detection accuracy is approximately 97.44% for the FFHQ dataset.

Then we evaluated the effectiveness of forensic and AF methods that works against each other on the FFHQ dataset. We trained forensic (FR) and AF methods five times. Forensic methods are updated using AF datasets, while AF methods are updated using FR as an ensemble of surrogates in a transferable fashion. Table 2 lists the detection results of FR (FR1, FR2, FR3, FR4, and FR5) and AF (AF1, AF2, AF3, AF4, and AF5) updated five times.

AF1 can successfully deceive FR1 because it is updated using FR1 as an ensemble of surrogates as given in bold in Table 2. AF1 can also deceive FR4 but it can not deceive FR2, FR3, and FR5. The same trend appeared for AF2, AF3, AF4, and AF5. That is, a specific AF method that is updated using a specific FR as an ensemble of surrogates can deceive only that FR or some of FR methods but not all.

To solve this problem, we built an improved AF (AF_Improved). Table 3 provides the detection results of the proposed AF_Improved method on the FFHQ dataset. The proposed AF_Improved successfully deceived all the forensic methods and achieved average detection accuracies of approximately 50% for all cases.

To show the generalizability of the proposed AF_Improved, we applied it to the LSUN dataset. Table 4 gives the average detection results of FR_Baseline on the LSUN dataset. The average detection accuracy is approximately 97.8%.

Similar to the FFHQ dataset, a specific AF method can deceive only a specific FR method used as an ensemble of surrogates, as given in Table 5 vacancy for the LSUN dataset. AF1 can only deceive FR1, but it can not deceive FR2, FR3, FR4, and FR5. In contrast, the proposed AF_Improved achieved average accuracies of approximately 50% for the LSUN dataset, as in Table 6.