SemINR: Semantic-Guided Implicit Neural Representation for Hyperspectral Compression

I. Introduction

Hyperspectral imaging captures the electromagnetic spectrum across hundreds of contiguous channels for each pixel, expanding beyond standard RGB capabilities to enable precise object detection, material analysis, and chemical identification^[1]. These properties make HSIs vital in fields including remote sensing, environmental monitoring, agriculture, medicine, and defense^{[2][3][4][5][6][7][8][9][10][11]}. However, acquiring hundreds of bands implies that HSIs demand orders of magnitude more storage space than conventional color images, creating massive transmission bottlenecks for satellite and airborne platforms^[12].

To mitigate these challenges, learning-based compression frameworks have shown considerable progress^[13][14][15]. Notably, in recent developments, implicit neural representations (INRs) formulate compression as a network overfitting process^[16]. Instead of saving a dense grid of values, a continuous mapping function parameterized by a neural network maps 2D spatial coordinates (x, y) directly to the corresponding channel vector [ch_0, ch_1, ⋯, ch_n]. Compression is achieved by storing the network parameters θ rather than the image itself.

Despite their success, standard INR^[17] models are strictly optimized using mean squared error (MSE), which focuses uniformly on low-level signal reconstruction. This ignores the thematic arrangement of the earth's surface. Because vanilla coordinate networks lack categorical awareness, they frequently oversmooth and blur fine-grained spectral distributions along material boundaries to satisfy global pixel averages.

To address this boundary degradation, we present a semantic-regularized coordinate network for HSI compression. Specifically, we insert an auxiliary semantic head that translates the shared internal features of the network into class label space. Using available classification ground-truth maps, we supervise this head during the continuous encoding phase via cosine-similarity-based semantic loss. Maximizing semantic cosine similarity between predicted and target class space forces the hidden layers of the model to learn a representation sensitive to categorical boundaries which minimizes inter-band spectral mixing and spatial blurring along material edges. It is noted that because of the auxiliary semantic head being completely deleted once the training concludes, the physical configuration, parameters, and storage constraints of the model during deployment remain identical to the baseline architecture. We achieve structural enhancement during decoding entirely for free.

Ⅱ. Background

For visual data such as a hyperspectral image $$ I\in {\mathbb{R}}^{H\times W\times B}$$containing H×W spatial locations across B spectral bands, the INR framework overfits a neural network Φ to that single specific data instance. The network acts as a continuous coordinate-to-value mapping function, parameterizing the image domain through the following forward pipeline:

where $$ p=\left(x,y\right)\in {\mathbb{R}}^2$$ represents the normalized continuous spatial coordinates of a queried pixel location within the image domain, typically $$ x,y\in \left[-\mathrm{1,1}\right]$$, Θ represents the network parameters and $$ \widehat{X}\in {\mathbb{R}}^B$$ denotes the predicted continuous output vector, capturing the full B-dimensional spectral intensity signature corresponding exactly to coordinate point p.

During the encoding phase, the objective is to optimize the parameter set Θ on a per-instance basis to minimize a structural reconstruction loss function, such as Mean Squared Error (MSE), over all pixel coordinates:

where N, p_i and X_i represent total number of pixels, i-th coordinates and the actual signal vector, respectively. Once optimization is complete, the compact network parameters Θ are stored as the final compressed representation of the image. To reconstruct or decode the data on demand, the saved neural network is queried across a grid of target coordinates, smoothly generating the original spectral values.

Ⅲ. Related Works

Hyperspectral image compression is generally organized into traditional coding standards and deep-learning paradigms. Traditional approaches eliminate spectral redundancies via PCA, compressing residual bands with spatial engines like JPEG or JPEG2000^[18].

Deep learning-based schemes have utilized autoencoders and variational frameworks to achieve compact latent code spaces^[13][14][15], however these require longer training and need more data for optimal learning. Recent advancements in coordinate-based deep learning have positioned implicit neural representations^[17] as a highly competitive paradigm for continuous data compression. Notably, Rezasoltani and Qureshi^[19] introduced an HSI compression framework using a multilayer perceptron optimized with periodic sinusoidal activations. Rather than preserving discrete spatial grids, their method overfits a compact neural network to map normalized 2D coordinates (x, y) directly to full multi-band spectral intensity vectors. This framework saves the network weights as a highly compressed continuous proxy that can be queried on demand to decode the original scene. While their evaluation across benchmarks like Indian Pines and Pavia University demonstrates that SIREN effectively bypasses traditional grid constraints and outperforms baseline standards like JPEG and JPEG2000, it relies strictly on a pixel-level Mean Squared Error (MSE) objective. Consequently, this standard INR approach treats all pixels as uniform signals, leaving it prone to spatial oversmoothing and high-frequency spectral cross-contamination along distinct land-cover material boundaries.

Ⅳ. Methodology

To overcome the spatial oversmoothing and inter-band spectral mixing typical of conventional coordinate networks, we propose a semantic-aware INR framework for HSI compression. The core objective of our method is to embed high-level thematic awareness directly into the network's parameter space during the encoding phase. We achieve this by introducing a temporary auxiliary semantic head consisting of a single linear layer optimized via a directional similarity constraint against the class distribution layout. By forcing the internal layers of the continuous network to align with these categorical boundaries, the model develops structural transitions. The proposed structural expansion of the baseline architecture and its corresponding optimization strategy is detailed below.

2. Multi-Objective Training Strategy

In addition to the reconstruction loss, we apply semantic loss which evaluates the mean cosine similarity between the model’s semantic output (S) and ground truth classes (T) as:

$\[ L_{semantic}=1.0-\frac{1}{N}\sum _{i=1}^N⟨S_i,T_i⟩/{‖S_i‖}_2{‖T_i‖}_2 \]$

(3)

The net objective function balancing fidelity and class structure behaves as follows:

$\[ L_{total}=L_{MSE}\left(X,\widehat{X}\right)+\alpha \cdot L_{semantic} \]$

(4)

where $$ L_{MSE}=\frac{1}{N}\sum {‖X_i-{\widehat{X}}_i‖}^2$$, and α is a scaling weight assigned as 10^-5 to guide the representations without dominating the primary visual recovery task.

Standard cross-entropy optimization makes the model map its outputs into an exact probabilistic distribution, maximizing the probability of the correct class index, this forces the model weights to continuously scale up so that the correct class score approaches positive infinity relative to other classes. This magnitude-driven pressure demands a significant portion of the network's capacity. In contrast, we optimize thematic alignment by constraining the directional correlation of the latent feature vectors within a multi-dimensional Euclidean space. By transforming ground-truth categorical labels into a series of mutually orthogonal axes via one-hot encoding, each land-cover material class is modeled as a definitive geometric direction. The semantic loss then explicitly calculates the mean cosine similarity complement across all pixel coordinates, without being affected by magnitude variance to optimize vector collinearity by forcing the network's auxiliary predictions to run parallel to the targeted ground-truth axis. Because these auxiliary predictions are derived from the shared internal parameters of the core continuous backbone, this purely directional restriction acts as a structural regulator, guiding the network to align with the material category changes, thereby regularizing the oversmoothing.

Once the network training finishes, the auxiliary module parameters are strictly excluded from storage files and inference execution ensuring that the bitstream budget remains strictly equivalent to the baseline. The overview of our framework along with encoding-decoding scheme of compression using INR is illustrated in Figure 1.

Fig. 1.

Overview of Methodology: (Top row) Overall framework where inference in right panel shows the semantic head is discarded after training. (Bottom row) Encoding-decoding flow of an HSI data cube via INR.

Ⅴ. Experiments and Results

The proposed framework was implemented using PyTorch and evaluated on three benchmark hyperspectral datasets, Indian Pines, Pavia University, and Houston13. Indian Pines comprises 145×145 pixels covering 200 spectral bands after removal of water absorption bands, containing 16 land-cover categories. Pavia University spans a 610 ×340 grid containing 103 spectral channels, with 9 classes. Houstor13 covers a 349×1905 grid with 48 spectral bands, containing 15 land-cover categories. Each network follows a SIREN backbone with periodic sinusoidal activations. We empirically selected w0 = 30.0 for the first layer and w0 = 50.0 for all subsequent hidden and output layers by tuning on a single network configuration on the Indian Pines dataset, then fixed these values as defaults across all remaining experiments. All networks were optimized using the Adam optimizer with a learning rate of 2×10⁻⁴for 8,000 iterations per scene, on a single RTX 3090 GPU. To generate the PSNR–bpppb curves in Figure 2, depth and width were varied, as this is the primary factor controlling model size (and thus bpppb) in INR-based compression, while all other hyperparameters were held constant. Reconstruction fidelity is assessed using Peak Signal-to-Noise Ratio (PSNR) measured in decibels (dB) with higher values representing better reconstruction. Spectral distortion is evaluated using the Spectral Angle Mapper (SAM), where lower values indicate better preservation of spectral curves. Storage consumption is measured via bits per pixel per band (bpppb).

Fig. 2.

PSNR and SAM vs. bpppb for our method and baseline on Indian Pines (top row), and Pavia University (middle row), and (bottom row) Houston13 datasets. The proposed method consistently improves both metrics on all three datasets.

These metrics are formally defined as follows.

where $$ \mathrm{M}\mathrm{S}\mathrm{E}=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{‖{\text{X}}_{\text{i}}-{\widehat{\text{X}}}_{\text{i}}‖}^2$$, with X_i and its reconstruction denoting the i-th ground-truth and reconstructed pixel spectra over all N pixel coordinates as in Eq. (2), and L denoting the maximum possible pixel intensity of the data or 1 for normalized data.

where bits(Θ) is the total bit length of the stored network parameters, and H, W, and B are the height, width, and number of spectral bands of the hyperspectral cube.

1. Evaluation and Discussion

To verify the efficacy of the proposed semantic-regularized optimization strategy, we evaluate our framework against the baseline using PSNR and SAM given a bpppb.

The experimental trends in Figure 2 demonstrate that incorporating a directional semantic constraint consistently outperforms the baseline on all three datasets yielding a consistent PSNR gain and BD-PSNR gain of up to +0.51dB. These gains validate our core hypothesis that forcing the shared internal layer representations to implicitly align with ground-truth categorical layouts prevent the oversmoothing, typical of purely pixel-level objectives, preserving fine-grained details. This structural preservation is further evidenced by SAM metric, which exhibits a decline when the semantic constraint is introduced. This indicates that the continuous coordinate mapping retains highly accurate, pure spectral signatures, successfully mitigating the inter-band cross-contamination along material borders. It is noted here that we define a boundary as the spatial edge of any labeled semantic region, regardless of what borders it (another labeled class or unlabeled background), since oversmoothing manifests at the edge of any structurally distinct region. Crucially, because the auxiliary semantic head is completely discarded prior to deployment, these architectural and spectral fidelity enhancements are achieved at zero additional storage or inference overhead, leaving the bits-per-pixel-per-band (bpppb) footprints strictly identical to the baseline model and successfully fulfilling the compression objective. Additionally, Tables 1 and 2 compare our method for two datasets with other standard compression schemes showing better performance for both datasets.

Table 1.

Comparison with other methods at bpppb 0.2 for Indian Pines dataset.

Table 2.

Comparison with other methods at bpppb 0.1 for Pavia University dataset.

Figure 3 provides a qualitative comparison across three hyperspectral benchmarks, Indian Pines, Pavia University, and Houston13, showing reconstructions from the baseline, and our method alongside ground truth (original data) and semantic label map for reference. For each scene, a small region spanning distinct semantic classes is magnified, allowing direct visual inspection of reconstruction fidelity. Across all three datasets, the baseline exhibits visible blurring and loss of structural detail at semantic boundaries, with fine object edges and small structures noticeably smeared into neighboring regions. In contrast, our method preserves sharper, more faithful boundaries that align more closely with the ground truth, consistent with our design goal of mitigating spatial oversmoothing via semantically meaningful regularization.

Fig. 3.

Qualitative Comparison: False color random crops from Indian Pines and Pavia University and Houston13 datasets with corresponding semantic label maps for reference and zoomed in regions for clerarer comparison.

2. Ablation Studies

For fair comparison in Table 3 we show that the improvement from the proposed method comes at a modest training-time cost: per-epoch training time increases by roughly a hundredth to a tenth of a second across datasets, reflecting the marginal added computation through the semantic head. Since the auxiliary head is discarded after training, this overhead is incurred only during encoding and has no effect on decoder size, inference time, or storage cost. We further analyze the sensitivity of our method to the semantic loss weight α using Indian Pines dataset in Figure 4, sweeping α across six orders of magnitude (10⁻⁶ to 10⁻²). Both PSNR and SAM show improvement as α decreases from 10⁻² toward 10⁻⁵, reaching an optimum at α = 10⁻⁵, before performance degrades slightly at α = 10⁻⁶. We adopt α = 10⁻⁵ for all experiments reported in this paper.

Table 3.

Comparison of performance and per-epoch training-time (seconds) with the baseline at 0.27 bpppb.

Fig. 4.

Sensitivity to the semantic loss weight α, swept from 10⁻⁶ to 10⁻². Both metrics improve as α decreases, reaching an optimum at α = 10⁻⁵ (dotted line), which is used for all reported experiments.

Ⅵ. Conclusion

In this work, we presented a semantic-aware INR framework for hyperspectral image compression. By successfully injecting categorical boundary awareness into the network’s latent parameters, we take a step towards resolving the oversmoothing issue inherent to MSE-driven architectures. Experimental evaluations conducted on the Indian Pines and Pavia University benchmarks demonstrate that our approach yields improved reconstruction fidelity while simultaneously reducing spectral distortion with zero added storage or inference computational overhead. A current limitation of our framework is its reliance on ground-truth class labels during training, which restricts direct applicability to unannotated HSI scenes. Future research will explore adapting the framework to leverage pretrained AI models for generating pseudo-labels or semantic descriptions in place of manual annotations, extending this self-regulating semantic framework to unannotated HSI datasets via self-supervised or zero-shot segmentation primitives.

Acknowledgments

This work was supported by a grant from Kyung Hee University in 2022 (KHU-20220914).

References

• Goetz, Alexander FH, et al. “Imaging spectrometry for earth remote sensing.” science 228.4704 (1985): 1147-1153. [https://doi.org/10.1126/science.228.4704.1147]
• Adam, Elhadi, Onisimo Mutanga, and Denis Rugege. “Multispectral and hyperspectral remote sensing for identification and mapping of wetland vegetation: a review.” Wetlands ecology and management 18.3 (2010): 281-296. [https://doi.org/10.1007/s11273-009-9169-z]
• Carrasco, Oscar, et al. “Hyperspectral imaging applied to medical diagnoses and food safety.” Geo-Spatial and temporal image and data exploitation III. Vol. 5097. SPIE, 2003. [https://doi.org/10.1117/12.502589]
• Edelman, Gerda J., et al. “Hyperspectral imaging for non-contact analysis of forensic traces.” Forensic science international 223.1-3 (2012): 28-39. [https://doi.org/10.1016/j.forsciint.2012.09.012]
• Fischer, Christian, and Ioanna Kakoulli. “Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications.” Studies in Conservation 51.sup1 (2006): 3-16. [https://doi.org/10.1179/sic.2006.51.Supplement-1.3]
• Ghamisi, Pedram, et al. “Advances in hyperspectral image and signal processing: A comprehensive overview of the state of the art.” IEEE Geoscience and Remote Sensing Magazine 5.4 (2018): 37-78. [https://doi.org/10.1109/MGRS.2017.2762087]
• Feng, Yao-Ze, and Da-Wen Sun. “Application of hyperspectral imaging in food safety inspection and control: a review.” Critical reviews in food science and nutrition 52.11 (2012): 1039-1058. [https://doi.org/10.1080/10408398.2011.651542]
• Govender, Megandhren, Kershani Chetty, and Hartley Bulcock. ”A review of hyperspectral remote sensing and its application in vegetation and water resource studies.” Water Sa 33.2 (2007): 145-151. [https://doi.org/10.4314/wsa.v33i2.49049]
• Gowen, Aoife A., et al.“"Hyperspectral imaging–an emerging process analytical tool for food quality and safety control.” Trends in food science & technology 18.12 (2007): 590-598. [https://doi.org/10.1016/j.tifs.2007.06.001]
• Liang, Haida. “Advances in multispectral and hyperspectral imaging for archaeology and art conservation.” Applied Physics A 106.2 (2012): 309-323. [https://doi.org/10.1007/s00339-011-6689-1]
• Schuler, Rebecca L., Paul E. Kish, and Cara A. Plese. “Preliminary observations on the ability of hyperspectral imaging to provide detection and visualization of bloodstain patterns on black fabrics.” Journal of forensic sciences 57.6 (2012): 1562-1569. [https://doi.org/10.1111/j.1556-4029.2012.02171.x]
• Dua, Yaman, Vinod Kumar, and Ravi Shankar Singh. “Comprehensive review of hyperspectral image compression algorithms.” Optical Engineering 59.9 (2020): 090902-090902. [https://doi.org/10.1117/1.OE.59.9.090902]
• Hinton, Geoffrey E., and Richard Zemel. “Autoencoders, minimum description length and Helmholtz free energy.” Advances in neural information processing systems 6 (1993). [https://doi.org/10.1117/1.OE.59.9.090902]
• Ballé, Johannes, Valero Laparra, and Eero P. Simoncelli. “End-to-end optimized image compression.” arXiv preprint arXiv:1611.01704 (2016). [https://doi.org/10.48550/arXiv.1611.01704]
• Ballé, Johannes, et al. “Variational image compression with a scale hyperprior.” arXiv preprint arXiv:1802.01436 (2018). [https://doi.org/10.48550/arXiv.1802.01436]
• Dupont, Emilien, et al. “Coin: Compression with implicit neural representations.” arXiv preprint arXiv:2103.03123 (2021). [https://doi.org/10.48550/arXiv.2103.03123]
• Sitzmann, Vincent, et al. “Implicit neural representations with periodic activation functions.” Advances in neural information processing systems 33 (2020): 7462-7473. [https://doi.org/10.48550/arXiv.2006.09661]
• Du, Qian, and James E. Fowler. “Hyperspectral image compression using JPEG2000 and principal component analysis.” IEEE Geoscience and Remote sensing letters 4.2 (2007): 201-205. [https://doi.org/10.1109/LGRS.2006.888109]
• Rezasoltani, Shima, and Faisal Z. Qureshi. “Hyperspectral image compression using implicit neural representations.” 2023 20th Conference on Robots and Vision (CRV). IEEE, 2023. [https://doi.org/10.48550/arXiv.2302.04129]