Self-supervised learning
Temporal Hyperspectral Representation Learning for Biotechnology
Overview
Temporal Hyperspectral Representation Learning for Biotechnology is a self-supervised framework for learning robust representations from multi-temporal hyperspectral imagery. The framework combines a masked autoencoder with factorized spectral, spatial, and temporal attention, FiLM-based metadata conditioning, and adversarial domain prediction to reduce the influence of collection geometry and acquisition effects in the latent space. Data-processing pipelines span greenhouse, UAS, and EO-1 Hyperion satellite observations, including the 4.5 TB ChronoEarth dataset of approximately 500,000 image chips across up to 235 timestamps over 17 years. The research investigates whether latent-space trajectories can distinguish healthy from stressed vegetation while suppressing observational and environmental variability; representation quality is evaluated through downstream tasks such as segmentation.
A closer look
Healthy and infected latent trajectories
This research leverages large volumes of available remote sensing data and self-supervised learning to distinguish healthy plant trajectories from disease- or infection-induced trajectories in the latent representation space.
Spatial-spectral-temporal masked-autoencoder architecture
Factorized spectral, spatial, and temporal attention, combined with FiLM metadata conditioning and adversarial domain learning, encourages latent representations to be invariant to collection geometry and acquisition effects while retaining the spectral state of observed materials.
Learning from missing observations
Masking demonstration for self-supervised hyperspectral pretraining.
Spatial reconstruction
Original images, masked inputs, and reconstructed spatial structure in selected examples from the ChronoEarth-492k dataset.
Spectral reconstruction
Target and predicted spectra at masked pixels. The highlighted spectral interval is withheld from the surrounding context in the ChronoEarth-492k dataset.