Envisioning Distant Worlds: Fine-Tuning a Latent Diffusion Model with NASA's Exoplanet Data

There are some 5,500 confirmed Exoplanets beyond our solar system. Though we know these planets exist, most of them are too far away for us to know what they look like. In this paper, we develop an algorithm and a model to translate any given exoplanet’s numeric data into a text prompt that can be input into a trained latent diffusion model to generate a predictive visualization of that exoplanet. This paper describes a novel approach of translating numeric data to textual descriptors formulated from prior accepted astrophysical research. These textual descriptions are paired with photographs and artistic visualisations from NASA’s public archives to build a training set for a latent diffusion model, which can produce new visualizations of unseen distant worlds.

Workshop on Machine Learning for Creativity and Design NeurIPS 2023 Workshop.