Academics | The Hong Kong University of Science and Technology

A key issue with pathological image-text alignment models is the insufficient number of image-text pairs, which prevents the models from achieving stronger performance. We propose an interpolation algorithm that infers the semantic vector of the text annotation for a candidate image based on several images most similar to it, thereby supplementing the data volume for image-text alignment.