Featured image of post Image to recipe retrieval

Image to recipe retrieval

Advanced Deep learning in computer vision project

Motivation

I realised midways during this project that we had essentially just recreated the CLIP model.

Data

alt text

Contrastive learning

Math

$$ L_{bi} (a^{n=i}, b^{n=i}, b^{b\neq i}, a^{n\neq i}) = \frac{1}{B} \sum_{j=1}^B L^\prime_{bi}(i,j)\delta(i,j) $$

alt text alt text

Model

alt text

Results

We can go both forwards and backwards between images and text.

Images and recipies

Out of distribution image

alt text

Built with Hugo
Theme Stack designed by Jimmy