Versions Compared

Old Version 1

changes.mady.by.user Oleksandr Liutyi

Saved on

compared with

New Version Current

changes.mady.by.user Oleksandr Liutyi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Intro

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

it is ML (machine learning) model used in Immich to do text search on media (photo and video) files

Immich support a lot of model, than can be found at https://huggingface.co/immich-app and than configured in settings

Different models a takes different time to index media. Some model are Multilanguage, some are not.

default CLIP model is ViT-B-32__openai (not multilanguage)

Table of Contents

Time of media storage processing

Time to process 320 000 photos and 17 000 videos with size of 5.4 TB

nllb-clip-large-siglip__mrl takes even more time than ViT-B-16-SigLIP-512__webli too process media

Quality of search by some random queries

Summary

Word/PhraseXLM-Roberta-Large-Vit-B-16PlusViT-B-16-SigLIP-512__weblinllb-clip-large-siglip__mrl
anxiesty
anxietycross markcheck markcheck mark
applecheck markcheck markcheck mark
apple treecheck mark+check markcheck mark
arguing peoplecheck markcheck markcheck mark
carcross markcheck markcheck mark - (infra red 50%)
car on a parking lotcheck markcheck markcheck mark
city fogcheck markcheck markcheck mark-(some screenshots)
exchange ratecheck mark+check mark (screenshots)check mark (screenshots)
feel sadcross mark check mark buttoncheck mark buttoncross mark buttoncross mark buttoncross mark button30%check markcross mark
fogcross markcheck mark-(some screenshots)check mark-(some screenshots)
green fencecheck markcheck markcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 60%+ (IR)
honda civiccheck markcheck markcheck mark
peachcheck markcheck markcheck mark
person feel anxietycheck markcheck markcheck mark
Range Rovercheck mark+UAZcross markcheck mark+Toyota
river fogcheck markcheck markcheck mark-(50% screenshots)
serenitycross markcheck mark (lot of screenshots)check mark
TVcheck markcheck markcheck mark
відчувати радістьcross mark cross mark buttoncross mark buttoncross mark buttoncross mark buttoncross mark button 0%cross markcheck mark check mark buttoncheck mark buttoncross mark buttoncross mark buttoncross mark button
cross mark button 20%
40%
відчувати сумcross mark check mark buttoncross mark buttoncross mark buttoncross mark buttoncross mark button
cross mark button 0%
20%cross markcross mark (bags/packing)
заправка wogcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 100%/40%cross markcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 100%/70%
заправка клоcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 100%/38%cross markcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 100%/10%
зелений заборcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncheck mark button 100%cross markcheck mark check mark buttoncheck mark buttoncheck mark buttoncheck mark buttoncross mark button 80%+ (IR)
курс обміну валютcheck mark  (8 photo / screenshots )cross mark check mark buttoncross mark buttoncross mark buttoncross mark buttoncross mark button 20%check mark (screenshots)
люди в лісіcheck markcheck mark (50% w/o people)cross mark (people+ forest-)
люди які сперечаються
check markcross mark (shops)check mark
людина
люди
що відчувають розчаруванняcheck markcross mark (ad boards)check mark
мати сумнівиcross markcross mark (mothers)cross mark
машина на стоянціcheck markcross markcross mark (machinery
машинаcheck markcross mark (city streets)cross mark (machinery
персикcheck markcross markcheck mark
почуття сумуcross mark check mark buttoncross mark buttoncross mark buttoncross mark buttoncross mark button 20%cross markcross mark
радістьcross markcross markcross mark
Рейндж Роверcheck mark+UAZcross markcross mark
Розчаруванняcross markcross markcross mark
спокійcross markcross markcross mark
сумcross markcross markcross mark
сумніватисьcross markcross markcross mark
сумуватиcross markcross markcross mark
твcheck markcross markcheck mark
туман на річціcheck markcheck mark/cross mark (fog+ river-)check mark
туман у містіcheck markcheck mark/cross mark (fog+ city?)check mark
туманcheck markcheck markcheck mark
Хонда Сівікcheck mark (only logo)cross markcheck mark
Хонда Цивікcheck mark (mostly logo)cross markcheck mark
Хонда Цівікcheck mark (mostly logo)cross markcheck mark
ціна на бензинcheck mark (30% exchange rates)cross markcheck mark
яблукоcheck markcross markcheck mark
яблуняcheck markcross markcheck mark/cross mark (more like aplle)

Image found Examples

Car (English)

ModelSearch Result
XLM-Roberta-Large-Vit-B-16Plus

Image Added

ViT-B-16-SigLIP-512__webli

Image Added

nllb-clip-large-siglip__mrl

Image Added

Green Fence (English)

ModelSearch Result
XLM-Roberta-Large-Vit-B-16Plus

Image Added

ViT-B-16-SigLIP-512__webli

Image Added

nllb-clip-large-siglip__mrl

Image Added


туман на річці ("River fog" Ukrainian)

ModelSearch Result
XLM-Roberta-Large-Vit-B-16Plus

Image Added

ViT-B-16-SigLIP-512__webli

Image Added

nllb-clip-large-siglip__mrl

Image Added