Evenbytes has built a system for one of Spain’s leading buy-and-sell platforms that, in real time, turns the images of every new listing into embedding vectors and extracts their safety metadata with Gemini. The result is a vectorhouse in BigQuery: a vector store that complements keyword search with search by what the photos show, and provides detailed insight into every item. It already holds around 4 TB of data.
The project is a proof of concept. With it, the platform’s team of Data Scientists is validating the architecture and the first use cases.
Why real time
Keyword search only finds what the seller has written. With a vector for every image, the platform can compare listings by what their photos show. And if every listing is in BigQuery shortly after it is published, the app can:
- Search by image similarity: find similar items from a photo.
- Detect duplicate listings, a common sign of fraud.
- Identify prohibited content with AI-generated safety metadata (nudity, weapons…).
How it works
The listings and their photos live on AWS and are processed on Google Cloud, with Dataflow as the orchestrator:
- The metadata of each listing is published as Kafka events.
- Images are copied from Amazon S3 to Cloud Storage in real time with Storage Transfer Service, and Eventarc signals each new image.
- Dataflow consumes the Kafka events and groups each listing with the events for its images. This is the trickiest part: two streams that arrive separately have to be joined in real time.
- For each listing, Dataflow calls three services on Cloud Run: one resizes the images, another gets the vector of each image with Gemini’s embedding model, and a third extracts the safety metadata with a Gemini Flash model.
- The result is stored in BigQuery, where Vector Search enables similarity searches with SQL. Listings that cannot be processed go to a separate table for review, without stopping the stream.
Six months of history in batch
So that search works from day one, the system also processed the listings from the last six months. This load was done in batch: Storage Transfer Service brought the images and the Kafka history over from S3, and several Cloud Run Jobs generated the vectors and metadata with the same services as the real-time stream. The results were combined in BigQuery and appended to the final table.
A model chosen with cost in mind
The system processes 1.2 million images a day. At that volume, a small change in the cost per call turns into a big expense on the final bill, and decides whether the solution is viable. To extract the safety metadata, several Gemini versions were compared: the output of the latest one was considerably more expensive and drove costs up, so both the real-time and batch flows use an earlier Flash model, cheaper and good enough for the task.
What comes next
With the architecture validated, the platform can bring image search to its app and explore new use cases on the same vectors. If your company wants to apply vector search or generative AI to its data, take a look at our data engineering and cloud services, or talk to an Evenbytes expert.