ViSenze Blog

How to evaluate visual search technology providers for an e-commerce business

Posted by ViSenze 02-November-2015

A Guide To Avoid Disappointment

With visual search as a gleaming new, promising and innovative technology, we understand that evaluating different providers can be an overwhelming task.

Having helped top e-commerce players in Asia such as Flipkart, Patsnap, Caratlane, Zalora, and Lazada implement visual search technology to their platforms, we understand the kind of questions you have when it comes to assessing potential vendors to partner with.

This guide serves to plug that gap so you can make a well-informed decision, giving you the confidence and peace of mind that you and your users will be in for a good experience.


1. Evaluate what is under the hood


Check if the technology is human-assisted or fully automated

A human-assisted system means that humans come in to fine-tune the search results delivered by technology when machines cannot deliver the most relevant results on their own.

This used to be a quite popular approach until recent years when visual technology developed enough to return relevant results in a fraction of a second.

Nowadays, there are solutions built with machine intelligence, which can return relevant search results to image queries without the need for humans to step in.

When it comes to the humans vs. machines battle, it is needless to say that a human-assisted technology will have speed and scalability issues. Not to mention that for large volumes, costs grow proportionally when humans need to tweak the results. On the other hand, fully automated systems do not face such problems.

So the first thing we recommend you to check when choosing a visual search technology vendor is if their solutions are based on R&D and data science and are using machine learning (more specifically deep learning) or they rely on human assistance.

Here are a few aspects you should investigate about your shortlisted vendors:

  • Is the solution built with deep learning and computer vision algorithms?

  • Do they have scientists or an R&D team to develop the technology?

  • What is the speed at which they display relevant results? (an indicator that they don’t use human assistance is a speed of 1-2 seconds)

  • Has the vendor proven that they can scale for large databases and multiple concurrent users with other customers?

Ask if the vendor has proprietary technology

It is best to go with the provider who invented the visual search technology in-house instead of one that acquired it through other means (resellers or those using open-source technology).

the vendor developed the technology, they are likely to be the true experts in the field. This means that they are capable of working fast with you to fine-tune and further develop the algorithms if needed be. It also indicates that they can be a reliable provider for the long term, and so you will not need to worry about them losing access to the technology.


2. Take a test drive

To start with, a serious vendor would not be shy to have a publicly available demo to play with.

Explore a demo to see if the technology actually works

Demos are a great opportunity to do an almost effortless preliminary assessment by gauging the search speed and evaluating the relevancy of results.

You need to understand though that the quality of results relies not only on the technology itself, but also on the database that is provided for demo purposes. This database is usually limited to one vertical so make sure your image search queries fit the category.

Also, do not expect matching results as most databases will probably not have a matching item. However, you can at least asses if the results are similar.

Start a trial

If you want to see how it works on your own database, the best option is for you to start a trial.

As soon as you upload or sync your database (or at least a portion of it), you can use a few images as search queries and see how long the technology takes to present accurate matching or similar results.

Here is a more systematic approach to testing:

  1. Build up a sample set of test queries, ideally including images of items that you have in the database, as well as similar items.

  2. Divide them into categories so that you can measure how well the visual search technology works on each product category.

  3. Get your search quality assurance team to score the technology:
  • What was the percentage of searches that returned the matching item as the first result? (in the case where it was actually in the database)
  • What was the percentage of searches that returned highly similar or relevant items as the top 3 results?

Do note that vendors are usually specialized in specific verticals. This being said, unless your vertical is one of them, you should know that the technology might need some tweaking. Thus, check with them if they are able to fine-tune the technology for your vertical.

Test for scalability, latency, and reliability

Assuming that you are satisfied after a few manual search tests, you should further evaluate the technology’s ability to scale without compromising on other parameters like speed and accuracy.

This is especially important in the mobile era, where the search speed offered should perform well even in conditions with low bandwidth and intermittent connectivity. Not forgetting that consumers expect a smooth user experience regardless of the situation. A study done by SOASTA indicates that in terms of conversions, the performance sweet spot for mobile pages loading time is 2.4 seconds.

Some things to look at include:

  • What is the average speed of search?

  • Does the search speed slow down with more concurrent searches?

  • How does the vendor’s technology deal with a database of a hundred million images as opposed to a thousand images?

  • What is the average uptime of the vendor’s service - is it 99% or 99.99%? (There is a world of a difference between the two when you break the percentage down into hours.)

  • Has the vendor’s technology handle peak periods before? Such as holiday sales or the Black Friday weekend, where e-commerce traffic can go up to four or five times more than usual.

But you shouldn’t stop here. You should stress test the solution by placing it under the highest level of stress you expect the technology to be used at when it is launched.

An engineering team can set up bots to run multiple parallel searches or simply a high volume of searches in a limited time period. See if the technology can keep up with the speed without compromising on the quality of results.


3. Examine the setup process

Ensure ease of implementation

You might want to check the following:

  • Does the visual search technology provide open APIs and SDKs?

  • Do they have mature and clear API and SDK documentation to smoothen the process?

  • Will implementing the technology be disruptive to your existing integrations?

  • How does it affect elements of your existing systems? Is it compatible with existing operations?

  • Does your vendor have a programmatic way of ingesting your catalog data and images?

  • Does their API allow you to push data across easily?

  • How long did the vendor take to onboard and integrate with their previous customers?

Some providers offer comprehensive, step-by-step guides on how you can integrate their publicly-available visual search API on your end, while others are severely lacking in this area.

Ensure ease of maintenance

In addition to ensuring that the implementation process will not be an issue, also question if maintaining your database would be hassle-free.

After integration, your inventory and product catalog would need to be synced and updated in real time. So instead of having to do this manually, it will be best if your vendor provides a Data API for a fully automated real time sync of your database.

This allows you to push data easily, allowing your product inventory updates, price fluctuations, and out-of-stock items to be synced across in real time without manual intervention.


4. Carry out due diligence

Check the technology provider’s credentials

When evaluating visual search technology vendors, look into their background and track record, and conduct your due diligence. Ask these questions:

  • How long have they been in business?

  • Are they growing or downsizing?

  • What existing applications and use cases have they have built?

It is also important to look for credibility by doing reference checks so ask the vendors for references in your industry. Happy customers will gladly share their experiences with you.

When speaking with their customers, be sure to ask if the provider has addressed their specific needs as a company doing business in their specific industry, and whether the customer support team is responsive and knowledgeable.

Inquire about quality of support

Find out if technical support or account support would be provided by the visual search technology provider you have in mind. This also contributes to how the vendor will support you after the sale.

Besides giving training and support on obtaining the necessary skill and experience sets for technology implementation, look for other ways the vendor can value add to your partnership.

For example, if they provide additional resources and best practices to help your company perform well in your industry, that would be the cherry on top of your visual search solution cake.



Using visual search technology for your e-commerce platform is one of the most powerful ways to generate a business advantage.

But as technology costs increase, so does the cost of making a wrong decision. The price isn’t simply money, but time and frustration, wasted training and maintenance fees, and forgone efficiencies and opportunities.

Thus, we hope that this guide acts as an additional resource for you to consider on top of your basic requirements in looking for a visual search technology provider of your dreams.

If you follow this article closely and do a careful assessment of the technology’s performance, system interoperability, costs, and its conformance with your business objectives, you are very likely to decide upon a winning solution that aligns with the goals of your company and meet the needs of the stakeholders.


  E-commerce,   Best practices