We've just added a great new application under the Tesco.com R&D Team account to the Apple App Store for iPhone - if you are reading this entry on a computer with iTunes, go straight to it here.
This app is the world's first visual search application that makes the discovery of wine interactive and fun just by sending your picture of a bottle of wine to our server - and if we find it, you can uncover loads of information about the wine, and have the option to buy it!
Have a look at this video we've uploaded to YouTube to see it in action:
We've been working with visual search specialist Cortexica Vision Systems. They are the creators of a bio-inspired vision system enabling intelligent image recognition using principles derived from the human visual cortex.
I'm excited about this application because of the opportunities that a visual searching mechanism could do. To make it happen, Cortexica have worked out how the human brain interprets the scene they eyes are observing - and Tesco Wine Finder has this technology enabled.
When you look at anything, there is a mass of information to take in, but the human brain is able to filter it out so it can concentrate on what it decides is the important elements. Supposing you are reading this blog entry on your computer screen. Your brain is reading this text - but that's a tiny fraction of the information your eyes are taking in right now. They can see the rest of your computer desktop, the edge surrounding the screen, the stuff on your desk, your keyboard, your hands and much more. But until I just mentioned those items you probably are not aware that your eyes are indeed seeing them all the time. Your brain has just 'thrown away' much of the information your eyes are sending it in order to concentrate on the important bit - the reading of these words.
Cortexia have achieved the same thing - you send it an image and its algorithm works out what the important parts of the image are. The database on the search server that is called by Tesco Wine Finder contains images of a thousand wine labels - but an index that only records the 'important bits'. When you use Tesco Wine Finder to send the wine label, the server receives the whole (compressed) image but analyses it to find just the 'important bits' to send to the search request to match with similar signatures.
From my basic understanding of this technique, you are able to take an image, create a small 'signature' representing the 'important bits' after which it's a case of matching one image's signature with similar image signatures in the database - and overall deal with a lot less data!
The 'magic' that Cortexica Visual Search brings is to write computer algorithms that decide what is important in an image to compare it with others - taking into account varying lighting conditions, colours, contrast, sharpness/blurriness, one or more dominant objects in the image, and digital artifacts such as noise and compression side-effects. Of course if you take an image of anything other than a wine label - or if you take an image of a label at an acute angle or in very low-light conditions, the ability to match will be lessened.
So please try out Tesco Wine Finder and let me know what you think - oh and of course you get access to our Tesco Wine by the case store too; choose from over 1,000 wines and champagnes including your in-store favourites as well as award winners and mixed cases. You can stock up and enjoy your wine sooner with next day delivery or choose a 2 hour delivery slot on a day that suits you. Which is nice!
Can we expect to see this on Android soon?
ReplyDeleteIs this the exciting product you alluded to the other day...?
ReplyDeleteVery interesting, thanks for the write up. I'd love to see some more tech details on this, is their a way to get overlays from an image to see what the system things are important areas?
ReplyDeleteAre their plans to widen this outside of wine? - in the future could I take a photo of a product and have it identified and the appropriate Tesco SKU so I can add it to my shopping list? how does it compare to barcode scanning with a iPhone camera?
I don't have an iPhone (I'm an android person :-D), how difficult is it to get a photo of suitable quality to be recognised?
Does it require much space to store the key for each product?
Hi "Anonymous" - we're gearing up to do a lot more than just iPhone, it;s just that we can try a few things quite easily with iPhone because it has so much power and functionality built right in.
ReplyDeleteBut we're not forgetting the Android, Nokia and Windows mobile owners either, we're just setting up a proper production service which takes a little time.
Hi Mally,
ReplyDeleteYes, sure is!
Hi Stephen,
ReplyDeleteWe don't have plans - yet - to go beyond wine, but it was chosen because it is a good 'control product group'.
I only understand the technology as an observer and from having some interesting conversations with the owners of the technology, Cortexica (link in the blog entry), who are keen to get their vision system used for all kinds of applications.
Sorry to Android fans (but see my response to 'Anonymous') - we chose iPhone because (as you now know) we are going to be selling them in Tesco Mobile!
This brings the fun back to shopping! I tested the app with a range of wines, challening it with dark or reflective lighting (and dust on older wines!) and I am very impressed with the underlying imaging software. I only set out to play, but ended up buying two cases of wine. If there was a 'one click' buy, or at least add to my basket for next time, I'd use this a lot, and other categories too, where I want to refresh my favourites.
ReplyDeleteI've used Red Laser and it's phenomenal. It Isnt tempernental as you described like other Barcode apps it gets you the barcode before it's fnished focussing. Very fast.
ReplyDeleteThis is a very visionary app. Especially on the retail ecom space, it makes a lot of sense to do something like this. And probably best done in an inspirational or gifting range like wines etc. (In my limited view, I doubt if you would apply this to your grocery ranges - unless you thought of having an app which builds a shopping list by taking a pic of products in your house which are just getting over). Overall, kudos on such progress...
ReplyDelete