Big improvements to service for Tesco Finder app

I've just sent into live an update to the API service supporting Tesco Finder application. The focus has been on improving performance and accuracy of the service for the ever-increasing number of Tesco Finder users.

1. The "nearest branches" feature is now much more accurate. Before the update, the API tended to favour nearest branches by longitude (east-west of your location) over latitude (north-south), resulting in branches further away but in an east-west direction ending up higher up the list than those north-south. The API was behaving like this because changes in longitude are much greater for the same distance on the ground than changes in latitude where UK and Ireland are located on the planet's surface. I had over-compensated for this effect in my arithmetic. In fact, for longitude the conversion to distance is the same as latitude except the value is multiplied by the cosine of the latitude. Lesson learned!
2. Obtaining product information is much faster. This works by pooling the search session to a special account on our grocery API so all Tesco Finder users share the same login session - since you don't have to login as anyone to use Tesco Finder, the API has to login to a special grocery account in order to search for products and provide pricing and offer information. Before now, each search was preceded by a login for that user - nice and safe in the multi-threaded computing environment of the API. Sharing the same login session is a lot trickier because that session has to be locked to each user in turn while the search takes place. Fortunately it's so quick that the queue of search requests waiting in turn to lock the session does not build up.

Sharing resources in a multi-tasking web environment can be a dangerous game. Each process (usually a computer script preparing a web page to be sent to your browser - we call this a 'thread') must lock the resource so it doesn't get altered by other threads until this thread is done. If one thread keeps the lock for too long, the backed-up threads (usually other people waiting for their web page to arrive) can grow quickly. If the backup is severe and default memory allocation is exhausted, the operating system starts trying to manage memory by swapping it to and from disk, hitting CPU load and disk response times and slowing things down more. Soon the server hits the digital cliff and the infamous HTTP 500/Too Many Users results, or the server just stops responding at all.

That's why it's taken some time and lot of testing to ensure that this programming pain is worth the performance improvement for Tesco Finder customers. I'm glad to say it has worked out well - it's now live as of 1:30pm today.

Does your Tesco Finder app crash when starting up?

I've just accepted an update of Tesco Finder and now it crashes as soon as I start the app! Help!

If you downloaded the Tesco Finder app in the past, then updated to the more recent version of the iPhone Operating System,  then take our Tesco Finder app update, our updated app seems to crash on some customers' phones.


There seems to be a further clue: it is mostly affecting customers who have upgraded to version 4 of iOS but not yet further upgraded to v4.2.1 (the very latest version at the time of writing).
You can find out your version by tapping on the Settings app, then selecting General - About - and reading the version number. My iPhone says: version 4.2.1 (8C148).

Customers downloading the app fresh for the first time are not having this problem - it's about the new app version using its old data store after an iOS upgrade.

To get round this problem, here's what you need to do:

1. Sync your iPhone with iTunes to make a backup copy of Tesco Finder (unless you want to download the app again),
2. Delete the app off your iPhone, 
3. Shutdown and restart your iPhone (this clears away all remnants of the app),
4. Re-sync with your iTunes program to copy the app back on to the phone, or re-download the app from the App Store - and please check your iPhone has the latest Apple operating system installed - iTunes will help you confirm this - at the time of writing it is v4.2.1 .
5. Start the app (it will ask you to re-select your home store). The freshly installed app will rebuild the data store compatible with your iPhone.

Our apologies for having to get you to do this. It results from the latest Apple iOS having a data store which does not seem to be fully backward-compatible. When you updated the iOS version (iPhone's operating system) when instructed to by iTunes, it protected the earlier app code and data store. However when we compiled the new app against the latest iOS, the 'earlier version protection' was no longer provided by your iPhone. The updated app tries to connect to a now incompatible data store left over from the earlier version of Tesco Finder and crashes.

When we are back at work, we'll look at how the app can survive the crash and attempt to rebuild the data store (or wipe it out and create a new data store). We'll need to do something because the current live version, 2.3 has an overall 1-star rating :(

It's also a learning curve to be fed through to our testing process. It looks like we need iPhones with different iOS versions in order to test the upgrade not just of our app but cope with the consequences of an iOS upgrade too.

Christmas is coming / Tesco Finder update

I knew there was a reason for the workload - I have been radio silent on this blog for the last two weeks because you you could say that work reached a crescendo as we put in the final preparations for Christmas trading.

It was not just that customers seemed to be waiting for us to release the coveted 'festive grocery delivery slots' adjacent to Christmas Eve. Nor was it just the fact that people were waiting, it seemed, to the last minute to get the Clubcard deals at the original 4:1 ratio before the date it switched to 3:1. Oh and the 'Double Up' Clubcard deals that proved so fantastically popular. But it all happened at the same time and our servers have been feeling the heat like never before.

In addition, our API received updates to improve performance and add new functionally, and Tesco Finder for iPhone (version 2.2) has just been uploaded to the Apple iTunes Store for approval.

I sent out Technika Freeview set-top boxes to triallists around the country engaged in the forthcoming Tesco Freeview Experiment, and they are getting their boxes set up ready so we can start soon.

Then I went on holiday to Ireland for a few days to offer my in-laws Yuletide greetings and drink Guinness (it just tastes best when sipped in a classic Irish pub with great company sat round a roaring fire).

But now we're settling down into a mix of research and support mode rather than try and add anything new as we guide our way through the next fortnight to the celebrations of the Yuletide season.

-

I'll talk more about the great new features of Tesco Finder in the next few days but in summary the new version includes:

• A complete re-design and re-program from the ground-up by Ben Martin (who also authored the Clubcard app for iPhone).
• A branch list that that stays up to date and now includes all Tesco stores in the Republic of Ireland.
• Filtering of nearby stores to only show those with certain facilities (such as petrol station, ATM, cultural food, those open 24 hours, and many more filter options).
• The ability to refresh your product list(s) to get the latest prices / availability / special offers and in-store product locations - especially useful when you change to a different Tesco branch. 
• A great new classy Tesco Finder icon showing a map of the British Isles - nice!

How to make "Sat-nav" work inside a Tesco Store

If you have been reading in the media about my proposal to have a form of in-store "satellite navigation"-style location based awareness for Tesco Finder app users, I thought I would take you on a dive into the sort of research we're doing to see if we can make this work.

Now we don't do 'tech' for its own sake so let's examine the use cases:

1. "As the customer, I wish to be guided to the product I am looking for in the store".
2. "As the customer, I wish to be alerted by my phone whenever I am close to a product I have stored in my shopping list".
3. "As the customer, I wish to be alerted by my phone whenever I am close to products on special offer that are similar to products in my shopping list".
4. "As the customer, I want you to show me a map of the store layout, and on it show where both I am and where all the products in my shopping list are located".

Wow, we need some good data to satisfy these requests! Let's see what we have:

1. We know where all grocery products are in every UK Tesco branch as long as it is bigger than an Express-sized format. This data is already provided to Tesco Finder users.
2. We are already obtaining good aisle and shelf location spatial data from software that is used by Tesco merchandising teams that creates planograms. We haven't made this live yet but it's looking good. Tesco Finder would take this data and draw out all the aisles on the screen. We're making sure we describe the layout using as few characters of data as possible (I hope other app writers think as carefully about your data plan limit and keep the amount of data transferred down as much as all Tesco app authors do).

So we know every product that every Tesco branch stocks, and where it is laid out spatially in that branch. We know what products are on special offer and what the nature of that offer is. We just need to update the Tesco API server interface to make the spatial/map data available in as few characters as we can, and code a version of Tesco Finder to support the store map. That work is in progress, and we have found everything we need to make it work.

So "all" that is left now is to work out where the customer is, on a phone that loses the GPS signal as soon as they enter the Tesco branch. Hmmm...!

As you can imagine, all of our Tesco stores have wifi wireless network access points built into various parts of the building. These provide staff with the ability to enjoy network connectivity from their various handheld devices as they go about their tasks, so this is a critically important part of our in-store infrastructure.

Each access point has a unique identifier - a Media Access Control (MAC) address - which it supplies in every piece of signal data it transmits. We can tune into the wireless data chatter and read the MAC address without actually having to connect the phone to the access point. Indeed the phone doesn't have to transmit anything - just listen.

If we were to:

1. Tell the phone (using the API) where the access points are on the store map and what their unique MAC addresses are, and
2. Get the phone to measure the relative signal strengths of the wireless signals coming from these MAC addresses...

..then we could get the Tesco Finder app to work out where it thinks it is on the map.

Some fairly simple mathematical formulae is all that's required but given that a picture paints a thousand vector symbols, you can see how it would work in this diagram (click image for larger version):

I have colour-coded 6 access points so you can see how the phone might work out where it is in the store. Given that the lower signal strength is most likely to mean a more distant access point, and because the phone knows where all the access points are located, it can work out where it is.

Job done? Not quite... radio waves are a finicky phenomenon (I should know as I am a licensed radio amateur). From a radio point of view, Tesco sells a diabolic mix of products that reflect, refract and absorb signals. As the phone moves around, these three corrupters of signal purity will be in full force wrecking the ability of the app to work out where it is.

For example, walk down the aisle full of bottled waters and it is quite possible that nearer access points will become weaker than more distant ones as these stronger signals are absorbed more by the water. Walk down the baked bean aisle and those tins are reflecting signal like you wouldn't believe. These rarely affect the staff equipment since the system allows staff devices to roam quickly between access points. It's only us who needs to know about each particular access point. 

So now you know why this an R&D project. Tesco Finder needs to work out (probably through some sort of averaging) where it thinks it is in a way that is quick and accurate enough to be credible to the customer. If we crack this, it means that we can provide in-store "satnav" style help with zero change in the infrastructure. We're going to have a damn good go!

P.S. Once again can I point out to journalists that I am Head of R&D for Tesco.com (which just so happens to that part of Tesco pushing apps out in an R&D context). I am not Head of R&D for Tesco. Somebody else is!

In the future use Tesco Finder app to "scan as you shop"

Tesco Finder, our R&D app currently available for iPhone, is ideal for trying out new ideas that come in from customers and staff. Not all ideas will be implemented but we like to try them out in a research context to test the art of the possible.

Here's one example: use Tesco Finder to scan items as you proceed to shop in a Tesco store then have the app transfer the scanned items to a self-scan checkout so you don't have to re-scan them.

To make it work, the self-scan checkout has a special barcode which Tesco Finder uses to identify it. When the customer arrives at the checkout, they scan the identifying barcode and then put the items into bags as normal. While they are bagging the products, the app connects to a wifi network and sends the scanned products contained in a file (named with the self-scan checkout's identifier) to a staging server.

All the checkouts monitor the server continuously waiting for a file 'with their name on it' whereupon it is downloaded and ingested into the checkout software in an instant. The scanned products are displayed on the screen as if the customer had scanned them in to the checkout manually.

All the customer has to do then is pay and go. Nice and quick - and fits snugly with Tesco Finder's philosophy of helping the customer get quickly to the products.

Now please remember that this is a proof of concept and not going into Tesco Finder's production version any time soon. R&D is here to prove that the technology can deliver an improvement in the customer service without major system rewriting - and so our job here is completed successfully.

We now have to leave to our security and production IT people to understand how Tesco could detect customer who doesn't quite scan everything that goes into their basket....

This special proof-of-concept addition to the features of Tesco Finder was developed by Ben Martin who has recently joined Tesco.com R&D from his 'old' Tesco PLC equivalent team - and then proceeded to work with both Dotcom and 'PLC together to help the deliver the project. Ben is also well known for delivering the Tesco Clubcard app for iPhone, so Tesco Finder is in capable hands.

Ribot have taught us the art of Simple.

It's mid-afternoon on a Saturday, and my copy of Tweetdeck is on the screen displaying my favourite filtered Twitter feeds. As you can imagine I have a column filtering "tesco iphone app" and I have to admit I'm enjoying the very many positive comments coming from 'the real world'.

The team at Ribot are enjoying it too. Given that they designed and built the Tesco Groceries for iPhone app, their enjoyment at watching this very real feedback is even greater than mine (click here and you'll see what I mean).

Ribot haven't finished with us yet, though. Their mobile expertise is affecting our thinking and what it means to actually make the customer's experience better by simplifying it.

It's a similar story for Tesco Finder - now customers can get directly to the set of products they are looking for and ignore the cacophony of presentation surrounding them. Indeed I'm beginning to identify a new demographic of customer who has avoided big Tesco stores (particularly unfamiliar ones) in the past because they think it would take too long to find what they are looking for, so don't even attempt to visit.

Teso Finder simplifies the journey to the product by enabling the customer to only look out for the navigation landmarks to get there (such as aisle number signs) and avoid the rest of the information hubbub. This new demographic of customer is now entering our doors - and together customers are using the app to find products at their chosen store at the rate of up to 5 requests a second according to Tesco Finder's server logs.

Now a confirmation message has come down from the very top of Tesco that we must start work soon to join Tesco Groceries and Tesco Finder together into a single app.

This means that in our work pipeline is your simple guide to Tesco groceries on your smart phone whether in-store, at-home or out-and-about - because, as Ribot have taught us: simpler mean better.

Survive an earthquake with Tesco Finder

A techfortesco reader has just alerted me to a very interesting article from a geography teacher's blog that is introducing Tesco Finder for iPhone/iPod Touch into geography lessons as part of an earthquake survival kit.

Chris Green, Head of Geography at a west midlands secondary school, has created the lessons to teach his pupils how to plan and execute a disaster preparedness kit on a fixed budget.

http://geographyipods.blogspot.com/2010/09/earthquake-survival-kit-on-budget.html

Some text snipped from the article:

Pupils are given a budget of £250 and asked to create an earthquake survival kit suitable for their family based at their current address (we need to be prepared in the West Midlands!! - read this article from Wired). Following the teacher introduction using the resources above, pupils use their knowledge of short and long term effects and generate a 'real' shopping list on their ipods using the apps below.

The Tesco Finder app allows pupils to search for products in their nearest store, make decisions about the price, add to a shopping basket (named by them) and generate a sub total.

They can use the maps app to plan a route from their house to the store (extension task).

It's great to see teachers using innovative ideas to teach kids, and here in Tesco.com R&D we're really chuffed that Mr. Green has chosen one of our apps to help him in his work.

Tesco Finder's one million requests

I always look forward to coming to work after a nice holiday break.

Perhaps I'm strange but Tesco.com, like many other organisations in e-commerce, is a rather exciting place to work. Every week there's always some remarkable new statistic that has surfaced which makes for a interesting talking point around the water-coolers (yes, we really do that).

It's even better being the author of such a statistic, especially when my holiday-mode brain is not quite ready for full scale work and normally makes me spend time doing no more than wading through the acres of email received during the vacation.

Tesco Finder, our store and product finding application for iPhone has now reached its one millionth request! The anonymous customer who made the request was duly advised to head to aisle 23, unit 4 on the left hand side and the 3rd shelf up from the floor to find the searched for 'bovril'.

Tesco Finder is coming up to its first birthday - a year of helping customers find our branches, and find the products in them!

Shareholder's question: Why aren't our mobile apps joined up?

As part of my job, I get to answer some questions that come in from Tesco shareholders who have attended recent Annual General Meetings, particularly those related to new technologies or services visible to customers.

One shareholder wanted to know why two of the mobile apps we have in the public domain, Tesco Finder and Tesco Clubcard, are not integrated together. The shareholder felt that it must be more costly running them separately. I have responded to the question via the Shareholders' response team but I thought I should echo it here as quite a few people must have asked themselves this question.

The reason for having two apps (in fact we have three apps in the iTune store - Tesco Finder, Tesco Clubcard, and Tesco Wine Finder) is because their origins and objectives are very different. However we will be integrating clubcard, finder and the forthcoming grocery application in a future build (hopefully by the end of 2010).

Tesco Finder and Tesco Wine Finder come from our R&D team - these applications were built as prototypes to see if customers would find them useful. They were created (and are supported) at very low cost inside the Tesco.com R&D team, and are listed under the "Tesco.com R&D Team" account on iTunes. We are still taking both these applications on a 'journey' based on continuous customer feedback and making sure that the supporting data is reliable and consistent. For example, we are hoping to include store layouts in a future edition so that customers can sat-nav their way around an unfamiliar branch to find all their favourite products easily.

Tesco Finder still has some work to improve the accuracy of finding barcodes in our product range, and it needs to be improved to easily answer the customer's question; "What are the nearest branches that currently stock 'this' product?", which it doesn't do at all well at the moment. It also doesn't have an up-to-date branch list, which is really annoying some customers and is a high priority to fix.

On the other hand, the Tesco Clubcard is much more of a known entity and was built as a production application from scratch. There aren't really any "unknowns" for this application. Joining it shortly will be the Tesco Grocery mobile application (currently in user-acceptance testing). These are full production applications, although once again built and supported at low cost.

In order to bring the functionality of all these applications into one integrated Tesco application, we are currently forming an internal mobile development team who will build these applications for all kinds of mobile phones, not just iPhone. We'll be appearing on all the major handset manufacturers offerings.

The R&D team exists to work with customers and staff to help improve their experience with Tesco - but where there are 'unknowns' we would rather keep the applications separate until we are happy that they are working flawlessly and customers find the functionality genuinely useful.

Apple iPhone adverts double Tesco Finder download rate

Tesco Finder, our store and product finder app for iPhone, has been featuring in Apple advertising on the back of the Sunday papers over the last four weeks (thank you Apple!).

You may be interested to know three interesting statistics:

1. Tesco Finder downloads on these Sundays have been double the normal download rates (and remember, each download is a new customer using the Finder service). In recent weeks Tesco Finder has secured an average of 760 downloads a day, but on the days the advertising has been shown, this has reached over 1,300 - pretty much double.
2. Use of the application (defined by the count of calls made by Tesco Finder to our servers to perform the search requests) increased by 20% on advertised days, returning to normal the following day.
3. On advertised days, our other apps also enjoyed an uplift: Tesco Wine Finder downloads increased from 145 downloads a day to 245 on advertised days (approx 60% increase), and Tesco Clubcard app increased from an average of 10,000 a day to just over 13,000 on advertised days (30% uplift).

Reviewing these stats, I draw the conclusion that Apple's featuring of apps as part of their advertising is highly effective at both bringing new users to an application, and also reminding existing users to start using an app they may have stopped using recently (or perhaps use it more) judging by the uplift in use on those days. It's also interesting that both downloads and usage returned to normal the next day once the awareness spike had passed.

There is also a 'halo effect' uplifting other apps in our collection in the app store. What makes this all the more interesting is that, before the advertising went live, we agreed with the Apple agency to change Tesco Finder's icon to show the full Tesco logo rather than the previous image of a "T" with a compass. Although the uplift in Tesco Finder was most effective, this may have helped our other apps too: People just glancing quickly at the advert would note the familiar Tesco logo and note that "there is a Tesco app in the app store" and search on just our name.

I would be interested to know if many (perhaps all?) iPhone app downloads increase on advertised days. I would also be interested to know if paid apps have the same percentage uplift as free apps do.

Apple use Tesco Finder in their iPhone advertising campaign

I've just received the artwork for the latest Apple iPhone campaign, and so I can reveal that recently Apple contacted us to ask if they could feature our R&D iPhone app Tesco Finder in their latest poster / newspaper advertisement (see above - closeup of Tesco Finder entry below).

I said, "Yes".

Actually, being ever the consummate professional, I may have toned my response down for your benefit. In reality it was more, "YES PLEASE OMG THANK YOU!!!", knowing me.

Thanks Apple :)

Tesco Finder - The Top 100 Searched Products

I'm a big fancier of statistics, me, so I thought I'd treat you to the most popular search for products from our Tesco Finder app for iPhone/iPod Touch!

All search requests are entirely anonymous and we throw away the server logs after 24 hours, but the database keeps a note of the actual search text, the date/time of the request, the Tesco store whose range was being searched, the time it took to process the request, and the number of products that were returned - all so I can keep a check on quality assurance.

There's been nearly a quarter of a million searches since we launched the app a few weeks ago, so there's a nice build of stats to reveal all.

Firstly, the top 100 most searched for products:

Rank	Product	Count
1	milk	5955
2	bread	4474
3	beans	2477
4	cheese	2464
5	eggs	2148
6	condoms	1870
7	beer	1483
8	coke	1481
9	DVD	1474
10	baked beans	1345
11	durex	1264
12	wine	1108
13	coffee	1037
14	sugar	1014
15	marmite	941
16	apples	889
17	tv tesco	836
18	iPod	829
19	chicken	780
20	pizza	764
21	crisps	724
22	vodka	673
23	tea	671
24	chocolate	652
25	(null)	638
26	Stella	634
27	apple	628
28	ps3	624
29	bananas	617
30	rice	602
31	xbox	600
32	bacon	597
33	cornflakes	597
34	wii	590
35	call of duty	589
36	condom	584
37	pasta	574
38	butter	552
39	jam	545
40	iPhone	522
41	fosters	490
42	soup	489
43	ketchup	485
44	onions	485
45	carrots	484
46	toothpaste	472
47	nappies	464
48	pringles	455
49	television	444
50	chips	417
51	laptop	403
52	fish	397
53	coca cola	392
54	weetabix	377
55	pepsi	371
56	lager	370
57	champagne	368
58	tampax	365
59	cake	357
60	salt	352
61	ham	351
62	banana	348
63	cat food	346
64	turkey	339
65	jack daniels	335
66	toilet roll	332
67	cider	330
68	shampoo	325
69	dog food	321
70	lynx	320
71	baileys	318
72	peas	312
73	olives	301
74	toys	297
75	tuna	296
76	coconut milk	295
77	sausages	295
78	potatoes	294
79	cereal	289
80	cd tesco	287
81	games	287
82	water	282
83	flour	282
84	pampers	281
85	cola	275
86	whisky	273
87	walkers	263
88	quorn	261
89	beef	258
90	egg	253
91	salmon	252
92	custard	251
93	Christmas	249
94	popcorn	248
95	ice cream	247
96	marmots	247
97	sony	246
98	biscuits	245
99	playstation	245
100	futex	241

Interesting commentary here: you'll note that amongst the more standard products are 'condoms', 'condom', 'durex' and the iPhone auto-corrected version of durex (unless you override it): 'futex'. Add up these entries and the count rises to 3,959 which would rank at position 3!

Hmmm! Filter out the time-of-day to between 9pm and 2am, and the combination of 'condomic' words easily hits the number one spot. Now I'm no social commentator but do you agree that this evidence points to a certain sub-group of iPhone user who may well be showing off the app to his mates - in the pub - than genuinely trying to find anything in their local Tesco?...

The most popular search hour:

Rank	Hour starting	Count
1	21:00	18508
2	22:00	17964
3	20:00	17777
4	19:00	17241
5	18:00	16211
6	17:00	14644
7	14:00	14177
8	13:00	14027
9	23:00	13591
10	15:00	13336
11	16:00	13212
12	12:00	12069
13	11:00	10861
14	10:00	10346
15	9:00	9377
16	0:00	8004
17	8:00	7044
18	7:00	4768
19	1:00	4142
20	2:00	2205
21	6:00	1784
22	3:00	1349
23	5:00	1302
24	4:00	1042

This is less of a surprise - the most searches are being conducted in the evening; again only my own thoughts but I think people are heading home from work and want to get in and out of Tesco quickly. The searches get more frequent as the evening progresses and I'm guessing Tesco Finder users are tired and just want to find the products they need quickly. It's interesting to note that people were searching happily for products right through the night!

Finally searches by day of week:

Rank	Day	Count
1	Sunday	40147
2	Saturday	39287
3	Friday	34754
4	Monday	34316
5	Tuesday	33226
6	Wednesday	31829
7	Thursday	31422

This is fairly consistent use throughout the week, although it's interesting to note that the most searches are being conducted during the restricted trading hours on a Sunday.

I'll be passing the full log to the insight team who, I'm sure, will come up with even more interesting information such as the patterns of product searching that vary by time, and searching for products by store. For example, if a particular store has more searches for a particular product than the 'norm', does that mean that the product concerned is more difficult to find than we thought?

Whatever the analysis, it's nice to think that Tesco Finder may well help to improve the ability for customers to find products in our stores.