IMV8 : Traditional Web Analytics is Dead

The following is a transcript for IMV8 : Traditional Web Analytics is Dead. The original podcast is located here.

Announcer:

Welcome to the Internet Marketing Voodoo podcast brought to you by MindComet. And now, here’s your host, Ted Murphy.

Ted Murphy:

Welcome to Internet Marketing Voodoo, Episode 8. I’m your host, Ted Murphy, and with me today is Avinash Kaushik, Senior Manager of Web Research and Analytics at Intuit. Welcome to the show today, Avinash.

Avinash Kaushik:

Hi Ted.

Ted Murphy:

Avinash and I met at a Frost & Sullivan show a couple weeks back, and we were both speakers. And I have to say Avinash, I really enjoyed your presentation. You were full of lots of great information and kept me laughing the whole time, too. So I am really happy to have you on the show today.

Avinash Kaushik:

Thanks, I know, I’m very happy to be here.

Ted Murphy:

Tell us a little bit about Intuit. I know that a lot of people know the products that you guys put out, but they may not associate all those products back to the company. So I wanted to give our listeners a little bit of background in terms of the magnitude of the things you are dealing with.

Avinash Kaushik:

Yeah, Intuit is actually a 2.2 billion dollar company, and we are in the financial services sector, and we have two main areas of focus. Our consumer business is things that people have heard a lot about with our TurboTax and Quicken brand of personal finance management software and tax preparation software. But we are also very heavily focused on the small business side with our offering of QuickBooks. And we have many different flavors of QuickBooks and Intuit is very interesting. It is a collection of about 10 or 12 companies that we have acquired over time with TurboTax, QuickBooks and Quicken primarily being the big one.

We actually sell our software on the Web; we have a phone channel, in stores. We also offer CD download versions of our products as well as online versions of our product. So both QuickBooks and TurboTax come in online versions where you complete all your interaction and manage everything exclusively on the web without ever having to install anything. So, it is a very unique company, yeah.

Ted Murphy:

A lot of web interactivity there.

Avinash Kaushik:

Definitely.

Ted Murphy:

So we hear a lot about web analytics and obviously your title is Senior Manager of Web Research and Analytics.

Avinash Kaushik:

Yep.

Ted Murphy:

What’s all the hype about?

Avinash Kaushik:

I think that as the Internet is growing up and as companies are beginning to spend more and more money on advertising and acquisition and getting more and more revenue, I think we are all beginning to wake-up and focus on figuring out why or why not are we are making money on the Web. And that is all the hype is about. I think we have had a free ride for a number of years so to speak. All companies were doing web because web was “cool.” Well, the Web is getting beyond “cool,” and becoming a normal business and there is a lot of accountability on all fronts and hence, there is a lot of focus on this term, web analytics.

Ted Murphy:

People were just happy in the early days just the fact that they were making any money online.

Avinash Kaushik:

Exactly. And in fact, you know, I think this is an interesting time to have sort of the discussion that we are having today because I just presented at Ad:tech in Seattle and, you know, I found this hypothesis that actually now, web analytics is dead. And we are entering this era of web insight. How can we get beyond clickstream, beyond web logs beyond how many hits are people getting. So, it will be interesting to see how all of this pans out over the next few years.

Ted Murphy:

Absolutely. I know that you are saying web analytics is dead. But what have you use in the past, how have you used web analytics to improve upon the customer experience, and what you are doing to increase sales?

Avinash Kaushik:

Yeah, so web analytics, traditionally, when people talk about web analytics a lot of times they are referring exclusively to the analysis and reporting that is done on either web log or Java Script tag data. I basically say that every time you say web analytics people automatically translate it into clickstream analysis. Where are people clicking on the page, how many times is the page being viewed, how long are people spending on the page, what paths do people take on my site? Just all this clickstream analysis is traditionally the web analytics sphere. And at Intuit, we started down a strategy about 15 months ago, to say that what are the real questions we want to know about our website? What are the answers that we are looking for? And we have embarked on a strategy of taking our clickstream data marrying it up with our outcome data and also, trying to figure out how we can bring in our experience date from the website. I like to say that not only do I want to know where you clicked and what you saw on my website, but what I also want to know is, what was in your head while you were having that interaction?

Ted Murphy:

And how do you get that experience date?

Avinash Kaushik:

So we have several different methodologies. Our flagship process for getting, what I call, experience data, we use the ACSI tool, the American Customer Satisfaction Index out of the University of Michigan. And we partnered with a company called, ForeSee Results. And we used a survey-based mechanism that is on top of the ACSI to measure customer satisfaction, future predictive behavior and also get satisfaction scores for the core element of our website.

So simple example, is on our e-commerce site we measure how satisfied are you as a result of the interaction. We measure what you think about the product information, the quality of the website, the navigation structure, the cost at which we are selling our products. And we throw this into the ACSI model and we try to figure out, as a result of your interaction with our website, how likely are you to buy from us? How likely are you to recommend our products? And behind all of this are actually open-ended views, we ask questions such as, “If you are not able to complete your task, why not? Please tell us.” And you’d be surprised Ted, they really tell you.

Ted Murphy:

That’s great! And what types of feedback have you gotten back? What’s the good, and what’s the bad?

Avinash Kaushik:

Well, overall, for example our sites actually get pretty good scores. They are very, very competitive with our, with the industry, we benchmark ourselves against the ACSI tech index and the other technology indexes which contain Apple, Ebay, Amazon and other company scores in them. The great thing is that many of our websites that exist for research and purchase perform very well in those functions helping people find the information that they are looking for, for our products and helping decide if they want to buy, and if they want to buy online or off-line. We make is easy for you to buy anywhere you want.

Ted Murphy:

Have they always scored that well? Or has that been since you implemented that new system that in the beginning the scores were a little bit lower and now they’ve gone higher or were you just lucky and from the beginning you nailed it?

Avinash Kaushik:

Oh no, not at all, not at all. I mean that would be too pompous. No, we actually started with okay scores, I would say, on our e-commerce websites, and we’ve made gradual progress to the extent that we’re actually nipping on the heels of some of the better websites now in our research and purchase category. On our support website, I think we share a common problem with all technology companies, and we don’t score as high as we would like to. And we are making ongoing continuous improvements to invite voice to the customer in figuring out: Why are you here? What are you not able to do? Please tell us. Please yell at us, let us know what is wrong. And we are starting down a path of making progress. But, we are actually; we have a long way to go let me put it that way.

Ted Murphy:

How closely do you think usability and experience are tied, in terms of overall customer satisfaction?

Avinash Kaushik:

Extremely closely. Let me put it this way, it is extremely closely. Every website is unique and every website has a value proposition. And I think what is more important to usability, customer satisfaction, talking to people, analyzing data, you can actually truly figure out: what is the value proposition of your website? You can have a completely crappy website, but if you are selling $400 products for $3, people would be happy.

Ted Murphy:

Absolutely.

Avinash Kaushik:

Now, not all of us can do that. It’s the value proposition. For us, we spend enormous amounts of time via usability or via our customer satisfaction measurements to actually figure out why do people come to our website and what tasks are they not able to complete. And one of the interesting things that Intuit is, we can actually tie usability data, customer satisfaction data back to clickstream, back to outcome. I can actually take transactions that have happened on the website, and we are able to match that back to our ForeSee usability data. If people ended up buying, what was the thing that they liked about our website? What pages were helpful? What information was helpful? How can we then go back and use our clickstream tool and do click analysis to improve our site so that more people are exposed to the same amount of, what we call here in the company, “convinced me to buy,” information.

Ted Murphy:

Now that bridge between those two sets of data is that something that you bought from a third party provider, or is that something you guys have created in house? How does that work?

Avinash Kaushik:

Actually, none of the tools that we use natively, so to speak, talk to each other. So ClickTracks, which is our primary clickstream analysis tool, what people would call traditional web analytics tool, that’s ClickTracks. And that tool doesn’t natively work with our survey data and our survey data doesn’t normally work with our outcome oracle database. I think that what we have done here at Intuit is put a lot of thought into the architecture of our products. And then as we have implemented them, made sure that we have ways to tie the data. We track people anonymously, 100 percent anonymously in our surveys on our clickstream data. But we make sure that when we are doing this, we pass anonymous values between these two things so we can, in the end, tie them together. So that was all a part of sort of our creation of, what I call, our trinity strategy with these three elements: behaviors, outcomes, and experience. And then as we implement them, make sure we put enough thought and love and attention to the primary keys that will tie these data together.

Ted Murphy:

Love and attention. Everybody loves, love and attention. Well we’ve got a question here in an e-mail from someone who is obviously a little bit smaller company, and the e-mail here says, “I’m a Webmaster of an E-commerce website that does about 5 million dollars a year in revenue. I have looked at several analytics packages to help improve conversion but I am not sure what the best choice is. They range from Free to several thousand dollars a month. Do you have a package you recommend?” And that is from Allison Hubbard in Fresno, California. Obviously, the things that you are doing, you know you are a massive organization with the resources to bring all this data together. But if someone is just starting out, and they are looking for a tool, do you have anything that you could recommend?

Avinash Kaushik:

Oh, definitely. It’s actually a misnomer that we, we are actually a really big company. But in my team we actually only have six people.

Ted Murphy:

Wow!

Avinash Kaushik:

It’s actually a really small group of people who are able to…

Ted Murphy:

Get a lot done.

Avinash Kaushik:

…who from efficiencies from tools and technologies who are able to do a lot done. I would say to Allison that there are actually a lot of really great analytics packages. The important thing for her to figure out is what kind of answers is she looking about? So, if she is actually trying to improve conversion rate, figuring out how best to tie the outcome, whatever they’re selling, to the clickstream data is critical. And, I would say that it’s not important what analytics package you use. What is important is that you have sort of in-house expertise of somebody who can do analysis. For example, a standard package from a big company costs, let’s say, about a hundred thousand dollars a year. If I were given a choice, and I had a hundred thousand dollars to analyze five billion dollars worth of revenue, and by the way a hundred thousand dollars is not too much, I would actually go out and buy a ten thousand dollar package and pay somebody ninety thousand dollars to do the analysis. Because in the end, that is where the value will come from, from having somebody who has some experience and is analytical enough to analyze the data. After that, you could throw any tool at it and it will work just fine.

In our company we use ClickTracks. I would definitely recommend that Allison look-up ClickTracks. She could send me an e-mail if she wants for information of how we use it, but one of the beautiful things about ClickTracks is it is a great tool to segment your customers. You can easily deport your SEM traffic, you can easily put success criteria, and you can truly understand what customers are doing on your website and how you’re making money. And it’s an extremely cost effective option, you can go to ClickTracks.com and I think you’ll be stunned at how cost effective it is.

Ted Murphy:

That’s great! Always good to get a customer testimonial like that. We should let the guys at ClickTracks know about this podcast.

Avinash Kaushik:

Yes, actually, it’s a very nice group of people who are always very helpful to us. And the interesting thing about ClickTracks, Ted, is that unlike many of the “standard” packages. ClickTracks does not puke data out. That is very important to know. A lot of these tools give you massive amounts of data and it’s left up to you to figure how to use it. ClickTracks’ approach is slightly innovative. It gives you a standard set of metrics and assumes your intelligence behind it and you can slice and dice them any which way to make intelligent decision so that is unique about them.

Ted Murphy:

That’s great. Well if anybody else has any questions about Internet marketing, they can call always call our 800 number. That’s 1-866-206-4461, or e-mail me at Ted.voodoo@mindcomet.com. So, I want to talk a little bit, Avinash, about what this change to focus on customer experience, what that has done for your organization. Have you seen a significant uplift?

Avinash Kaushik:

Oh absolutely positively, absolutely. Actually, one of the problems including with us and many of our peers in the industry, we always talk about hits, we always talk about visits, we always talk about cookies and cookies and cream and ice-cream. Our focus is all these sort of non-human things, right? It doesn’t somehow let you connect to the fact that real human beings interact with your website. These are real people. And one of the things that has been tremendously beneficial to Intuit is it’s helped us transform our thinking. It’s very subtle, but it transformed our thinking from visits and cookies and Internet Explorer and Firefox to human beings that are coming for particular purposes to our website.

The other thing is one of the big downfalls of every web analytics tool every one of them ClickTracks included by the way, full dis-closure, is that none of them can actually tell you why people come to your site. You can infer it by figuring out where they’re clicking but, well, everybody could be clicking on the wrong link! And what the big benefit of focusing on customer satisfaction and the ACSI tool is it is helping us to understand primary purpose; Why is it that people come to our site?

Ted Murphy:

And how are you finding that? Is that through simple surveys?

Avinash Kaushik:

Exactly. Our survey actually we’ve inspired from Mr. Nielsen’s philosophy. His philosophy is on a website you should only ask people three questions in a survey. The first question is why are you here? The second question is were you able to complete your tasks today? And the last questions is, if not, why not? And we ask them. We really ask people why are you here? It helps us understand why people come to our site. For research, for downloading white papers, for buying our products, for getting support, for complaining about things to look up error codes, to get rebates. Well we can really get an excellent customer view to see why people are coming to our websites. And after that, we have actually asked them and have analyzed data to figure out what percent of people in each category are able to complete their tasks. And quite simply after that Ted, we go back and figure out, from their own words, what task are they not able to complete. And this gives us a perfect prescription of where to focus our energy. No web analytics tool can give you this level of insight about the customer’s mind. We do then take all this information, go back into our clickstream tool, look at the overlay and figure out how to optimize. More importantly, after we implement chances we go back to our clickstream tool and see if the change is working.

Ted Murphy:

And how often do you do those changes? Is that like a quarterly thing? Is that like a weekly thing? How often do you take all that data, analyze it and say, “Okay its time to make a major change?”

Avinash Kaushik:

The thing is, one of my mantras that has been beaten into me is on the Internet revolutions rarely work and evolution always works. So, we used to be, many companies are in a space where they keep blowing up websites and building new ones and we’ve learned that that rarely works. Our approach is evolution. We can slowly evolve our website and achieve a greater impact. So, the answer to your questions is: It depends.

At the moment, as you know, we are in the thick of TurboTax season. Everyone is worried about refunding taxes. Changes on a TurboTax support and eCommerce websites are happening every single day, sometimes within hours. Massive amounts of change constantly going on, based on the outcomes that we are analyzing, based on the customer voice that we are analyzing.

And at the other end, we have websites where we are changing at a much slower pace. But Intuit over the last few years has gone from changing many of our websites once a year, to changing our websites significantly frequently. And I think now we are between the smaller sites to about once a quarter to some of our bigger sites changing, quite literally, a couple of times a day.

Ted Murphy:

And how many sites are we talking about?

Avinash Kaushik:

Overall Intuit has about 15 big websites, but in all Intuit has Internet presence in roughly 50+ websites.

Ted Murphy:

Wow.

Avinash Kaushik:

We actually have 50+ customer-facing websites, external websites.

Ted Murphy:

So, Avinash, we’re coming to the end of the podcast here, so much good information though. But I wanted to ask you real quick, what would you say are the top three most important things that people should know when it comes to creating a positive customer experience? If there is three things to take away from the podcast today what are they?

Avinash Kaushik:

The most important thing is that, 80% of the time we are wrong about what our customers want. 80% percent of the time we are wrong, and I’m always embarrassed about how wrong I am about what our customers want. And its quite simply that we don’t know what our customers want. And we should figure out, all of us in this industry, how can we bring the customer to the table when it comes to design decisions, when it comes to navigation structure, when it comes to information architecture on a web page. And there are ways to do that. So that’s my number one recommendation to everyone. Figure out how to involve the customer.


The second thing is, that while it is important to analyze clickstream data, and it is very important, we should not expect revolutionary insight from clickstream data. For example, everyone loves to do path analysis. On any given website of a decent size, not a huge size, but a decent size, let’s say a couple hundred pages, less than 1-5 percent of the traffic will follow the same path. If you do path analysis, would you ever make changes to your website based on what 5% of the people are doing? So clickstream analysis is important, but I think it’s important to realize the limitations of clickstream analysis, and look for other ways.

And the last thing that I definitely would want people to take away, is that when it comes to analysis, when it comes to improving website, when it comes to doing all of this great work, perhaps the most important thing is not the tools and the data you have available, but the people you have working.

People and perceptions are extremely important, and I think, I observe that, we don’t put enough love and attention to actually have the right people decked up against the right questions that we want answered. And we try and do that at Intuit. Intuit is an extremely, extremely employee-focused company and we have the privilege of trying different things and we’ve been lucky enough to get the right people organized in the right spaces. People matter way more than tools. That’s very very important.

Ted Murphy:

I definitely agree with you, and that is one of the things that I really liked about the speech I originally heard you give. Because a lot of people just kind of put up a website and they think, well the website is going to do all the work and all we have to do is look at little bits of date.

Avinash Kaushik:

That’s right.

Ted Murphy:

But, I think that people forget that on the other side of the human as you said. And if people took that same approach in a physical world, I don’t think you would have any customers at any of these stores to be honest.

So I really appreciate you coming onto the show today and spending some time with us. Hopefully we can have you back again in the future to talk about the next big thing. And for all of those people listening on the podcast, you can go to Internetmarketingvoodoo.com to learn a little bit more about Avinash and his experiences at Intuit. Thanks and have a great day!

Avinash Kaushik:

Thanks very much Ted. It was a pleasure.

Announcer:

For more information on this week's topic, visit InternetMarketingVoodoo.com. This podcast has been brought to you by MindComet, the Relationship Agency.


[End of Audio]

Marketing Resources
Download Top Ten Things to Know About Web Analytics (PDF - 674.4KB)

Listen to the Web Analytics is Dead podcast.

Contact MindComet about our Internet marketing services.

Technorati Tags
web analytics, analytics, google analytics