People Companies Advertise Archives Contact Us Jason Dowdell

Marketing Home » Archives » 2005 » April » Week 3

April 2005, Week 3 Marketing Archives

Friday, April 22, 2005

Search Engine Comparison Study Continued

When I got into the office today I saw there was an email from Danny Sullivan. He emailed me about the study and my post to him about the misunderstanding surrounding my comparison of search results between the 3 major engines and Become.

Danny said that if he thought he neede more information from me then he would've emailed me directly to get it. And he felt I had put enough information in my post and it didn't require followup or clarification from me. But that he mistakenly thought I was comparing Shopping.com, Froogle.com and Become.com. Danny went on to apologize if his review was taken as a dig at me. He said he updated the original blog post on the SEW blog and there is a decent forum thread going about qualitative search results.

Danny's a good guy and I knew he wasn't trying to make me look bad and I didn't take it that way. I sent him an email letting him know that so all is right with the search world, for now. Danny addes some excellent commentary to the original post and I recommend it as a good read.

Search Engine Comparison Study Continued By Jason Dowdell at 01:35 PM
Comments (0)

Wednesday, April 20, 2005

Danny Sullivan Read This

Earlier today Danny Sullivan posted on the Search Engine Watch blog that my shopping related search comparison wasn't fair because he said I compared Froogle and Shopping.com to Become.com.

Danny,

I was comparing Google's normal search results, not froogle, and none of Shopping.com's search engine results were used because it's not a true search engine, it's a feed based search tool. There is a good chance I didn't make that clear in my post and I apologize for that but the results were pulled from Google's natural search results, Yahoo's natural search results, MSN's natural search results and Become's natural search results. If you look at the graphs in my comparison post you'll see that I clearly label each search engine with the number of unique urls each had.

The confusion probably arose from the fact that I used 3 separate sources for the keywords.

Let me know if you have any questions.

Best Regards,
Jason Dowdell

Danny Sullivan Read This By Jason Dowdell at 05:29 PM
Comments (0)

7-Eleven In Silicon Valley

I guess it's part of normal human behavior to want to find something that reminds us of home when we're in an area we're not use to. Although I've been to the Valley before, it was for a very brief time and I didn't have a chance to feel my way around and really see what there is to be seen here. Well, as many of you know, my favorite convenience store is 7-11. It's my dream [not really but humor me here] to own a chain of 7-11 stores because I love them so much. They're the king of convenient store branding, they invented the Big Gulp for heavens' sake! How many other Gulp competitors are out there playa hatin against the Gulp? A bunch I tell ya. They also invented the Big Bite, other stores can't touch 7-11 from a branding perspective. So anyway, I was overjoyed when I realized there are 7-11's out here in Silicon Valley. I found one over near Google [on the other side of the 101 in a little suburban area - don't recall the name] and I stopped in. My fear was they wouldn't have the two things I so desperately needed, a Super Big Gulp of Mountain Dew and a Big Bite hotdog. Then I saw both and immediately said...
Oh thank heaven for 7-11!
Anyway, I got my goods and headed out to my car to eat my lunch. It made me feel like I was at my home away from home. I typically hit a 7-11 in Merritt Island or Cape Canaveral back home to grab a quick lunch and this felt almost the same. Anyway, Silicon Valley isn't that much different than Merritt Island, at least from a convenience store perspective. Noah, why don't you post a comment here about our many surf trips after work and the visit to 7-11 on the way home and how we were able to create a meal out of a bag of chips and a super big gulp;)

7-Eleven In Silicon Valley By Jason Dowdell at 11:52 AM
Comments (0)

Q & A With A9 CEO Udi Manber

Amazon A9Yesterday there was a lively keynote by Udi Manber, the ceo of Amazon's A9 search engine. For clarification purposes, A9 is a wholey owned subsidiary of Amazon and the questions were given by the audience at the conference and not by me personally. I'm merely the messenger this time and not the questioner. The best answer Udi gave was when he was asked what the difference between Google's keyhole application and A9's image features, read on to see his responses.

Q: What happens when a business shuts down? How do you know and what do you do about it?

A: Customers and users give us feedback and once we drive the entire country we'll drive it again. We're not stopping after the first passthrough.


Q: Are you going to charge for yellow page listings?

A: Currently sponsored links fund this section and it is free. No comment on future business development plans.


Q: How does it compare to Google's keyhole?

A: Google comes from above and A9 comes in from the side. [the crowd erupted in laughter after that response]


Q: Are you considering licensing data?

A: Yes, if the right partnership presents itself then yes.


Q: Is the click per call feature free?

A: Yes and the technology that drives it is from eStara.


Q: What about the privacy issues of people being photographed that don't want to be? How are you dealing with those?

A: The person needs to contact A9 and we'll do whatever we need to do in order to remedy the situation, including removing the image from A9. However, taking pictures on the streets is part of the public domain.


Q: How do you validate the data consumers put up when they review a business or post images of a business?

A: It's free for anyone in general. [huh?] A9 highlight the users real name and have algorithms in place to perform quality checks in real time so the editorial process is as human free as possible.


Q: Do you plan on competing with Google and how do you plan on making money? [from Barbara Coll ;)]

A: We don't think of Google as a competitor, we're only focusing on our own product. And I have no comment about forward thinking business development or features.


Q: What kind of feedback are you getting?

A: Lots of great feedback. People are using Flickr to post their most interesting images found on A9 and others have created ways to make panoramic pictures of streets from the images found on A9. The feedback has been really well received.

Q & A With A9 CEO Udi Manber By Jason Dowdell at 11:40 AM
Comments (0)

Tuesday, April 19, 2005

YO, Steve Rubel

Steve, just wanted to let you know I felt compelled to mention your post about my Captain Morgan prediction during my panel at the conference today. I think the audience got a good chuckle from my comments regarding Captain Morgan and how he's not a real A List blogger. Thanks for the props Sunday. I must admit that I did give you a hard time about the fact that you have the word "blog" in pretty much every post on MicroPersuasion but that's about it.

YO, Steve Rubel By Jason Dowdell at 05:18 PM
Comments (0)

Greg Sterling For President

So I spoke on my assigned panel today at the conference and it was a blast. I just wish we would've had another 3 or 4 hours to talk about the impact of blogs, online word of mouth and rss are going to be on local advertisers and the net in general. There is so much that is yet to be charted by anyone and it's really an exciting time. There is so much going on right now that nobody's come up with a great business model or even the best business model for connecting advertisers to bloggers to social networks to local markets to end users. MAN I love this stuff!

On a different note, I really can't say enough good things about Greg Sterling or The Kelsey Group in general, even if John Kelsey and I differ on our opinions of what bloggers and journalists have in common and how they should be treated :) Greg has introduced me to so many people in such a short period of time and every one of them has said "e;If Greg says you're someone I should know then you're someone I should know". Man what a good feeling that is.

Greg takes the term "Word Of Mouth" to a whole new level. He's genuinely humble and is more embarassed about being complemented as I am. He's the first to admit that he's not a smart guy, even though he is, but everyone around him knows the truth and likes him for being so humble. Anyway, I'm just glad to know him and hopefully he and I will still talk now that he's seen me in action on stage haha.

Greg Sterling For President By Jason Dowdell at 05:09 PM
Comments (0)

Monday, April 18, 2005

Comparing Search Engine Results - My Experiment

A couple months back I was covering the launch of a new shopping search engine. As part of my event coverage I was allowed to be part of the beta group, prior to the launch only a handful of people [mostly employees and family members of employees] were able to take part in the Become beta test. But, Michael gave me an open invite and said he would welcome my feedback, good or bad. With that I said, hmmmm, can I break this? After I played around trying to break it I realized my time might be better spent doing some actual research. So that's what I did. Initially, I created a spreadsheet comparing the search results of the top search engines for a generic shopping related term "furniture" but when I saw what the data looked like I decided to dig a bit deeper and limited the scope of the engines I compared but broadened the scope of the terms I used in my comparison.

Before I tell you what terms I used and how I picked them I'd like to first let you in on a little secret. I had a hypothesis. Yes, that's right, the guy that never brought is high school lab notebook to class actually had a hypothesis before he conducted an experiment. Dr. Scott McCord would be proud.

My Hypothesis On Shopping Search Result Comparison
I believe there is no significant difference in the search results of Google, Yahoo and Microsoft. This statement applies to their search results in general but, for sake of limiting the scope of this experiment, I will be focusing soley on shopping related queries.

The Method
Step 1: Getting A List of Objective Shopping Related Search Terms

First, I selected keyword phrases from Froogle's year end Zeitgest list for 2004. This consisted of 6 different categories of queries with 10 keyword phrases per category. So that gave me a total of 60 terms and 600 results I could use in my experiment.

Second, I used the top 20 queries on Shopping.com for each week starting in June 2004 and going through February 2005. Even though there are 20 queries for each week there are a lot of duplicate queries in those lists. So, in order to get 60 unique queries from Shopping.com I had to go all the way back to June 2004. I thought that was pretty funny in and of itself.

Lastly, I used the top 100 searches on MySimon for the week ending February 25, 2005. This list contained 100 queries that were completly unique.

Step 2 Data Aggregation and Comparison of Search Results
In order to determine the similarity of search results it's important to know how unique their results are. To accomplish this, I performed 3 different sets of comparisons [because there were 3 unique sources of data used as noted in step 1 above]. Then, within each report I compared the top 10 search results of each keyword phrase per search engine. If a url was found within the search results of a single search engine [engine A] but the url didn't exist in the top 10 results of either one of the other engines [engines B, C & D] then that url was considered unique for the search engine in which it was found [engine A]. This comparison could be drilled down to even further if people request that. For example, I could compare the number urls in the search results of Search Engine A to the urls of Search Engine B to determine how 2 specific search engines fare against one another. For my purposes, I only wanted to compare the results each engine tested against all the engines so I'd have an unbiased report.


Let me give an example to clarify that.
For the term furniture, the url "http://www.nhfa.org/consumer.asp" was found within the results of Become but not in the results of Google, Yahoo or MSN so this would count as a "unique result" for become but not for any of the other search engines.

The graphs below show the uniqueness of each search engine per keyword set. The keyword sets [explained in step 1] are labeled Froogle, Shopping.com and MySimon for clarification. Some terms overlapped between each of those 3 sources of keywords but I felt it was important to not merge the lists and run a single report for objectivity purposes. I wanted to see how each search engine fared when compared on 3 completely different sets of shopping related keyword searches without trying to create one master list.
Froogle comparison of 60 shopping terms and 600 results
mySimon.com comparison of 100 shopping terms and 1,000 results
Shopping.com comparison of 60 shopping terms and 600 results

From these charts I think it's fairly clear that the top search engines have a long way to go before they can ever be considered truly unique. On the other hand, since I started this little experiment as a way to test Become's algorithm and whether or not they were bringing something truly unique to the table, I think the results speak for themselves.

When compared to the major search engines they delivered the most unique urls on every list of keywords tested. They also tended to return more deep results in their top 10 than the other engines. By deep results, I mean pages other than the homepage or main domain of a web site. When I did a "spot check" of the actual quality of the results from Become versus the other Big 3 search engines I felt [I know, I know, you're not suppose to have feeling when you're conducting an experiment] the results were much better from the standpoint of shopping research. I was frustrated, however, that I couldn't do any comparison shopping when I found a product that met my needs but I'm told that feature is currently in development.

On the "big picture" side of things, it's a bit concerning that Google had the least amount of unique results when compared to the other search engines. I'm guessing this is because the other engines do indeed crawl Google's index to seed their own from time to time and to conduct qualitative checks of their own. The mentality of most search engines is "we need to beat Google team, now go do it!" and I'm certain many a search engineer has crawled the results of their competitors as a guage of how well they're actually doing about uncovering those hidden gems [urls] of the web. Another culprit of Google's low uniqueness factor is the fact that they tend to return the primary domain of a site in their results rather than the deep pages found within a site. I'm not sure why this behaviour exists but I've spoken with many an seo that said their client was in the first position for their main keyword but only for the homepage and not for the page they want visitors to come to. I'm predicting there will be a change along these lines at Google and the other major engines because it would result in higher conversion ratios for their natural results which means their advertisers and users would be happier as well. Advertisers will think "heck, if my natural results did that good, just think how good my ppc campaign will do" and consumers will be more satisfied in general because they'd be getting better results right from the start instead of having to drill down into the sites found in the results themselves and trying to find what they were looking for all over again.

I hope you find this data and my interpretation of this data interesting. Feel free to comment on this post or to email me directly [marketingshift at gmail.com] with your thoughts. As always, I'm open to criticism. If you'd like a copy of the 3 lists of terms I used to conduct my experiment then let me know.

Comparing Search Engine Results - My Experiment By Jason Dowdell at 07:11 PM
Comments (0)

Best Marketing Blog Awards

Well, I just got an email from MarketingSherpa asking for nominations for the best marketing blog award. I know I have no chance of appearing on the list but I'm sure many of you have blogs you really like so I would encourage you to nominate them. Nothing was more encouraging in school than when I'd get an A on an exam and nothing is more encouraging to a blogger than being nominated for an award, cheesy at it may seem.

Best Marketing Blog Awards By Jason Dowdell at 01:42 PM
Comments (0)

In Santa Clara Speaking at Kelsey Group Conference

Okay, so I finally made it back to the left coast, this time Santa Clara. I got maybe 2 hours sleep Saturday night and left the house at 4:45 am and headed to the airport. Funny part is that my car said I had 1 mile left before I needed to refill and the nearest gas station was 3 miles away. So, being the good citizen I am, I sped the entire way to the exit where the nearest gas station was and had to run a red light to get there before my car stalled. Unbelievably, I made it to the gas station and was so thankful.

Then my flight left at 7 am and I arrived in the valley a little after 2pm my time. Then I slept for 15 hours [no idea how I did that since I haven't slept that much in 3 weeks]. Now I just have a few things to take care of before I can go out in public [not including work].


  • Shave this awful beard off my face.

  • Get some Pepcid.

  • Find a grocery store.

  • Iron my clothes for the week.

  • Gather my thoughts about a piece I've had in the works for 2 months.

  • Register for the conference.

  • Change the time on my watch.

  • Charge the battery on my[shannons] camera.

  • Charge the battery on my cell phone

  • Take a shower.

  • Unpack



That was boring
As I composed that list I realized how obvious it is to Shannon that I need to shave every once in a while. I look pretty nasty when I don't shave and am often embarassed when I meet new people and they have never seen me before and they see me with this nasty beard. Hmmm, note to self, shave once in a while for Shannon's sake and your own.

Why am I here?
The Kelsey Group Conference brings me here and good friends and cool places keep me here. I'll be speaking at the local comedy clubs all week [scratch that] at the conference on Tuesday, and will be interviewing cool and interesting folks through Wednesday. Let me know if there is someone from the conference you'd like to ask a question and I'll be sure to include them and your question in my posts.

In Santa Clara Speaking at Kelsey Group Conference By Jason Dowdell at 01:33 PM
Comments (0)

« April 2005 Week 2 April 2005 Week 4 »

  • Week 1 (3 entries) April 1-9
  • Week 2 (4 entries) April 10-16
  • Week 3 (9 entries) April 17-23
  • Week 4 (7 entries) April 24-30

Subscribe to Marketing Shift PostsSubscribe to The MarketingShift Feed