Yahoo Stores Trojan Dynamically Creates
Duplicate Content for Spiders and Robots

Greetings,

In my work as a search engine optimizer, one of the first things I do when I start working on an existing website is perform a remote crawl of the site, doing the same basic thing that search engine spiders and robots do. Can I find all the pages? Are there any broken links, missing files or conflicts with the Google index?

If a website it not structurally correct and working well with spiders and robots, then any other optimization work may very well be wasted. Like any good effort, a website needs a solid structural foundation.

Week before last I took on a new client from Louisiana who has a Yahoo Store ecommerce web. During the past 6 months his rankings particularly with Google have been dropping or vanishing and he could not figure out why. HELP, the holidays are coming up and I am in trouble was the plea. My client is a manufacturer and sells most of his production online to the home consumer market.

So, I put my crawler to work and low and behold we discovered that every time he creates or edits a page in his website, his Yahoo Stores account is creating a duplicate page that looks like this:

His page name = http://www.mywebsite.com/page-file-name.html

The Yahoo Store account creates another page =

http://www.mywebsite.com/www.acompetitorswebsite.com/page-file-name.html

The alarm bells went off. This page is exact duplicate content other than the file name and Google despises duplicate content.

We started the SEO forensics and discovered three things: 100 percent duplicate content in the index at Microsoft live.com, about 80 percent duplicate content at Google and that the Yahoo bot only had about 27 out of over 200 pages in their index for one of their own store accounts - had not been to visit his web in almost a year.

So, the client immediately contacted Yahoo Store support. The initial reply we received is that it must be something the client is doing wrong, they don't have a problem. That's Bovine Scatology. We sent them copies of the crawl files and copies of the index pages from the other engines.

We also did a reverse IP lookup and determined that the Yahoo Store network has a single IP address for almost 47,000 websites. We got the domain names of 4-5 of these accounts, ran crawls on them and could not duplicate the problem. Why? Because Yahoo Stores has multiple servers and on the server where my client's web resides, the server is infected with a Trojan (like a virus) that is dynamically rewriting a file name when a spider or robot crawls a page. Otherwise, the page file does not exist. It only shows up for spiders and robots. Really sneaky?

My client mentioned that he had another Yahoo Store website that he sold about 8 months ago, thought it was on the same server. We did a crawl on it and got the same results, except the name of the competitor is different.

Yahoo Store support has been too slow and technically insufficient to date, and they do know they have a problem but are not admitting it or any liability. They would not provide the client with the names of any other Yahoo Store domains on this server so that we could see what the Trojan is doing to them. Don't you love big companies?

After four days of investigation here is what Yahoo Support sent to the client:

QUOTE

[A Yahoo engineer gave this message to support] “There are no links to acompetitorswebsite.com anywhere on his store nor are there any links with acompetitorswebsite.com in the URL itself. I don't know how they (meaning me) are managing to get these results as I'm unable to install the program they are using, but I highly suspect that they may have accidentally typed acompetitorswebsite.com into the configuration options somehow. [This is massive Bovine Scatology]

It is a known issue that adding some nonexistent directory after the main URL will change the URLs for other relative links in their store and this is being fixed. However, without those links hardcoded in their account, no search engine is finding them and they are not being penalized for duplicate content. Basically, they have no problem.”

It doesn’t appear to be an issue on our end according to my engineer he scoured your site, checked the server, etc. Was hoping to find something but I think we have done all we can on this end to assist in this issue.

Please let me know if you have any other questions.

Best Regards,
Guy's name at Yahoo Support

END QUOTE

How the guy at Yahoo support got the title of engineer is beyond comprehension. It took 5 days to get this totally unintelligent answer and we told them on day one that they have a Trojan. Perhaps the engineer thought we were referring to those little round latex thingies that IT engineers wear in hope that they might one day loose their virginity?????

So, yesterday afternoon my client sent Yahoo Stores Support another email saying I estimate that this defect in your server security has cost me to ton of money in sales in the past 6 months, I have lost almost all of my search engine rankings with Google and most of my pages are in Google's Supplemental Index, I have 100 percent duplicate content at www.live.com, your search engine spider has not crawled my website in a year and I don't have a problem?

Perhaps by this time next week I will be able to tell you the rest of the story. Either Yahoo will fix the problem early next week or we will move the entire website contents to a real ecommerce web.

What I can tell you right now is that if you have a Yahoo Store ecommerce website, you need to get it checked out ASAP because this could be happening to you, and you might be able to tell from the Support message above, Yahoo is not smart enough to figure it out. Yet.

Cheers,

Ron Castle