Business News

But did the spider actually visit? Maybe not. Dynamic page content is often invisible to most search engine spiders, so it never gets indexed. Increase the traffic to your dynamic site by making your valuable content visible to search engine spiders.

Dynamic Pages Are Easy To Maintain

The content of static pages doesn’t change unless you actually code the changes into your HTML file: open the file, edit the content, save the file, and upload it to the server. All search engine spiders can index static Web pages.

A dynamic Web page is a template that displays specific information in response to queries. Most of the page content comes from the database connected to the Web site. Visitors love them since they get quick access to the information they want. These sites are easy for webmasters to update: as product offerings or prices change, just edit your database instead of hundreds of individual Web pages.

Search engine spiders have a much tougher time with dynamic sites. Some get stuck because they can’t supply the information the site needs to generate the page. Other spiders deliberately stay away from dynamic pages to avoid getting trapped in the site.

What Was The Question Again?

Visitors find information in a dynamic site by using a search query. That query can either be typed into a search form by the visitor or already be coded into a link on the home page - making the link a pre-defined search of the site’s catalog. In that later case, the portion of the link containing the search parameters is called a ‘query string.’

But a search engine spider doesn’t know to use your search function - or what questions to ask. Dynamic scripts often need certain information before they can return the page content: cookie data, session id, or a query string are common requirements. Spiders usually stop indexing a dynamic site because they can’t answer the question.

If the spider does accidentally wander deeper into your site, it could inadvertently get caught in a “spider trap”: a badly written CGI script that requests information the spider can’t supply. Then the spider and your server navigate a never-ending loop where a request for a page is met with a request for information.

Getting a spider trapped inside your server is not just bad for the spider. The repeated requests for pages can crash the server. If you share server space with other Web sites and have a problem with site downtime, ask your Web host to check for CGI script problems on other sites.

It’s All In The Name

A page’s actual URL address often poses a problem too because most dynamic page URLs contain query strings. Here’s an example of the URL for a book search result page on Barnes and Noble’s Web site:

http://shop.barnesandnoble.com/booksearch/isbnInquiry.asp?
userid=2IMXLT5XN1&mscssid=QEUFGRFF5X2G9H2UCMJQLAKJ8
JV83FMD&isbn=0452269350

Look closely at the URL. See the question mark after /isbnInquiry.asp? Most search engine spiders get to the “?” in the query string and stop indexing because of the probability of getting caught in a spider trap.

Attract Spiders To Your Web

So, you’ve got all this invisible content - what do you do? Search engines know about the problem, but most have shown very little interest in addressing it. Infoseek and HotBot are the exception. Their search engine spiders can index dynamic page content, but don’t do it automatically. You have to invite them in.

HotBot recommends that you submit your dynamic page with all the arguments added onto the URL (www.website.com/products/search/product_query.asp?prod_id=22929). You can also submit a static page that contains links to the dynamic URLs you need indexed.

Infoseek’s spider, called Slurp, will index dynamic pages that you submit, but won’t crawl through your dynamic Web site by default.

You do have options to get indexed by the other search engines, but no matter which you select, you’ll have to spend some time and effort to make sure your dynamic content gets indexed.

Add Dynamic Links To Static Pages

Include links to important dynamic content on your static pages. The simplest way is a straightforward table of contents page that links to your most important dynamic pages. It gives spiders a way to index content without having to answer any questions. If you have a small site with few products, this is a quick way to get more of your content indexed.

However, the table of contents won’t help you with search engine spiders that stop at query strings. Increase your chances by including good, descriptive links to your major product categories on a static products page. Search engines that stop at query strings will still index the content of the products page - including your link titles. Other search engines that can follow dynamic links can visit the actual dynamic page content without a query.

Remove Query Strings From Dynamic URLs

Amazon.com uses this method to get its product selections indexed by search engines. For instance, a search on Google for Rachael Carson’s book, Silent Spring, returns a result that takes you directly to the appropriate dynamic page at Amazon: www.amazon.com/exec/obidos/ISBN=0395683297/thenatiowomensha/. Because the URL doesn’t contain any query strings, all search engines can index Amazon’s product line.

This method works, but it’s also the most technically demanding solution. If you decide to use this method, you can select from several different options, depending on the type of Web server you use and the software you’re using to integrate your database with your Web site:

Cold Fusion: Reconfigure your Cold Fusion setup to replace the “?” in a query string with a ‘/’ and pass the value to the URL. The browser interprets that as a static URL page.

Instead of http://www.mystore.com/products.cfm?prod_id=22343, you get a string like this: http://www.mystore.com/products.cfm/22343.

CGI Scripts: Path_Info (or Script_Name) is a variable in a dynamic application that contains the complete URL address (including the query string information). Write a script that strips out all the information before the query string and set the balance of the information equal to a variable. You can then use that variable in your URL address.

Apache: has a special rewrite module that allows you to translate URL’s containing query strings into URL addresses that search engine spiders can follow. The module, mod_rewrite, isn’t automatically installed with Apache software. Check with your Web host or administrator and see if it’s available on your server.

Visit the Apache Web site for more information on the mod_rewrite module.

Active Server Pages: Most search engines will index .asp pages if the “?” is removed from the URL. XQASP offers a product that will automatically remove the query strings from your .asp pages and replace them with “/” marks.

A note of caution: these four methods make your dynamic page appear to have its own sub-directory, so the browser will look for images and links there. You can completely avoid broken links and pages by using all absolute URL addresses on your page, but that will make maintenance more difficult later. Alternatively, use URL addresses that are relative to the root directory of your site, not the document. Use /homepage.htm instead of ../homepage.htm and you’ll be fine.

Remember The Rules

Don’t get so caught up in modifying your page design or URL addresses that you forget the basic rules for search engine optimization. Your pages need to have good content, META tags, a high link popularity score, appropriate keywords, and more before you can climb to the top of the search engine ranking.

http://webdevelopersjournal.com/

AOL’s disclosure last week of the Web-search habits of more than 650,000 of its unwitting members has fanned privacy fears and rekindled the debate over the collection and storage of this often private information.

The incident revived concerns that surfaced last year when AOL and other major search companies received subpoenas from the Justice Department for their search data. Time Warner Inc.’s AOL, Yahoo Inc. and Microsoft Corp. cooperated, while Google Inc. challenged the request.

Most search engines record the terms users enter, along with the time of the requests and information on the computer used. While the sites have adopted privacy policies that restrict how this information can be used or shared, there are few laws or regulations that address this issue.

Earlier this year, Rep. Ed Markey (D., Mass.) proposed a bill that would require Internet companies to delete “obsolete” data containing personal information. Meanwhile, Rep. Diana DeGette (D., Colo.) proposed legislation that would require Internet service providers to retain subscriber information for potential law-enforcement use.

Should Internet companies store search queries? Are new laws or closer oversight of the industry warranted? The Wall Street Journal Online invited Kevin Bankston, a lawyer at the Electronic Frontier Foundation, a privacy rights group, to debate the issue with Markham Erickson, executive director of NetCoalition, a lobby group for Internet firms including Google and Yahoo. Their conversation, carried out over email, is below.

Mr. Erickson begins: Search queries are stored and used by Internet companies for internal purposes. Unfortunately, AOL made a mistake in publishing its subscribers’ online search requests. This should not have happened, and I hope every Internet company is evaluating its information procedures and policies to ensure that kind of thing does not happen again.

(more…)

If you’re a webmaster, you’ve probably heard about Google’s Adsense. Perhaps you’re even earning from it already. This article will show you a method to increase your Adsense revenue dramatically.

Google’s Adsense allows webmasters to place links from Google’s Adwords advertisers on their webpages by inserting a special code. When a visitor clicked on a link, the webmaster gets a cut of the profit Google made from the advertiser. The unique feature about Adsense is that the links it generates on your webpage are relevant to your page content. Thus, your keywords and topics become very important determinant for the type of links that will appear on your webpage.

There are only two factors that determine how much a webmaster can make from Adsense. The first factor is the number of visitors who click on the links. The more people who clicked on the links, the more you make. Naturally, this means that the more targetted traffic to the website, the more you’ll make. This is where most webmasters focus their efforts on.

The second factor is the value of the links they clicked on. Here’s where you can make a great difference to your earnings from Adsense and is our focus in this article.
(more…)

Is this topic necessary? After all, it is the Internet; just put up a site, offer content, optimize for search engines, and voila! You have hits. Well, maybe you have hits, and maybe you don’t.

And if you do have hits, how long will your traffic grow? How far can you expand? How big can you become? What is the difference between a small, profitable operation like www.internetmarketingsecrets.com, or www.freewebmasterhelp.com, and a large household name like www.devshed.com. I mean, they all offer great content, they all optimize their sites for the SERPs (Search Engine Ranking Pages) but only one of them is a household name, relatively speaking, on the Internet.

The Bad News

In the real (as opposed to virtual) world of traditional marketing, it is the biggest brands that have the largest market share. In fact, market share is not determined by quality of service or excellence of product. It is a question of branding. Does Daimler Chrysler make the best cars? Or does Nokia really make the best phones? If they do, will they still make such great products tomorrow? I answer, who cares? I have my mind made up. Don’t confuse me with the facts!

On the Internet, Google has branded itself as the search engine. The terms “google” and “googling” even made it into the Merriam-Webster dictionary recently to signify using the Google search engine to search online. Despite the distinction of using a lower case letter and the fact that it specifically refers to just the Google search engine, Google risks its name becoming like Levis, which I found in the Oxford Advanced Learners dictionary defined as “jeans.” Amazon has branded itself for books, and eBay for auctions; devshed has branded itself through its network of sites for content — specifically, for information about open source software.

A brand will beat a better product or service ALL the time, not once, not nine out of ten times, but all the time. In traditional marketing, if the customer wants to choose any other product apart from the brand, he will easily find an alternative (a second brand that competes in that space, like Coke and Pepsi). In the online world, however, there usually is no second. Second in a category can spell death (read as “dependence on SERPs”). (more…)