Food For Thought...
Spider Food That Is

By Andrew Hazen

As searching remains the second most popular activity and function of the Internet (e-mail is number one), prime search engine visibility and strategic Internet marketing have proven to be vital to any business that wants to succeed online.  Interestingly enough however, most website designers and programmers have little understanding of search engine optimization and most businesses wait until after their website is developed before they seek search engine optimization services.  This is simply poor planning and has negative effects on the success of any Internet venture.  Why design and develop your hotel’s website if it will not be found in the top of the major Internet directories and search engines?  Websites that utilize FLASH, frames, dynamically generated web pages, or a plethora of graphics may look impressive but they are far from being search engine friendly! Directories such as Yahoo! and LookSmart are maintained by human editors who review and index site submissions.  Search engines on the other hand use technology referred to as “spidering”. Spiders crawl the web and read web pages and then follow the links to other web pages within the site.  Everything spiders find go into an index, which search engine software sifts through to find matches to search queries and rank them in relevant order.  Spiders crawl websites that it finds via either Submit URL link, a Paid Inclusion Program, or a link from another web page.  Each search engine has its own algorithm and set of rules relating to how often the web page can be submitted, how much weight the META data receives, how the pages should be indexed, and so on.  Additionally, each search engine has its own schedule as to how often it updates and refreshes by crawling and spidering its index and the web.  Therefore, it is possible for your web page to be spidered but not yet indexed.  Ideally, you want to have as many optimized web pages indexed as you can in any given top search engine.  The following search engines use spiders to maintain their index:  Google, Inktomi,  MSN, Lycos, HotBot, AllTheWeb (FAST). When optimizing your website for a search engine spider, the following should be taken into account:          -          page content is crucial -          link analysis is becoming vital -          title tags, file names and link architecture are important -          META tags may help -          design issues have an impact To succeed in the search engines, your website must have text, especially your home page and service pages.  Spiders can only read and crawl text; spiders cannot read images, FLASH files, dynamically generated web pages, or pages within websites that use frames.  Spiders also crawl, read and index those links found on the web pages it’s crawling.  Therefore, a site with the right link architecture can have its entire site crawled by submitted its home page via the proper submission vehicles. Many B2C and B2B websites use shopping cart systems that are most likely dynamic or database driven.  This is fine as long as there are enough static pages with text throughout the main navigation. Because so many websites are being developed without proper title tags, META data, alt tags, site architecture and page names their online visibility, traffic and sales are not up to par.  However, numerous case studies confirm that companies implementing search engine optimization and search engine marketing receive a positive return on their investment.  All Internet ventures should be strategically planned from inception.  From procuring the right domain name and choosing the proper site architecture to implementing effective marketing tactics; every decision should be made with the search engine spiders in mind - just some food for thought.  Andrew S. Hazen andrew@primevisibility.com is the Founder and CEO of Prime Visibility LLC http://www.PrimeVisibility.com, a leader in search engine optimization and search engine marketing.