Search engine spiders and crawlers are little workers that search engines send out to your site in order to see what you have on your site.  Content is usually scanned about 200-300 words deep depending on the size of your page.  Pictures and videos may not be scanned right away on the first visit.  Depending on how many pages your site has, the search engine may not get to all your pages on the first glance.  Having a lot of data or content on your site gives the search engine spiders and crawlers a lot of data and links to scan, and there are trillions of sites out there other than yours.  Don’t be worried that your site will never be fully scanned, it may be scanned throughout on the first pass, or it may not, but either way you need to focus on the SEO for each page one step at a time.  In fact if people are linking to the internal pages within your site, chances are when the search engine crawls the site linking in, they will make it to your site.  So be sure to link to relevant sites to what you do/provide this way the search engine spiders will find relevance to the content on your site and give you a better ranking than what you would get if it was an irrelevant site.  Don’t get me wrong you can still link to sites that are irrelevant, however if your site is filled with sites that have nothing to do with what you do, search engines may consider it “link farming” or a way to cheat the system in order to gain higher rankings.

A search engine spider and a crawler are really the same thing.  They both go out looking for new websites in order to provide the best search results for any given query at any given time.  You could even use code on your site’s source to set rates at which the crawler comes back to your site.  However this could backfire if you don’t change your content or your links on your site from time to time.  And not necessarily change the actual content, but if you were to add a page, or maybe post some news about the company.  Don’t allow the content on your site to get stale and outdated, unless what you are doing is never gonna change, you will be finding yourself lower and lower if you don’t give yourself some prominice over all the other sites that are out there.

So in short, give the search engine crawlers and spiders something to talk about when they come to your site, give them something they are looking for which is good, clean, informative content that allows their searchers to find what they are looking for when they want to find it causing the search engines to give you high rankings, only cause they are looking for the searchers best interest all the time, not the SEO specialist nor the website owner.