Skip to main content

Library Home: Deep Research

Deep Web Video

Image from http://salaswildthoughts.blogspot.com/2008_04_01_archive.html

The deep web includes content in searchable databases available to web users but not accessible by popular search engines. This video provides an introduction to the deep web search engine and how it can be used to access research and development information contained in previously hard-to-find databases.

Books in our Library

2010 Statistics

Size estimates from Google.com, Yahoo.com, Cyberatlas, and MIT. These stats are current to Spring 2010:

  • Google.com indexes 23.5 billion public web pages.

  • 80+ billion static web pages are publicly-available. These pages can easily be found by Google and other search engines. (e.g. www.honda.com, www.australia.gov.au)

  • 9.5 billion static pages are hidden from the public. As private intranet content, these are the corporate pages that are only open to employees of specific companies. (e.g. employees.honda.com, secure.australia.gov.au)

  • 300+ billion database-driven pages are completely invisible to Google. These invisible pages are not the regular web pages you and I can make. Rather, these are dynamic database reports that exist only when called from large databases.
    (e.g. custom online car quote for Shelly, Australian government discussion on aboriginal taxation)

From about.com, What is the 'Invisible Web'? by Paul Gil

Tools to Access the Deep Web

Invisible Web Research Tools

 

http://www.weblens.org/invisible.html

Hidden Web Tools

 

Deep Web / Invisible Web / Hidden Web

What is the Invisible Web?

Google, MSN and Yahoo only access the top 20% of the information available on the web - the remainder is either hidden but freely available with the right search tools, or it is locked behind password-protected websites - eg our school databases.

If you learn to use some of the search tools mentioned below, you will be able to access a lot more high-quality information.

Image from http://www.juanico.co.il/index.html

Follow these links for more information:

Invisible Web: What it is, Why it exists, How to find it

About's Invisible Web Pages

BrightPlanet White Paper

Marcus Zillman's Deep Web Research

What is the 'Invisible Web'?

Deep Web Search Tools

 There are a lot of different search tools freely available to help you access the Deep Web.

Don't be put off by the number of different ones available - read about them and try a few different ones.

  • For database searches, you can begin with CompletePlanet or Geniusfind.
  • For reference material searches, try LibrarySpot.com, INFOMINE or Internet Public Library.
  • There are many specialty sites. For instance, science.gov provides links to the U.S. government's scientific deep web site. FreeLunch.com contains economic data from across the world. NHS Direct has health information and advice within England. You can find a good starter list of specialty sites at Online Education Database, or use one of the search engines below.
  • You can use a search engine that helps you find the right search engine, such as Search Engine Guide or Geniusfind. Search Engine Guide works best if you choose a topic before you do a search, while Geniusfind requires you pick a topic first.
  • If you aren't sure what you need, there are some general search sites for the deep web such as Internet Archive and Clutsy that make a good starting point.

Information from WikiHow

 

 

Useful Tools for Accessing the Deep Web :

CompletePlanetTM uses a query based engine to index 70,000+ deep Web databases and surface Web sites. Appendix A lists 60 of the largest deep Web databases which contain 10% of the information in the deep Web, or 40 times the content of the entire surface Web. These 60 databases are included in CompletePlanet’s indexes. CompletePlanet is sponsored by BrightPlanet® Corporation, a leader in deep Web searches. The interface is intuitive and easy to use. You can do a keyword search on all 70,000+ databases to find which databases to use for your search. You can also browse by category, and then search databases of interest.

ProFusion is a combination of query based engine and a deep Web directory portal. The directory structure is accessed by clicking on Specialized Searches. With an account, you can setup custom “My Search Groups” to search customized lists of websites and/or databases of your choice. For example, you could create a group called Technology and add all the databases and websites of interest to you. This group is saved to your profile. You could then, at any future time, search this group on a research topic with keywords. This is a great time saver. Their query based engine is called SmartDiscovery®. 

 

SurfWax also uses a site's existing search capability as part of the meta-search process to tap the deep Web. They use proprietary algorithms to interpret the site's search criteria (Boolean, etc). With an account, you can also setup custom SearchSets to search customized lists of websites and/or databases of your choice. Surfwax also has a news accumulator feature with over 50,000 news topics in 84 categories. This news accumulator feature is a godsend providing high quality results. These are some useful news accumulator categories: all topics, networking, technology, telecommunication, and web services. In addition this site has WikiWax which takes the online encyclopedia Wikipedia to the next level. WikiWax does advanced look-aheads on Wikipedia searches to speed your keyword choices.

 

Information from http://techdeepweb.com/4.html