|
| Home | Accessibility | Courses | Diary | The Mouth | Forum | Resources | Site Map | About Us | Contact |
 Well House Consultants
You are on the site of
Well House Consultants
who provide
Open Source Training Courses
and business
hotel accommodation. You are welcome to browse and use
our resources subject to our copyright statement and to add in links from your pages to ours.
Other subject areas - resources
Java ResourcesWell House Manor Resources Perl Resources Python Resources PHP Resources Object Orientation and General topics MySQL Resources Linux / LAMP / Tomcat Resources Well House Consultants Resources Extras Resources C and C++ Resources Ruby Resources Tcl/Tk Resources Web and Intranet Resources | Robots, Crawlers and Spiders - Perl module P608
Although most users of the web will be seated at a browser and will call up pages one by one, there's also a requirement for automated browsing tools. For example, a search engine such as Google will methodically visit a site page by page, indexing the entire content for its customers, and a web site validation program will visit each page in turn in order to find any broken links before live users do. Perl is an excellent language for writing automated browsing tools such as these, and in this module we study the techniques you may wish to use, and also the etiquette involved in writing socially acceptable automata.
This topic is presented on public courses Using Perl on the Web, Perl Extra Background information
Some modules are available for download as a sample of our material or under an Open Training Notes License for free download from http://www.training-notes.co.uk.
Topics covered in this module
Retrieving pages automatically.
Checking links. Checking and analysing a site. Providing a Web frontend for these facilities. Retrieving a site for later viewing. Acceptable and unacceptable behaviour of automated clients. Complete learning
If you are looking for a complete course and not just a information on a single subject, visit our Listing and schedule page. Well House Consultants specialise in training courses in Python, Perl, PHP, and MySQL. We run Private Courses throughout the UK (and beyond for longer courses), and Public Courses at our training centre in Melksham, Wiltshire, England. It's surprisingly cost effective to come on our public courses - even if you live in a different country or continent to us. We have a technical library of over 600 books on the subjects on which we teach. These books are available for reference at our training centre. Also available is the Opentalk Forum for discussion of technical questions. |
PH: 0800 043 8225 or 01225 708225 • FAX: 0845 8382 405 or 01225 707126 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho