This Practice Note explains the manner in which intellectual property and certain other rights, as well as data protection obligations, apply in the context of web crawling, indexing, spidering, caching and scraping on the web, with specific regard to the perspective of a website operator. It explains each of the key terms of crawling, indexing, caching, and scraping. It examines legal issues relating to copyright infringement, database right infringement, website terms of use, compliance with the Robot Exclusion Protocol and instructions in metatags, the Computer Misuse Act 1990 and data protection issues.