Web Scraping with Python

Nonfiction, Computers, Internet, Web Development, Programming
Cover of the book Web Scraping with Python by Richard Lawson, Packt Publishing
View on Amazon View on AbeBooks View on Kobo View on B.Depository View on eBay View on Walmart
Author: Richard Lawson ISBN: 9781782164371
Publisher: Packt Publishing Publication: October 28, 2015
Imprint: Packt Publishing Language: English
Author: Richard Lawson
ISBN: 9781782164371
Publisher: Packt Publishing
Publication: October 28, 2015
Imprint: Packt Publishing
Language: English

Successfully scrape data from any website with the power of Python

About This Book

  • A hands-on guide to web scraping with real-life problems and solutions
  • Techniques to download and extract data from complex websites
  • Create a number of different web scrapers to extract information

Who This Book Is For

This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved.

What You Will Learn

  • Extract data from web pages with simple Python programming
  • Build a threaded crawler to process web pages in parallel
  • Follow links to crawl a website
  • Download cache to reduce bandwidth
  • Use multiple threads and processes to scrape faster
  • Learn how to parse JavaScript-dependent websites
  • Interact with forms and sessions
  • Solve CAPTCHAs on protected web pages
  • Discover how to track the state of a crawl

In Detail

The Internet contains the most useful set of data ever assembled, largely publicly accessible for free. However, this data is not easily reusable. It is embedded within the structure and style of websites and needs to be carefully extracted to be useful. Web scraping is becoming increasingly useful as a means to easily gather and make sense of the plethora of information available online. Using a simple language like Python, you can crawl the information out of complex websites using simple programming.

This book is the ultimate guide to using Python to scrape data from websites. In the early chapters it covers how to extract data from static web pages and how to use caching to manage the load on servers. After the basics we'll get our hands dirty with building a more sophisticated crawler with threads and more advanced topics. Learn step-by-step how to use Ajax URLs, employ the Firebug extension for monitoring, and indirectly scrape data. Discover more scraping nitty-gritties such as using the browser renderer, managing cookies, how to submit forms to extract data from complex websites protected by CAPTCHA, and so on. The book wraps up with how to create high-level scrapers with Scrapy libraries and implement what has been learned to real websites.

Style and approach

This book is a hands-on guide with real-life examples and solutions starting simple and then progressively becoming more complex. Each chapter in this book introduces a problem and then provides one or more possible solutions.

View on Amazon View on AbeBooks View on Kobo View on B.Depository View on eBay View on Walmart

Successfully scrape data from any website with the power of Python

About This Book

Who This Book Is For

This book is aimed at developers who want to use web scraping for legitimate purposes. Prior programming experience with Python would be useful but not essential. Anyone with general knowledge of programming languages should be able to pick up the book and understand the principals involved.

What You Will Learn

In Detail

The Internet contains the most useful set of data ever assembled, largely publicly accessible for free. However, this data is not easily reusable. It is embedded within the structure and style of websites and needs to be carefully extracted to be useful. Web scraping is becoming increasingly useful as a means to easily gather and make sense of the plethora of information available online. Using a simple language like Python, you can crawl the information out of complex websites using simple programming.

This book is the ultimate guide to using Python to scrape data from websites. In the early chapters it covers how to extract data from static web pages and how to use caching to manage the load on servers. After the basics we'll get our hands dirty with building a more sophisticated crawler with threads and more advanced topics. Learn step-by-step how to use Ajax URLs, employ the Firebug extension for monitoring, and indirectly scrape data. Discover more scraping nitty-gritties such as using the browser renderer, managing cookies, how to submit forms to extract data from complex websites protected by CAPTCHA, and so on. The book wraps up with how to create high-level scrapers with Scrapy libraries and implement what has been learned to real websites.

Style and approach

This book is a hands-on guide with real-life examples and solutions starting simple and then progressively becoming more complex. Each chapter in this book introduces a problem and then provides one or more possible solutions.

More books from Packt Publishing

Cover of the book Joomla! 1.5 Content Administration by Richard Lawson
Cover of the book Mastering Management Styles: Expert Guidance for Managers by Richard Lawson
Cover of the book Getting Started with Meteor.js JavaScript Framework - Second Edition by Richard Lawson
Cover of the book Hacking Android by Richard Lawson
Cover of the book Apache Solr Essentials by Richard Lawson
Cover of the book CCENT/CCNA: ICND1 100-105 Certification Guide by Richard Lawson
Cover of the book Installing and Configuring Windows 10: 70-698 Exam Guide by Richard Lawson
Cover of the book PrimeFaces Cookbook - Second Edition by Richard Lawson
Cover of the book Microsoft SharePoint 2010 and Windows PowerShell 2.0: Expert Cookbook by Richard Lawson
Cover of the book Arduino by Example by Richard Lawson
Cover of the book Continuous Delivery with Docker and Jenkins by Richard Lawson
Cover of the book Learning Network Forensics by Richard Lawson
Cover of the book Node Web Development, Second Edition by Richard Lawson
Cover of the book Microsoft SQL Server 2012 with Hadoop by Richard Lawson
Cover of the book Neural Networks with Keras Cookbook by Richard Lawson
We use our own "cookies" and third party cookies to improve services and to see statistical information. By using this website, you agree to our Privacy Policy