Description

The VBA Web Scraping Kit is a collection of VBA scripts that can be used for Web Scraping in a variety of scenarios making it possible to scrape almost any content. Why waste time Googling, writing scripts through trail and error when you can get a whole package of ready and tested script together with working live examples!

What the VBA Web Scraping kit contains

The kit contains a collection of scripts that support most Web Scraping scenarios including both direct HTTP calls (GET and POST) and simulating browser interaction (via Internet Explorer).

Examples in the kit

The kit comes with several examples that showcase the capabilities and functions included in the Web Scraping kit. These examples include:

Scraping a HTML Table

HTML Tables and Divs are typical structures from which you need to scrape data sets. The first example shows how you can use Regex and XPath to loop through the rows and items of a HTML table with ease and print the results.

Scraping a news headline

Is scraping a news headline hard? Not in the least. In just 6 lines of code learn how to scrape that headline right into your script.

Filling online forms

Forms, buttons, drop-downs – all these require often actual user input. It is often not so easy to use HTTP calls (GET or POST) to submit a form and interact with a web page. The examples shows how you can leverage simulated user interaction to input a value into a web form, hit submit and recover the results.

Check page status

Are you a Page Web master looking to make sure your webpage is up? Or maybe you are a SEO expert looking to find broken links (404)? The kit includes a simple example of how to check that web page status together with an entire list of HTTP status codes and descriptions.

HTTP GET or POST?

Not all pages/API calls submit parameters via GET (the URL), especially since you don’t face similar limits when using HTTP POST. This will show you the importance of learning how to encode and decode string params.

Who will find it useful?

Beginner Web Scrapers will find simple Web Scraping examples that are reusable and well adaptable to most web scraping scenarios – like scraping data of Wikis, Blogs and other static websites, as well as filling forms via user interaction simulation. VBA is available via Excel/Access in all office workstations without any installation consent needed from IT, this makes the kit useful as it requires no installation.

Advanced Web Scrapers will certainly appreciate a well prepared collection of various tested VBA web scraping tools. No more waiting on Internet Explorer’s the non-reliable IE “is busy” property – simply wait until a control is available on the webpage with a defined timeout. Why not use XPath or Regex to scrape those elements instead of looping through elements one-by-one. Whether you are a practiced SEO, Web Master or Web Scraper, you will definitely find this kit useful.

Trouble-shooting

The VBA Web Scraping Kit leverages Microsoft Internet Explorer built-in components. Hence, in case of issues e.g. “Access Denied” VBA error please first make sure your Internet Explorer settings are configured correctly:
Internet Options -> Security Tab > Custom Security Level -> Miscellaneous -> Access Data Sources Across Domains -> Enable

Having issues with purchasing/downloading the AddIn? Write directly to me at analystcave(at)gmail.com!