Hello fellow dashdashers
Let me introduce you to the holy grail of scraping a.k.a the Google Custom Search Engine (GCSE). It is a super powerful tool that can help you generate sales leads or support your talent acquisition process
These are the steps:
- Get your Google Cloud Platform API key
- Setup a Google Custom Search Engine
- Query your searches via the GCSE integration
- Build cool stuff!
Setup a Google Custom Search Engine
Navigate to the GCSE login page, sign in and be ready to create your very first custom search engine.
Entering a site to search is mandatory in this step. You can delete your choice later, ultimately enabling you to search the entire web. So go ahead and enter any website or multiple ones that you’d like to search on
In your search engine’s setup page you can fine tune the settings and find the ID for this custom search engine. Store it safely—you’ll need it later.
If you’d like to search the entire web, don’t forget to delete any sites you might have previously entered and toggle the search the entire web button.
Query via the GCSE integration
A1 = 01663930225133... A2 = automobile manufacturers Germany A4 = CUSTOM_SEARCH_GOOGLE(A2,A1)
In the JSON object, you can find your search results via
[items:]. In this case, the fourth search result ended up being the Volkswagen Group
Build cool stuff!
The possibilities are endless! Here are some use cases we have already documented for you:
- Build company lists (coming soon)
- Generate leads through LinkedIn
- Identify company workforce size (coming soon)
Alternatively: Query your searches via
If you don’t want to use the GCSE integration, you will have to build a
GET request in order to pull data from the API. Within the GCSE API documentation we can find the standard URL with the mandatory parameters:
https://www.googleapis.com/customsearch/v1?KEY=[YOUR API KEY]&CX=[CSE ID]&Q=[YOUR QUERY]
There a further optional parameters, which you can find here.
Off to the fun part, let’s put everything together and send out the
A1 = AIzaSyDSh7q... A2 = 01663930225133... A3 = automobile manufacturers Germany A4 = GET("https://www.googleapis.com/customsearch/v1?KEY="&A1&"&CX="&A2&"&Q="&A3&"")