How To Use Keyword Planner From Google Ads API With Python
In this post, we’re going to make a Python script for fetching keyword ideas with Google Ads API. Furthermore, the functionality this API offers us goes beyond the limits of what we can get from Keyword Planner.
There is a twist to it though. Before we can dive into code, you’ll need to set up your Google Ads Manager Account. Actually, you’ll need to create two of them. One for testing (closed) and another for production (open). You’ll get your API Developer key from the production one.
Once you set up your production account and get the developer token, you will only have access to permissions that Test accounts offer. One of which is, that you can only access your Google Manager test account.
In case you’re confused, test account is different than the manager account that has API access type for test accounts. However, you’ll be able to tell if you’re on a test account by the Test account label inside a red rectangle in the dashboard menu.
Okay, before I get ahead of myself, let’s first list all the information we’re going to need.
- Developer token
- Client ID
- Client secret
- Refresh token
- Customer ID
Setting up
Alright, if you followed my instructions so far, you should be able to acquire the Developer token from the Google manager account’s API Center. Keep in mind that you’ll unlock API Center, to make it appear in the menu, by activating the Google manager account.
Next thing we need to take care of, is getting the client ID and client secret from the Google Cloud Console. You can also access a variety of APIs from there, but for the sake of this project, we’ll focus only on Google Ads API.
Google Cloud Console
I’ll assume that you’ve never heard of this service before and start from the very top. Alright, first things first, you need to create a project. This will allow you to create various different access tokens, like API keys, credentials and service accounts.
Next, you’ll need to click on the APIs & Services and go the OAuth consent screen options. If you don’t have it already, set the Publishing status to Testing. After that, you’ll need to add test users, where you add your email address.
Okay, now let’s get to creating the credentials. In order to do this, go to the Credentials link in the sidemenu. Once you open the credentials settings, click on top the CREATE CREDENTIALS button and choose Oauth client ID.
After you create the OAuth client ID, you can download the json file containing all your credentials information. I also recommend you rename the file into something simple like credentials.json.
Before we move on, you should also enable the Google Ads API, which you can find in the API library.
Refresh token
In the next step of our lenghty setup process, we’ll need to get the refresh token from a tool called OAuth2l (pronounced “OAuth Tool”). In order to make this work on Windows, I downloaded the pre-compiled binary for Windows and unzipped it.
Once I had the oauth2l.exe sitting in a folder on my hard drive, I added the folder path among the PATH variable in the Environment Variables. You can find these settings by typing in the Windows search bar “Environment Variables”. For different languages these options will be translated so keep that in mind if you can’t find it on your computer.
Once you open these options, a window will pop up, where you’ll need to navigate to Advanced tab and click on the Environment Variables… button.
There, you need to find Path under System variables (bottom section) and double click on it. Another window will open, where you’ll be able to add the OAuth2l folder path at the bottom.
Finally, open the command prompt on your Windows with Admin privilages and run the following command.
oauth2l fetch --credentials credentials.json --scope adwords --output_format refresh_token
It is also important that you navigate to the folder where you saved your credentials.json file in command prompt, before you run the command.
Once you run the command, you should see an output from oauth2l in a json format. From there you should find the refresh token and copy its value. You will also need to confirm access of your Google project application when the tool automatically opens your browser.
Creating config file for Google Ads API
If you made it this far, GREAT! We’re almost done setting up the credentials and whatnot to gain access to the Google Ads API. Last thing we need to do is create a .yaml configuration file, where we’ll store all these tokens.
We also need to set whether or not we want to use proto-plus messages. In case you missed this step, you might run into an error.
Alright the contents of your google-ads.yaml configuration file should look something like the following snippet, you just need to input your own information.
developer_token: <your-developer-token>
client_id: <your-client-id>
client_secret: <your-client-secret>
refresh_token: <your-refresh-token>
use_proto_plus: "True"You can find Client ID and Client Secret in the credentials.json file you downloaded earlier. Or you can get it from Google Cloud Console under credentials section of your project.
Amazing, for this project I also created a .env file where I saved the Customer ID. This should be the ID number of your Test Google manager account, without the dashes (“-“) between the numbers.
Coding Keyword Idea Generator
Amazing, now we’re finally ready to start coding our project. Therefore, like with any other Python project, we need to import all the necessary modules and tools.
import os
import argparse
import pandas as pd
from dotenv import load_dotenv
from google.ads.googleads.client import GoogleAdsClient
from google.ads.googleads.errors import GoogleAdsExceptionNext, we’ll load the .env file and define a couple of constants for later use.
load_dotenv()
ROOT = os.path.dirname(__file__)
CUSTOMER_ID = os.getenv('CUSTOMER_ID')
LOCALE = 'en'
COUNTRY_CODE = 'US'
_DEFAULT_LOCATIONS = ['United States']
_DEFAULT_LANGUAGE_ID = '1000'Getting keywords using Google Ads API with Python
Now, let’s focus on getting those keyword ideas and create a method that will return them. Furthermore, this method is going to be the heart of this operation, containing all the API requests.
def get_keywords(client, keyword_texts,
locations=_DEFAULT_LOCATIONS,
language_id=_DEFAULT_LANGUAGE_ID,
page_url=None):
keyword_plan_idea_service = client.get_service('KeywordPlanIdeaService')
keyword_plan_network = (
client.enums.KeywordPlanNetworkEnum.GOOGLE_SEARCH_AND_PARTNERS
)
location_rns = []
gtc_service = client.get_service('GeoTargetConstantService')
gtc_request = client.get_type('SuggestGeoTargetConstantsRequest')
gtc_request.locale = LOCALE
gtc_request.country_code = COUNTRY_CODE
gtc_request.location_names.names.extend(
locations
)
gtc_results = gtc_service.suggest_geo_target_constants(gtc_request)
for suggestion in gtc_results.geo_target_constant_suggestions:
location_rns.append(suggestion.geo_target_constant.resource_name)
language_rn = client.get_service('GoogleAdsService').language_constant_path(language_id)
if not (keyword_texts or page_url):
raise ValueError(
'At least one of keywords or page URL is required'
)
request = client.get_type('GenerateKeywordIdeasRequest')
request.customer_id = CUSTOMER_ID
request.language = language_rn
request.geo_target_constants = location_rns
request.include_adult_keywords = False
request.keyword_plan_network = keyword_plan_network
if not keyword_texts and page_url:
request.url_seed.url = page_url
if keyword_texts and not page_url:
request.keyword_seed.keywords.extend(keyword_texts)
print('Keywords', request.keyword_seed.keywords)
if keyword_texts and page_url:
request.keyword_and_url_seed.url = page_url
request.keyword_and_url_seed.keywords.extend(keyword_texts)
keyword_ideas = keyword_plan_idea_service.generate_keyword_ideas(request)
return keyword_ideasPutting the whole algorithm together
Here we’re going to define everything that should happen when we run the script. Furthermore, we’re going to add a couple of arguments like seed keywords, locations and more.
This will allow us to use the script directly from command prompt. Furthermore, we’ll also set some default values for locations and languages. This way, we can have optional arguments and only input the keywords.
However, before we get to that, it’s important that we declare a client for Google Ads API in our Python project, which will take the path of the google-ads.yaml configuration file as an argument.
google_ads_client = GoogleAdsClient.load_from_storage(os.path.join(ROOT, 'google-ads.yaml'))Now, we’ll get to declaring arguments, generating keyword ideas and saving them in an excel file including the average monthly searches and competition level.
We’re also going to wrap this into a try and except statement in case there is any trouble with sending requests to the API.
parser = argparse.ArgumentParser(
description='Returns keyword ideas including their Google search metric data.'
)
parser.add_argument(
'-k',
'--keyword_texts',
type=list_of_items,
required=False,
default=[],
help='List of starter keywords'
)
parser.add_argument(
'-l',
'--locations',
type=list_of_items,
required=False,
default=_DEFAULT_LOCATIONS,
help='List of location names'
)
parser.add_argument(
'-i',
'--language_id',
type=str,
required=False,
default=_DEFAULT_LANGUAGE_ID,
help='The language criterion ID'
)
parser.add_argument(
'-p',
'--page_url',
type=str,
required=False,
help='A URL address for extracting relevant keywords from'
)
args = parser.parse_args()
try:
keyword_ideas = get_keywords(
client=google_ads_client,
keyword_texts=args.keyword_texts,
locations=args.locations,
language_id=args.language_id,
page_url=args.page_url
)
result = []
for idea in keyword_ideas:
result.append([
idea.text,
idea.keyword_idea_metrics.avg_monthly_searches,
idea.keyword_idea_metrics.competition.name
])
df = pd.DataFrame(result)
df.to_excel(
os.path.join(ROOT, 'output.xlsx'),
header=['Keyword', 'Average monthly searches', 'Competition'],
index=False
)
except GoogleAdsException as ex:
print(
f'Request with ID "{ex.request_id}" failed with status '
f'"{ex.error.code().name}" and includes the following errors:'
)
for error in ex.failure.errors:
print(f'\tError with message "{error.message}".')
if error.location:
for field_path_element in error.location.field_path_elements:
print(f"\t\tOn field: {field_path_element.field_name}")Thats it! Now all we have to do is run the script in command prompt like the following example.
python generate-kw-ideas.py -k "google ads api"
Code for Keyword Ideas Generator Using Google Ads API with Python
Here is the entire code of the project.
import os
import argparse
import pandas as pd
from dotenv import load_dotenv
from google.ads.googleads.client import GoogleAdsClient
from google.ads.googleads.errors import GoogleAdsException
load_dotenv()
ROOT = os.path.dirname(__file__)
CUSTOMER_ID = os.getenv('CUSTOMER_ID')
LOCALE = 'en'
COUNTRY_CODE = 'US'
_DEFAULT_LOCATIONS = ['United States']
_DEFAULT_LANGUAGE_ID = '1000'
def get_keywords(client, keyword_texts,
locations=_DEFAULT_LOCATIONS,
language_id=_DEFAULT_LANGUAGE_ID,
page_url=None):
keyword_plan_idea_service = client.get_service('KeywordPlanIdeaService')
keyword_plan_network = (
client.enums.KeywordPlanNetworkEnum.GOOGLE_SEARCH_AND_PARTNERS
)
location_rns = []
gtc_service = client.get_service('GeoTargetConstantService')
gtc_request = client.get_type('SuggestGeoTargetConstantsRequest')
gtc_request.locale = LOCALE
gtc_request.country_code = COUNTRY_CODE
gtc_request.location_names.names.extend(
locations
)
gtc_results = gtc_service.suggest_geo_target_constants(gtc_request)
for suggestion in gtc_results.geo_target_constant_suggestions:
location_rns.append(suggestion.geo_target_constant.resource_name)
language_rn = client.get_service('GoogleAdsService').language_constant_path(language_id)
if not (keyword_texts or page_url):
raise ValueError(
'At least one of keywords or page URL is required'
)
request = client.get_type('GenerateKeywordIdeasRequest')
request.customer_id = CUSTOMER_ID
request.language = language_rn
request.geo_target_constants = location_rns
request.include_adult_keywords = False
request.keyword_plan_network = keyword_plan_network
if not keyword_texts and page_url:
request.url_seed.url = page_url
if keyword_texts and not page_url:
request.keyword_seed.keywords.extend(keyword_texts)
print('Keywords', request.keyword_seed.keywords)
if keyword_texts and page_url:
request.keyword_and_url_seed.url = page_url
request.keyword_and_url_seed.keywords.extend(keyword_texts)
keyword_ideas = keyword_plan_idea_service.generate_keyword_ideas(request)
return keyword_ideas
def list_of_items(arg):
return arg.split(',')
if __name__ == '__main__':
google_ads_client = GoogleAdsClient.load_from_storage(os.path.join(ROOT, 'google-ads.yaml'))
parser = argparse.ArgumentParser(
description='Returns keyword ideas including their Google search metric data.'
)
parser.add_argument(
'-k',
'--keyword_texts',
type=list_of_items,
required=False,
default=[],
help='List of starter keywords'
)
parser.add_argument(
'-l',
'--locations',
type=list_of_items,
required=False,
default=_DEFAULT_LOCATIONS,
help='List of location names'
)
parser.add_argument(
'-i',
'--language_id',
type=str,
required=False,
default=_DEFAULT_LANGUAGE_ID,
help='The language criterion ID'
)
parser.add_argument(
'-p',
'--page_url',
type=str,
required=False,
help='A URL address for extracting relevant keywords from'
)
args = parser.parse_args()
try:
keyword_ideas = get_keywords(
client=google_ads_client,
keyword_texts=args.keyword_texts,
locations=args.locations,
language_id=args.language_id,
page_url=args.page_url
)
result = []
for idea in keyword_ideas:
result.append([
idea.text,
idea.keyword_idea_metrics.avg_monthly_searches,
idea.keyword_idea_metrics.competition.name
])
df = pd.DataFrame(result)
df.to_excel(
os.path.join(ROOT, 'output.xlsx'),
header=['Keyword', 'Average monthly searches', 'Competition'],
index=False
)
except GoogleAdsException as ex:
print(
f'Request with ID "{ex.request_id}" failed with status '
f'"{ex.error.code().name}" and includes the following errors:'
)
for error in ex.failure.errors:
print(f'\tError with message "{error.message}".')
if error.location:
for field_path_element in error.location.field_path_elements:
print(f"\t\tOn field: {field_path_element.field_name}")Conclusion
To conclude, we made a Python script for generating keyword ideas by using Google Ads API. We went through the whole process of gaining access. I learned a lot while working on this project and I hope you will find it helpful as well.
