9

Im querying google ads api and need to save results as json. What is the best way to convert GoogleAdsRow type into json?

The result of the google ads appi look like this (the campaign and customer ids are fake):

campaign {
  resource_name: "customers/752830100/campaigns/22837002"
  id {
    value: 22837002
  }
  name {
    value: "test"
  }
}
metrics {
  clicks {
    value: 51
  }
  impressions {
    value: 33
  }
}

type = <class 'google.ads.googleads_v1.types.GoogleAdsRow'>

5 Answers 5

12

Google.protobuf has a method called json_format which can do exactly that. Below is a code example to help:

# Import the method 
from google.protobuf import json_format
import json

# Query the Ads API, 
ga_service = client.get_service('GoogleAdsService', version='YOUR_VERSION')
query = ('YOUR_QUERY')

# Response as an iterator made up of GoogleAdsRow  
response = ga_service.search(customer_id, query)

# Converting each of the GoogleAdsRow to json
for row in response : 
    json_str = json_format.MessageToJson(row)
    d = json.loads(json_str)
3
  • 2
    Amazing, thanks for the share! And in case anyone ended up here because they wanted to convert google ads api result to a dataframe, you just need to add the following line to the above: df = pd.json_normalize(data=d['results'])
    – Saint
    Commented Dec 11, 2020 at 14:34
  • 4
    This worked for me but with the addition of ._pb to my row object, e.g. MessageToJson(row._pb). Otherwise I was getting a "DESCRIPTOR" error message. See this answer: stackoverflow.com/a/68216092, section 3. Commented Oct 12, 2021 at 2:59
  • 3
    change MessageToJson to MessageToDict and you can skip the loading with json Commented May 3, 2022 at 1:26
3

Right now I am also looking for the same problem. One workaround that I have found is that use a CustomEncoder in json.dumps. Here is a sample:

class CustomEncoder(json.JSONEncoder):
def default(self, obj):
    if isinstance(obj, (google.protobuf.wrappers_pb2.StringValue,
                        google.protobuf.wrappers_pb2.Int64Value, 
                        google.protobuf.wrappers_pb2.DoubleValue)):
        return obj.value

    elif isinstance(obj, google.protobuf.pyext._message.RepeatedCompositeContainer):
        data = []
        try:
            while True:
                item = obj.pop()
                data.append(self.default(item))
        except IndexError:
            return data

    elif isinstance(obj, google.ads.google_ads.v1.proto.common.custom_parameter_pb2.CustomParameter):
        return { 
                self.default(obj.key): self.default(obj.value) 
                }
return json.JSONEncoder.default(self, obj)

Use the above encoder in json.dumps(data, cls=CustomEncoder)

This is the only solution I have so far come with. Will update it if I find a better solution.

Edited: Found the Solution. Here is the New Encoder class.

class GoogleProtoEncoder(json.JSONEncoder):
"""
Custom JSON Encoder for GoogleAdsRow. 

Usage: json.dumps(data, cls=GoogleProtoEncoder)
"""
def default(self, obj):
    """
    Overriden method. When json.dumps() is called, it actually calls this method if 
    this class is specified as the encoder in json.dumps().
    """
    if isinstance(obj, google.protobuf.message.Message) and hasattr(obj, 'value'):
        # This covers native data types such as string, int, float etc
        return obj.value

    elif isinstance(obj, google.protobuf.pyext._message.RepeatedCompositeContainer):
        # This is basically for python list and tuples
        data = []
        try:
            while True:
                item = obj.pop()
                data.append(self.default(item))
        except IndexError:
            return data

    elif isinstance(obj, google.ads.google_ads.v1.proto.common.custom_parameter_pb2.CustomParameter):
        # Equivalent to python dictionary
        return { 
                self.default(obj.key): self.default(obj.value) 
                }

    elif isinstance(obj, google.protobuf.message.Message):
        # All the other wrapper objects which can have different fields.
        return {key[0].name: getattr(obj, key[0].name) for key in obj.ListFields()}

    return json.JSONEncoder.default(self, obj)

Thanks.

Edited: Updated solution. Works in V7

import proto

response = ga_service.search_stream(search_request)
for batch in response:
    for row in batch.results:
        logging.debug(proto.Message.to_dict(row))
3
  • This works great when using previous version of the API. Do you know what has changed in V3? What needs to be updated?
    – Dave Davis
    Commented Jul 10, 2020 at 22:45
  • Hi, I ran into an issue, using the code. Although it works for the vast majority of the time I ran into an issue. Im running v5 as they now allow retrieval of discovery campaigns. I ran into this issue in one of my extractions TypeError: Object of type RepeatedScalarContainer is not JSON serializable
    – spak
    Commented Sep 28, 2020 at 15:26
  • @DaveDavis @spak Apologies.. I haven't looked into the newer versions.. I would recommend you to check the structure of RepeatedScalarContainer and add another elif statement to convert that accordingly. Commented Oct 5, 2020 at 6:45
2

If you don't have to use the SDK, another option to get the json, is to query the API sending a post request directly, which returns a json response:

r = requests.post('https://googleads.googleapis.com/v3/customers/YOUR_CUSTOMER_ID/googleAds:searchStream', 
                 headers={'Authorization': f'Bearer {access_token}',
                         'developer-token' : developer_token,
                         'login-customer-id' : 'YOUR_CUSTOMER_ID',
                         'Content-Type': 'application/json'},
                params={"query" : query})
2

You can use the custom protobuf helper methods now as outlined here: https://developers.google.com/google-ads/api/docs/client-libs/python/library-version-10#wrapped_versus_native_protobuf_messages

campaign = client.get_type("Campaign")
json = type(campaign).to_json(campaign)
campaign = type(campaign).from_json(json)
1

Step 1: Import this lib

from google.ads.googleads.errors import GoogleAdsException

Step 2: Send request

keyword_ideas = keyword_plan_idea_service.generate_keyword_ideas(
    request=request
)

Step 3: Convert response to json

keyword_ideas_json = MessageToDict(keyword_ideas)

Step 4: Do whatever you want with that json

print(keyword_ideas_json)

Note: This might though an attribute error: "DESCRIPTOR" error, so look at this answer: here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.