API Clients

HTTP Sessions

Todo

Update prose of “…you can pass in your own session into an API client” with specific API client class names once they’re defined. Also update the example code using one API client.

By default, the HTTP session used for getting credentials is reused for API calls (recommended if there are many). If this is not desired, you can pass in your own aiohttp.ClientSession instance into an API client or AIOConnection. The auth client GAuthClient may also take an explicit session object, but is not required to assert a different HTTP session is used for the API calls.

import aiohttp
import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
session = aiohttp.ClientSession()  # optional
auth_client = gordon_gcp.GAuthClient(
    keyfile=keyfile, session=session
)

new_session = aiohttp.ClientSession()

# basic HTTP client
client = gordon_gcp.AIOConnection(
    auth_client=auth_client, session=new_session
)

Asynchronous GCP HTTP Client

Module to interact with Google APIs via asynchronous HTTP calls. AIOConnection is meant to be used/inherited by other product-specific API clients as it handles Google authentication and automatic refresh of tokens.

Todo

Include that it also handles retries once implemented.

To use:

import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
auth_client = gordon_gcp.GAuthClient(keyfile=keyfile)

client = AIOConnection(auth_client=auth_client)
resp = await client.request('get', 'http://api.example.com/foo')
class gordon_gcp.AIOConnection(auth_client=None, session=None)[source]

Async HTTP client to Google APIs with service-account-based auth.

Parameters:
  • auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
  • session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.
get_all(url, params=None)[source]

Aggregate data from all pages of an API query.

Parameters:
  • url (str) – Google API endpoint URL.
  • params (dict) – (optional) URL query parameters.
Returns:

Parsed JSON query response results.

Return type:

list

get_json(url, json_callback=None, **kwargs)[source]

Get a URL and return its JSON response.

Parameters:
  • url (str) – URL to be requested.
  • json_callback (func) – Custom JSON loader function. Defaults to json.loads().
  • kwargs (dict) – Additional arguments to pass through to the request.
Returns:

response body returned by json_callback() function.

request(method, url, params=None, headers=None, data=None, json=None, token_refresh_attempts=2, **kwargs)[source]

Make an asynchronous HTTP request.

Parameters:
  • method (str) – HTTP method to use for the request.
  • url (str) – URL to be requested.
  • params (dict) – (optional) Query parameters for the request. Defaults to None.
  • headers (dict) – (optional) HTTP headers to send with the request. Headers pass through to the request will include DEFAULT_REQUEST_HEADERS.
  • data (obj) – (optional) A dictionary, bytes, or file-like object to send in the body of the request.
  • json (obj) – (optional) Any json compatible python object. NOTE: json and body parameters cannot be used at the same time.
  • token_refresh_attempts (int) – (optional) Number of attempts a token refresh should be performed.
Returns:

(str) HTTP response body.

Raises:

GCPHTTPError – if any exception occurred, specifically a GCPHTTPResponseError, if the exception is associated with a response status code.

valid_token_set()[source]

Check for validity of token, and refresh if none or expired.

GCP Auth Client

Module to create a client interacting with Google Cloud authentication.

An instantiated client is needed for interacting with any of the Google APIs via the AIOConnection.

The GAuthClient supports both service account (JSON Web Tokens/JWT) authentication with keyfiles, and default credentials. To setup a service account, follow Google’s docs. More information on default credentials can be found here. To setup default credentials, follow Application Default Credentials.

If a keyfile is not provided, the Application Default Credentials will be used.

To use:

>>> import asyncio
>>> import google_gcp
>>> loop = asyncio.get_event_loop()
>>> keyfile = '/path/to/service_account_keyfile.json'
# with keyfile
>>> auth_client = google_gcp.GAuthClient(keyfile=keyfile)
# with Application Default Credentials
>>> auth_client = google_gcp.GAuthClient()
>>> auth_client.token is None
True
>>> loop.run_until_complete(auth_client.refresh_token())
>>> auth_client.token
'c0ffe3'
class gordon_gcp.GAuthClient(keyfile=None, scopes=None, session=None, loop=None)[source]

Async client to authenticate against Google Cloud APIs.

SCOPE_TMPL_URL

str – template URL for Google auth scopes.

DEFAULT_SCOPE

str – default scope if not provided.

JWT_GRANT_TYPE

str – grant type header value when requesting/refreshing an access token.

Parameters:
  • keyfile (str) – path to service account (SA) keyfile.
  • scopes (list) – (optional) scopes with which to authorize the SA. Default is 'cloud-platform'.
  • session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests.
  • loop – (optional) asyncio event loop to use for HTTP requests. NOTE: if session is given, then loop will be ignored. Otherwise, loop will be used to create a session, if provided.
refresh_token()[source]

Refresh oauth access token attached to this HTTP session.

Raises:
google.auth._default.default(scopes=None, request=None)[source]

Gets the default credentials for the current environment.

Application Default Credentials provides an easy way to obtain credentials to call Google APIs for server-to-server or local applications. This function acquires credentials from the environment in the following order:

  1. If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set to the path of a valid service account JSON private key file, then it is loaded and returned. The project ID returned is the project ID defined in the service account file if available (some older files do not contain project ID information).

  2. If the Google Cloud SDK is installed and has application default credentials set they are loaded and returned.

    To enable application default credentials with the Cloud SDK run:

    gcloud auth application-default login
    

    If the Cloud SDK has an active project, the project ID is returned. The active project can be set using:

    gcloud config set project
    
  3. If the application is running in the App Engine standard environment then the credentials and project ID from the App Identity Service are used.

  4. If the application is running in Compute Engine or the App Engine flexible environment then the credentials and project ID are obtained from the Metadata Service.

  5. If no credentials are found, DefaultCredentialsError will be raised.

Example:

import google.auth

credentials, project_id = google.auth.default()
Parameters:
  • scopes (Sequence[str]) – The list of scopes for the credentials. If specified, the credentials will automatically be scoped if necessary.
  • request (google.auth.transport.Request) – An object used to make HTTP requests. This is used to detect whether the application is running on Compute Engine. If not specified, then it will use the standard library http client to make requests.
Returns:

the current environment’s credentials and project ID. Project ID may be None, which indicates that the Project ID could not be ascertained from the environment.

Return type:

Tuple[Credentials, Optional[str]]

Raises:

DefaultCredentialsError – If no credentials were found, or if the credentials found were invalid.

GCP Cloud DNS HTTP Client

Client module to interact with the Google Cloud DNS API.

This client makes use of the asynchronus HTTP client as defined in AIOConnection, and therefore must use service account/JWT authentication (for now).

To use:

import asyncio
import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
auth_client = gordon_gcp.GAuthClient(keyfile=keyfile)
client = gordon_gcp.GDNSClient(
    project='my-dns-project', auth_client=auth_client)

async def print_first_record(client)
    records = await client.get_records_for_zone('testzone')
    print(records[0])

loop = asyncio.get_event_loop()
loop.run_until_complete(print_first_record(client))
class gordon_gcp.GDNSClient(project=None, auth_client=None, api_version='v1', session=None, default_zone_prefix=None)[source]

Async HTTP client to interact with Google Cloud DNS API.

BASE_URL

str – base call url for the DNS API

Parameters:
  • project (str) – Google project ID that hosts the managed DNS.
  • auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
  • api_version (str) – DNS API endpoint version. Defaults to v1.
  • session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.
get_managed_zone(zone)[source]

Get the GDNS managed zone name for a DNS zone.

Google uses custom string names with specific requirements for storing records. The scheme implemented here chooses a managed zone name which removes the trailing dot and replaces other dots with dashes, and in the case of reverse records, uses only the two most significant octets, prepended with ‘reverse’. At least two octets are required for reverse DNS zones.

Example

get_managed_zone(‘example.com.’) = ‘example-com’ get_managed_zone(‘20.10.in-addr.arpa.) = ‘reverse-20-10’ get_managed_zone(‘30.20.10.in-addr.arpa.) = ‘reverse-20-10’ get_managed_zone(‘40.30.20.10.in-addr.arpa.) = ‘reverse-20-10’

Parameters:zone (str) – DNS zone.
Returns:str of managed zone name.
get_records_for_zone(dns_zone, params=None)[source]

Get all resource record sets for a managed zone, using the DNS zone.

Parameters:
  • dns_zone (str) – Desired DNS zone to query.
  • params (dict) – (optional) Additional query parameters for HTTP requests to the GDNS API.
Returns:

list of dicts representing rrsets.

is_change_done(zone, change_id)[source]

Check if a DNS change has completed.

Parameters:
  • zone (str) – DNS zone of the change.
  • change_id (str) – Identifier of the change.
Returns:

Boolean

publish_changes(zone, changes)[source]

Post changes to a zone.

Parameters:
  • zone (str) – DNS zone of the change.
  • changes (dict) – JSON compatible dict of a Change.
Returns:

string identifier of the change.

GCRM Client

Client classes to retrieve project and instance data from GCE.

These clients use the asynchronous HTTP client defined in AIOConnection and require service account or JWT-token credentials for authentication.

To use:

import asyncio

import aiohttp
import gordon_gcp

loop = asyncio.get_event_loop()

async def main():
    session = aiohttp.ClientSession()
    auth_client = gordon_gcp.GAuthClient(
        keyfile='/path/to/keyfile', session=session)
    client = gordon_gcp.GCEClient(auth_client, session)
    instances = await client.list_instances('project-id')
    print(instances)

loop.run_until_complete(main())
# example output
# [{'hostname': 'instance-1', 'internal_ip': '10.10.10.10',
#   'external_ip': '192.168.1.10'}]
class gordon_gcp.GCRMClient(auth_client=None, session=None, api_version='v1')[source]

Async client to interact with Google Cloud Resource Manager API.

You can find the endpoint documentation here.

BASE_URL

str – Base endpoint URL.

Parameters:
  • auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
  • session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.
  • api_version (str) – version of API endpoint to send requests to.
list_all_active_projects(page_size=1000)[source]

Get all active projects.

You can find the endpoint documentation here.

Parameters:page_size (int) – hint for the client to only retrieve up to this number of results per API call.
Returns:all active projects
Return type:list(dicts)

GCE Client

Client classes to retrieve project and instance data from GCE.

These clients use the asynchronous HTTP client defined in AIOConnection and require service account or JWT-token credentials for authentication.

To use:

import asyncio

import aiohttp
import gordon_gcp

loop = asyncio.get_event_loop()

async def main():
    session = aiohttp.ClientSession()
    auth_client = gordon_gcp.GAuthClient(
        keyfile='/path/to/keyfile', session=session)
    client = gordon_gcp.GCEClient(auth_client, session)
    instances = await client.list_instances('project-id')
    print(instances)

loop.run_until_complete(main())
# example output
# [{'hostname': 'instance-1', 'internal_ip': '10.10.10.10',
#   'external_ip': '192.168.1.10'}]
class gordon_gcp.GCEClient(auth_client=None, session=None, api_version='v1', blacklisted_tags=None, blacklisted_metadata=None)[source]

Async client to interact with Google Cloud Compute API.

BASE_URL

str – base compute endpoint URL.

Parameters:
  • auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
  • session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.
  • api_version (str) – version of API endpoint to send requests to.
  • blacklisted_tags (list) – Do not collect an instance if it has been tagged with any of these.
  • blacklisted_metadata (list) – Do not collect an instance if its metadata key:val matches a {key:val} dict in this list.
list_instances(project, page_size=100, instance_filter=None)[source]

Fetch all instances in a GCE project.

You can find the endpoint documentation here.

Parameters:
  • project (str) – unique, user-provided project ID.
  • page_size (int) – hint for the client to only retrieve up to this number of results per API call.
  • instance_filter (str) – endpoint-specific filter string used to retrieve a subset of instances. This is passed directly to the endpoint’s “filter” URL query parameter.
Returns:

data of all instances in the given

project

Return type:

list(dicts)