API Clients¶

HTTP Sessions¶

Todo

Update prose of “…you can pass in your own session into an API client” with specific API client class names once they’re defined. Also update the example code using one API client.

By default, the HTTP session used for getting credentials is reused for API calls (recommended if there are many). If this is not desired, you can pass in your own aiohttp.ClientSession instance into an API client or AIOConnection. The auth client GAuthClient may also take an explicit session object, but is not required to assert a different HTTP session is used for the API calls.

import aiohttp
import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
session = aiohttp.ClientSession()  # optional
auth_client = gordon_gcp.GAuthClient(
    keyfile=keyfile, session=session
)

new_session = aiohttp.ClientSession()

# basic HTTP client
client = gordon_gcp.AIOConnection(
    auth_client=auth_client, session=new_session
)

Asynchronous GCP HTTP Client¶

Module to interact with Google APIs via asynchronous HTTP calls. AIOConnection is meant to be used/inherited by other product-specific API clients as it handles Google authentication and automatic refresh of tokens.

Todo

Include that it also handles retries once implemented.

To use:

import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
auth_client = gordon_gcp.GAuthClient(keyfile=keyfile)

client = AIOConnection(auth_client=auth_client)
resp = await client.request('get', 'http://api.example.com/foo')

The keyfile is optional. If not provided the default service account will be used.

class gordon_gcp.AIOConnection(auth_client=None, session=None)[source]¶

Async HTTP client to Google APIs with service-account-based auth.

Parameters:	auth_client (GAuthClient) – client to manage authentication for HTTP API requests. session (aiohttp.ClientSession) – (optional) `aiohttp` HTTP session to use for sending requests. Defaults to the session object attached to `auth_client` if not provided.

get_all(url, params=None)[source]¶

Aggregate data from all pages of an API query.

Parameters:	url (str) – Google API endpoint URL. params (dict) – (optional) URL query parameters.
Returns:	Parsed JSON query response results.
Return type:	list

get_json(url, json_callback=None, **kwargs)[source]¶

Get a URL and return its JSON response.

Parameters:	url (str) – URL to be requested. json_callback (func) – Custom JSON loader function. Defaults to `json.loads()`. kwargs (dict) – Additional arguments to pass through to the request.
Returns:	response body returned by `json_callback()` function.

request(method, url, params=None, headers=None, data=None, json=None, token_refresh_attempts=2, **kwargs)[source]¶

Make an asynchronous HTTP request.

Parameters:	method (str) – HTTP method to use for the request. url (str) – URL to be requested. params (dict) – (optional) Query parameters for the request. Defaults to `None`. headers (dict) – (optional) HTTP headers to send with the request. Headers pass through to the request will include `DEFAULT_REQUEST_HEADERS`. data (obj) – (optional) A dictionary, bytes, or file-like object to send in the body of the request. json (obj) – (optional) Any json compatible python object. NOTE: json and body parameters cannot be used at the same time. token_refresh_attempts (int) – (optional) Number of attempts a token refresh should be performed.
Returns:	(str) HTTP response body.
Raises:	`GCPHTTPError` – if any exception occurred, specifically a `GCPHTTPResponseError`, if the exception is associated with a response status code.

valid_token_set()[source]¶: Check for validity of token, and refresh if none or expired.

GCP Auth Client¶

Module to create a client interacting with Google Cloud authentication.

An instantiated client is needed for interacting with any of the Google APIs via the AIOConnection.

The GAuthClient supports both service account (JSON Web Tokens/JWT) authentication with keyfiles, and default credentials. To setup a service account, follow Google’s docs. More information on default credentials can be found here. To setup default credentials, follow Application Default Credentials.

If a keyfile is not provided, the Application Default Credentials will be used.

To use:

>>> import asyncio
>>> import google_gcp
>>> loop = asyncio.get_event_loop()
>>> keyfile = '/path/to/service_account_keyfile.json'
# with keyfile
>>> auth_client = google_gcp.GAuthClient(keyfile=keyfile)
# with Application Default Credentials
>>> auth_client = google_gcp.GAuthClient()
>>> auth_client.token is None
True
>>> loop.run_until_complete(auth_client.refresh_token())
>>> auth_client.token
'c0ffe3'

class gordon_gcp.GAuthClient(keyfile=None, scopes=None, session=None, loop=None)[source]¶

Async client to authenticate against Google Cloud APIs.

SCOPE_TMPL_URL¶: str – template URL for Google auth scopes.

DEFAULT_SCOPE¶: str – default scope if not provided.

JWT_GRANT_TYPE¶: str – grant type header value when requesting/refreshing an access token.

Parameters:

keyfile (str) – path to service account (SA) keyfile.
scopes (list) – (optional) scopes with which to authorize the SA. Default is 'cloud-platform'.
session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests.
loop – (optional) asyncio event loop to use for HTTP requests. NOTE: if session is given, then loop will be ignored. Otherwise, loop will be used to create a session, if provided.

refresh_token()[source]¶

Refresh oauth access token attached to this HTTP session.

Raises:	`GCPAuthError` – if no token was found in the response. `GCPHTTPError` – if any exception occurred, specifically a `GCPHTTPResponseError`, if the exception is associated with a response status code.

google.auth._default.default(scopes=None, request=None, quota_project_id=None, default_scopes=None)[source]¶

Gets the default credentials for the current environment.

Application Default Credentials provides an easy way to obtain credentials to call Google APIs for server-to-server or local applications. This function acquires credentials from the environment in the following order:

If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set to the path of a valid service account JSON private key file, then it is loaded and returned. The project ID returned is the project ID defined in the service account file if available (some older files do not contain project ID information).

If the environment variable is set to the path of a valid external account JSON configuration file (workload identity federation), then the configuration file is used to determine and retrieve the external credentials from the current environment (AWS, Azure, etc). These will then be exchanged for Google access tokens via the Google STS endpoint. The project ID returned in this case is the one corresponding to the underlying workload identity pool resource if determinable.

If the environment variable is set to the path of a valid GDCH service account JSON file (Google Distributed Cloud Hosted), then a GDCH credential will be returned. The project ID returned is the project specified in the JSON file.
If the Google Cloud SDK is installed and has application default credentials set they are loaded and returned.

To enable application default credentials with the Cloud SDK run:
```
gcloud auth application-default login
```
If the Cloud SDK has an active project, the project ID is returned. The active project can be set using:
```
gcloud config set project
```
If the application is running in the App Engine standard environment (first generation) then the credentials and project ID from the App Identity Service are used.
If the application is running in Compute Engine or Cloud Run or the App Engine flexible environment or the App Engine standard environment (second generation) then the credentials and project ID are obtained from the Metadata Service.
If no credentials are found, DefaultCredentialsError will be raised.

Example:

import google.auth

credentials, project_id = google.auth.default()

Parameters:	scopes (Sequence[str]) – The list of scopes for the credentials. If specified, the credentials will automatically be scoped if necessary. request (Optional[google.auth.transport.Request]) – An object used to make HTTP requests. This is used to either detect whether the application is running on Compute Engine or to determine the associated project ID for a workload identity pool resource (external account credentials). If not specified, then it will either use the standard library http client to make requests for Compute Engine credentials or a google.auth.transport.requests.Request client for external account credentials. quota_project_id (Optional[str]) – The project ID used for quota and billing. default_scopes (Optional[Sequence[str]]) – Default scopes passed by a Google client library. Use ‘scopes’ for user-defined scopes.
Returns:	the current environment’s credentials and project ID. Project ID may be None, which indicates that the Project ID could not be ascertained from the environment.
Return type:	Tuple[Credentials, Optional[str]]
Raises:	`DefaultCredentialsError` – If no credentials were found, or if the credentials found were invalid.

GCP Cloud DNS HTTP Client¶

Client module to interact with the Google Cloud DNS API.

This client makes use of the asynchronus HTTP client as defined in AIOConnection, and therefore must use service account/JWT authentication (for now).

To use:

import asyncio
import gordon_gcp

keyfile = '/path/to/service_account_keyfile.json'
auth_client = gordon_gcp.GAuthClient(keyfile=keyfile)
client = gordon_gcp.GDNSClient(
    project='my-dns-project', auth_client=auth_client)

async def print_first_record(client)
    records = await client.get_records_for_zone('testzone')
    print(records[0])

loop = asyncio.get_event_loop()
loop.run_until_complete(print_first_record(client))

class gordon_gcp.GDNSClient(project=None, auth_client=None, api_version='v1', session=None, default_zone_prefix=None)[source]¶

Async HTTP client to interact with Google Cloud DNS API.

BASE_URL¶: str – base call url for the DNS API

Parameters:

project (str) – Google project ID that hosts the managed DNS.
auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
api_version (str) – DNS API endpoint version. Defaults to v1.
session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.

get_managed_zone(zone)[source]¶

Get the GDNS managed zone name for a DNS zone.

Google uses custom string names with specific requirements for storing records. The scheme implemented here chooses a managed zone name which removes the trailing dot and replaces other dots with dashes, and in the case of reverse records, uses only the two most significant octets, prepended with ‘reverse’. At least two octets are required for reverse DNS zones.

Example

get_managed_zone(‘example.com.’) = ‘example-com’ get_managed_zone(‘20.10.in-addr.arpa.) = ‘reverse-20-10’ get_managed_zone(‘30.20.10.in-addr.arpa.) = ‘reverse-20-10’ get_managed_zone(‘40.30.20.10.in-addr.arpa.) = ‘reverse-20-10’

Parameters:	zone (str) – DNS zone.
Returns:	str of managed zone name.

get_records_for_zone(dns_zone, params=None)[source]¶

Get all resource record sets for a managed zone, using the DNS zone.

Parameters:	dns_zone (str) – Desired DNS zone to query. params (dict) – (optional) Additional query parameters for HTTP requests to the GDNS API.
Returns:	list of dicts representing rrsets.

is_change_done(zone, change_id)[source]¶

Check if a DNS change has completed.

Parameters:	zone (str) – DNS zone of the change. change_id (str) – Identifier of the change.
Returns:	Boolean

publish_changes(zone, changes)[source]¶

Post changes to a zone.

Parameters:	zone (str) – DNS zone of the change. changes (dict) – JSON compatible dict of a Change.
Returns:	string identifier of the change.

GCRM Client¶

Client classes to retrieve project and instance data from GCE.

These clients use the asynchronous HTTP client defined in AIOConnection and require service account or JWT-token credentials for authentication.

To use:

import asyncio

import aiohttp
import gordon_gcp

loop = asyncio.get_event_loop()

async def main():
    session = aiohttp.ClientSession()
    auth_client = gordon_gcp.GAuthClient(
        keyfile='/path/to/keyfile', session=session)
    client = gordon_gcp.GCEClient(auth_client, session)
    instances = await client.list_instances('project-id')
    print(instances)

loop.run_until_complete(main())
# example output
# [{'hostname': 'instance-1', 'internal_ip': '10.10.10.10',
#   'external_ip': '192.168.1.10'}]

class gordon_gcp.GCRMClient(auth_client=None, session=None, api_version='v1')[source]¶

Async client to interact with Google Cloud Resource Manager API.

You can find the endpoint documentation here.

BASE_URL¶: str – Base endpoint URL.

Parameters:	auth_client (GAuthClient) – client to manage authentication for HTTP API requests. session (aiohttp.ClientSession) – (optional) `aiohttp` HTTP session to use for sending requests. Defaults to the session object attached to `auth_client` if not provided. api_version (str) – version of API endpoint to send requests to.

list_all_active_projects(page_size=1000)[source]¶

Get all active projects.

You can find the endpoint documentation here.

Parameters:	page_size (int) – hint for the client to only retrieve up to this number of results per API call.
Returns:	all active projects
Return type:	list(dicts)

GCE Client¶

Client classes to retrieve project and instance data from GCE.

These clients use the asynchronous HTTP client defined in AIOConnection and require service account or JWT-token credentials for authentication.

To use:

import asyncio

import aiohttp
import gordon_gcp

loop = asyncio.get_event_loop()

async def main():
    session = aiohttp.ClientSession()
    auth_client = gordon_gcp.GAuthClient(
        keyfile='/path/to/keyfile', session=session)
    client = gordon_gcp.GCEClient(auth_client, session)
    instances = await client.list_instances('project-id')
    print(instances)

loop.run_until_complete(main())
# example output
# [{'hostname': 'instance-1', 'internal_ip': '10.10.10.10',
#   'external_ip': '192.168.1.10'}]

class gordon_gcp.GCEClient(auth_client=None, session=None, api_version='v1', blacklisted_tags=None, blacklisted_metadata=None)[source]¶

Async client to interact with Google Cloud Compute API.

BASE_URL¶: str – base compute endpoint URL.

Parameters:

auth_client (GAuthClient) – client to manage authentication for HTTP API requests.
session (aiohttp.ClientSession) – (optional) aiohttp HTTP session to use for sending requests. Defaults to the session object attached to auth_client if not provided.
api_version (str) – version of API endpoint to send requests to.
blacklisted_tags (list) – Do not collect an instance if it has been tagged with any of these.
blacklisted_metadata (list) – Do not collect an instance if its metadata key:val matches a {key:val} dict in this list.

list_instances(project, page_size=100, instance_filter=None)[source]¶

Fetch all instances in a GCE project.

You can find the endpoint documentation here.

Parameters:

project (str) – unique, user-provided project ID.
page_size (int) – hint for the client to only retrieve up to this number of results per API call.
instance_filter (str) – endpoint-specific filter string used to retrieve a subset of instances. This is passed directly to the endpoint’s “filter” URL query parameter.

Returns:

data of all instances in the given: project

Return type:

list(dicts)

API Clients¶

HTTP Sessions¶

Asynchronous GCP HTTP Client¶

GCP Auth Client¶

GCP Cloud DNS HTTP Client¶

GCRM Client¶

GCE Client¶

Table Of Contents

Related Topics