Managing quota for Google Analytics Data API

Minhaz Kazi, Developer Advocate, Google Analytics – February 2023

If you are developing applications using the Google Analytics Data API, you should understand how the quotas and limits for the API work. If your application is well designed, users are less likely to hit quota limits. Some of the relevant best practices also lead to performant queries to the API. This can speed up reports and dashboards in your application and result in a more desirable user experience. This article discusses the quota system and best practices for implementing the Google Analytics Data API.

Understanding the quota system for Google Analytics Data API

As Google Analytics is used by millions of developers and users, quota on API requests protect the system from processing more data than it can handle while ensuring equitable distribution of system resources. The Data API for Google Analytics 4 properties uses a token bucket system for managing API quotas. To understand the concept: imagine there is a bucket that can hold up to a maximum number of tokens. Any API request will first check the bucket. If there are no tokens left, the request will fail. Otherwise, the request will get executed and will consume one or more tokens from the bucket depending on the complexity of the request. The tokens are replenished in the bucket to the maximum at fixed time intervals.

Depending on which Data API method you are using, there are three separate quota categories:

• Realtime (for runRealtimeReport)
• Funnel (for runFunnelReport)
• Core (for all other methods)

And the Data API methods will check multiple buckets for quota tokens:

1. Per property per day
2. Per property per hour
3. Per project per property per hour
4. Concurrent requests per property
5. Server errors per project per property per hour

These five buckets are checked anytime a Data API request comes in for a property. If any of the buckets are empty, the request would immediately fail with a 429 error. If none of the buckets are not empty, a single token will get consumed from the Concurrent requests per property bucket and then the API request will get executed. Based on the complexity of the request, a certain amount of tokens will be consumed from each of the first three buckets once the execution completes. The Concurrent requests per property will also get a token replenished at this time.

The Per project per property per hour quota ensures that quota depletion for one or more users won't affect other users of your application. Here, project refers to your application's GCP project. The Per property per hour quota is usually four times the Per project per property per hour quota. So for end users, a property has to be accessed by at least four different projects before the Per property per hour quota can be exhausted. Quota enforcement at both project and property level ensures that quota issues are limited to a single property and won't affect other properties that your application is accessing.

Warning: Developers are strongly discouraged from sharding quota tokens across multiple API projects. This is a violation of Google APIs Terms of Service.

The Server errors quota refers to API responses with 500 or 503 codes. If your application generates too many errors while accessing a property, it will exhaust the Server errors per project per property per hour quota.

All quota tokens are replenished to the limit at stated intervals. Refer to Google Analytics Data API Quotas for updated quota information. For example, Core methods get 1,250 quota tokens in the Per project per property per hour bucket. Assuming an average request from your application consumes 10 quota tokens, your application will be able to make 125 Core requests per hour for a standard property and 10x that amount (1250 Core requests) for any Analytics 360 property. The higher quota token limit is one of the major benefits of Analytics 360 properties.

Since token consumption for the first three buckets depends on the complexity of the request, it is difficult to predict the exact token usage prior to request execution. The following will usually increase the complexity of a request, thus resulting in token usage:

• Requesting more dimensions
• Querying higher time range
• Including dimensions with higher cardinality
• Querying a property with higher event count

Therefore, the same query for two different properties might result in completely different token usage since the cardinality of the dimensions might vary or the volume of traffic might be different. However, you can expect properties with similar levels of traffic and similar configuration to have similar token usage. You can use this assumption to predict customer token usage during planning and application design phases.

Note: If you had previously used the Reporting API for Universal Analytics, you will notice that Reporting API enforced quota at the Application/API project level. But for Data API, the enforcement is at the Google Analytics property level as well as at the API project level.

Monitoring quota usage

To monitor quota usage and convey that information to your end user, you can add "returnPropertyQuota": true to the API request body. This will return the PropertyQuota object along with the API response. The PropertyQuota object will contain consumption amounts and remaining quota status for all five buckets. Here's an example request body and response:

Request

{
  "dimensions": [
    {
      "name": "medium"
    }
  ],
  "metrics": [
    {
      "name": "activeUsers"
    }
  ],
  "dateRanges": [
    {
      "startDate": "yesterday",
      "endDate": "yesterday"
    }
  ],
  "returnPropertyQuota": true
}

Response

{
  "dimensionHeaders": [
    {
      "name": "medium"
    }
  ],
  "metricHeaders": [
    {
      "name": "activeUsers",
      "type": "TYPE_INTEGER"
    }
  ],
  ...
  
  "propertyQuota": {
    "tokensPerDay": {
      "consumed": 1,
      "remaining": 24997
    },
    "tokensPerHour": {
      "consumed": 1,
      "remaining": 4997
    },
    "concurrentRequests": {
      "consumed": 0,
      "remaining": 10
    },
    "serverErrorsPerProjectPerHour": {
      "consumed": 0,
      "remaining": 10
    },
    "potentiallyThresholdedRequestsPerHour": {
      "consumed": 0,
      "remaining": 120
    },
    "tokensPerProjectPerHour": {
      "consumed": 1,
      "remaining": 1247
    }
  },
  
  "kind": "analyticsData#runReport",
  ...
}

Thus, after each successful Data API request, you can expect to see how much quota the request consumed and how much quota is left for the property. It is also possible to surface this information to the user via your application interface.

Note: In the current stage of the Data API, you need to successfully execute an API request to get back the PropertyQuota object. If the property has already exhausted a quota token bucket, your request will fail with a 429 error and the response will not contain the PropertyQuota object. If you are affected by this and think your application would benefit from having a separate way to access the quota information, +1 this feature request on the issue tracker.

Quota management

We recommend implementing quota management best practices detailed below to get the most out of the Data API. Also upgrading your properties to 360 can increase the amount of data accessed through the API.

Best practices

There are broadly two ways to reduce quota usage for your application:

• Sending fewer API requests
• Sending less complex API requests

Keeping these two principles in mind, here are the practices you can implement:

• Caching: Implementing a caching layer will benefit both usability and quota management for your application. Google Analytics itself will cache your API requests but repeated requests will still incur quota tokens. By caching the API response, you can drastically reduce the number of repeated requests. For example, intraday data for standard properties can have 4 hours or higher cache expiration time. See Data freshness for Google Analytics.
• Merging requests: Try to merge multiple API requests into a single one. For example, 5 requests for data within a 2 day time frame could use 3 times the quota tokens compared to 1 request for a 10 day time frame. If you have multiple requests that vary only by a single dimension, consider merging them into a single request.
• Simplifying requests: Limit your requests to the minimum amount of data required by your application and the user. Large number of rows/columns or complex filter criteria will consume more quota tokens. Longer date ranges are usually more expensive (e.g. changing date range from 28 days to 365 days can consume 3 times the quota tokens). You can also consider using dimensions with lower cardinality whenever possible (e.g. request dateHour instead of dateHourMinute).
• Effective usage of limit: Changing the limit in the API request to reduce the number of rows returned does not significantly affect quota tokens consumed. For example, 5 requests with limits of 10k rows can consume five times quota tokens compared to 1 request with 50k limit.
• Using the right method category: As mentioned above, quota limits are spread across three method categories. Using the right method for the right use case can save quota on other categories. For example, rather than creating your own funnel in your application using data from Core methods, use the runFunnelReport method for creating funnels.
• Update default settings: When authoring or customizing reports on your platform, users might not update the default options presented by your application and only change them at runtime. If your application has a default date range of 365 days and the user usually looks at 28 days report, this will end up consuming more quota than required on a regular basis. Consider limiting the ranges and selections in default settings and let users select the optimal settings for their use cases. Or in some instances, you can also limit which defaults the users can change.
• Queuing requests and lazy loading: Be mindful of the Concurrent Requests Per Property token limit. Your application should not send off too many requests at the same time. If your application has a large number of UI elements resulting in a significant number of API requests, consider paginating the UI, lazy loading, and queuing up requests with exponential backoff for retries. Use returnPropertyQuota method to aggressively monitor your application's Concurrent Requests Per Property token usage.

Managing User Experience and Expectations

• Give user feedback before they run queries with potential high token usage. For example, queries with multiple high cardinality dimensions or with a large time frame could use a large number of tokens. Providing warning and a confirmation prompt for such queries can prevent users from making unnecessary changes to reports and help them limit the scope to their queries.
• For customized reporting solutions, provide a way for users to understand query usage of each element in their report. For example, you can provide a debug view listing the quota token usage for each report element.
• Provide feedback on the specific type of quota error and prescribe user action.
• Since Google Analytics 360 properties get 5-10x times quota limit compared to standard properties, you get more flexibility with Google Analytics 360 properties.

API quota increases above default limits are not available for the Data API for Google Analytics 4. Google Analytics 360 provides quotas higher limits for Google Analytics 4 properties. If your users are hitting the quota limits even after implementing best practices, they should consider upgrading their properties to 360. Another option for the users is to use the Google Analytics BigQuery export. That will let users export event level data to BigQuery and run their own analysis.

For further questions around the Data API quotas, head over to GA Discord or ask on Stack Overflow. If you have specific feature requests around the Data API, you can post them on our issue tracker.