Refresh API

How Refresh Jobs Work

In general, refresh jobs work a bit like search engine crawlers. We request every cached HTML file with an anonymous and fresh session (no cookies etc. will be sent to your origin) from your servers and compare the content. If the content has changed, the whole cache hierarchy is updated.

With growing numbers of files, this procedure takes a bit more time (also because we do not want to overwhelm your server with requests). To speed up the process, you can use a type: "INSTANT" Refresh Job. This means the very first request to an HTML file that has not been refreshed yet, will return the currently cached version and trigger an immediate update in the background. All requests and users after that will get the fresh content from the cache. This ensures that popular sites are updated instantly while maintaining an optimal cache hit rate.

With type: "DEPLOYMENT" you can even force the CDN to be purged hard so no cached content is served afterwards.

Generate API Token

Log in to the Dashboard, go to settings and generate your token under “Refresh API Token”. This token has to be sent with every request.

authorization: BAT <your-token>

Refresh REST Endpoint

All refresh requests are issued against the revalidation endpoint:

https://<appname>.app.baqend.com/v1/asset/revalidate

Initiating Refresh Operations via POST Requests

With a POST request against the endpoint, you can start a new refresh. The POST body should contain the information on what to refresh. On top of that you can set options on how to carry out the refresh. The body is given as JSON:

Syntax Overview

The body may include the following attributes:

  • filter - Defines the scope of assets to refresh. This can include:

    • origins- Specify origins to target.

    • mediaTypes - Restrict to specific media types.

      • contentTypes - Restrict to specific content types, like ["script", "style"]. Possible values are:

        • document

        • style

        • script

        • feed

        • audio

        • video

        • track

        • image

        • font

      • urls - Match an array of URLs (supports * wildcards). Example: ["https://example.com/html/example-*"]

      • prefixes - Match URLs by prefixes.

      • tags - Match assets by tags set via the Baqend-Tags HTTP header.

      • query - Apply a MongoDB query to combine attributes using AND, OR, or regular expressions. Supported fields:

        • url

        • eTag

        • lastModified

        • contentType

        • mediaType

        • contentHash

        • variation

        • origin

        • tags

  • triggeredBy - Name of the routine or person initiating the job. Defaults to api.

  • allowDelete - Indicates if unused or gone assets can be deleted. Defaults to true.

  • type - Specifies the refresh mode. Default is "REFRESH". Possible values:

    • "REFRESH" - A standard refresh job. Updates resources while serving cached versions until refresh completes.

    • "INSTANT" - Soft purges the CDN; stale cached content is served initially, with subsequent requests refreshed in the background. This is a good trade off to keep cache hit rates high and still update popular resources very fast. Note: should be used to refresh a large amount of HTML files

    • "DEPLOYMENT" - A deployment refresh purges the cache immediately and rebuilds the cache afterwards. This means cache entries are only served again if they got refreshed. This will cause a performance drop and should only be used if the HTML structure has been changed. (e.g. after a deployment of the origin) Note: only supported for HTML files.

    • "PURGE" - Our caches will receive a complete purge for the marked assets. This will drop all resources from our caches. The cache will not be build up again for the purged resources. Note: this will dramatically impact the performance and should only be used to delete unwanted cached resources from Speed Kit caches.

  • forceUpdate - Forces to also update resources that are marked as immutable because they contain a hash and have high caching headers. Default is false

  • delay - Defines a wait time between batches of refreshed resources. This option can be used to minimize the traffic to the origin server if necessary but makes the refreshes take much longer. Default is 0

  • scheduleIn - Allows to define a time span in milliseconds the job scheduler will wait until the refresh job is actually scheduled to run. The delivery of cached assets is suspended while the job is scheduled. Default is 0

  • name - Set a name for the refresh job to make it easier identifiable. Default is null

Example:

{
  "filter": {
    "origins": [
      "https://www.example.com"
    },
    "mediaTypes": [
      "text/plain",
      "video/mpeg"
    ],
    "contentTypes": [
      "image",
      "video"
    ],
    "urls": [
      "https://example.com/assets/main.js"
    ],
    "prefixes": [
      "https://www.example.com/de-de/"
    ],
    "query": {
      "origin": "https://www.example.com",
      "url": { "$regex": "https://www.example.com/assets/\\d+" }
    },
  },
  "type": "REFRESH",
  "revalidationDelay": 0
}

  • Response Syntax

    The response of this POST request returns a content-type: application/json;charset=utf-8 containing a statusId that can be used to query the status of the refresh via a GET request to the same endpoint.

    Example Response

    { "statusId" : "bc87f0aa-6013-4d2a-bdc2-9e96c8e86118" }

Example I (refreshing JS and CSS)

{
  "filter": {
    "contentTypes": ["script", "style"],
    "origins": [ "https://www.example.com" ]
  },
  "type": "refresh"
}

Refreshes all scripts and styles on the given domain and purges the CDN.

Example II (HTML Refreshing)

{
  "filter": {
    "contentTypes": ["document"],
    "origins": [ "https://www.example.com" ]
  },  
  "type": "instant"
}

Refreshes all HTML files and performs a soft purge on the CDN. With this approach, the first request to an HTML file that hasn't been refreshed yet will return the current cached version while simultaneously triggering an immediate background update. Subsequent requests will then serve the updated content from the cache. This method ensures that popular sites are refreshed quickly while maintaining a high cache hit rate for optimal performance.

Request as CURL

curl -X POST -d '{ "contentTypes": ["document"], "type": "instant", "origins": ["https:/www.example.com"] }' -H "authorization: BAT <tour-token>" -H "Content-type: application/json" https://<appname>.app.baqend.com/v1/asset/revalidate

Example III (Deployment job for a specific origin)

{
  "filter": {
    "contentTypes": ["document"],
    "origins": [ "https://www.example.com" ]
  },  
  "type": "deployment",
  "scheduleIn": 300000
}

Deployment refreshes are ideal for scenarios where your system undergoes a deployment. This type of job immediately purges the Speed Kit caches and halts the delivery of any matching assets until they are refreshed. To accommodate situations where the deployment process requires additional time to complete, you can use the scheduleIn parameter to set a delay in milliseconds (e.g., 5 minutes in the example). During this delay period, Speed Kit suspends the delivery of matching assets. Once the specified delay has elapsed, the refresh process begins, and only the refreshed assets will be delivered again.

Request as CURL

curl -X POST -d '{ "type": "deployment", "scheduleIn": 300000, "contentTypes": ["document"], "origins": ["https:/www.example.com"] }' -H "authorization: BAT <tour-token>" -H "Content-type: application/json" https://<appname>.app.baqend.com/v1/asset/revalidate

Checking Refresh Status via GET

If you need to know the progress of a refresh that you started, you can query it via a GET requests including your statusId like this:

https://<appname>.app.baqend.com/v1/asset/revalidate/<statusId>

  • Response Syntax

    The response contains the following attributes

    • revId - The revalidation ID is equal to the statusId from the request

    • filter - Contains all the details of the refresh that was created via POST

    • assetCount - Contains the number of resource matched by the refresh

    • refreshedCount - Contains how many resources have been updated so far

    • changedCount - Contains how many resources have changed during the update

    • state - Contains the state of the refresh, value are

      • FINISHED - For refreshes that are completed

      • CREATED - The job was created and assets are being marked

      • RUNNING - The marking finished, the job is passed on to the revalidation loop

      • ERROR - An error occurred. The error property will hold more information on the cause.

      • CANCELLED - The job was cancelled either via API or by the job scheduler because another Instant job was running

      • SCHEDULED - The scheduler moved the job aside and waits the time specified in the scheduleIn property

      • DEFERRED - Another job with the same filter is currently active, this job is deferred until the previous job finishes. Subsequent jobs with the same filter as the DEFERRED job are ignored

    • time - The current runtime of the refresh

    • deleteAll -

    • deleteMissing -

    Example

    {
  "revId": "bc87f0aa-6013-4d2a-bdc2-9e96c8e86118",
  "filter": {
    "mediaTypes": [
      "text/plain",
      "video/mpeg"
    ],
    "contentTypes": [
      "image",
      "video"
    ],
    "urls": [
      "<http://localhost:8080/v1/file/www/video*>"
    ],
    "prefixes": [
      "<http://localhost:8080/v1/file/www/vi>"
    ],
    "query": {
      "url": "<http://localhost:8080/v1/file/www/video>"
    }
  },
  "assetCount": 1,
  "refreshedCount": 1,
  "changedCount": 0,
  "final": true,
  "deleteErrors": false,
  "state": "FINISHED",
  "error": null,
  "startedAt": "2021-03-26T07:17:58.295214311Z",
  "updatedAt": "2021-03-26T07:18:25.582033896Z",
  "finishedAt": "2021-03-26T07:18:25.582033896Z",
  "triggeredBy": "api",
  "forceUpdate": false,
  "allowDelete": true,
  "type": "REFRESH",
  "duration": 27287,
  "scheduleIn": null,
  "delay": 0,
  "name": null
}

The Baqend-Tags header

When content is fetched from the origin, a special Baqend-Tags header can be set as well to tag the to be cached resource with some specific tags.

Those tags can be later used in a refresh job, to refresh all resources tagged with a specific value. The tag header should contain tags separated by white spaces, tags can contain any allowed HTTP header chars.

Example:

 Baqend-Tags: some-tag some/other/tag ThirdTag

That resource can be then refreshed with the following Job:

{
  "filter": {
    "tags": ["some-tag", "some-other-resource"]
  },
  "type": "refresh"
}

If you want to use a different header or another separator, then this can be configured by the Baqend staff.

PreviousAccelerating Dynamic DataNextPage Speed Analyzer

Last updated