Handling Firebase HTTPS Functions - Returning a 200 status code Before Executing any Background Tasks

Question

Handling Firebase HTTPS Functions - Returning a 200 status code Before Executing any Background Tasks

We are currently initiating a cloud function with an HTTPS request to:

Provide a quick response (<10s) to the caller with a 200 Response in order to prevent additional retries from our caller.
THEN carry out two other time-sensitive POST requests before concluding the function.

What are some optimal solutions to address this issue? We have explored:

PubSub as a way for subscribers to complete the POST requests asynchronously after sending the 200 response.
Utilizing Cloud Tasks and enqueueing them to a task queue. However, we are unsure if these tasks are impacted by the HTTPS termination, similar to our previous attempts.
Implementing an Event-Based Trigger such as Cloud Firestore to write to a document with a specified ID that triggers an onCreate event to finalize the POST requests.

Additional information includes the fact that we currently experience low traffic due to being a smaller app (less than 1000 QPS), which may eliminate Pub Sub as an option. We aim to create the background task within <1-2 seconds to maintain a positive user experience (specifically for a messaging app).

Initially, our code was structured like this:

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} =
      verifyAuthHeader(authHeader, token);
    // Respond quickly to the caller within 10 seconds to avoid additional retries.
    response.status(statusCode).send(
     {
      message: httpsResponseMessage,
     }
    );
    // Perform other POST requests
  }

We also attempted the following:

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} =
      verifyAuthHeader(authHeader, token);
    // Provide a 200 response to the caller within 10 seconds to prevent further retries.
    response.write(
      JSON.stringify(
        {
          message: httpsResponseMessage
        }
      )
    );
    // Carry out additional POST requests
    res.end()
  }

However, we encountered inconsistent behavior with our code AFTER

response.status(statusCode).send()

, as mentioned in the tips & tricks section of Firebase docs. Sending a partial result with response.write did not prevent future retries from the caller.

typescript firebase google-cloud-platform google-cloud-firestore google-cloud-functions

Answer 1

Answer №1

It is crucial to exercise caution when handling HTTP functions and ensure that all tasks are completed before initiating a response. This requirement is outlined in The Node.js Runtime:

When dealing with asynchronous tasks involving callbacks or Promise objects, it is essential to explicitly notify the runtime once these tasks have finished executing. Various methods can achieve this, as demonstrated in the code snippets below. The key point is that your code must wait for the completion of the asynchronous task or Promise before returning to prevent premature termination of the async component.
// Correct: awaiting a Promise before sending an HTTP response
await Promise.resolve();

// Incorrect: HTTP functions should send an
// HTTP response instead of merely returning.
return Promise.resolve();

// HTTP functions should indicate completion by returning an HTTP response.
// This should only happen after all background tasks are done.
res.send(200);
res.end();

// Incorrect: this section may not run if an
// HTTP response has already been sent.
return Promise.resolve();

In practical terms, both examples presented will not function as intended since the response is triggered before completing the work described as // do other POST requests.

Based on your explanation:

You require prompt responses to the caller
You need to trigger two additional functions when calls are made to this function
The responses from these two functions are not utilized in the original caller's response

If this scenario applies, I suggest utilizing Pub/Sub-triggered background functions and structuring the function as follows:

const { PubSub } = require('@google-cloud/pubsub')

const pubSubTopic1 = new PubSub('your-project-id').topic('the-first-topic-name')
const pubSubTopic2 = new PubSub('your-project-id').topic('the-second-topic-name')

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} = verifyAuthHeader(authHeader, token);

    // Publish messages to the Pub/Sub topics to trigger those functions
    await Promise.all([
      pubSubTopic1.publishMessage('message-1'),
      pubSubTopic2.publishMessage('message-2')
    ]);

    // Send 200 response to our caller within 10 seconds to avoid more retries.
    response.status(statusCode).send(
     {
      message: httpsResponseMessage,
     }
    );

    // Do nothing after calling .send() because the function has terminated
  }

Pub/Sub delivers fast performance with minimal latency, ensuring your functions stay warm and execute swiftly even with high concurrency levels.

Avoid using Firestore for this purpose due to cost implications associated with creating documents and triggering functions. Furthermore, excessive document creation leads to increased costs and operational complexities. Firestore triggers operate by emitting Pub/Sub messages, introducing document writes would add unnecessary latency.

Sample PubSub Function

Firebase-functions being deemed heavy and cumbersome, here's a simplified example of a PubSub-triggered function:

Assuming this function expects a JSON string as the message sent to the PubSub topic, in the earlier HTTP function example, you could publish a message similar to this:

const message1 = {
  "foo": "bar",
  "bar": "baz"
}
let statusCode = 202 // Accepted

try {
  await pubSubTopic1.publishMessage({ json: message1 })
} catch (error) {
  statusCode = 502 // Bad Gateway
} finally {
  res.status(statusCode).send({ message: httpsResponseMessage })
}

Your receiving PubSub function can be concise like so:

export const examplePubSubFunction = message => {
  const data = JSON.parse(Buffer.from(message.data, 'base64').toString())
  console.log(data.foo) // output: "bar"
  console.log(data.bar) // output: "baz"
}

To deploy the function, use the following command:

gcloud functions deploy examplePubSubFunction --runtime=nodejs18 --trigger-topic=the-first-topic-name

Answer 2

It is crucial to exercise caution when handling HTTP functions and ensure that all tasks are completed before initiating a response. This requirement is outlined in The Node.js Runtime:

When dealing with asynchronous tasks involving callbacks or Promise objects, it is essential to explicitly notify the runtime once these tasks have finished executing. Various methods can achieve this, as demonstrated in the code snippets below. The key point is that your code must wait for the completion of the asynchronous task or Promise before returning to prevent premature termination of the async component.
// Correct: awaiting a Promise before sending an HTTP response
await Promise.resolve();

// Incorrect: HTTP functions should send an
// HTTP response instead of merely returning.
return Promise.resolve();

// HTTP functions should indicate completion by returning an HTTP response.
// This should only happen after all background tasks are done.
res.send(200);
res.end();

// Incorrect: this section may not run if an
// HTTP response has already been sent.
return Promise.resolve();

In practical terms, both examples presented will not function as intended since the response is triggered before completing the work described as // do other POST requests.

Based on your explanation:

You require prompt responses to the caller
You need to trigger two additional functions when calls are made to this function
The responses from these two functions are not utilized in the original caller's response

If this scenario applies, I suggest utilizing Pub/Sub-triggered background functions and structuring the function as follows:

const { PubSub } = require('@google-cloud/pubsub')

const pubSubTopic1 = new PubSub('your-project-id').topic('the-first-topic-name')
const pubSubTopic2 = new PubSub('your-project-id').topic('the-second-topic-name')

exports.example = functions
  .https
  .onRequest(async (request, response) => {
    const authHeader = request.get("authorization");
    const {statusCode, httpsResponseMessage} = verifyAuthHeader(authHeader, token);

    // Publish messages to the Pub/Sub topics to trigger those functions
    await Promise.all([
      pubSubTopic1.publishMessage('message-1'),
      pubSubTopic2.publishMessage('message-2')
    ]);

    // Send 200 response to our caller within 10 seconds to avoid more retries.
    response.status(statusCode).send(
     {
      message: httpsResponseMessage,
     }
    );

    // Do nothing after calling .send() because the function has terminated
  }

Pub/Sub delivers fast performance with minimal latency, ensuring your functions stay warm and execute swiftly even with high concurrency levels.

Avoid using Firestore for this purpose due to cost implications associated with creating documents and triggering functions. Furthermore, excessive document creation leads to increased costs and operational complexities. Firestore triggers operate by emitting Pub/Sub messages, introducing document writes would add unnecessary latency.

Sample PubSub Function

Firebase-functions being deemed heavy and cumbersome, here's a simplified example of a PubSub-triggered function:

Assuming this function expects a JSON string as the message sent to the PubSub topic, in the earlier HTTP function example, you could publish a message similar to this:

const message1 = {
  "foo": "bar",
  "bar": "baz"
}
let statusCode = 202 // Accepted

try {
  await pubSubTopic1.publishMessage({ json: message1 })
} catch (error) {
  statusCode = 502 // Bad Gateway
} finally {
  res.status(statusCode).send({ message: httpsResponseMessage })
}

Your receiving PubSub function can be concise like so:

export const examplePubSubFunction = message => {
  const data = JSON.parse(Buffer.from(message.data, 'base64').toString())
  console.log(data.foo) // output: "bar"
  console.log(data.bar) // output: "baz"
}

To deploy the function, use the following command:

gcloud functions deploy examplePubSubFunction --runtime=nodejs18 --trigger-topic=the-first-topic-name

Handling Firebase HTTPS Functions - Returning a 200 status code Before Executing any Background Tasks

Answer №1

Sample PubSub Function

Similar questions

Discover every conceivable combination of items that results in the total values adding up to a specific number

What is the best way to transition from using merge in a pipe in v5 to v6?

Tips for generating a subprocess with exec within a TypeScript Class

Make sure accordion items stay open even when another one is clicked

Utilizing Typescript for parsing large JSON files

Is it possible to automatically correct all import statements in a TypeScript project?

Error: Module not found - TypeScript + Playwright

The concept of `object()` does not function properly in the context of utilizing redux

What is the best way to apply a CSS class to a ComponentRef that has been generated in Angular 5

Restricting types through property union types

Sending a specific object and its corresponding key as parameters to a reusable component in Angular: T[K]

I am puzzled by the Angular production build error I encountered - specifically, the ./src/app/app.module.ngfactory.js

Problem with logging into Google account on Firebase authorization

My goal is to prevent users from using the Backspace key within the input field

Can anyone explain why the Splice function is removing the element at index 1 instead of index 0 as I specified?

An object in typescript has the potential to be undefined

What is the best way to retrieve a specific field from the observable data stream?

The input 'Query' cannot be matched with the type '[(options?: QueryLazyOptions<Exact<{ where?:"

ts:Accessing the state within a Redux store

Issue encountered with UglifyJs - Unexpected token: title (Subject)