Yesterday I needed to make some kind of proxy for my project. Basically, I needed to make an endpoint that makes a GET request to another server, and return the result.

I had heard of Firebase Functions and decided to write it in that. The meat of the code should look somewhat like this:

const requestHandler = async (request, response) => {
  const result = await fetch(
    "https://some-service.com/.../geocode?query=" + requestQueryParam,
    {
      headers: {
        [some API keys]
      },
    }
  );
  response.send(await result.text());
};

Let's get started by reading the documentation. The entry-point is here: https://firebase.google.com/docs/functions. They document how to set up the project and make a basic function that returns "Hello world". They also give a repository that contains many examples: https://github.com/firebase/functions-samples.

However, none of those examples demonstrate how to do a fetch. They all use some npm package that provides some API.

fetch is not defined

fetch is really nice. See the API here. I try using it, run it locally with npm run serve, open the end-point and get this runtime error:

functions: ReferenceError: fetch is not defined
    at /Users/anhtuan/git/cargo/functions/lib/index.js:11:20
    at cloudFunction (/Users/anhtuan/git/cargo/functions/node_modules/firebase-functions/lib/v1/providers/https.js:51:16)
    at /Users/anhtuan/.nvm/versions/node/v16.17.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:532:16
    at runFunction (/Users/anhtuan/.nvm/versions/node/v16.17.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:15)
    at runHTTPS (/Users/anhtuan/.nvm/versions/node/v16.17.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:531:11)
    at /Users/anhtuan/.nvm/versions/node/v16.17.0/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:690:27
    at Layer.handle [as handle_request] (/Users/anhtuan/.nvm/versions/node/v16.17.0/lib/node_modules/firebase-tools/node_modules/express/lib/router/layer.js:95:5)

request

Googling how to make a GET request on Firebase functions will return very few results. One of which was telling the reader to use the request package. However, that package is has been deprecated for 3 years.

The author also published a lengthy post explaining their decision. Surprisingly, this package is still being downloaded 18m times a week.

In this thread, I found another post that lists out alternatives. There are many to choose from, but who knows what might work with Firebase Functions?

node-fetch? request-promise?

Some posts like this one mention node-fetch and some other npm packages. Apparently, one could have used request-promise but it was deprecated some time before May 2020. They recommend axios but I've never heard of it. As for node-fetch, it seems to not work either.

Even the documentation in Google Functions, which is what Firebase Functions depends on, shows an example with node-fetch, which does not actually work anymore:

Send HTTP requests | Cloud Functions Documentation | Google Cloud

Shows how to make an HTTP request from a Cloud Function.

Google Cloud

Still, I tried using node-fetch. I ran npm i node-fetch --save and imported it.

import * as functions from "firebase-functions";
import fetch from "node-fetch";

When I ran the server, it said:

Failed to load function definition from source: FirebaseError: Failed to load function definition from source: Failed to generate manifest from function source: Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/anhtuan/git/cargo/functions/node_modules/node-fetch/src/index.js from /Users/anhtuan/git/cargo/functions/lib/index.js not supported.
Instead change the require of /Users/anhtuan/git/cargo/functions/node_modules/node-fetch/src/index.js in /Users/anhtuan/git/cargo/functions/lib/index.js to a dynamic import() which is available in all CommonJS modules.

Which is the same as what a user reported in this StackOverflow post. One workaround apparently is to use an old version of node-fetch. As of writing, the major version number is 3, but they recommend using 2. That does not sound like a good idea at all. But hey I'm desperate, so I try it.

I change the dependency to "node-fetch": "^2.0.0" in package.json, run npm update and try running the function locally.

Now my import fetch from "node-fetch"; statement is broken:

src/index.ts:2:19 - error TS7016: Could not find a declaration file for module 'node-fetch'. '/Users/anhtuan/git/cargo/functions/node_modules/node-fetch/lib/index.js' implicitly has an 'any' type.

 Try `npm i --save-dev @types/node-fetch` if it exists or add a new declaration (.d.ts) file containing `declare module 'node-fetch';`

I change it to const fetch = require("node-fetch"); and it now compiles. I run it locally with npm run serve, and the end-point actually works!

So I try to deploy it: npm run deploy. Now I get this error:

/Users/anhtuan/git/cargo/functions/src/index.ts

 2:15  error  Require statement not part of import statement  @typescript-eslint/no-var-requires

✖ 1 problem (1 error, 0 warnings)

So I am guessing they are not happy with require statements and I have to use import instead.

https Node module

I guess I should have realized from the start. fetch is implemented in Web browsers, but in Node.js, one has to use the Node.js modules such as http or https. Let's take a look at the http documentation:

http.get('http://localhost:8000/', (res) => {
  const { statusCode } = res;
  const contentType = res.headers['content-type'];

  let error;
  // Any 2xx status code signals a successful response but
  // here we're only checking for 200.
  if (statusCode !== 200) {
    error = new Error('Request Failed.\n' +
                      `Status Code: ${statusCode}`);
  } else if (!/^application\/json/.test(contentType)) {
    error = new Error('Invalid content-type.\n' +
                      `Expected application/json but received ${contentType}`);
  }
  if (error) {
    console.error(error.message);
    // Consume response data to free up memory
    res.resume();
    return;
  }

  res.setEncoding('utf8');
  let rawData = '';
  res.on('data', (chunk) => { rawData += chunk; });
  res.on('end', () => {
    try {
      const parsedData = JSON.parse(rawData);
      console.log(parsedData);
    } catch (e) {
      console.error(e.message);
    }
  });
}).on('error', (e) => {
  console.error(`Got error: ${e.message}`);
});

I guess if you remove most of the boilerplate, this is not too bad. However, I hate that I have to add a listener on the result to listen to data and end events and collect each chunk myself.

The res object passed in the callback is of type IncomingMessage (see doc). It states that IncomingMessage implements the stream.Readable interface.

Importing the IncomingMessage type

So at this point my code looks like this:

const https = require("node:https");

export const geocode = functions.https.onRequest(async (request, response) => {
  const options = {
    headers: {
      [...]
    },
  };
  https.get(
    url,
    options,
    (res) => {
      [do something with the response]
    }
  );
});

And the line that defines the callback throws this error:

Parameter 'res' implicitly has an 'any' type.

I guess it wants me to specify the type of res. Well, I know it's an http.IncomingMessage, so let's import that and set the type:

const http = require("node:http");
const https = require("node:https");

export const geocode = functions.https.onRequest(async (request, response) => {
  const options = {
    headers: {
      [...]
    },
  };
  https.get(
    url,
    options,
    (res: http.IncomingMessage) => {
      [do something with the response]
    }
  );
});

And it throws 2 errors:

• At the first line, it tells me

'http' is declared but its value is never read.

• At the callback definition, it tells me

Cannot find namespace 'http'.

Well, I am trying to declare http and use it down there. I imported http just the same as I imported https. What's wrong with my code?

It turns out changing the import to import * as http from "node:http"; fixed it. I am so rusty with all the different ways to set up and require modules and imports. Eventually I probably should brush up on them. See the MDN doc.

Streaming the result

Back to the Firebase Functions doc: the res object is of type Response as defined in ExpressJS, which itself is a subclass of Node.js's Response class:

The res object is an enhanced version of Node’s own response object and supports all built-in fields and methods.

On Node.js's documentation, I learn that http.ServerResponse extends `http.OutgoingMessage` and that OutgoingMessage extends Stream.

That means that since both https.get's response and Firebase Functions's response are streams, I do not have to listen to "data" and "end" and concatenate everything, then call res.send.

My call back looks like this:

(res: http.IncomingMessage) => {
  res.pipe(response);
}

And it seems to work, except for some encoding issue:

{"status":"OK","meta":{"totalCount":1,"page":1,"count":1},"addresses":[{"roadAddress":"�쒖슱�밸퀎��"

Fixing the encoding

To finish I had to fix the encoding. It could either have been due to the server I am calling not returning the result in the right encoding, or my own function returning it in the wrong encoding, or my own function returning it in the right encoding but Chrome displaying it in the wrong encoding. After a few tries, I ended up finding that if I set the response's Content-Type, it works.

(res: http.IncomingMessage) => {
  response.setHeader("Content-Type", "application/json; charset=utf-8");
  res.pipe(response);
}

Fixing the last import

Even though it works locally, npm run deploy fails with this error:

Require statement not part of import statement  @typescript-eslint/no-var-requires

because of how I imported https. I cannot use require and should use the standard import statement.

So here's the final version of the code.

import * as functions from "firebase-functions";
import * as http from "node:http";
import * as https from "node:https";

export const geocode = functions.https.onRequest(async (request, response) => {
  const options = {
    headers: {
      [...]
    },
  };
  https.get(
    "https://some-endpoint/geocode?query=" +
      encodeURI(request.query.query + ""),
    options,
    (res: http.IncomingMessage) => {
      response.setHeader("Content-Type", "application/json; charset=utf-8");
      res.pipe(response);
    }
  );
});

I have to say, this was much harder than expected. And I cannot believe no one posted that on StackOverflow.