ERROR: error fetching storage source: generic::unknown... When deploying GCP Cloud Function via GitHub Actions

Question

I'm deploying a Firebase Cloud Function via this GitHub Action. When I deploy from my own machine using a service account it works fine. When running the Action using the same service account I'm met with the below error.

Final Error:

ERROR: error fetching storage source: generic::unknown: retry budget exhausted (3 attempts): fetching gcs source: unpacking source from gcs: source fetch container exited with non-zero status: 9

Preceding Logs:

End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of /tmp/source-archive.zip or /tmp/source-archive.zip.zip, and cannot find /tmp/source-archive.zip.ZIP, period.

Here's my Action yml:

name: Deploy
on:
  pull_request:
    branches: 
        - master

jobs:
  integration-test:
    name: Run tests
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: checkout repo and set up Node.js v12.x
      uses: actions/setup-node@v1
      with:
        node-version: 12.x
    - name: Install dependencies
      run: cd ./functions/ && npm install
    - id: deploy
      name: Deploy my_function to Cloud Functions
      uses: google-github-actions/[email protected]
      with:
        name: my_function
        runtime: nodejs12
        credentials: ${{ secrets.GCP_ACCOUNT_JSON }}
        source_dir: ./functions/
    - id: echo_url
      name: Echo deployed Cloud Function URL
      run: echo ${{ steps.deploy.outputs.url }}

The function itself doesn't require GCS access, but the service account is an editor so it has it anyway. I guess this issues has more to do with the way GitHub source code is uploaded to GCS and then sourced from there to deploy? Any hints?

Answer 1

If anyone is using Cloud Build to deploy and ran into this issue, here's a potential cause and solution.

CAUSE

I was following the steps in the official Cloud Build documentation for Firebase but kept getting this error. There have been multiple answers to similar questions that said to use Node 12, but that wasn't actually helpful because I was using the provided image.

It finally occurred to me that it was actually the build step image provided by Google that had the incompatible Node version. I had a look at the Docker file used to build the image, and it doesn't specify what version of Node. When I checked the Node version used in the image on Cloud Build, it was Node 15. I suspect this was what was causing the issue.

SOLUTION

In order to create an image that can deploy Firebase functions using Node 12 on Cloud Build, I followed the steps to clone the community builder image as per the documentation:

git clone https://github.com/GoogleCloudPlatform/cloud-builders-community.git

cd cloud-builders-community/firebase

But before I submitted the build to my project, I made a change in the Dockerfile of the repo to specifically use Node 12:

FROM node:12

The followed the rest of the steps to submit the image. Once I did that, I was able to deploy Firebase functions on Cloud Build!

As a reference, this is what my cloudbuild.yaml looked like:

timeout: 1600s

steps:
  # I am using Typescript, so I had to install dependencies
  # and build my function source before I deployed
  - name: gcr.io/$PROJECT_ID/firebase
    entrypoint: yarn
    args: ["install-functions"]

  - name: gcr.io/$PROJECT_ID/firebase
    entrypoint: yarn
    args: ["build:functions"]

  # Deploy using the build image's default entry point
  - name: gcr.io/$PROJECT_ID/firebase
    args: ["deploy", "--project=$PROJECT_ID", "--only=functions"]

Answer 2

Usually these kind of scenarios are related to the size of the Github repository. Currently there is a size limit of 500 MB for the whole repository, if that limit is exceeded the file reading would end and the resulting tarball will be invalid, therefore causing the operation to fail.

Take a look at the official quotas documentation.

Answer 3

I had this issue when my cloud functions were set to Node 14 (in the engines field in functions/package.json), but my local machine was running Node 16 when I deployed the functions.

I fixed this issue by changing my Node version temporarily to 14 using nvm.

nvm use 14

By this method, your local Node version will reset to nvm's default when you restart your terminal.

Answer 4

I just experienced the same issue on Google Cloud Build & GitHub for a trigger I'd set up a month earlier.

Upon retrying, I noticed the GitHub authenticated failed three (03) times.

I deleted the trigger and set up a new one which seems to work just great.

NOTE: It also solved another issue I was looking after into those logs. Cloud Build was failing to access my secrets from the secret manager, which I found to be weird, given that I'd set the permissions for those secrets.