How to clone public submodule in "Github Actions"

Question

Reason for this Q&A-Style question: It took me a few hours to get this to run because I had some typos and thought the solution is more complicated. If I would have found a tutorial like this on google or Stackoverflow I would have checked for typos.

Git Repository Setup:

• Private repository A - name: repoA
• with submodule B (public repository) - name: repoB

Goal:

• I want to run a gradle build in repository A > Github Actions

Github Action Workflow

name: Test
on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1

Problem:

• The actions/checkout@v1 step fails to access the submodule

.gitmodules

[submodule "library"]
    path = library
    url = [email protected]:organization/repoB.git

Github Actions Step Build with Gradle error

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':repoA:compileReleaseAidl'.
> Could not resolve all task dependencies for configuration ':repoA:releaseCompileClasspath'.
   > Could not resolve project :repoBSubmodule1.
     Required by:
         project :repoA

What I tried:

---

1. Add with: submodules: true to actions/checkout@v1

    - uses: actions/checkout@v1
      with: 
        submodules: true

Github Actions Step Run actions/checkout@v1 error

(...)
git submodule sync
git -c http.https://github.com.extraheader="AUTHORIZATION: basic ***" submodule update --init --force
Submodule 'repoB' ([email protected]:organization/repoB.git) registered for path 'repoB'
Cloning into '/home/runner/work/repoA/repoA/repoB'...
Host key verification failed.
##[error]fatal: Could not read from remote repository.

---

2. Use 3rd party Github Actions like textbook/[email protected]

Run textbook/[email protected] error:

fatal: Not a git repository (or any parent up to mount point /github/workspace)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
##[error]Docker run failed with exit code 128

---

3. Add a personal access token to the actions/checkout

   • The token was generated from a github user which has access to that repository
   • The repoA is owned by an organization
   • the token has full repo permissions

    - uses: actions/checkout@v1
      with: 
        submodules: true
        token: ${{ secrets.GITHUB_REPO_TOKEN }}

Run actions/checkout@v1 error:

git submodule sync
git -c http.https://github.com.extraheader="AUTHORIZATION: basic ***" submodule update --init --force
Submodule 'repoB' ([email protected]:organization/repoB.git) registered for path 'repoB'
Cloning into '/home/runner/work/repoA/repoA/repoB'...
Host key verification failed.
##[error]fatal: Could not read from remote repository.

I.e. with the that token which has access to both, repoA and repoB I was not even able to checkout the parent repoA.

Answer 1

Now you can use actions/checkout@v2

As https://github.com/actions/checkout#checkout-submodules said:

- uses: actions/checkout@v2 # checkout root
- name: Checkout submodules # checkout rest
  shell: bash
  run: |
    # If your submodules are configured to use SSH instead of HTTPS please uncomment the following line
    # git config --global url."https://github.com/".insteadOf "[email protected]:"
    auth_header="$(git config --local --get http.https://github.com/.extraheader)"
    git submodule sync --recursive
    git -c "http.extraheader=$auth_header" -c protocol.version=2 submodule update --init --force --recursive --depth=1

Answer 2

You can use actions/checkout@v2 without additional scripting. I used this on Mac, Linux and Windows. Meanwhile v3 is available:

- uses: actions/checkout@v3
  with:
    submodules: true

For recursive submodules (where a submodule requires another submodule), use

- uses: actions/checkout@v3
  with:
    submodules: recursive

Answer 3

Changing the submodule URL from the SSH to the HTTPS format fixed it:

.gitmodules

[submodule "repoB"]
    path = repoB
#Before:
    url = [email protected]:organization/repoB.git
# After:
    url = https://github.com/organization/repoB.git 

Answer 4

For Windows based runner the following worked: Created a bot-user, generated Access Token, included the user into private (submodules) repos and added this in actions.yml:

  - name: Checkout submodules using a PAT
    shell: bash
    run: |
      git config --file .gitmodules --get-regexp url | while read url; do
      git config --file=.gitmodules $(echo "$url" | sed -E "s/[email protected]:|https:\/\/github.com\//https:\/\/${{ secrets.GHACTIONS_PAT }}:${{ secrets.GHACTIONS_PAT }}@github.com\//")
      done
      git submodule sync
      git submodule update --init --recursive