When branch is ahead by X commits, try to pull and push again

Question

I have multiple VMs with a crontab which pushes a file into the same git repository.

The issue is that the VMs are pushing at the same time (crontab every 5 minutes), so sometimes after a git pull, it was already changed, so when it tries to push, it says the branch is ahead by X commits.

This is the script:

VAPP_NAME="OC-MS-N1"
VAPP_IP="10.238.96.64"
FILE_NAME="$VAPP_NAME":"$VAPP_IP"
FILE_PATH="vApps/"
if cd oc_healthcheck; then git pull; else git clone https://example.com/repo.git && cd oc_healthcheck; fi
cd "$FILE_PATH"
echo "`date`" > "$FILE_NAME";  df -h >> "$FILE_NAME"
git add "$FILE_NAME"
git commit -m "`date`"
git push

I need a way to check, if the branch is ahead by X commits, it will try to pull and then push again. Or is there a better approach?

Answer 1

The issue you're hitting is that git maintains an unambiguous, ordered, history of changes, while your cron script does not care or ensure the order in which the clients update their state. Ultimately the "branch is ahead by X commits" message means that you're trying to send changes to the history ledger out of order, and rejects them because of lack of ordering.

One solution is to have a distinct branch for each reporting client. Changes on each branch can only be in order because only one client modifies each branch. The clients need not clone the server's repo.

Example:

#bin/bash

VAPP_NAME="OC-MS-N1"
VAPP_IP="10.238.96.64"
FILE_NAME="$VAPP_NAME"
FILE_PATH="vApps/"

# Initialize local heartbeat history repo
[[ -d .git ]] || {
    git init --initial-branch "$VAPP_IP" && git commit --allow-empty -m "Healthcheck branch for $VAPP_IP"
    git remote add -m "$VAPP_IP" heartbeat-server https://example.com/repo.git
    git push --set-upstream heartbeat-server "$VAPP_IP"
}
    
# generate/collect data
echo "`date`" > "$FILE_PATH/$FILE_NAME" ;  df -h >> "$FILE_PATH/$FILE_NAME"

# report collected data
git add "$FILE_PATH/$FILE_NAME"
git commit -m "`date`" --only "$FILE_PATH/$FILE_NAME"
git push heartbeat-server

Each client will report its status to the same filename, but in a different branch. On the server, you can access the status file from arbitrary branches with git show 10.238.96.64:$FILE_PATH/$FILE_NAME

Answer 2

I need a way to check, if the branch is ahead by X commits, it will try to pull and then push again.

That would not be a reliable solution, because during the time between the check and the push, another instance might already have pushed. So the way to go is sort of like this pseudo-code:

push
while "push failed" {
  pull
  push
}

To check whether the push failed you can check the exit-code or use git push --porcelain and parse the output.