Keep workspace when switching stages in combination with agent none

Question

I have a Jenkins pipeline where I want to first build my project (Stage A) and trigger an asynchronous long running external test process with the built artifacts. The external test process then resumes the Job using a callback. Afterwards (Stage B) performs some validations of the test results and attaches them to the job. I don't want to block an executor while the external test process is running so I came up with the following Jenkinsfile which mostly suites my needs:

#!/usr/bin/env groovy

pipeline {
    agent none
  
    stages {
        stage('Stage A') {
            agent { docker { image 'my-maven:0.0.17' } }
            steps {
                script {
                    sh "rm testfile.txt"                   
                    sh "echo ABCD > testfile.txt"
                    sh "cat testfile.txt"
                }
            }
        }
        stage('ContinueJob') {
            agent none
            input { message "The job will continue once the asynchronous operation has finished" }
            steps { echo "Job has been continued" }
        }
        stage('Stage B') {
            agent { docker { image 'my-maven:0.0.17' } }
            steps {
                script {
                    sh "cat testfile.txt"
                    def data = readFile(file: 'testfile.txt')
                    if (!data.contains("ABCD")) {
                        error("ABCD not found in testfile.txt")
                    }
                }
            }
        }
    }
}

However, depending on the load of the Jenkins or the time passed or some unknown other conditions, sometimes the files that I create in "Stage A" are no longer available in "Stage B". It seems that Jenkins switches to a different Docker node which causes the loss of workspace data, e.g. in the logs I can see:

[Pipeline] { (Stage A)
[Pipeline] node
Running on Docker3 in /var/opt/jenkins/workspace/TestJob
.....
[Pipeline] stage
[Pipeline] { (Stage B)
[Pipeline] node
Running on Docker2 in /var/opt/jenkins/workspace/TestJob

Whereas with a successful run, it keeps using e.g. node "Docker2" for both stages.

Note that I have also tried reuseNode true within the two docker sections but that didn't help either.

How can I tell Jenkins to keep my workspace files available?

Answer 1

As pointed out by the comment from @Patrice M. if the files are not that big (which is the case for me) stash/unstash are very useful to solve this problem. I have used this combination now since a couple of months and it has solved my issue.