Excel VBA macro slows down unpredictably, usually at DoEvents

Question

What can be happening that slows down Excel to a crawl, sometimes for a minute or two, sometimes for hours, and its workload is spread across calls to DoEvents?

Unfortunately, I can't post code because there seem to be no code causing the problem. It happens in different places without apparent pattern. Below there are details about my investigation.

Last week Excel on my computer started slowing down when running a macro that has been running without problems for years. I tested the same macro on 2 other computers, but they don't seem to have the same problem. It's difficult to test because it doesn't always slow down and it never happens on the same place. Sometimes it does it every minute, sometimes it works for 10 minutes and then it does it again. I tried two other computers for about 30 minutes and it never happened.

If I set breakpoints, it never slows down.

If I sprinkle prints it does, but never at the same place.

If I press Ctrl+Break the macro stops after a few minutes, the debugger shows the current line highlighted as usual, but Excel keeps using 25% CPU (that is 100% of two cores). At this point every click on Excel or on the VBA IDE is responsive, but very slow. It usually takes minutes before you see the cursor moving to the clicked spot. Sometimes if you wait a few minutes the control comes back and it's possible to press F8 or F5 to continue the execution, but I usually kill Excel if the CPU usage doesn't go down in 2-3 minutes.

During the investigation I created this function:

Sub DoEvents2()
  Dim T As Single
  T = Timer
  DoEvents
  Debug.Print Format(Timer - T, "0.000")
End Sub

and replaced the calls to DoEvents with calls to DoEvents2, and I see on the debug window that the time required for the DoEvents is always a few thousands of a second, sometimes a few hundreds and once in a while, without apparent pattern, the time goes up. Often it goes up to ~90 seconds, sometimes less, a few times it lasted almost an hour, once it was still running the next day.

Here is an example of the output on the debug window after running the macro with the above defined DoEvents2:

0.000
0.289
0.000
0.004
0.066
88.324
26.727
20.699
28.762
4.359
0.789
0.090
0.297
0.141
0.000
0.070
0.000
0.043
[...]
0.016
0.035
0.004
0.199
1.852
0.066
0.023
0.004
0.000
31.309
104.438
1.449
0.785
0.020
0.004
0.547
0.000
0.000
0.055

The macro is large and it's difficult to remove one piece without breaking it.

At first I thought the problem was with a form, but I removed all the forms and the problems is still there.

Then I tried working on volatile functions: I removed all the volatile functions and the problem is still there. Plus, it doesn't seem to be triggered by changes to Application.CalculationMode or Application.EnableEvents.

I checked if there are globals that could trigger long running garbage collection, but I didn't find anything. Plus, I don't see how garbage collection could run 12 hours (I left it running once the whole night and it was still there in the morning).

Sometimes I click on a button that runs a macro that uses JsonConverter, regular expressions, http requests, forms, volatile functions, etc., it runs for a minute or two as expected, and has no problems. Then I click on another button that runs 10 lines of code, and it's done in a few hundreds of a second. I click the same button again and again, and after a few times Excel hangs. I don't know where it hangs because it never hangs at the same line.

The problem started last week while Windows updates were going on, but I don't know if it's related. Since then I've been working full time chasing this problem without success.

I can't post any code, because the macro has 14,000 lines, and it happens every where, wherever there is a DoEvents, even if it's a small 5 line function. But it seems to be affected by something that happened earlier.

So, my question is, what can be happening that slows down Excel to a crawl, sometimes for a minute or two, sometimes for hours, and its workload is spread across calls to DoEvents.

Edit

It happened again about 40 minutes ago, I decided to let it vent hoping it would stop while I was working on something else. After 30 minutes I pressed Ctrl+Break to try to stop it, after a minute or two it did stop, then I did a burst of clicks on the save button of the VBA IDE, hoping that one of them would work and after a minute or two a popup showed asking if I wanted to save the file with unfinished calculation. After a few seconds, while this popup was visible, the CPU usage went to zero. I asked to save the file without completing the calculation, it did (I can see the " - Saved" on the title bar), then the CPU went on sucking 2 cores with the IDE still showing the current line highlighted. I tried with a burst of clicks on the stop button, but didn't work (yet?).

Edit 2

I found a place in the macro where if I set a breakpoint and press F5 it works consistently, but if I remove the breakpoint it works once, then it hangs.

This is the code:

T0 = Timer
Debug.Print Application.CalculationState,
DoEvents ' Here I set the breakpoint
Debug.Print Application.CalculationState, Timer - T0

This is the output on the Immediate window after running it a few times with a breakpoint (you see the time it takes me to press F5 after seeing it stopping) and twice without the breakpoint. I am pasting a snapshot because I couldn't copy from a hanging Excel ready to be killed. While that code is executed the calculation is manual and the events are disabled. I don't think it was calculating.

Tomorrow I will try to reinstall Office.

Edit 3

Uninstalling and reinstalling Office didn't help.

But I think I found one factor that seems to reliably allow Excel to work without problems or to hang: the VPN.

If I start Excel without VPN connection, then I can work without problems. If I then start the VPN connection, the first click (and the following macro) has no problems, the second click is slow, the third one hangs and Excel needs to be killed. Disconnecting the VPN after the second click doesn't help. I tested this scenario 5-6 times, always with the same result.

I can't think of any reason why Excel would be affected by the VPN connection. The only addins installed are the ones I am struggling with, they are on the local drive, no 3rd party addins installed. My macros do not access any network drive. There is one class with one member Req As New MSXML2.XMLHTTP60 which is never used during my tests.

Edit 4

Nope, nothing to do with the VPN. Today I'm in the office, without VPN, and the problem is still there.

After reinstalling Office I had the same settings as before.

Is it possible to reset everything and make a new clean Office installation that doesn't remember anything from its previous life?

Answer 1

Here is a little update. I don't know if this is the answer that allows to fix the problem, but it's the last thing I have tried before the problem disappeared.

Thinking that the file was corrupted, I started creating a new file, importing the code and the forms from the allegedly corrupted one, creating the sheets from scratch (rather than copying the old ones to avoid any risk of duplicating the corruption) and I realized that I had both one global variable and one sheet called ShNesting. The duplicated names are not a problem because the global variable has narrower scope than the sheet object, so VBA never saw the sheet ShNesting. I checked on the git repository and I see that it has been working for 5 years with duplicated names without problems.

I renamed the sheet to Sheet4 and the problem disappeared.

I tried to reproduce the problem with backup copies of the corrupted file, but I wasn't able to reproduce it. I don't know if I wasn't able to reproduce it because the backup copies were saved in a condition that doesn't reproduce the problem and I wasn't able to recreate it, or if my computer decided to heal itself.

I waited a few days, I never had the problem, so I thought to leave a little update here.

Answer 2

Not really a solution, but too long for comment.
I know I've seen better stack tracing procedures, but maybe modify you're DoEvents2 and add a call to TraceRoutines at the top of all the UDF, SheetChange or whatever routines you think are running. Use the Timeout on the TraceRoutines as a circuit breaker.

Public DoingEvents As Boolean
Public TraceList As String
Public LastTime As Single
Public StartTime As Single

Sub DoEvents2()
    StartTime = Timer
    LastTime = StartTime
    If DoingEvents Then
        Debug.Print "Nested DoEvents calls?"
        TraceList = TraceList & "DoEvents" & " (@ " & Format(Timer - LastTime, "0.000") & "s)" & vbCrLf
        Debug.Assert False
    Else
        TraceList = vbNullString
    End If
    DoingEvents = True
    DoEvents
    DoingEvents = False
    Debug.Print Format(Timer - StartTime, "0.000") & " Doing Events"
    Debug.Print TraceList
End Sub

Sub TraceRoutines(Name As String, Optional Timeout As Long = 20)
'Static LastTime As Single
Static TimeoutTriggered As Boolean
    If Timeout = 0 Then TimeoutTriggered = True
    If DoingEvents Then
        TraceList = TraceList & Name & " (@ " & Format(Timer - LastTime, "0.000") & "s)" & vbCrLf
        LastTime = Timer
        If LastTime - StartTime > Timeout Then
            If Not TimeoutTriggered Then
                Debug.Print TraceList
                MsgBox "What is going on here?"
                Debug.Assert False
                TimeoutTriggered = True
            End If
        Else
            'Reset
            TimeoutTriggered = False
        End If
    End If
End Sub

Sub MyFunction()
    TraceRoutines "MyFunction"
End Sub

Answer 3

I have recently seen VBA code which exhibited very similar behaviour, that is it was code which had worked reliably for many years but then started to occasionally hang Excel. Eventually I isolated the problem to calls to Application.DoEvents that rather than taking the expected few milliseconds to execute could take up to three minutes.

The PCs in question had Netskope anti-virus installed and I discovered that the problem went away if Netskope was de-activated. Though for me a permanent solution was to amend the code to no longer use DoEvents.

So in answer to the question: Have you tried switching off any anti-virus?