IDA Hex Rays can't decompile function in automation

Question

when I reverse the binary with IDA gui, all the functions get decompiled without a problem.

but when I am running an automatic script on ida without gui, there is always the same function, that refuses to be decompiled. (when I am openning the same IDB that the automation script worked on, the function get decompiled without a problem)

I am using bip. and using BipFunc.can_decompile to check if a function can get decompiled.

EDIT:

according to an answer bellow, I have tried to add the following:

if not func.can_decompile:
    print(f"can't decompile function 0x{func.ea:04x}, trying again")
    decomp_all()
    if not func.can_decompile:
        print(f"can't decompile function 0x{func.ea:04x}, trying again")
        decomp_all_twice_cacheclear()
        if not func.can_decompile:
            print(f"can't decompile function 0x{func.ea:04x}, skipping...")
            return

sadly it did not work, I get all 3 prints every time, even on different binaries

it seems to be fixed on IDA Pro 7.6

Answer 1

There is several reason you can get an error on the decompilation from IDA. If it works on some case and other it does not it is probably because of the call analysis. When decompiling a function IDA will try to gather information on the function called by this one and in some case fail to get those information which will make the decompilation fail. But once that function has been decompiled, the information fetched by IDA will be updated, and so the decompilation of the caller function might now work. So basically it means you have to decompile the function in an order, which is a pain, for fixing that the simplest way is to just decompile everything twice, but it can take quite some time if you do it on "big" binaries.

I though I put that in the Bip repository somewhere but I can't find it, so here is a small plugin/code which should allows to do that:

from bip import *

class DecompileAll(BipPlugin):
    """
        Plugin for decompiling all the function in the binary.
    """

    @menu("Bip/DecompileAll/", "Invalidate hexrays caches")
    def clear_hxcCache(self):
        HxCFunc.invalidate_all_caches()

    @menu("Bip/DecompileAll/", "Decompile all func")
    def decomp_all(self):
        count = 0
        for f in HxCFunc.iter_all():
            count += 1
        print("0x{:X} functions decompiled".format(count))

    @menu("Bip/DecompileAll/", "Decompile twice with cache clear")
    def decomp_all_twice_cacheclear(self):
        HxCFunc.invalidate_all_caches()
        self.decomp_all()
        self.decomp_all()

Just for information the basic reason for decompilation error, is that it is not able to make a correct translation of some piece of code because it does not understand the assembly, this is typically true if there is a problem during the analysis and the code is not correctly detected (also happens a lot if you are dealing with obfuscation). You can typically view this case by an error telling you "failed analysis at ADDR" in the IDAPython console, and then look at the problem. Probably not your case but might still help.

Glad to hear you are using bip. So about the BipFunc.can_decompile function: like indicated in the documentation (https://synacktiv.github.io/bip/build/html/base/func.html#bip.base.BipFunction.can_decompile) it will just try to decompile the function and see if an error occurs. The code is pretty straight forward (https://github.com/synacktiv/bip/blob/master/bip/base/func.py#L372), this is mostly written for being done while using one-liner, its the same thing as catching the exception when trying to decompile.