'How to get all transaction from an address via the etherscan API in R?

I've been looking for a way to download all transaction from a specific contract since their start up until now. It's around 25k transactions as of today. I've tried different libraries such as ether and etherscanr but none seem to be working. I have extracted and adapted the code from etherscanr::etherscan_transactions() and I got this (API key removed!):


      txns <- data.frame()
      p <- data.frame()
      more <- TRUE
      page <- 1
      
      while(more == TRUE){
        message("Getting page ", page)
        while(dim(p)[1] == 0){
          #message("DEBUG: Into the loop")
          url <- paste0("https://api.etherscan.io/api?module=account&action=txlist", 
                        "&address=", address, 
                        "&page=", page, 
                        "&offset=", 1000, 
                        "&apikey=XXXXXXX") # not real api key in this example
          
          res <- jsonlite::fromJSON(url)
          
          message(" Results show message ", print(res$mes))
          
          p <- data.frame(blockNumber = as.integer(res$result$blockNumber), 
                              timeStamp = as.POSIXct(as.numeric(res$result$timeStamp), 
                                                     origin = "1970-01-01"), hash = res$result$hash, 
                              nonce = as.integer(res$result$nonce), blockHash = res$result$blockHash, 
                              transactionIndex = as.integer(res$result$transactionIndex), 
                              from = res$result$from, to = res$result$to, value = res$result$value, 
                              gas = res$result$gas, gasPrice = res$result$gasPrice, 
                              isError = res$result$isError == "1", input = res$result$input, 
                              contractAddress = res$result$contractAddress, cumulativeGasUsed = res$result$cumulativeGasUsed, 
                              gasUsed = res$result$gasUsed, confirmations = as.integer(res$result$confirmations), 
                              stringsAsFactors = FALSE)
          message("Page ", page, " retrieved is size ", dim(p)[1])
          Sys.sleep(5)
        }
         # message("DEBUG: Out of the loop")
        
        txns <- bind_rows(txns, p)
        page <- page + 1
        p <- data.frame() # reset
        
        Sys.sleep(7)
        
        
        if(page == 25){
          message("Done parsing up until page ", page, ".")
          page <- 1
          p <- data.frame()
          break
        } 
      }

This works fine up to page 10, in which I get the following message from res$mes:

[1] "Result window is too large, PageNo x Offset size must be less than or equal to 10000"

I understand the error message, but could there be any workaround to this? I heard about using the block number and iterate through that but that would increase the number of API calls beyond than the free plan.

Any other alternative is greatly appreciated, thanks!



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source