Use URLs from List to save zip file

Question

Trying to use urllib.request to read a list of urls from a shapefile, then download the zips from all those URLs. So far I got my list of a certain number of URLs, but I am unable to pass all of them through. The error is expected string or bytes-like object. Meaning theres prob an issue with the URL. As a side note, I also need to download them and name them by their file name/#. Need help!! Code below.

import arcpy
import urllib.request
import os
    
os.chdir('C:\\ProgInGIS\\FinalExam\\Final')
lidar_shp = 'C:\\ProgInGIS\\FinalExam\\Final\\lidar-2013.shp'
zip_file_download = 'C:\\ProgInGIS\\FinalExam\\Final\\file1.zip'
    
    
data = []
with arcpy.da.SearchCursor(lidar_shp,"*") as cursor:
    for row in cursor:
        data.append(row)
data.sort(key=lambda tup: tup[2])
    
i = 0
with arcpy.da.UpdateCursor(lidar_shp,"*") as cursor:
    for row in cursor:
        row = data[i]
        i += 1
        cursor.updateRow(row)
    
counter = 0
url_list = []
with arcpy.da.UpdateCursor(lidar_shp, ['geotiff_ur']) as cursor:
    for row in cursor:
        url_list.append(row)
        counter += 1
        if counter == 18:
            break
for item in url_list:
    print(item)
    urllib.request.urlretrieve(item)

Answer 1

I understand your question this way: you want to download a zip file for each record in a shapefile from an URL defined in a certain field.

---

It's easier to use the requests package which is also recommended in the urllib.request documentation:

The Requests package is recommended for a higher-level HTTP client interface.

Here is an example:

import arcpy, arcpy.da

import shutil
import requests

SHAPEFILE = "your_shapefile.shp"

with arcpy.da.SearchCursor(SHAPEFILE, ["name", "url"]) as cursor:
    
    for name, url in cursor:
        
        response = requests.get(url, stream=True)
        if response.status_code == 200:
            with open(f"{name}.zip", "wb") as file:
                response.raw.decode_content = True
                shutil.copyfileobj(response.raw, file)

There is another example on GIS StackExchange:

• https://gis.stackexchange.com/a/392463/21355