How to fetch blob from azure

Question

I am trying to download some data from Azure Blob Storage in Go via the official azure-sdk-for-go.

To setup my development environment I have successfully logged in via az login. I have verified that the blob can be accessed via CLI:

az storage blob download --container-name [container-name] --name [blob-name] --account-name [storage-account-name] -f out.txt

This works as expected. To fetch the file unsing go I am using the following snippet (as a reproducer):

func getBlob(account, container, object string) ([]byte, error) {
        blobPath := fmt.Sprintf("https://%s.blob.core.windows.net/%s/%s", uri.Host, container, object)
        ctx := context.Background()

        credential, err := azidentity.NewDefaultAzureCredential(nil)
        if err != nil {
            return []byte{}, err
        }

        blobClient, err := azblob.NewBlockBlobClient(blobPath, credential, nil)
        if err != nil {
            return []byte{}, err
        }

        get, err := blobClient.Download(ctx, nil)
        if err != nil {
            return []byte{}, err
        }

        downloadedData := &bytes.Buffer{}
        reader := get.Body(&azblob.RetryReaderOptions{})
        _, err = downloadedData.ReadFrom(reader)
        if err != nil {
            return []byte{}, err
        }
        err = reader.Close()
        if err != nil {
            return []byte{}, err
        }
        data = downloadedData.Bytes()
        return data, nil
}

Being logged in via az login I would expect azidentity.NewDefaultAzureCredential(nil) to use this session/cerdentials (see https://docs.microsoft.com/en-us/azure/developer/go/azure-sdk-authentication?tabs=bash#-option-3-sign-in-with-azure-cli), however that appears not no work as expected. The error I get is the following:

===== RESPONSE ERROR (ErrorCode=AuthorizationPermissionMismatch) =====
Description=This request is not authorized to perform this operation using this permission.
RequestId:b078ec61-xxxx-xxxx-xxxx-604682000000
Time:2022-05-05T10:24:18.8093649Z, Details: (none)

exit status 255

What am I missing?

(I am coming from a AWS background so chances are I am making assumptions on how things should work based on that experience.)

Answer 1

Apparently interacting with blobs does not work with credentials provided by azidentity.NewDefaultAzureCredential(). Azure requires either SAS Tokens or Shared Keys in order to work with blobs. Here's an example function that can be used to get a client for specific blob:

func getBlobClient(account, container, object string) (*azblob.BlockBlobClient, error) {
    accountKey, ok := os.LookupEnv("AZURE_STORAGE_ACCOUNT_KEY")
    if !ok {
        return nil, errors.New("AZURE_STORAGE_ACCOUNT_KEY could not be found")
    }

    credential, err := azblob.NewSharedKeyCredential(account, accountKey)
    if err != nil {
        return nil, err
    }

    accountPath := fmt.Sprintf("https://%s.blob.core.windows.net/", account)
    serviceClient, err := azblob.NewServiceClientWithSharedKey(accountPath, credential, nil)
    if err != nil {
        return nil, err
    }

    containerClient, err := serviceClient.NewContainerClient(container)
    if err != nil {
        return nil, err
    }

    blobClient, err := containerClient.NewBlockBlobClient(object)
    if err != nil {
        return nil, err
    }

    return blobClient, nil
}

This uses the AZURE_STORAGE_ACCOUNT_KEY environment variable for the credentials.

The examples that can be found are rather confusing (and possibly wrong), an issue is opened here:

https://github.com/Azure-Samples/storage-blobs-go-quickstart/issues/7