Boto3 scan not scanning whole table

Question

I want to fetch a list of objects from a DynamoDB table with the following scan

video_list = video_table.scan(
        FilterExpression="attribute_not_exists(title)",
)

But I receive something like this

{'Items': [], 'Count': 0, 'ScannedCount': 948 ...}

By performing the exact same scan from the DynamoDB console I get many results.

I'm correctly connected to the table because I've received results before, it just stopped receiving them recently.

The script is in a Lambda function, deployed with the Serverless framework.

Answer 1

The Count and ScannedCount indicate that DynamoDB read 948 items, but none of these items matched the attribute_not_exists(title). I can think of two possibilities why this might happen:

1. Maybe you have a typo in your query - maybe the attribute name title is mispelled? Maybe you meant attribute_exists, not attribute_not_exists?

2. As always, a Scan operation is paged. This operation read just one page, of 948 items, and none of those specific items matched. Are these 948 items the entire database, or just the first 948 of a much larger number? If it's the latter, it's entirely possible (but I don't know how likely in your case...) that due to pure luck, the first 948 items all do not match, and you have to continue reading the next pages to get any matches.