Skip to content

Commit

Permalink
Skip bad files in consolidation
Browse files Browse the repository at this point in the history
  • Loading branch information
palewire committed Jan 11, 2025
1 parent eaf6418 commit f64b549
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions newshomepages/extract/consolidate.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,11 @@ def consolidate(
with zf.open(file_name) as fp:
item_data = json.loads(fp.read())

# If the JSON doesn't have the right keys, skip it
if "metadata" not in item_data:
print(f"!!! Skipping {file_name} because it doesn't have the right keys")
continue

# Pull out the data we want to keep
identifier = file_name.replace(".json", "")
handle = identifier[:-5]
Expand Down

0 comments on commit f64b549

Please sign in to comment.