We are getting 401 authentication failures from Cloudflare on purge requests
https://muckrock.sentry.io/issues/7692498953/events/23f1212c611b4ef2819a26bbf17382e1/?project=2689648&query=is%3Aunresolved&referrer=next-event
From the Sentry events:
28589020 is deleted (fails on tags)
28323016 is public and live (failed on files)
28329122 is public and live (failed on files)
28392265 is public and live. (failed on files)
I verified the credentials are set in Heroku environment variables.
From my understanding of the code:
The invalidate_cache task calls invalidate_cache_batch from documentcloud/documents/cache.py which calls the
_invalidate_cloudflare and _invalidate_cloudfront helpers.
It is returning errors from Cloudflare, so that is where the issue must be.
It does
for key, values in (("files", files), ("tags", tags)):
files is constructed
cloudflare_files.extend(
host + document.get_absolute_url() for host in settings.CLOUDFLARE_HOSTS
)
cloudflare_files.append(settings.PUBLIC_ASSET_URL + doc_path[1:])
PUBLIC_ASSET_URL is s3.documentcloud.org.
I think this is an intermittent issue with Cloudflare's auth endpoint returning 401s even though our credentials are legit. This seems to be an issue people have had with workers:
cloudflare/workers-sdk#13854
I've emailed Cloudflare support.
We are getting 401 authentication failures from Cloudflare on purge requests
https://muckrock.sentry.io/issues/7692498953/events/23f1212c611b4ef2819a26bbf17382e1/?project=2689648&query=is%3Aunresolved&referrer=next-event
From the Sentry events:
28589020 is deleted (fails on tags)
28323016 is public and live (failed on files)
28329122 is public and live (failed on files)
28392265 is public and live. (failed on files)
I verified the credentials are set in Heroku environment variables.
From my understanding of the code:
The invalidate_cache task calls invalidate_cache_batch from documentcloud/documents/cache.py which calls the
_invalidate_cloudflare and _invalidate_cloudfront helpers.
It is returning errors from Cloudflare, so that is where the issue must be.
It does
for key, values in (("files", files), ("tags", tags)):
files is constructed
cloudflare_files.extend(
host + document.get_absolute_url() for host in settings.CLOUDFLARE_HOSTS
)
cloudflare_files.append(settings.PUBLIC_ASSET_URL + doc_path[1:])
PUBLIC_ASSET_URL is s3.documentcloud.org.
I think this is an intermittent issue with Cloudflare's auth endpoint returning 401s even though our credentials are legit. This seems to be an issue people have had with workers:
cloudflare/workers-sdk#13854
I've emailed Cloudflare support.