[minor] Add verified Common Crawl bypass

[joecorall]

Allow a config option (enabled by default) to allow Common CRAWL bot through.

The config follows the docs at https://commoncrawl.org/ccbot by grabbing the list of IPs at https://index.commoncrawl.org/ccbot.json and doing a reverse DNS lookup on the IPv4 ranges before adding the client IPs to the list of good bots.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 86.91589% with 14 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/helper/commoncrawl.go	84.90%	8 Missing ⚠️
main.go	88.88%	6 Missing ⚠️

Files with missing lines	Coverage Δ
main.go	85.38% <88.88%> (+4.05%)	⬆️
internal/helper/commoncrawl.go	84.90% <84.90%> (ø)

... and 8 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.