New MCP URL filtering needs wildcard support

[cyanidium]

Is your feature request related to a problem? Please describe.
The MCP allowed_urls is not very useful: you need to specify exactly which URLs are allowed, so it would be better/just as much work to set the value in [[features.mcp.servers]].

Describe the solution you'd like
allowed_urls should support wildcards in both the path and domain. This still allows admins to prevent SSRF, but also allows them to say "all the MCP servers at this location are fine".

Describe alternatives you've considered
The main alternative would be to remove allowed_urls completely; as an admin if I already have to configure the URL, then I would rather fully setup the server than leave it up to an end user.

[dokterbob]

I would rather fully setup the server

On first reflection, this seems the wiser posture, in which case we'd remove allowed_urls. But let me sleep on it some time. Happy to hear other user's feedback. Particularly those with an information security background/hacker mindset.

[cyanidium]

Small correction after reviewing the code again: there is already sufficient wildcarding for the path component of the URL.

A small change here to convert from:

and parsed.host == allowed_parsed.hostname

to

and fnmatch(parsed.host, allowed_parsed.hostname)

would be all that is needed to give domains a similar capability. fnmatch.fnmatch() is probably better for the task than a regex because . wouldn't have two meanings in the same context.

Let's say my organisation hosts approved MCP servers as subdomains of mcp.example.com, like date.mcp.example.com and math.mcp.example.com. Then I can add an allowed_urls entry for https://*.mcp.example.com and not have to configured them individually. This allows teams to make their own tools without having to change the chainlit code to get them working.

This appears to already be supported for paths, so if we used mcp.example.com to host multiple tools, like mcp.example.com/date and mcp.example.com/math then it seems we can already add an allowed_urls entry for https://mcp.example.com and those sub-paths should be valid.

Yes, there are terrible ways to apply wildcards, like at the end of the domain; my.domain.* would also match my.domain.attacker.domain.com. I can't think of a reason why anyone would even want to try that, though, so it doesn't seem like a dangerous foot-gun.

[dokterbob]

Small correction after reviewing the code again: there is already sufficient wildcarding for the path component of the URL.

A small change here to convert from:

and parsed.host == allowed_parsed.hostname
to

and fnmatch(parsed.host, allowed_parsed.hostname)
would be all that is needed to give domains a similar capability. fnmatch.fnmatch() is probably better for the task than a regex because . wouldn't have two meanings in the same context.

Let's say my organisation hosts approved MCP servers as subdomains of mcp.example.com, like date.mcp.example.com and math.mcp.example.com. Then I can add an allowed_urls entry for https://*.mcp.example.com and not have to configured them individually. This allows teams to make their own tools without having to change the chainlit code to get them working.

This appears to already be supported for paths, so if we used mcp.example.com to host multiple tools, like mcp.example.com/date and mcp.example.com/math then it seems we can already add an allowed_urls entry for https://mcp.example.com and those sub-paths should be valid.

Yes, there are terrible ways to apply wildcards, like at the end of the domain; my.domain.* would also match my.domain.attacker.domain.com. I can't think of a reason why anyone would even want to try that, though, so it doesn't seem like a dangerous foot-gun.

You'd be surprised what people do when handed a footgun.

[dokterbob]

Have been thinking a bit; perhaps allowing wildcards only below domain name level, e.g. through regexp validation, erroring hard otherwise?