cloaked_req

cloaked_req is a Req adapter on the Rust wreq crate. It sends each request with the TLS and HTTP/2 fingerprint of a real browser, and you keep the Req API.

Docs: https://hexdocs.pm/cloaked_req

Installation

def deps do
  [
    {:cloaked_req, "~> 0.7.0"}
  ]
end

Precompiled NIFs

The package downloads a precompiled NIF for these targets:

• aarch64-apple-darwin
• aarch64-unknown-linux-gnu, glibc 2.34 or newer
• x86_64-unknown-linux-gnu, glibc 2.34 or newer

The release checks each Linux NIF and fails when it needs a glibc newer than 2.34.

On any other platform, for example Intel macOS or Alpine (musl), build the NIF from source:

1. Add {:rustler, "~> 0.38.0"} to your deps.
2. Install Rust 1.98 or newer, a C and C++ compiler, cmake, libclang, and git.
3. Set CLOAKED_REQ_BUILD=1 when you compile. On musl, also set RUSTFLAGS="-C target-feature=-crt-static". Without it, Rust cannot build the NIF as a shared library.

Usage

Attach the adapter to a request:

request =
  Req.new(url: "https://tls.peet.ws/api/all")
  |> CloakedReq.attach(impersonate: :chrome_136)

response = Req.get!(request)

Set impersonation later on an existing request:

request =
  Req.new(url: "https://example.com")
  |> CloakedReq.impersonate(:firefox_136)

Adapter options

Option	Type	Default	Description
:impersonate	atom	nil	Browser profile, for example :chrome_136
:cookie_jar	CookieJar.t()	nil	Stores and sends cookies across requests
:insecure_skip_verify	boolean	false	Skip TLS certificate verification
:local_address	IP string or IP tuple	nil	Source IP for outbound requests
:max_body_size	pos_integer | :unlimited	10 MB	Max request and response body size
:pool	Pool.t()	nil	Dedicated client with its own connection pool

:max_body_size applies in both directions. The adapter rejects a larger request body before it sends it, and returns an error when a response body grows past the limit. It counts decompressed bytes.

Req's :receive_timeout (default 15 s) starts with the request. Until the response headers arrive, it is one window that does not reset. DNS, connect, TLS, and the upload of the request body all count against it, so a large upload on a slow link needs a larger value. After the headers, it limits each wait for the next body chunk, so a body that keeps arriving has no total limit.

Req connect options

CloakedReq supports these Req :connect_options:

• :timeout: connect timeout in milliseconds for DNS, TCP, the proxy tunnel, and TLS. The default is 30 s. :receive_timeout also runs during the connect, so the lower of the two applies.
• :proxy: a {:http | :https, host, port, []} tuple.
• :proxy_headers: headers for the proxy, for example for proxy authentication.

Any other connect option returns an adapter error.

Req.new(
  url: "https://example.com",
  connect_options: [
    timeout: 5_000,
    proxy: {:http, "proxy.example.com", 8888, []},
    proxy_headers: [{"proxy-authorization", "Basic " <> Base.encode64("user:pass")}]
  ]
)
|> CloakedReq.attach(impersonate: :chrome_136)
|> Req.get!()

Req.new(url: "https://example.com")
|> CloakedReq.attach(local_address: {127, 0, 0, 1})

Errors and retries

A timeout, a refused connection, or a closed connection returns %Req.TransportError{} with the reason :timeout, :econnrefused, or :closed, the same struct Req's own Finch adapter uses. Req's default retry: :safe_transient therefore retries it with the usual backoff. Every other failure, such as a TLS or DNS error, an invalid option, or a body over :max_body_size, returns %CloakedReq.AdapterError{} and is not retried.

Cookie jar

The jar stores the cookies from set-cookie response headers and sends them with later requests that use the same jar. It checks the cookie domain against the public suffix list and rejects a cookie set on a public suffix or on a cross-origin domain. When Domain equals a request host that is itself a public suffix, such as localhost, the jar keeps the cookie as host-only. An explicit cookie header on a request wins, and the jar adds no cookies to that request. Over HTTP/2 the jar sends one cookie field per cookie.

jar = CloakedReq.CookieJar.new()

# Login: server sets session cookie
Req.new(url: "https://example.com/login")
|> CloakedReq.attach(impersonate: :chrome_136, cookie_jar: jar)
|> Req.post!(body: "user=admin&pass=secret")

# Dashboard: session cookie sent automatically
Req.new(url: "https://example.com/dashboard")
|> CloakedReq.attach(impersonate: :chrome_136, cookie_jar: jar)
|> Req.get!()

Connection pooling

By default every request goes through a shared client cache of limited size. Requests with the same impersonation profile, TLS verification, and connect timeout share one client and its connection pool. This needs no setup.

To keep identities apart, build a CloakedReq.Pool. A pool is a dedicated client with its own connections, TLS session cache, and HTTP/2 multiplexing. Build one per identity (per account, per proxy persona, per crawl), so no connection opened for one identity carries a request for another. Hold the pool in a worker's state and pass it to every request that worker makes.

pool = CloakedReq.Pool.new!(impersonate: :chrome_136)

Req.new(url: "https://example.com")
|> CloakedReq.attach(pool: pool)
|> Req.get!()

A pool fixes its client when you build it. The pool sets the impersonation profile, TLS verification, and connect timeout. On a request through a pool, the adapter still validates :impersonate, :insecure_skip_verify, and the :connect_options timeout, but ignores them. The proxy, source address, headers, body, cookie jar, and receive timeout of each request still apply. Each pool keeps up to 20 idle connections per host.

The BEAM garbage-collects the client when nothing references the pool struct, and its idle connections close. A worker that crashes without a teardown cannot leak the pool. To change a pool's identity, for example after its proxy exit changes, build a new pool and drop the old struct. :pool_idle_timeout (milliseconds) sets how long an idle connection stays open. Without it, wreq's default applies.

Impersonation profiles

The profiles come from wreq-util 0.2.0. Quote a profile atom that has a dot, for example :"safari_17.4.1".

Chrome

:chrome_100, :chrome_101, :chrome_104, :chrome_105, :chrome_106, :chrome_107, :chrome_108, :chrome_109, :chrome_110, :chrome_114, :chrome_116, :chrome_117, :chrome_118, :chrome_119, :chrome_120, :chrome_123, :chrome_124, :chrome_126, :chrome_127, :chrome_128, :chrome_129, :chrome_130, :chrome_131, :chrome_132, :chrome_133, :chrome_134, :chrome_135, :chrome_136, :chrome_137, :chrome_138, :chrome_139, :chrome_140, :chrome_141, :chrome_142, :chrome_143, :chrome_144, :chrome_145, :chrome_146, :chrome_147, :chrome_148, :chrome_149

Edge

:edge_101, :edge_122, :edge_127, :edge_131, :edge_134, :edge_135, :edge_136, :edge_137, :edge_138, :edge_139, :edge_140, :edge_141, :edge_142, :edge_143, :edge_144, :edge_145, :edge_146, :edge_147, :edge_148

Opera

:opera_116, :opera_117, :opera_118, :opera_119, :opera_120, :opera_121, :opera_122, :opera_123, :opera_124, :opera_125, :opera_126, :opera_127, :opera_128, :opera_129, :opera_130, :opera_131

Firefox

:firefox_109, :firefox_117, :firefox_128, :firefox_133, :firefox_135, :firefox_private_135, :firefox_android_135, :firefox_136, :firefox_private_136, :firefox_139, :firefox_142, :firefox_143, :firefox_144, :firefox_145, :firefox_146, :firefox_147, :firefox_148, :firefox_149, :firefox_150, :firefox_151

Safari

:"safari_15.3", :"safari_15.5", :"safari_15.6.1", :safari_16, :"safari_16.5", :"safari_17.0", :"safari_17.2.1", :"safari_17.4.1", :"safari_17.5", :"safari_17.6", :safari_18, :"safari_18.2", :"safari_18.3", :"safari_18.3.1", :"safari_18.5", :safari_26, :"safari_26.1", :"safari_26.2", :"safari_26.3", :"safari_26.4", :safari_ipad_18, :"safari_ipad_26", :"safari_ipad_26.2", :"safari_ios_16.5", :"safari_ios_17.2", :"safari_ios_17.4.1", :"safari_ios_18.1.1", :safari_ios_26, :"safari_ios_26.2"

OkHttp

:"okhttp_3.9", :"okhttp_3.11", :"okhttp_3.13", :"okhttp_3.14", :"okhttp_4.9", :"okhttp_4.10", :"okhttp_4.12", :okhttp_5

Limitations

• No HTTP/3 or QUIC. wreq speaks HTTP/1.1 and HTTP/2 only, so QUIC transport fingerprinting (JA4QUIC) is out of reach. When you need it, look at the Go library surf, which fingerprints QUIC; reaching it from Elixir means a sidecar or Port instead of a NIF.
• No streaming. The adapter rejects into:. The request body must be a binary or iodata, so a stream fails. A form_multipart body with a File.Stream is a stream.
• Do not set compressed: true. The profile sends its own accept-encoding, and wreq decompresses the response. compressed: true replaces the profile header and breaks the fingerprint. raw: true has no effect, because the body is already decompressed.
• No Finch-only options. :inet6, :unix_socket, and :request_timeout have no effect. Use :receive_timeout and connect_options: [timeout: ...] for timeouts.