# ntex framework guide Learn how ntex services fit together, run servers, manage state and I/O, and build web applications. - [Service Model](./1-service.md) - [Service Pipelines](./2-pipeline.md) - [Service State](./3-state.md) - [Runtime](./4-runtime.md) - [Server](./5-server.md) - [Worker and Request State](./6-server-app.md) - [I/O Abstraction Layer](./7-io.md) - Web Application Framework - [Applications and Routing](./8.1-web-app.md) - [Application State](./8.2-web-app.md) - [Filters and Middleware](./8.3-web-app.md) - [Extractors and Responders](./8.4-web-app.md) - [Error Handling](./8.5-web-app.md) - [WebSocket Support](./8.6-web-app.md) - Integrations - [HTTP Client](./13-client.md) - [HTTP/2 Client and Server](./14-h2.md) - [gRPC Services and Clients](11-grpc.md) - [MQTT Client and Server](15-mqtt.md) - [Securing Connections with TLS](./10-tls.md) - [Byte Buffers](./9-bytes.md) - [Timers and Time Utilities](./12-time.md) ## API reference - [API documentation for the `main` branch](https://docs.ntex.rs/api/) - [API documentation for released versions on docs.rs](https://docs.rs/ntex/) ## LLM documentation - [llms.txt](https://docs.ntex.rs/llms.txt): rules and templates for generated code - [llms-full.txt](https://docs.ntex.rs/llms-full.txt): this guide as a single text file ## Migration - [Migrating from ntex 3 to ntex 4](./migration_3to4.md) # Component and Service Model The term *component model* can make a simple idea sound more complicated than it really is. It often brings to mind IoC containers, layers of dependency injection, and plenty of indirection. That is not what we are trying to build. Our goal is much simpler: reusable components that are easy to understand and, most importantly, easy to combine. Imagine a service that receives an operation, performs some work, and returns a result. Its implementation might be quite complex, but callers should not need to know about any of that. From the outside, the service should have a small, predictable interface. In Rust, the most natural way to express this is with a function: ```rust,ignore async fn execute(op: Operation) -> Result { // Execute the operation. // ... } ``` That is all the caller needs to see. Give the function an `Operation`, and it either returns an `OperationResult` or reports an `Error`. How it produces that result is an implementation detail. This small abstraction already gives us almost everything we need: one input, one output, and a clear contract. There is no hidden framework behavior or unnecessary ceremony. It is also easy to understand, straightforward to test, and naturally composable. ## Building an HTTP Endpoint Suppose we want to expose our execution service through an HTTP endpoint. What else do we need? At a minimum, we need a thin layer to handle the HTTP-specific details. It must deserialize the incoming request, call the execution service, and serialize the result into an HTTP response. Once again, the shape can stay simple. The endpoint can be just another function: ```rust,ignore async fn endpoint(req: HttpRequest) -> Result { // Extract an operation from the request. let operation = load_operation(req).await?; // Execute the operation. let result = execute(operation).await?; // Convert the result into an HTTP response. into_response(result).await } ``` There is nothing particularly fancy happening here. The endpoint is simply glue code: 1. Turn the HTTP request into a domain value—an `Operation`. 2. Pass the operation to the execution service. 3. Turn the resulting `OperationResult` into an HTTP response. Each part has a single responsibility, and the boundaries are explicit. The execution engine knows nothing about HTTP, while the HTTP layer does not need to know how the operation is executed. More importantly, these pieces do not depend on one another directly. Each one simply transforms one type into another. That is the key idea. ## Composing Services To compose two services, the output of one simply needs to match the input of the next. When the types line up, the services naturally form a transformation chain. In this example, the overall transformation is from an HTTP request to an HTTP response: ```text HttpRequest -> HttpResponse ``` Everything that happens in between is an implementation detail. We can add more steps, such as authentication and authorization, without changing the overall shape of the system: ```rust,ignore async fn endpoint(req: HttpRequest) -> Result { // Authenticate the request. let req = authenticate(req).await?; // Extract the operation from the request. let operation = load_operation(req).await?; // Check whether the operation is allowed. let operation = authorize(operation).await?; // Execute the operation. let result = execute(operation).await?; // Convert the result into an HTTP response. into_response(result).await } ``` The processing flow now looks like this: ```text HttpRequest -> authenticate -> load operation -> authorize -> execute -> build response -> HttpResponse ``` Each step is small and focused. It receives a value, performs its work, and returns the value expected by the next step. There is no tight coupling between the components. Once we start thinking in these terms, the component model becomes almost trivial: components are functions, and composition means connecting compatible outputs and inputs. Despite its simplicity, this model scales surprisingly far. ## Defining a Service Now that we have the basic idea, we can formalize what a *service* actually is. At its core, a service accepts an input and produces an output. The operation may be asynchronous, and it may fail. That should already sound familiar—it is the same shape as the functions in the previous examples. We can describe this idea with a small `Service` trait that resembles Rust's `Fn` traits: ```rust,ignore trait Service { /// Response produced by the service. type Res; /// Error produced by the service. type Error; async fn call(&self, req: Req) -> Result; } ``` This trait gives us a common way to describe any transformation from a request to a response. The real value lies in what this common interface allows us to build. We are no longer limited to ordinary functions. A service can be: - an asynchronous function; - a struct with its own configuration or internal state; - a wrapper around another service; or - a chain composed of several services. As long as every component follows the same `Service` contract, we can write generic tools that work with all of them. This is what makes middleware, combinators, and reusable pipelines possible. Instead of calling a collection of unrelated functions, we now have a system in which every component is composable by design. ## Services in Networking This model is not limited to HTTP endpoints. If you look closely at networking code—especially the generic, reusable parts—you will find the same pattern almost everywhere. Consider a TCP connection handler: ```rust,ignore impl Service ``` It receives a `TcpStream` and either handles the connection successfully or returns an error. Once connection handling follows this interface, we can build a generic server that works with any compatible TCP service. This is the basic idea behind [ntex-server](https://crates.io/crates/ntex-server). A TCP connector fits the same model: ```rust,ignore impl Service ``` It receives a socket address and returns an established TCP connection. From the caller's point of view, it is simply another transformation: ```text SocketAddr -> TcpStream ``` A TLS handshake is another example: ```rust,ignore impl Service, Error = io::Error> ``` It receives a plain stream and returns a TLS-enabled stream: ```text T -> TlsStream ``` The TLS service does not need to know whether the stream came from a server, a client, or an in-memory transport. It only needs a compatible stream as input. All these components follow the same basic pattern: ```text input -> service -> output ``` The types differ, but the model remains the same. A connector creates a stream, a TLS service transforms it, and a connection handler consumes it. Because these components share a common interface, they can be wrapped, reused, and composed without needing to understand one another's implementation. This is the same idea we explored with HTTP requests, applied to the rest of the networking stack. ## From a Simple Trait to a Practical Framework The [ntex-service](https://github.com/ntex-rs/ntex/tree/main/ntex-service) crate turns this model into a practical abstraction for real applications. The entire [ntex framework](https://github.com/ntex-rs/ntex) builds on it, from low-level connection handling to high-level HTTP and web services. Of course, a production-ready service model needs more than the minimal `Service` trait shown here. It must also support concerns such as: - service initialization and configuration; - readiness and backpressure; - graceful shutdown; - middleware; - service composition; and - pipeline state. These features add some complexity, but they do not change the core idea. A service still accepts an input and produces an output. Everything else exists to make that simple model reliable, reusable, and practical at scale. # Service Pipelines In the previous section, we described what a `Service` is and how to define one. Consider the HTTP endpoint from our earlier example: ```rust async fn endpoint(req: HttpRequest) -> Result { // Extract an operation from the request. let operation = load_operation(req).await?; // Execute the operation. let result = execute(operation).await?; // Convert the result into an HTTP response. into_response(result).await } ``` This function is a sequence of fallible, asynchronous transformations. Each step takes the output of the previous step and either produces the next value or returns an error. That structure maps naturally to a service pipeline. We can compose the same steps with an `and_then` combinator, similar to `Result::and_then()`: ```rust let pipeline = service(authenticate) .and_then(load_operation) .and_then(authorize) .and_then(execute) .and_then(into_response); ``` Each service runs only if the previous one succeeds. If any stage returns an error, the remaining stages are skipped and the pipeline returns that error. The completed pipeline is itself a service: ```rust Service ``` From the outside, it behaves just like the original `endpoint` function. It accepts an `HttpRequest` and produces an `HttpResponse`. Everything that happens between those two types remains an implementation detail. The benefit of this approach is that each stage remains independent. We can replace, remove, or insert a service without rewriting the rest of the pipeline. For example, adding request throttling is a local change: ```rust let pipeline = service(authenticate) .and_then(throttle) // <- throttling service .and_then(load_operation) .and_then(authorize) .and_then(execute) .and_then(into_response); ``` The external interface has not changed. The pipeline still represents the same overall transformation: ```text HttpRequest -> HttpResponse ``` The only requirement is that the types line up: the output of one service must match the input expected by the next. A throttling service can be designed specifically for HTTP requests: ```rust Service ``` Such a service inspects a request and returns it unchanged when the request is allowed to continue. The service can also be generic over its input: ```rust Service ``` A generic throttling service can act as a reusable concurrency or rate-limiting boundary anywhere in a pipeline. It does not need to know anything about HTTP or the application's domain types. This is what makes service pipelines useful: they let us build larger behavior from small, focused transformations while preserving a simple interface for the result. ## Implementing `and_then` Given the `Service` trait, we can describe `and_then` as a generic operation. It accepts two services and connects them so that the output of the first becomes the input of the second: ```rust fn and_then(first: A, second: B) -> impl Service where A: Service, B: Service, { AndThen { first, second } } ``` The type constraints describe the rules of composition: - `first` accepts the original request. - `second` accepts the response produced by `first`. - Both services use the same error type. - The combined service returns the response produced by `second`. The implementation is equally straightforward: ```rust struct AndThen { first: A, second: B, } impl Service for AndThen where A: Service, B: Service, { type Res = B::Res; type Error = A::Error; async fn call(&self, req: Req) -> Result { let intermediate = self.first.call(req).await?; self.second.call(intermediate).await } } ``` The first service processes the request and produces an intermediate value. If it succeeds, that value is passed to the second service. If it fails, the `?` operator returns the error immediately and the second service is never called. This is ordinary sequential composition with short-circuiting on error. Other familiar combinators, such as `then`, `map`, and `map_err`, follow the same general pattern. In practice, this machinery is already available in the [ntex-service](https://github.com/ntex-rs/ntex/tree/main/ntex-service) crate. Its [`ServiceChain`](https://docs.ntex.rs/api/ntex_service/dev/struct.ServiceChain.html) type provides a fluent, strongly typed API for composing services: ```rust let chain = service(authenticate) .and_then(load_operation) .and_then(authorize) .and_then(execute) .and_then(into_response); ``` Each call wraps the existing chain in another service combinator. The resulting `ServiceChain` is itself a service, so it can be extended further, wrapped in middleware, or placed in a `Pipeline`. The entire chain remains strongly typed. If one service produces a value that the next service cannot accept, the code fails to compile. Invalid pipelines are therefore caught while the application is being built rather than after it starts running. ## Readiness and Backpressure So far, `Service::call()` describes what a service does, but it says nothing about when the service is able to accept more work. Real services are not always ready. A service may have reached its limit for in-flight requests, filled an internal queue, or be waiting for an external resource such as a database connection. If callers can invoke `call()` at any time, the service must either buffer an unlimited amount of work or implement backpressure through a separate, non-composable mechanism. We can make backpressure part of the service contract by adding a readiness check: ```rust trait Service { type Res; type Error; async fn ready(&self) -> Result<(), Self::Error>; async fn call(&self, req: Req) -> Result; } ``` Conceptually, `ready()` acts as a gate. It waits until the service can accept more work without exceeding its internal limits. If the service cannot become ready—for example, because an underlying resource has failed—it returns an error. A caller would follow this pattern: ```rust service.ready().await?; let response = service.call(request).await?; ``` For a composed service such as `AndThen`, readiness is no longer local to one component. A request may pass through both services, so the combined service is ready only when both components are ready. A simplified implementation looks like this: ```rust impl Service for AndThen where A: Service, B: Service, { type Res = B::Res; type Error = A::Error; async fn ready(&self) -> Result<(), Self::Error> { self.first.ready().await?; self.second.ready().await?; Ok(()) } async fn call(&self, req: Req) -> Result { let intermediate = self.first.call(req).await?; self.second.call(intermediate).await } } ``` This captures the basic rule: the composed service should not accept a request unless every stage needed to process it is ready. A production implementation can check the services concurrently rather than waiting for them one at a time. More importantly, it must coordinate readiness across all concurrent calls to the same service graph. That coordination is the responsibility of the `Pipeline`. ## Pipeline Readiness A `Pipeline` turns a service chain into a runnable service graph. It owns the graph's state and coordinates readiness, calls, and shutdown. This matters because several requests may be in flight at the same time. Whether the pipeline can accept more work may depend on active calls, queue capacity, connection limits, or external resources used by any of its inner services. Readiness is therefore a cross-cutting concern. It cannot be managed reliably by looking at each call or service in isolation. Before dispatching a request, the pipeline waits until the required services are ready. If one of them reaches capacity, processing pauses until that service can accept more work. This gives us a useful distinction: - `ServiceChain` describes how services are connected. - `Pipeline` owns the runnable service graph and coordinates its lifecycle. ### Shared Readiness Concurrent calls use the same underlying pipeline and therefore share its readiness state. Each active call receives its own pipeline binding, which identifies the call while readiness is being coordinated. A normal call creates this binding internally: ```rust let response = pipeline.call(request).await?; ``` Sometimes a call must be stored, moved into another task, or allowed to outlive the borrow of `pipeline`. In that case, `Pipeline::call_static()` returns an owned future that keeps its binding alive: ```rust let call = pipeline.call_static(request); let response = call.await?; ``` Both methods use the same service graph and shared readiness state. The difference is how the lifetime of the call is managed. This coordination prevents multiple callers from independently driving the same readiness check. It also avoids hiding excess work in unbounded internal queues: when the pipeline is not ready, callers wait. The pipeline owns a readiness scope. A successful readiness check, such as `Pipeline::poll_ready()`, is cached by that scope until a service notifies it that its readiness has changed. Every binding and every call of the pipeline reuses the cached result, and calls do not consume it. A dispatcher that waits for readiness before each call therefore re-checks the services only after something has changed. ### The Service Tree A composed pipeline is a directed tree of services. The pipeline is the root, combinators are inner nodes, and the services that do the actual work are the leaves: ```rust let pipeline = service(authenticate) .and_then(ready(throttle)) .and_then(execute) .pipeline(()); ``` ```text Pipeline <- scope └── AndThen ├── AndThen │ ├── authenticate │ └── ready(throttle) <- scope └── execute ``` Requests flow down the tree, from the root to the leaves. Readiness flows in the opposite direction: every node depends on the readiness of the nodes below it, so readiness links point backward from the leaves to the root. The pipeline is ready only when `authenticate`, `throttle`, and `execute` are all ready. A service shared through `Rc` can be bound under several parents. Its readiness then feeds every one of them, which turns the tree into a directed acyclic graph. #### Building Dependencies in `Service::bind()` Readiness links are not discovered at runtime. Each service declares them once in `Service::bind()`. The pipeline calls `bind()` when it is constructed, and `Ctx::create()` calls it for services created later. In both cases, `bind()` runs synchronously before any `ready()`, `call()`, or `shutdown()`. The [`BindCtx`](https://docs.ntex.rs/api/ntex_service/struct.BindCtx.html) passed to `bind()` registers dependencies in the current readiness scope: - `ctx.subscribe(¬ifier)` registers a `ReadyNotifier` that the service uses to report changes in its own readiness. - `ctx.bind(&child)` binds a child service. It must be called for every child the service uses, including children that are only called and never checked for readiness. A raw child shares the current scope. A `ReadyBinding` or `ready()` child links its own scope below the current one. - `ctx.on_notify(&callback)` optionally registers a `ReadyCallback` that runs when a notification reaches the current scope. - A child pipeline is bound with `pipeline.bind_to(&ctx)`. ```rust struct Limited { inner: S, changed: ReadyNotifier, } impl> Service<(), u8> for Limited { type Res = S::Res; type Error = S::Error; fn bind(&self, ctx: BindCtx<'_>) { ctx.subscribe(&self.changed); // changes of our own readiness ctx.bind(&self.inner); // readiness of the child } // ready() and call() ... } ``` When a child implements `Service` for several request types, pass the types explicitly, for example `ctx.bind::<_, St, Req>(&self.first)`. Services that wrap child services can use `forward!(St, field, ...)`, which forwards `bind()`, `ready()`, and `shutdown()` to every listed field. Use `map_err = f` as the last argument to convert readiness errors. Register `Pipeline` fields with `pipeline.bind_to(&ctx)`. The default `bind()` does nothing, which is correct for leaf services that have no notification sources. A few rules follow from this design: - All dependencies must be known at bind time. `ready()` and `call()` cannot register new ones. - Using a `ReadyBinding` that its parent did not bind panics, so a missing `bind()` forward is detected instead of silently losing notifications. The panic message names the fix: call `ctx.bind(&child)` in the parent's `bind`, or use `forward!`. - Shared services can be bound more than once. Repeated subscriptions are idempotent. - Children created lazily at runtime must use `ctx.create(&factory).await`, which binds the new service in the current scope and invalidates it. ### Readiness Scopes A readiness scope caches a successful readiness check of its subtree until it is notified that readiness has changed. Not every node of the tree has its own scope. Scopes exist only at: - the pipeline root, - each [`ReadyBinding`](https://docs.ntex.rs/api/ntex_service/struct.ReadyBinding.html), - each service wrapped with `ready()` or created by `ready_factory()`. Plain services and combinators such as `and_then`, `then`, and `apply` store their children raw, so those children share the nearest scope above them. A scope's cached readiness remains valid only while every service below it is still ready: - When a service calls `ReadyNotifier::notify()`, the scope it is subscribed to and every scope above it are invalidated before any waiting task is woken. Scopes in unrelated branches stay valid. In the example above, a notification from `throttle` invalidates both scopes, while one from `execute` invalidates only the pipeline scope. - A notification never marks a service ready. It only forces the next `ready()` or `call()` to check again. A service whose readiness can change must therefore notify on every change, such as taking the last free slot, releasing a slot, a timer firing, or an external failure, before another caller can reuse the stale result. Waking a task is not the same as invalidating a scope: a wake only re-polls a pending check. ```rust fn acquire(&self) { self.in_flight.set(self.in_flight.get() + 1); self.changed.notify(); // may have become not ready } fn release(&self) { self.in_flight.set(self.in_flight.get() - 1); self.changed.notify(); // may have become ready } ``` ### Observing Readiness Notifications Use a reusable [`ReadyCallback`](https://docs.ntex.rs/api/ntex_service/struct.ReadyCallback.html) to observe explicit readiness notifications. A service registers its handle during `Service::bind`: ```rust fn bind(&self, ctx: BindCtx<'_>) { ctx.subscribe(&self.changed); ctx.on_notify(&self.on_notify); ctx.bind(&self.inner); } ``` Pipelines also expose `pipeline.on_notify(&callback)`, as do `PipelineBinding`, `PipelineState`, and `PipelineStateBinding`. For example: ```rust use ntex::service::ReadyCallback; let callback = ReadyCallback::new(|| println!("readiness notified")); # let pipeline = ntex::Pipeline::new((), async |_: ()| Ok::<_, ()>(())); pipeline.on_notify(&callback); pipeline.bind().on_notify(&callback); // Same scope and identity: one registration. ``` The callback runs synchronously when `ReadyNotifier::notify()` reaches its scope, including notifications propagated from child scopes and repeated notifications while the scope is already invalid. It does not run for successful readiness checks, cached calls, ordinary task wakeups, or internal resets such as lazy service creation and shutdown. All affected scopes are invalidated before callbacks run, outside internal borrows and locks. Nested notifications still invalidate every affected scope, but skip callback delivery on scopes already executing callbacks. Registrations made during that delivery are retained for later notifications. Delivery reuses each scope's registered callback storage without allocating a snapshot or cloning handles. Callbacks are registered on the nearest scope: raw services share their enclosing one, while `ready(svc)` gives the service its own. Registering one handle on two affected scopes invokes it once for each scope. Cloning a handle preserves its identity. Registrations persist until the scope is dropped, independently of readiness checks and runtime binding handles. Callbacks must own their captures; avoid strong reference cycles through a pipeline that retains the callback. ### Readiness Between Services Readiness must also be respected while a request moves through the service chain. A service should not invoke the next service directly, because doing so would bypass that service's readiness check. Instead, services call one another through an execution context: ```rust let result = ctx.call(&next_service, request).await?; ``` In ntex-service, this execution context is represented by `Ctx`. It carries the pipeline-level information needed to coordinate otherwise independent service instances, including the nearest readiness scope above the current service. `Ctx` uses that scope to decide whether a readiness check is needed: | Operation | Behavior | |---|---| | `ctx.call(&svc, req)`, scope valid | Dispatches immediately, without a check. | | `ctx.call(&svc, req)`, scope invalid | Waits for `svc.ready()` only. The scope above is not marked ready. | | `ctx.call(&binding, req)` | Checks the binding's own scope, independent of the scope above. | | `ctx.ready(&svc)` | Checks the raw service every time and does not update any scope. | | `ctx.ready(&binding)` | Reuses the binding's scope, re-checking only after a notification. | The same rules apply to services wrapped with `ready()`, which use a `ReadyBinding` internally. This allows backpressure to propagate through the chain, even if readiness changes while a request is being processed. Use `ready()` when a child needs its own readiness, independent of its siblings. A typical case is a second stage called after an `.await`: its readiness may change in the meantime, and with its own scope it is re-checked only when it notifies, not every time a sibling changes. A useful mental model is: - [`Service`](https://docs.ntex.rs/api/ntex_service/trait.Service.html) defines what happens at each step. - `ServiceChain` describes how the steps are connected. - [`Pipeline`](https://docs.ntex.rs/api/ntex_service/pipeline/struct.Pipeline.html) owns the runnable graph and its root readiness scope. - `ReadyBinding` and `ready()` add independent readiness scopes inside the graph. - [`Ctx`](https://docs.ntex.rs/api/ntex_service/struct.Ctx.html) controls how execution moves safely between services. ## The Complete `Service` Trait Once we include state, readiness, and shutdown, a simplified version of the ntex `Service` trait looks like this: ```rust trait Service { type Res; type Error; fn bind(&self, ctx: BindCtx<'_>) {} async fn ready(&self, ctx: Ctx<'_, Self, St>) -> Result<(), Self::Error>; async fn call(&self, req: Req, ctx: Ctx<'_, Self, St>) -> Result; async fn shutdown(&self, ctx: Ctx<'_, Self, St>); } ``` The context serves several related purposes: - It provides access to the pipeline state. - It connects services to the pipeline that owns them. - It coordinates readiness between services through the nearest readiness scope. - It allows one service to call another without bypassing backpressure. - It participates in orderly shutdown. The important point is that readiness is not merely a method that callers are expected to use correctly. It is built into the way requests move through a pipeline. This makes backpressure explicit, composable, and enforceable without relying on hidden queues or coordination outside the service model. The `St` parameter represents the service state made available through the context. We will explore service and pipeline state in the next section. ## Shutdown The `shutdown()` method represents the final stage of the service lifecycle: ```rust async fn shutdown(&self, ctx: Ctx<'_, Self, St>); ``` It is called by the service's owner—typically a `Pipeline`—when the service is being taken out of operation. A service can use `shutdown()` to: - stop background tasks gracefully; - release external resources; - flush buffered data; and - allow in-flight work to finish cleanly. Simple services may not need to do anything during shutdown. Services that own resources or run background tasks, however, should use this method to clean up properly rather than stopping abruptly. Composite services should also propagate shutdown to their inner services so that the entire pipeline stops cleanly. # Service State Services often need access to information that is not part of an individual request. Examples include connection metadata, a database pool, application configuration, or a channel to another subsystem. The `Service` trait represents this information with its `St` type parameter: ```rust,ignore trait Service { type Res; type Error; async fn call(&self, req: Req, ctx: Ctx<'_, Self, St>) -> Result; async fn ready(&self, ctx: Ctx<'_, Self, St>) -> Result<(), Self::Error>; async fn shutdown(&self, ctx: Ctx<'_, Self, St>); } ``` A service borrows the pipeline state through [`Ctx`] instead of owning it. The service can still own its own fields, such as configuration or inner services. The pipeline state is available during calls, readiness checks, and shutdown: ```rust,ignore async fn call(&self, req: Req, ctx: Ctx<'_, Self, AppState>) -> Result { let state: &AppState = ctx.st(); // ... } ``` Because the state type is part of `Service`, Rust verifies that a service is used with compatible state. A service requiring `AppState` cannot be placed directly in a pipeline that supplies an unrelated type. ## State-aware functions An asynchronous function whose first argument is `&St` can be converted into a state-aware service. This is the simplest way to define one: ```rust use std::convert::Infallible; use ntex::Pipeline; struct AppState { prefix: &'static str, } async fn format_value(state: &AppState, value: usize) -> Result { Ok(format!("{}-{value}", state.prefix)) } #[ntex::main] async fn main() { let service = Pipeline::new(AppState { prefix: "item" }, format_value); assert_eq!(service.call(10).await.unwrap(), "item-10"); } ``` The function receives `&AppState` for each call. The [`IntoService`] implementation performs this conversion automatically. You can call [`fn_service_st`] explicitly when the service value must be named or passed through an API before creating the pipeline. ## Testing state-aware function services A state-aware function service can be tested without starting a server. Create the service with [`fn_service_st`], place it and the test state in a [`Pipeline::new`], then call it with the request value. This exercises the same state access and readiness handling used in production. When the state contains external dependencies, a mocking library can verify how the service uses them: - [`tests/mockall`] defines the state behavior as a trait. Mockall creates a mock state and checks calls to its accessor and operation methods. - [`tests/shimforge`] keeps the concrete state type. Shimforge replaces its methods during the test, including a method on an object nested in the state, and checks the receiver's attributes and call arguments. Both examples configure call counts and ordering, invoke the service through a pipeline, and assert its response. ## Pipeline-owned state [`Pipeline::new`] stores one service and one state value together. Every call, readiness check, and shutdown operation on that pipeline uses the same state instance. The state belongs to that pipeline rather than to the process. Creating another pipeline creates another state instance unless the application explicitly shares a resource. For example, several worker-local states can each hold a clone of the same `Arc`. Pipeline bindings share the original pipeline and its state; cloning a [`PipelineBinding`] does not clone the state value. Bindings also participate in the pipeline's coordinated readiness and shutdown handling. ## Calling nested services `Ctx` does more than provide `ctx.st()`. It connects a service call to the pipeline's readiness and shutdown machinery. A service that wraps another service should call it through the context: ```rust,ignore async fn call(&self, req: Req, ctx: Ctx<'_, Self, St>) -> Result { ctx.call(&self.inner, req).await } ``` [`Ctx::call`] reuses valid readiness from the nearest upstream scope, checking only the target service on a cache miss. A child wrapped with `ready(svc)` also checks its own independent cache. Cached success is not consumed by calls, and the valid-cache path adds no readiness allocations. Use `ctx.call(&self.inner, req).await` even after an explicit readiness check: it retains the cached fast path while observing intervening invalidation. `Ctx` also provides `ready()` and `shutdown()` operations. Wrappers must forward `Service::bind`, `Service::ready`, and `Service::shutdown` to their children, for example with `forward!`. Services whose readiness changes must subscribe to a `ReadyNotifier` in `bind` and notify on every relevant change. A raw `Ctx::ready` call does not populate an upstream cache; use `ready(svc)` when a child needs independent persistent readiness. Readiness checks and their wake sources run on the pipeline's thread. This local-only requirement is specific to readiness; ordinary call, shutdown, runtime, and I/O task wakers are unchanged. `Service::call` remains the primitive dispatch operation and is not deprecated. For ordinary nested calls, use `Ctx::call` to retain context and readiness coordination rather than bypassing them. ## Substituting state Sometimes one service in a larger chain needs a different state type. [`map_state`] wraps that service with a fixed state value while allowing the outer pipeline to use another state type: ```rust use std::convert::Infallible; use ntex::{Pipeline, service::map_state}; struct Metrics { prefix: &'static str, } async fn record(metrics: &Metrics, value: usize) -> Result { Ok(format!("{}-{value}", metrics.prefix)) } #[ntex::main] async fn main() { let service = map_state(Metrics { prefix: "metric" }, record); // The wrapped service uses Metrics even though the outer pipeline uses (). let pipeline = Pipeline::new((), service); assert_eq!(pipeline.call(10).await.unwrap(), "metric-10"); } ``` ## State supplied per operation [`PipelineState`] stores a service without owning its state. The caller supplies a state reference for each readiness check, call, or shutdown operation: ```rust use std::convert::Infallible; use ntex::service::pipeline::PipelineState; struct RequestContext { prefix: &'static str, } async fn format_value(state: &RequestContext, value: usize) -> Result { Ok(format!("{}-{value}", state.prefix)) } #[ntex::main] async fn main() { let service = PipelineState::new(format_value); let first = RequestContext { prefix: "first" }; let second = RequestContext { prefix: "second" }; assert_eq!(service.call(1, &first).await.unwrap(), "first-1"); assert_eq!(service.call(2, &second).await.unwrap(), "second-2"); } ``` Use `Pipeline` when one state value should remain attached to the service. Use `PipelineState` when the same service and readiness machinery must operate with a state selected by each caller. `PipelineState::bind_state()` can attach an owned state value and produce a normal `PipelineBinding`, the state type must implement `Clone`. All of these handles share one readiness cache and pending checker, even when they supply different state values. Passing a different state does not invalidate cached success. `Service::ready` may inspect the state, but its result must describe the shared pipeline-wide gate; use separate pipelines or service bindings for independent state-specific gates. ## Carrying state with a request Some protocol boundaries receive both a request and the state that a nested pipeline should use. The [`RequestState`] trait represents such an input. [`State`] and `(St, Req)` implement it by splitting the value into a state and a request: ```rust use ntex::service::{RequestState, State}; let input = State { state: "connection-1", req: "request", }; let (connection, request) = input.unpack(); assert_eq!(connection, "connection-1"); assert_eq!(request, "request"); ``` For example, ntex HTTP services use `RequestState` to extract state associated with an accepted connection and then provide that state to the HTTP request and control-service pipelines. This keeps connection setup state separate from the HTTP request value while preserving its concrete type. A plain `Io` also implements `RequestState` with `()` as its state, so an ordinary server can pass accepted connections to `HttpService` without wrapping them. See [Passing Connection State to the HTTP Service](6-server-app.md#passing-connection-state-to-the-http-service) for an example that wraps an accepted `Io` in `State`. [`Ctx`]: https://docs.ntex.rs/api/ntex/struct.Ctx.html [`Ctx::call`]: https://docs.ntex.rs/api/ntex/struct.Ctx.html#method.call [`fn_service_st`]: https://docs.ntex.rs/api/ntex/service/fn.fn_service_st.html [`IntoService`]: https://docs.ntex.rs/api/ntex/trait.IntoService.html [`map_state`]: https://docs.ntex.rs/api/ntex/service/fn.map_state.html [`Pipeline::new`]: https://docs.ntex.rs/api/ntex/struct.Pipeline.html#method.new [`PipelineBinding`]: https://docs.ntex.rs/api/ntex/struct.PipelineBinding.html [`PipelineState`]: https://docs.ntex.rs/api/ntex/service/pipeline/struct.PipelineState.html [`RequestState`]: https://docs.ntex.rs/api/ntex/service/trait.RequestState.html [`State`]: https://docs.ntex.rs/api/ntex/service/struct.State.html [`tests/mockall`]: https://github.com/ntex-rs/examples/tree/main/tests/mockall [`tests/shimforge`]: https://github.com/ntex-rs/examples/tree/main/tests/shimforge # Runtime Each ntex runtime thread runs its own single-threaded event loop. A task stays on the thread where it was started, so services and futures can freely use `Rc`, `Cell`, `RefCell`, and other types that are not `Send` or `Sync`. This does not limit an application to one CPU core. A server normally starts several worker threads, each with its own single-threaded runtime. State shared between workers still needs thread-safe types such as `Arc`, atomics, or locks. Most applications only need `#[ntex::main]`, `#[ntex::test]`, and `ntex::rt::spawn`. The lower-level `System` and `Arbiter` APIs are there for explicit startup, shutdown, and cross-thread coordination. ## Choosing a runtime backend [`DefaultRuntime`] picks the async runtime and the matching ntex network reactor together. Cargo features decide which backend it uses: * `tokio` uses Tokio. If a Tokio runtime handle is already entered on the current thread, ntex uses it; otherwise it creates a current-thread Tokio runtime. ntex tasks run inside a Tokio `LocalSet`. * `compio` creates a Compio runtime on each runtime thread. Compio chooses the I/O driver for the platform and configuration. * With neither feature, ntex uses its [native runtime](#native-runtime). On Linux it tries io_uring and falls back to polling; other Unix platforms use polling, and Windows uses IOCP. If both `tokio` and `compio` are enabled, Tokio wins. With the native backend, `neon-polling` forces polling on Unix and `neon-uring` requires io_uring on Linux. Do not enable both. For example: ```toml [dependencies] ntex = { version = "4", features = ["tokio"] } ``` Tokio and Compio are convenient when an application also uses libraries from those ecosystems. The native backend avoids running a second general-purpose runtime and lets ntex use the platform reactor directly. `SystemRunner::block_on` blocks the current thread. Do not call it from inside an async Tokio task in an attempt to nest one runtime inside another. The cleanest setup is usually to let `#[ntex::main]` own the process entry point. [`DefaultRuntime`]: https://docs.ntex.rs/api/ntex/rt/struct.DefaultRuntime.html ## Spawning tasks [`ntex::rt::spawn`] starts a future on the current runtime thread: ```rust #[ntex::main] async fn main() { let task = ntex::rt::spawn(async { // `Rc`, `RefCell`, and other `!Send` values may be used here. 10usize }); assert_eq!(task.await.unwrap(), 10); } ``` Neither the future nor its result needs to be `Send`, but the future must be `'static`, so use `async move` when it captures owned values. `spawn()` panics if it is called outside a running ntex runtime. The returned [`JoinHandle`] can be awaited, canceled with `cancel()`, detached with `detach()`, or checked with `is_finished()`. It is **neither `Send` nor `Sync`**, even when the result is `Send`, and must stay on the thread that created it. If the task panics or is canceled, awaiting the handle returns a `JoinError` instead of the task's output. Dropping a `JoinHandle` detaches the task; it does not cancel it. Keep the handle and await it when you need the result. A detached task is not guaranteed to finish before its arbiter or the whole system shuts down. ### Spawning on another thread To run work on another runtime thread, get that arbiter's `Handle` and call `spawn()` on it. Both the future and its output must be `Send + 'static`. **`Handle::spawn()` returns a `RemoteJoinHandle`**, not a `JoinHandle`. Like the local handle, it is neither `Send` nor `Sync` and stays on the thread that called `spawn()`. ```rust async fn example(handle: ntex_rt::Handle) { let task = handle.spawn(async { 42 }); assert_eq!(task.await.unwrap(), 42); } ``` Called on the runtime's own thread, `Handle::spawn()` simply spawns a local task. From any other thread, the future is placed in a single allocation, which later also holds its output, and handed to the runtime thread, where it is polled and dropped. This works the same way on the native, Tokio, and Compio backends. Dropping a `RemoteJoinHandle` detaches the task, and `RemoteJoinHandle::cancel()` cancels it on the runtime thread. If the runtime shuts down or the task is canceled before it produces a result, the handle returns `JoinError`. A rejected submission is never polled, and its future may be dropped on the submitting thread. The two handle types are separate so the common local case stays small: a native `JoinHandle` is a single pointer. `RemoteJoinHandle` supports both paths, so a same-thread submission still avoids the shared allocation. For fire-and-forget work, use `Handle::spawn_detached()`. It returns `()`, and because the output is discarded on the runtime thread, the output does not need to be `Send`. A submitted future can start non-`Send` work with `ntex::rt::spawn`. It cannot hold that local `JoinHandle` across an `.await`, though, because the submitted future must stay `Send`. To await the local result, do it inside a local task. [`JoinHandle`]: https://docs.ntex.rs/api/ntex/rt/struct.JoinHandle.html [`RemoteJoinHandle`]: https://docs.ntex.rs/api/ntex/rt/struct.RemoteJoinHandle.html [`ntex::rt::spawn`]: https://docs.ntex.rs/api/ntex/rt/fn.spawn.html ## Timers and timeouts Use `ntex::time` rather than a backend-specific timer, so the code works with all three backends: ```rust use ntex::time::{Millis, sleep, timeout}; #[ntex::main] async fn main() { sleep(Millis(10)).await; let result = timeout(Millis(100), async { "done" }).await; assert_eq!(result.unwrap(), "done"); } ``` ntex timers are meant for scheduling and timeouts, not precise measurement. Their resolution is about 16 milliseconds, and even a zero-duration `sleep` or `timeout` waits for at least one timer tick. Use `timeout_checked` if a zero timeout should mean "no timeout". [Timers and Time Utilities](12-time.md) covers the timer design and the full API. ## System lifecycle [`System`] is the top-level runtime context. It holds the shared runtime configuration, keeps track of arbiters, delivers signals, and provides the blocking thread pool. `#[ntex::main]` builds a system for you. Building one by hand gives you a [`SystemRunner`]: ```rust fn main() { let result = ntex::rt::System::build() .name("worker") .build(ntex::rt::DefaultRuntime) .block_on(async { 10usize }); assert_eq!(result, 10); } ``` A `SystemRunner` can be driven in two ways: * `block_on(future)` starts the system, runs one root future, and returns its output when the future completes; * `run(callback)` calls a synchronous startup callback inside the running system, then keeps the event loop going until `System::stop()` or `System::stop_with_code()` is called. For a process that runs until something explicitly stops it: ```rust,no_run use std::io; use ntex::rt::{self, DefaultRuntime, System}; use ntex::time::{Millis, sleep}; fn main() -> io::Result<()> { System::build() .name("my-app") .build(DefaultRuntime) .run(|| { rt::spawn(async { sleep(Millis(10)).await; System::current().stop(); }); Ok(()) }) } ``` `run_until_stop()` is `run()` without a startup callback. Each of these methods consumes the runner, so pick one; you cannot call `block_on()` and then `run_until_stop()` on the same runner. A non-zero stop code is returned as an I/O error. Signal and panic handling are off by default. Turning on signal handling makes process signals available through `ntex::rt::signals`, but it does not decide how the application shuts down. ntex servers listen for signals while they run and handle their own shutdown. [`System`]: https://docs.ntex.rs/api/ntex_rt/struct.System.html [`SystemRunner`]: https://docs.ntex.rs/api/ntex_rt/struct.SystemRunner.html ## Arbiters and runtime threads An [`Arbiter`] is one runtime thread with its own event loop. Every system has a primary arbiter. `Arbiter::new()` and `Arbiter::with_name()` start another OS thread with the same runtime configuration. ntex server workers are arbiter threads too. `Arbiter::current()` returns the arbiter for the current thread and panics outside one. For an arbiter created with `new()` or `with_name()`, `stop()` asks it to shut down and `join()` waits for its thread to exit. The primary arbiter runs on the thread that started the system. It cannot be stopped on its own, has no thread handle, and its `join()` returns immediately. Stop the `System` to end it. The two levels shut down different things: * `Arbiter::stop()` stops one separately created runtime thread; * `System::stop()` stops every registered arbiter and ends the system. There are also two scopes for typed storage. `System::get_value()` stores a thread-safe value shared by the whole system, while `Arbiter::get_value()` stores a cloneable value local to one runtime thread. The latter is handy for per-worker clients or caches that should not be shared. [`Arbiter`]: https://docs.ntex.rs/api/ntex_rt/struct.Arbiter.html ## Blocking work Blocking an arbiter thread stalls connection handling, timers, and every other task on that thread. Move CPU-heavy work and blocking system calls to [`spawn_blocking()`]: ```rust #[ntex::main] async fn main() { let total = ntex::rt::spawn_blocking(|| (0..1_000u64).sum::()) .await .unwrap(); assert_eq!(total, 499_500); } ``` The closure and its result must be `Send`. Inside a system, the work runs on a blocking pool that grows and shrinks as needed. By default the pool allows up to 256 threads, and an idle thread exits after 60 seconds. Change these with `thread_pool_limit()` and `thread_pool_recv_timeout()`: ```rust,no_run use std::time::Duration; fn main() { ntex::rt::System::build() .thread_pool_limit(32) .thread_pool_recv_timeout(Duration::from_secs(30)) .build(ntex::rt::DefaultRuntime) .block_on(async {}); } ``` Outside a running system, `spawn_blocking()` does **not** start a background thread. The closure runs immediately on the calling thread, and the returned future is already complete. Treat this as a fallback, not as a way to start work before the runtime is up. Dropping the returned future stops queued work from starting, but it cannot interrupt a closure that is already running. Call `detach()` if the work should go ahead even when nobody needs the result. Cancellation and panics in the closure are reported as `BlockingError`. [`spawn_blocking()`]: https://docs.ntex.rs/api/ntex_rt/fn.spawn_blocking.html ## Runtime diagnostics The system pings each spawned arbiter every two seconds and keeps the ten most recent round-trip times, which `System::list_arbiter_pings()` returns. Set `ping_interval(0)` to turn pings off. Each arbiter also measures its scheduling latency every `latency_interval()`, 100 milliseconds by default. `Arbiter::latency()` returns a histogram of how late those measurements woke up. A watchdog thread marks an arbiter as stalled when it records no measurement for longer than the interval plus `ping_threshold()` (one second by default), and `Arbiter::latency()` counts the stalls. Set `latency_interval(0)` to turn off both measurement and stall detection. On the native (neon) runtime, `Handle::current_task()` returns the spawn location of the task being polled, and the stall log includes it. A stall with no running task happened in the driver. Tasks that ntex spawns itself report a location inside ntex. On Tokio and Compio, `current_task()` returns `None`. The native runtime also counts self-wakes, that is, wakes a task triggers while it is being polled, in `Handle::self_wake_count()`. If a task wakes itself 10,000 times in a row over at least one second without waiting for anything, it is logged once, with its spawn location, as a possible busy loop. Adjust the limits with `RuntimeBuilder::self_wake_threshold()`; a count of zero turns the warning off. On Linux and Windows, `System::set_latency_callback()` can receive a backtrace of a stalled arbiter. If the same task is still running after the capture, the callback also gets its spawn location. Without a callback, the backtrace is logged. On Linux, capturing a backtrace requires ntex signal handling and uses `SIGUSR2`, so an embedding application should not use that signal for anything else. On Windows, the watchdog briefly suspends the stalled thread and walks its stack (x86_64 and aarch64 only). ## Custom runners `System::build().build(...)` accepts any implementation of `ntex_rt::Runner`. This is an advanced integration point, not just a way to swap executors: ntex networking also expects a compatible reactor on every runtime thread, and a custom runner must set up everything the chosen backend relies on. Stick with `DefaultRuntime` unless the application deliberately provides both halves of that integration. ## Runtime attributes `#[ntex::main]` turns an async function into a synchronous entry point. It builds a `System` and drives the function's future: ```rust #[ntex::main] async fn main() { // Start the application. } ``` It accepts these options: * `name = "..."` sets the system name; * `signals = true` or `false`; * `panic_handling = true` or `false`; * `ping_interval = N` in milliseconds, where zero turns off arbiter pings; * `rt = Type` selects a custom runtime runner. For example: ```rust #[ntex::main( name = "my-service", signals = true, panic_handling = true, ping_interval = 250, )] async fn main() { // Start the application. } ``` The async function can return a value such as `Result<(), E>`, and the generated synchronous function returns it. `#[ntex::test]` creates a fresh system named after the test, marks it as a test system, turns off signal and panic handling, and sets up ntex test logging: ```rust #[ntex::test] async fn runtime_test() { let value = ntex::rt::spawn(async { 10usize }).await.unwrap(); assert_eq!(value, 10); } ``` The `no-test-logging` feature skips the logging setup. The main points to remember: * local tasks can be non-`Send`, but work sent to another arbiter must be `Send`; * dropping a join handle does not stop its task; * blocking closures belong in `spawn_blocking`; * ntex timers keep code independent of the backend; * stop an `Arbiter` you created to end one runtime thread, and the `System` to end the whole runtime; * use `DefaultRuntime` unless the application also provides the matching ntex network reactor. ## Native runtime The native runtime, called neon (hence the `neon-polling` and `neon-uring` features), has two parts: a task runner that polls futures, and a reactor that performs I/O. The task runner is the same on every platform. The reactor depends on the platform: io_uring or polling on Linux, polling on other Unix systems, and IOCP on Windows. Like the rest of ntex, the runtime is single-threaded and shares nothing. Each runtime thread runs its own tasks, and a task never moves to another thread. What sets it apart from Tokio or async-task is how tasks are woken. Rust's standard `Waker` must be `Send + Sync`, because any thread can hold a clone and call `wake()`. Tokio and async-task meet that requirement by synchronizing every waker operation: reference counts and task state are atomics, so each clone, wake, and drop pays for synchronization, even on the task's own thread. The native runtime separates local and remote access instead. A waker detects whether it is used on the thread that owns the task or on another thread: * **On the owner thread**, task state and reference counts are plain values wrapped in `Cell`. Cloning, waking, and dropping a waker need no atomic operations, and a woken task goes straight onto the local ready queue. * **From another thread**, the waker never touches the task directly. It pushes a command onto a Crossbeam queue, and the owner thread applies it on its next turn. This is a trade-off. Local wakes are cheaper than in Tokio or async-task, and most wakes in an ntex application are local, because a connection's I/O, timers, and services all live on one thread. Remote wakes are slower than in Tokio or async-task: each one goes through the queue, and the owner thread finishes the work later. The rest of this section covers the mechanics in detail. You do not need it to use ntex. ### Owner-thread wake handles [`ntex_rt::LocalWaker`] is how ntex code takes advantage of local wakes. It is an owned wake handle that stays on its owner thread. Do not confuse it with `ntex::task::LocalWaker`, which is a notification slot. Timers, I/O waiters, and notification slots store these handles internally. Creating a `LocalWaker` from a borrowed standard waker checks what kind of waker it is, which thread owns the task, and which runtime it belongs to, and then takes a local reference to the task. After that, cloning, waking, and dropping the handle are local operations with no further thread checks. The savings come from keeping the handle: acquiring a new one and converting it back every time is not free. `into_waker()` consumes the handle and returns a thread-safe standard waker; for native tasks it takes the standard-waker reference before releasing the local one. The handle itself cannot cross threads. On 64-bit targets, the handle, `Option`, and the notification slot are each 16 bytes, and neither native nor fallback handles allocate a wrapper. A native task is a single allocation that holds its routing metadata, its pinned execution core, and the storage for its future and result. There is no separate wake-token allocation and no per-task lookup table. The remote command queue does allocate segments, and arbitrary waker callbacks, timers, and I/O storage may allocate as they grow. All of this speeds up a hot path; it does not guarantee faster scheduling or request handling. Registering the same waker again still pays for checking its kind. Tokio, Compio, and wrapped or custom wakers keep their original callbacks and may pay a small extra dispatch cost. ### Remote wakes Only the owner thread changes a task's reference count, even after the task completes. A standard native waker used on another thread therefore never touches the count. Instead, it publishes reference and wake commands to the owner through a Crossbeam FIFO. On 64-bit targets each command is one pointer-sized word, with the operation stored in the three low bits that task alignment leaves free. The encoding preserves pointer provenance, and the tag is removed before the task is accessed. Other targets store the pointer and the operation separately, without increasing task alignment. Each task also has a sticky atomic flag. It is set just before the first remote waker command for the task is published, and it is never cleared. Tasks without the flag release their references directly, including the last one, without looking at the FIFO. Submitting a task from another thread does not set the flag, because its initial reference passes to the owner before the task can hand out a waker. Once the flag is set, ordering matters. The owner's final release waits behind acquisitions that are already queued, and while commands are pending, the owner's other releases go through the queue too. Otherwise a release could overtake an earlier queued release and invalidate a later wake or acquisition. When a remotely cloned waker returns to the owner, the flag store is synchronized; until then, the waker it was cloned from keeps the allocation alive. Processing a queued command also synchronizes with the flag store. The flag is only shared metadata; remote threads never update the reference count. ### Task completion When a task completes, its future is dropped. Its storage stays allocated while any native standard waker, local handle, or join handle still points to it, and a join handle also owns the result until it is taken. This leftover storage is not a running future. Dropping the join handle frees the result, whether or not wake handles remain. The list of unfinished tasks does not own them, so it never keeps an orphaned future alive. ### Routing remote commands A global registry maps runtime IDs, which are never reused, to runtimes, so a standard waker can find where to send its commands. Each remote thread caches an owned reference to the last runtime endpoint it used. A cache hit skips the registry lookup and the per-operation reference counting on the queue. Remote waker commands and unpolled submissions reserve admission before they publish, so shutdown waits for work that was already accepted, even when the publisher bypassed the registry. A cached endpoint that has closed rejects new work. Notifications run only after admission, registry locks, and cache borrows are released. When the cache evicts an endpoint, it drops the old one outside any locks or borrows and before admitting the new operation, so driver destructors can safely re-enter. Wakers used while thread-locals are being destroyed fall back to the registry. The owner thread has its own cache: the last runtime ID it checked and whether that endpoint is open. A hit also proves that the current thread owns the runtime, which saves a separate thread-ID lookup; a miss checks ownership before consulting the registry. Shutdown records the closed state in this cache before running callbacks, and because IDs are never reused, that entry stays correct. This cache holds no queue reference. Switching between runtimes on one thread goes back to the registry. Unpolled `Send` submissions travel through the same FIFO as remote wakes. Once a task is running, all of its state, references, join results, and awaiters are owner-local, and local wakes use the intrusive local ready queue. Native join awaiters hold `LocalWaker`s, which can still be released after they unregister. Each task stores a raw pointer to its queue that only the owner thread uses, rather than a strong or weak reference. Unfinished tasks use it while their runtime or runner keeps the queue alive; completed tasks need a live endpoint before they borrow it. Deferred releases on a busy owner publish without taking a temporary strong queue reference, but releases that may send a notification or run shutdown cleanup still take one. Cleanup that can run callbacks resolves the runtime ID to a strong queue reference outside the registry lock. ### Limits and memory layout Runtime IDs and owner-thread indexes are 16 bits, which keeps the task core at 80 bytes on 64-bit targets. A process can create at most 65,535 native runtimes, and at most 65,535 threads can own one. Neither is reused, and the process aborts if either runs out. Threads that only wake tasks or spawn through a `Handle` do not use an index. The fields used by wakes, runs, and releases sit in the first 64 bytes of the core, and the task-list links, used only on spawn, completion, and shutdown, come last. Task allocations are not cache-line aligned, so this makes single-cache-line access more likely but does not guarantee it. After running local tasks, the scheduler checks whether the remote FIFO is empty before trying to pop from it, avoiding the full memory fence that an empty pop costs. It still rechecks readiness afterward, so a command published in the meantime either keeps the runtime polling or wakes the driver. A non-empty FIFO is drained in order, within the same per-turn budget as before. ### Shutdown Shutdown closes admission, disables the runtime's wake endpoint, drains accepted commands, and drops live futures on their owner thread. The closed registry entry stays available for owner cleanup, including panic cleanup, until draining finishes. Only then is it removed, so owner releases cannot overtake pending commands. After that, standard-waker operations are ignored, and cloning such a waker produces an inert one. **Standard wakers still alive at shutdown permanently keep their task allocations**, even after they are dropped. This is retained memory, not continued execution: task allocations do not keep the driver or the queue's backing memory alive. Separately, a remote thread's endpoint cache can keep one closed queue and its driver alive until it caches a different endpoint or the thread exits. Owner-local handles can still be released after shutdown without touching the possibly dangling queue pointer. If reclaiming task memory matters, drop standard wakers and let the owner process their queued releases before shutdown. [`ntex_rt::LocalWaker`]: https://docs.ntex.rs/api/ntex_rt/struct.LocalWaker.html # Server ntex servers use several single-threaded workers to make use of multiple CPU cores. A dedicated accept thread owns the listening sockets and sends each connection to an available worker: ```text listeners -> accept thread -> worker-local service ``` The server does not know whether a connection speaks HTTP, MQTT, or a custom protocol. It manages sockets and workers; the service created for each worker decides how to handle the resulting `Io`. For encrypted connections, compose a TLS acceptor before the protocol service. The listener and protocol implementation do not need backend-specific TLS logic. See [Securing Connections with TLS](10-tls.md) for backend configuration, HTTPS helpers, and connection-level client-certificate authentication. ## Starting a server Create a server inside an active ntex `System`, register at least one listener, and call `run()`: ```rust,no_run use ntex::http::{HttpService, Response}; #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build() .bind( "http", "127.0.0.1:8080", ntex::SharedCfg::default(), async |_| { HttpService::new(async |_| { Ok::<_, std::io::Error>( Response::Ok().body("Hello world!"), ) }) }, )? .run() .await } ``` `server::build()` needs a current `System`, which `#[ntex::main]` creates in this example. `run()` panics if no listener has been registered. `bind()` receives: 1. a service name used in logs and to associate listeners with services; 2. an address that implements `ToSocketAddrs`; 3. a `SharedCfg` attached to accepted connections; 4. an asynchronous factory that builds the connection service for each worker. `run()` returns a cloneable `Server` controller. Awaiting it waits for the server to stop. The future always resolves to `Ok(())`; it is a completion notification, not a health report for every connection or worker. ## Binding and existing listeners `bind()` creates and owns the listening socket. If an address resolves to several socket addresses, ntex tries all of them and registers every listener that binds successfully. The call succeeds when at least one address binds. Use a concrete `SocketAddr` when the application needs exactly one listener. Use `listen()` when the application already owns a `TcpListener`, for example with socket activation or custom socket options. The builder's `backlog()` setting applies only to listeners created by later `bind()` and `configure()` calls; it cannot change the backlog of an existing listener. On Unix, `bind_uds()` and `listen_uds()` provide the same choices for Unix domain sockets. `bind_uds()` removes an existing file at the socket path before binding and removes the socket file again when the server stops. A server may register several listeners and several protocols. Keep service names clear and unique when using modular `configure()` callbacks, because those callbacks attach worker services to listeners by name. ## Workers and service factories By default, the server starts one worker for each available logical CPU. Set a different number with `workers()`: ```rust,no_run let builder = ntex::server::build() .workers(4); ``` Use at least one worker. With no workers, the server has nowhere to dispatch connections and remains paused. Each worker runs on its own arbiter thread and owns its own service instance. The factory passed to `bind()` is called for every worker. It may be called again if a worker restarts or a service is recreated after a readiness failure, so factory setup should be safe to repeat. The factory must be `Send`, `Clone`, and `'static`, but the service it creates stays on one worker and does not need to implement `Send` or `Sync`. This is why a worker-local service may use `Rc`, `Cell`, or `RefCell`. Data shared across workers must be shared explicitly with thread-safe types such as `Arc`, atomics, or locks. For richer process-wide and worker-local state, use `build_with_config()` and `ServerAppConfig`; the next chapter explains that model in detail. Connections are distributed across workers that currently report ready. If a worker reaches its connection limit, or one of its registered services is not ready, that worker temporarily stops receiving new connections. If no worker is available, the accept loop pauses until one becomes ready again. The default limit is 25,600 concurrent connections per worker: ```rust,no_run let builder = ntex::server::build() .workers(4) .max_connections(10_000); ``` This limit is process-wide and shared by every ntex server. Configure it before starting servers; each worker takes the current value when its services are created. ## Server configuration The commonly useful `ServerBuilder` settings are: * `name()` sets the accept and worker thread-name prefix. It defaults to the current system name. * `workers()` sets the worker count. It defaults to the number of logical CPUs the process may use, across all processor groups on Windows. * `backlog()` sets the listen backlog for listeners created afterward. * `max_connections()` sets the process-wide per-worker connection limit. * `affinity()` sets the CPU placement of workers, see [Worker placement](#worker-placement). The default is `Affinity::Soft` on Windows and `Affinity::Core` elsewhere. * `load_balancing()` selects how new connections are distributed between workers, see [Connection load balancing](#connection-load-balancing). The default `LoadBalancing::IncomingCpu` sends a connection to the worker on the CPU that receives its network packets unless that worker is busier than others, `LoadBalancing::LeastLoaded` to the less loaded of two random workers, and `LoadBalancing::RoundRobin` to the workers in turn. * `graceful_shutdown_timeout()` bounds how long a graceful stop waits for workers. The default is 30 seconds. * `stop_on_panic()` stops the server instead of restarting a worker that fails. * `graceful_shutdown()` makes `stop_on_panic` worker failures, `SIGQUIT`, fatal signals, and application panics use graceful rather than immediate server shutdown. * `disable_signals()` disables the server's built-in signal handling. * `stop_runtime()` stops the complete ntex `System` after this server stops. For example: ```rust,no_run use ntex::time::Seconds; let builder = ntex::server::build() .name("api") .workers(4) .backlog(1024) .max_connections(20_000) .graceful_shutdown_timeout(Seconds(15)) .stop_on_panic(); ``` `stop_runtime()` is useful when one server owns the complete process lifecycle. Do not enable it when other independent work or servers must keep using the same `System`. `Server::status()` returns the server status, and `Server::status_changed()` waits for the next change, e.g. to connect readiness to a supervisor or health check. The server is `Starting` until a worker becomes available, then `Ready`. It is `Unavailable` while no worker is available, `Paused` after `Server::pause()` until `Server::resume()`, and `Stopping` and then `Stopped` when it stops. Listeners accept connections only while the server is `Ready`. The status describes acceptance state, not whether every worker is healthy. Changes in quick succession may be reported once, with the latest status. ## Worker placement and load balancing Two settings decide where a connection is processed. `affinity()` places each worker thread on a CPU, and `load_balancing()` chooses the worker for each new connection. With the defaults, a connection is handled on the CPU, or at least the physical core, that receives its network packets. The packet data, the socket state and the worker's own data then stay in the same CPU caches and on the same NUMA node, and the network stack does not have to wake a thread on another CPU for every received packet. ```rust,no_run use ntex::server::{Affinity, LoadBalancing}; let builder = ntex::server::build() .workers(8) .affinity(Affinity::Core) .load_balancing(LoadBalancing::IncomingCpu); ``` ### Worker placement When the server starts, ntex reads the CPU topology and orders the logical CPUs the process may use: 1. On hybrid CPUs, faster cores come first: performance cores before efficiency cores. 2. Within each class, one logical CPU of every physical core, interleaved across the L3 caches of a NUMA node and then across NUMA nodes. With four workers on a two-socket machine, each socket gets two workers, on different L3 caches if the socket has several. 3. Then the remaining SMT siblings (hyper-threads), in the same order. Worker `i` takes entry `i` of that list, and the list starts over when there are more workers than CPUs. Workers never share a physical core until every core has a worker. A restarted worker keeps the placement of the worker it replaces. The placement is applied with one of the `Affinity` modes: | Mode | Windows | Linux and Android | |------|---------|-------------------| | `Disabled` | not placed | not placed | | `Cpu` | pinned to one logical CPU (`SetThreadGroupAffinity`) | pinned to one logical CPU (`sched_setaffinity`) | | `Core` | pinned to the logical CPUs of one physical core | pinned to the logical CPUs of one physical core | | `Soft` | preferred CPU only, the ideal processor (`SetThreadIdealProcessorEx`) | same as `Core` | The default is `Soft` on Windows and `Core` on Linux and Android. `Core` lets the scheduler move a worker between the SMT siblings of its core, which share its L1 and L2 caches. Other platforms have no topology support and ignore affinity. Placement is applied on the worker thread before the worker's runtime is created. The runtime, its timers and the worker's buffer page cache are therefore allocated on the NUMA node of the worker's CPU. Each placement is logged at `info` level: ```text Set affinity to Soft cpu 2 (core 2 3) for worker "api:worker:1" ``` `Worker::start()` takes a logical CPU index for workers started directly. On Windows, the CPUs of processor group 1 are numbered after those of group 0. #### Windows * The topology comes from `GetLogicalProcessorInformationEx`: physical cores with their efficiency class, L3 caches and NUMA nodes, across all processor groups. * Windows splits machines with more than 64 logical CPUs into processor groups, and a thread runs in one group at a time. `std::thread::available_parallelism()` counts only the current group, ntex counts all of them for the default number of workers and spreads the workers over all groups. A `Soft` worker whose ideal processor is in another group also gets the affinity of that whole group, because the ideal processor must be in the thread's group. * The CPUs the process may use come from an explicit process affinity mask (`start /affinity`, job object affinity limits), which also keeps the workers in the process's processor group, the group affinity of a job object the process runs in, and the default CPU sets of the process (`SetProcessDefaultCpuSets`, container CPU limits). The processor groups the process happens to run in are not a restriction, so without these all groups are used. Only Windows 10 APIs are used. * `Soft` is a hint. The scheduler runs the worker on its ideal processor when that CPU is free, and on another CPU of the group otherwise, so a worker is not stuck behind a busy thread or interrupt-heavy CPU. Use `Core` or `Cpu` on dedicated machines when strict isolation matters more. #### Linux and Android * The topology comes from sysfs, `/sys/devices/system/cpu/cpuN/`: `topology/core_cpus_list` (or `thread_siblings_list`) for the physical core, `cache/index3/shared_cpu_list` for the L3 cache (the package if there is none), the `nodeN` link for the NUMA node, and `cpu_capacity` for big.LITTLE cores. * The CPUs the process may use come from `sched_getaffinity()`, so `taskset`, cgroup cpusets (`docker run --cpuset-cpus`) and systemd's `CPUAffinity=` are honored. * Linux has no preferred-CPU hint, so `Soft` works like `Core`. * The default number of workers follows `std::thread::available_parallelism()`, which also honors a cgroup CPU quota (`docker run --cpus`). A quota does not restrict which CPUs run the process, so the workers are still spread over all allowed CPUs. ### Connection load balancing The accept thread accepts a connection, chooses a worker and queues the connection to it. Items submitted with `Server::process()` are dispatched the same way, on the submitting thread. While no worker is available, items go to the server's command loop, which runs on the thread that started the server, and wait there until a worker is ready. The worker turns the socket into an ntex `Io` and registers it with its own runtime, so all later I/O of the connection runs on that worker. Only workers that are ready receive connections, see [Workers and service factories](#workers-and-service-factories). Among them, `LoadBalancing` decides: * `RoundRobin` sends connections to the workers in turn, skipping workers blocked by a task while another worker is available. * `LeastLoaded` picks two random workers and takes the one with the lower runtime utilization, the share of time the worker's runtime was busy, averaged over about 250 ms and sampled at most every 10 ms. If the two differ by less than 10%, the one with fewer open and queued connections wins. Workers blocked by a task are skipped while another worker is available, see [Stall detection](#stall-detection). On tokio, busy time is available only for current-thread runtimes, otherwise the connection count decides. * `IncomingCpu`, the default, prefers the worker on the CPU that receives the connection's packets, and works like `LeastLoaded` otherwise. For each connection, `IncomingCpu`: 1. asks the operating system which CPU receives the connection's packets, once, when the connection is accepted; 2. finds the workers placed on that CPU, or, if none is, the workers whose physical core contains it. This includes SMT siblings the process may not use, for example with `taskset`, because the network stack may still process packets on them; 3. takes the one that is not blocked and has the lowest utilization, then the fewest connections; 4. compares it with the `LeastLoaded` choice, and uses it unless its utilization is more than 20% higher or its load exceeds the other worker's load by more than half plus 16 connections. The limits keep the preference from overloading workers when the network adapter delivers traffic to only a few CPUs, when its hash spreads connections unevenly, or when some connections are much more expensive than others. Under normal load nearly every connection stays local. Connections without an incoming CPU use the `LeastLoaded` choice. This covers loopback connections on Windows, platforms other than Windows, Linux and Android, `Affinity::Disabled`, and custom `ServerConfiguration` implementations that do not override `incoming_cpu()`. A worker whose placement could not be applied is never preferred as local. Connections that arrive while no worker is available wait in the server and are balanced the same way once a worker becomes available. The incoming CPU is read once. If the operating system later moves the connection's traffic to another CPU, for example when RSS is rebalanced or irqbalance moves an interrupt, the connection stays on its worker. `ServerMetrics::incoming_cpu_hits` and `incoming_cpu_misses` show how well this works, see [Incoming CPU](#incoming-cpu). #### Rescheduling A connection waits in its worker's queue until the worker takes it. The server moves queued connections to other workers when their worker: * fails, for example its service panics. Connections that are queued while no other worker is available wait in the server for the restarted worker; * is not ready, for example its service is re-created; * is blocked by a task, see [Stall detection](#stall-detection). With two or more workers, the server checks for blocked workers every 50 ms and moves their queued connections while another worker is not blocked. Only queued connections move. A connection the worker has taken is registered with that worker's runtime and stays there, a blocked worker handles it once it is unblocked, and a failed worker drops it. Moved connections are balanced again without their incoming CPU, and are not counted in `incoming_cpu_hits` and `incoming_cpu_misses` again. `ServerMetrics::rescheduled` counts them, see [Rescheduled connections](#rescheduled-connections). #### Windows: receive-side scaling A network adapter with receive-side scaling (RSS) hashes each connection's addresses and ports into an indirection table that selects a receive queue, and each queue is served by one CPU, where NDIS runs the receive processing. ntex reads that CPU with `WSAIoctl(SIO_QUERY_RSS_PROCESSOR_INFO)`, which returns its processor group and number. * With `Affinity::Soft`, the ideal processor of the chosen worker is the RSS CPU, so the receive processing that completes the worker's I/O and the worker thread it wakes usually run on the same CPU. * By default, RSS uses one logical CPU per physical core, starting at the adapter's base processor. Worker placement also takes one logical CPU of every core first, so with a worker per core the RSS CPUs have their own workers. * Loopback connections (`127.0.0.1`, `::1`) do not pass through an adapter and have no RSS CPU. Connections to one of the machine's own adapter addresses, including connections from WSL, do have one, which is useful to test this on a single machine. Adapters without RSS, or with RSS disabled, report none. * `Get-NetAdapterRss` shows the adapter's RSS settings, including `BaseProcessorNumber`, `MaxProcessors` and the processors the indirection table uses. If many connections miss, align the RSS processors with the worker CPUs, for example: ```powershell Set-NetAdapterRss -Name "Ethernet" -BaseProcessorNumber 0 -MaxProcessors 8 ``` #### Linux and Android: `SO_INCOMING_CPU` ntex reads `getsockopt(SOL_SOCKET, SO_INCOMING_CPU)` (Linux 3.19 or later), the CPU on which the network stack processed the last packet received for the socket. For a newly accepted connection, that is the CPU that processed the last packet of the TCP handshake. * That CPU is set by the interrupt affinity of the adapter's receive queue (`/proc/irq/N/smp_affinity_list`), and changed by Receive Packet Steering (`/sys/class/net//queues/rx-N/rps_cpus`), which moves packet processing to other CPUs, and by Receive Flow Steering, which moves it to the CPU where the consuming thread last ran. * Loopback packets are processed on the sending CPU, so loopback connections report the client's CPU. Test servers on loopback therefore count hits and misses too. * `ethtool -l ` shows the number of receive queues, `ethtool -L combined N` changes it, and `/proc/interrupts` shows which CPUs handle the queues. For the best locality, use one queue per worker and pin the queue interrupts to the worker CPUs. irqbalance may move them again, unless it is stopped or told to leave those interrupts alone. ### Choosing the settings * The defaults suit most servers, including servers without a multi-queue network adapter, where `IncomingCpu` behaves like `LeastLoaded`. * Use `LoadBalancing::LeastLoaded` if per-worker `active_connections` stay uneven under `IncomingCpu`, for example when an adapter has fewer queues than the server has workers and its CPUs carry most connections. * On machines shared with other busy processes, prefer `Affinity::Soft` on Windows, or `Affinity::Disabled`, so the scheduler can move workers away from busy CPUs. * With more workers than CPUs, several workers share a CPU, and `IncomingCpu` chooses the least loaded of them. * `LoadBalancing::RoundRobin` distributes connections the way earlier ntex versions did, which keeps tests that count connections per worker predictable. ## Controlling a running server Clone the controller when another task needs to pause, resume, or stop the server: ```rust,no_run use ntex::server::Server; async fn manage(server: Server) -> std::io::Result<()> { let control = server.clone(); ntex::rt::spawn(async move { control.pause().await; // Perform maintenance or wait for an external readiness condition. control.resume().await; control.stop(true).await; }); server.await } ``` The server starts in a paused state and resumes once its first worker becomes ready. `pause()` stops accepting new connections without closing existing ones; new connection attempts normally wait in the operating system's listen backlog. `resume()` enables the listeners again. Use `stop(true)` for a graceful stop. The accept loop closes its listeners, then the server asks workers currently available for dispatch to finish active work and waits up to the configured graceful-shutdown timeout. A worker still initializing or recreating its service is not part of that wait, so long-lived factory setup should have its own cancellation or ownership boundary. Use `stop(false)` when the controller should not wait for graceful worker draining. Available worker services are still asked to shut down and get a default three-second local timeout, but that cleanup may continue after the server controller reports completion. An immediate stop is therefore not a guarantee that every in-flight operation has finished. Repeated stop requests join the stop already in progress. Await either `stop(...)` or the original `Server` when later work must begin only after the server has reached its stopped state. ## Server metrics `Server::metrics()` returns a `ServerMetrics` snapshot. The call is synchronous and cheap enough to run periodically. Counters and durations are totals since the server started, so take two snapshots and use their differences to get rates. Gauges (`active_connections`, `accept_backlog`, the `*_active` and `*_buffered` values, cached pages) describe the moment of the snapshot. ```rust,no_run use ntex::server::Server; use ntex::time::{Seconds, sleep}; async fn report(server: Server) { let mut prev = server.metrics(); loop { sleep(Seconds(10)).await; let cur = server.metrics(); let paused = cur.accept_paused_duration - prev.accept_paused_duration; println!("accept paused {paused:?}, backlog {}", cur.accept_backlog); for (w, p) in cur.workers.iter().zip(&prev.workers) { println!( "worker {:?}: {} new, {} active, not ready {:?}", w.id, w.items - p.items, w.active_connections, w.not_ready_duration - p.not_ready_duration, ); } prev = cur; } } ``` The worker list has one entry per worker slot, in the same order in every snapshot. A restarted worker keeps the metrics of the worker it replaces. ### Accept loop These fields are on `ServerMetrics`. | Metric | Kind | Meaning | |--------|------|---------| | `accept_pauses` | counter | Number of times the accept loop stopped accepting, because no worker was ready or the server was paused. | | `accept_paused_duration` | duration | Total time the accept loop was not accepting, including the current pause. | | `accept_backlog` | gauge | Accepted connections waiting for a ready worker. | ntex does not reject connections at the limit. While the accept loop is paused, new connections wait in the operating system's listen backlog, so these metrics are the main saturation signal: * If `accept_paused_duration` grows steadily, all workers are at capacity. Add workers (`workers()`), raise `max_connections()` if workers have spare CPU and memory, or scale out. * A non-zero `accept_backlog`, or short frequent pauses, means the server runs at its limit. Clients then see higher connect latency. If load comes in bursts, a larger `backlog()` keeps the operating system from dropping connection attempts. * Pauses caused by `Server::pause()` are counted too. Account for them during maintenance windows. ### Incoming CPU These fields are on `ServerMetrics`, see `LoadBalancing::IncomingCpu`. | Metric | Kind | Meaning | |--------|------|---------| | `incoming_cpu_hits` | counter | Connections sent to the worker on the CPU that receives their packets. | | `incoming_cpu_misses` | counter | Connections with a known incoming CPU sent to another worker, because no worker is placed on that CPU or its worker was busier. | Connections without a known incoming CPU, e.g. loopback connections on Windows or servers on platforms other than Windows, Linux and Android, are counted in neither. Many misses mean the network adapter delivers packets to CPUs without workers, for example CPUs outside the process affinity, or that the workers on the receiving CPUs are busier than the others, for example the adapter uses fewer queues than the server has workers. Check the adapter's RSS settings (`Get-NetAdapterRss` on Windows, `ethtool -l` and `/proc/interrupts` on Linux) against the worker placement. ### Rescheduled connections This field is on `ServerMetrics`, see [Connection load balancing](#rescheduling). | Metric | Kind | Meaning | |--------|------|---------| | `rescheduled` | counter | Connections taken from the queue of a failed, not ready or blocked worker and sent to another worker. | An occasional increase follows a worker restart or a stall. A steady increase means workers are blocked often, check `latency.stalls` and `current_task` in [Runtime](#runtime) for the blocking code. ### Listeners `ServerMetrics::listeners` has one `ListenerMetrics` entry per bound socket. | Metric | Kind | Meaning | |--------|------|---------| | `name`, `addr` | label | Service name and address passed to `bind()`. | | `accepted` | counter | Connections accepted on the listener. | | `connection_errors` | counter | Per-connection accept errors, for example a connection reset before it was accepted. | | `resource_errors` | counter | Accept failures caused by resource exhaustion, for example running out of file descriptors. After each one the listener pauses briefly. | The rate of `accepted` is the connection rate per service; compare it across listeners to see where load comes from. A few `connection_errors` are normal on the internet; a rising rate suggests clients that give up early or a network problem. Treat any `resource_errors` as a capacity problem: raise the file descriptor limit (`ulimit -n`) or lower `max_connections()`. ### Workers `ServerMetrics::workers` has one `WorkerMetrics` entry per worker slot. | Metric | Kind | Meaning | |--------|------|---------| | `id` | label | Worker id. | | `items` | counter | Connections (or other items) the worker received. | | `active_connections` | gauge | Open connections on the worker. Network servers only. | | `not_ready` | counter | Number of times the worker's services stopped being ready, for example on reaching `max_connections()`. | | `not_ready_duration` | duration | Total time the worker was not ready, including the current period. | | `restarts` | counter | Number of times the worker was restarted after a failure. | | `io`, `io_other` | | I/O statistics, see below. | | `page_cache` | | Buffer page cache statistics of the worker thread, see below. | | `runtime` | | Runtime statistics of the worker thread, see below. | * Compare `active_connections` and the rate of `items` across workers to check load balancing. With the default `LoadBalancing::IncomingCpu` or with `LoadBalancing::LeastLoaded`, they should be close. A worker with much higher `not_ready_duration` than the others points to long-lived or expensive connections pinned to it. * If `not_ready_duration` grows on all workers together with `accept_paused_duration`, the server is at capacity. * `active_connections` at the `max_connections()` value with idle CPU means the limit is too low for the workload, for example with many idle keep-alive or WebSocket connections. * `restarts` should stay at zero. Any increase means a worker failed; alert on it and check the logs. ### Runtime `WorkerMetrics::runtime` is a `RuntimeMetrics` with the statistics of the worker thread's runtime. A field is `None` when the runtime does not provide it. Totals are kept when a worker restarts; `current_turn`, `current_task` and the `latency` gauges (`last`, `last_sample`, `stalled`) describe the current worker thread. | Metric | Kind | Runtimes | Meaning | |--------|------|----------|---------| | `busy_duration` | duration | all | Time the runtime spent running tasks and handling I/O events. | | `polls` | counter | neon, compio | Runtime turns that ran scheduled tasks. | | `polls_again` | counter | neon, compio | Runtime turns that left scheduled tasks for the next turn. | | `self_wakes` | counter | neon | Polls in which a task woke itself, for example by yielding. | | `poll_histogram` | histogram | neon, compio | Duration of runtime turns. | | `current_turn` | duration | neon, compio | How long the current runtime turn has been running, `None` while the runtime is idle. | | `current_task` | location | neon | Spawn location of the task being polled now. | | `latency` | | all | Scheduling latency and stalls, see below. `None` if sampling is disabled. | `latency` is a `LatencyMetrics`. A task on each worker sleeps for `interval` and records how late it wakes up: | Metric | Kind | Meaning | |--------|------|---------| | `interval` | setting | Sampling interval. | | `last` | gauge | Lateness of the most recent sample. | | `histogram` | histogram | Lateness of all samples. | | `last_sample` | gauge | Time of the most recent sample. | | `stalls` | counter | Times the worker recorded no sample for longer than `interval` plus the stall threshold. | | `stall_duration` | duration | Total duration of finished stalls. | | `stalled` | gauge | Whether the worker is stalled now. | On tokio, `busy_duration` is tokio's worker busy time, which excludes handling I/O events. * Utilization is the share of time the worker was busy, see [Worker utilization](#worker-utilization). A worker close to 100% is CPU bound: add workers or cores. Uneven utilization across workers points to expensive connections pinned to a few workers. * Scheduling latency is what every request on the worker waits in addition to its own processing. Watch high percentiles of the `histogram` difference; values of several milliseconds mean the worker is overloaded or a task runs too long between `.await` points. * Long turns in `poll_histogram` delay every task woken during the turn. They come from tasks that run too long between `.await` points; move such work to `spawn_blocking()` or split it. * A ratio of `polls_again` to `polls` close to 1 means tasks are scheduled faster than the worker runs them, the worker has a backlog. A high `self_wakes` rate points to tasks that busy-loop by yielding. * `stalls` should stay at zero. A stall means blocking code, such as a synchronous call or a lock, held the worker. Alert on it, see [Worker stalls and busy loops](#worker-stalls-and-busy-loops). The sampling interval and the stall threshold are system builder settings, see [Worker stalls and busy loops](#worker-stalls-and-busy-loops). #### Worker utilization `busy_duration` is a running total, kept across worker restarts. To get utilization, take two snapshots and divide the growth of `busy_duration` by the time between them: ```rust,no_run use std::time::Instant; use ntex::server::Server; use ntex::time::{Seconds, sleep}; async fn utilization(server: Server) { let mut prev = server.metrics(); let mut prev_time = Instant::now(); loop { sleep(Seconds(10)).await; let cur = server.metrics(); let now = Instant::now(); let elapsed = (now - prev_time).as_secs_f64(); for (w, p) in cur.workers.iter().zip(&prev.workers) { let (Some(busy), Some(prev_busy)) = (w.runtime.busy_duration, p.runtime.busy_duration) else { continue; // not reported by this runtime }; let stalled = w.runtime.latency.as_ref().is_some_and(|l| l.stalled); let pct = if stalled { 100.0 } else { 100.0 * busy.saturating_sub(prev_busy).as_secs_f64() / elapsed }; println!("worker {:?}: {pct:.1}% busy", w.id); } prev = cur; prev_time = now; } } ``` * Use the same window for all workers. 5 to 10 seconds smooths out short spikes; shorter windows show bursts. * On neon and compio `busy_duration` includes the turn that is running now, so a worker held by a long task shows up as 100% busy right away. * On tokio, `busy_duration` is reported only for current-thread runtimes. It excludes handling I/O events, so values are lower than on neon or compio for the same load. Tokio adds busy time only after a batch of tasks finishes, so a stalled worker can show low utilization; treat a worker with `latency.stalled` set as fully busy, as the example does. * Compare utilization with the rate of `items` and `active_connections`. High utilization with few connections points to expensive requests; high utilization on one worker only points to an imbalance, see the bullets above. ### I/O metrics I/O metrics are collected per I/O configuration (`IoConfig` in a `SharedCfg`) on each worker thread. Every listener configuration is registered automatically, so `WorkerMetrics::io` has one `IoConfigMetrics` entry per listener configuration, identified by `id`, `tag` and `service` of the `SharedCfg`. Other configurations, for example the configuration of an HTTP client used by the application, get their own entry when created with `IoConfig::set_metrics(true)`. Connections whose configuration has no metrics of its own are counted together in `io_other`. I/O metrics are always collected: `set_metrics(false)`, the default, only removes the separate entry of a configuration, and does not affect the registered entries of listener configurations. To get the totals of a service across the server, add up its entries from all workers. Each entry is an `IoMetricsSnapshot`: | Metric | Kind | Meaning | |--------|------|---------| | `bytes_read` | counter | Bytes read from the transport. | | `bytes_written` | counter | Bytes written to the transport. | | `read_buffered` | gauge | Bytes held in read buffers, not yet consumed by the application. | | `read_buffers_held` | gauge | Connections that keep a read buffer allocated for unconsumed input, such as the start of a frame that has not fully arrived. | | `read_buffer_capacity` | gauge | Bytes allocated by those read buffers, a whole page for a pooled buffer. | | `write_buffered` | gauge | Bytes in write buffers, or held by the transport and not yet written. | | `read_backpressure` | counter | Number of times read backpressure was enabled: the read buffer reached `set_read_backpressure()` and reading stopped. | | `read_backpressure_active` | gauge | Connections with read backpressure enabled now. | | `read_backpressure_time` | duration | Total connection-time spent under read backpressure. | | `write_backpressure` | counter | Number of times write backpressure was enabled: the write buffer reached `set_write_backpressure()`. | | `write_backpressure_active` | gauge | Connections with write backpressure enabled now. | | `write_backpressure_time` | duration | Total connection-time spent under write backpressure. | | `eager_writes` | counter | Writes started in place, as soon as buffered output reached `set_write_buf_threshold()`. | | `eager_writes_partial` | counter | Eager writes that could not write all output and left the rest to the write task. | | `scheduled_writes` | counter | Writes left to the write task. With io-uring all writes use the write task. | | `closed` | | Closed connections by reason, see below. | Backpressure time is connection-time: ten connections under backpressure for one second add ten seconds. Divide its rate by the number of active connections to get the share of time a connection spends blocked. * **Throughput.** Rates of `bytes_read` and `bytes_written` give the bandwidth per service and worker. Divided by the `items` rate they give the average traffic per connection. * **Memory.** `read_buffer_capacity` plus `write_buffered` is the memory held in connection buffers. If it is high, lower `set_read_backpressure()` and `set_write_backpressure()`, or lower `max_connections()`. * **Partial input.** A read buffer is released once its input is consumed, so busy connections are counted in `read_buffers_held` only briefly; a snapshot mostly shows idle connections waiting for the rest of a frame. Each keeps a whole page for a few bytes, so `read_buffer_capacity` far above `read_buffered` means memory is held for small leftovers. Many such connections point to slow or stalled clients; `set_frame_read_rate()` bounds how long they can hold a buffer, and a smaller minimum page in `set_read_size()` lowers the cost of each. Buffers held by a pending io-uring read, and frames split off a buffer that keep its page alive, are not counted; [Io buffer attribution](#io-buffer-attribution) counts them. * **Slow clients.** Growing `write_backpressure_time` means clients read slower than the application writes. A few slow clients are normal; a steady high `write_backpressure_active` holds memory and connection slots. Use `set_write_timeout()` to close clients that stop reading, and watch `closed.write_timeout`. * **Slow application.** Growing `read_backpressure_time` means the application processes input slower than it arrives. Look at the service's processing time or worker CPU; a larger `set_read_backpressure()` only helps with short bursts. * **Write path.** Most writes should be eager on fast networks. A high share of `eager_writes_partial` means the socket send buffer fills, which is normal for large responses and slow clients. If `scheduled_writes` dominate, output is written in many small pieces; raising `set_write_buf_threshold()` batches more output per system call, lowering it reduces latency. ### Close reasons `IoMetricsSnapshot::closed` is a `CloseStats`. Each closed connection is counted once in `total` and once under the first reason reported for it: | Metric | Meaning | |--------|---------| | `total` | Closed connections. | | `keepalive` | The connection was idle for longer than the keep-alive timeout. | | `read_timeout` | Input did not arrive within the read timeout, for example `set_frame_read_rate()`. | | `write_timeout` | Write backpressure stayed enabled for longer than `set_write_timeout()`. | | `peer_gone` | The peer closed the connection. | | `io` | The transport failed with an I/O error. | | `decoder` | Incoming data could not be decoded, for example a malformed request. | | `encoder` | Outgoing data could not be encoded. | | `service` | The service failed or closed the connection. | | `shutdown` | The server was shutting down. | | `custom` | Application-defined reasons `Custom1` to `Custom8`, counted in `custom[0]` to `custom[7]`. | | `other` | No reason was reported, for example the application closed the connection or the response had no keep-alive. | * A high share of `keepalive` closes means connections sit idle. Shorten `set_keepalive_timeout()` to free connection slots sooner, or lengthen it if clients reconnect often (compare with the `accepted` rate). * Rising `read_timeout` closes point to slow or idle clients, or to slow-read attacks; rising `write_timeout` closes to clients that stopped reading. Check the timeouts against legitimate slow clients before tightening them. * `decoder` closes count protocol errors. A sudden increase usually means a broken client release, a scanner, or abuse. * `io` and `peer_gone` closes reflect network quality and client behavior. An `io` spike together with `connection_errors` points to a network problem. * `service` closes come from the application. Compare their rate with the application's error logs. * `shutdown` closes count connections closed because the server or worker was stopping. Expect them during deploys and restarts; elsewhere they point to unexpected stops. Applications can track their own reasons, for example authentication failures, rate-limited clients or upgrades, by setting a reason before closing the connection or failing the service: ```rust,ignore use ntex::io::CloseReason; // in a service that owns the connection's `Io` io.set_close_reason(CloseReason::Custom1); // for example "auth failed" io.close(); ``` The first reported reason is kept, so a reason set by the application is not replaced by one the dispatcher reports later, such as `Service`. ### Page cache `WorkerMetrics::page_cache` holds the statistics of the worker thread's buffer page cache. I/O buffers are allocated as pages of fixed sizes, and released pages are kept in a per-thread cache for reuse. Use `page_cache.get(size)` for one page size, `page_cache.iter()` for all sizes, or `page_cache.total()` for the sum. Each entry is a `PageCacheStats`: | Metric | Kind | Meaning | |--------|------|---------| | `size` | label | Page size, `BytePageSize::Unset` for totals. | | `hits` | counter | Pages taken from the cache. | | `misses` | counter | Pages allocated because the cache was empty. | | `cached` | counter | Released pages returned to the cache. | | `freed` | counter | Released pages freed because the cache was full. | | `pages` | gauge | Pages in the cache now. | | `bytes` | gauge | Memory held by the cached pages. | * A high miss rate together with a high `freed` rate for the same size means the cache is too small for the workload: pages are freed and then allocated again. Raise its size for that page size with `ntex_bytes::set_page_cache_size()` on each worker thread, for example from the worker's service factory. * If `bytes` stays high while traffic is low and misses are rare, the cache is larger than needed; lowering it frees memory. After a worker restart these statistics may be slightly off, because the old and new worker threads can update them at the same time. ### Io buffer attribution The I/O metrics count only the read buffers a connection holds. Input that leaves the connection is not counted: `Bytes` frames split off a read buffer and kept by the application keep the whole buffer alive, and so do buffers held by a pending io-uring read. Io buffer attribution charges each read buffer allocation to a `BytesTracker` for as long as the allocation lives, wherever its data ends up. It answers questions such as "how much memory do request bodies of this listener hold" or "which connection holds the most input". Attribution needs the `tracker` feature, which adds a pointer to the header of every heap buffer: ```toml ntex = { version = "4", features = ["tracker"] } ``` It is configured per `IoConfig` with `set_buf_tracker()`: | `BufTracker` | Behavior | |------------------|----------| | `Disabled` | Read buffers are not attributed. This is the default. | | `PerConnection` | Each connection charges its read buffers to its own tracker, created when first used. | | `Shared(tracker)` | All connections of the configuration, on all workers, charge their read buffers to `tracker`. | A shared tracker gives a total per listener or service. Create it before the server starts and keep a clone to read it: ```rust,ignore use ntex::io::{IoConfig, BufTracker}; use ntex::time::{Seconds, sleep}; use ntex::util::BytesTracker; use ntex::SharedCfg; let api_input = BytesTracker::new(); let cfg = SharedCfg::new("API").add( IoConfig::new().set_buf_tracker(BufTracker::Shared(api_input.clone())), ); let server = ntex::server::build() .bind("api", "0.0.0.0:8080", cfg, async |_| { /* service */ })? .run(); ntex::rt::spawn(async move { loop { sleep(Seconds(10)).await; let snap = api_input.snapshot(); println!( "api input: {} bytes in {} buffers", snap.live_bytes, snap.live_allocations ); } }); ``` With `PerConnection`, `IoRef::buf_tracker()` (and `IoContext::buf_tracker()` in filters) returns the connection's tracker, for example from `HttpRequest::io()` in a handler. It returns `None` when attribution is disabled. The returned reference must not be used after `Io::set_config()`; clone the tracker to keep it longer. A clone keeps reporting buffers that outlive the connection, but does not keep the connection alive. ```rust,ignore if let Some(tracker) = req.io().and_then(|io| io.buf_tracker()) { if tracker.snapshot().live_bytes > 64 * 1024 * 1024 { // the connection holds too much input, reject or close it } } ``` The tracker is not limited to the connection's own read buffers. The application can charge its own buffers to it, so memory a connection causes outside the I/O layer shows up in the same metrics, for example a body assembled from many frames, a decompressed payload, or a buffer received from another source on behalf of the connection. This works with `PerConnection` and `Shared` trackers: * `BytesMut::with_page_size_in(size, tracker)` takes a new buffer from the page cache, charged to `tracker`. * `BytesMut::set_tracker(tracker)` charges an existing buffer. If the allocation is already charged to another tracker, the charge moves, also while `Bytes` frames split off it are still alive. * `BytesMut::tracker()` returns the tracker a buffer is charged to. ```rust,ignore use ntex::util::{BytePageSize, BytesMut}; let mut body = match req.io().and_then(|io| io.buf_tracker()) { Some(tracker) => BytesMut::with_page_size_in(BytePageSize::Size16, tracker), None => BytesMut::with_page_size(BytePageSize::Size16), }; while let Some(chunk) = payload.next().await { body.extend_from_slice(&chunk?); } ``` Such buffers follow the same rules as read buffers: the charge lasts until the last handle is dropped, a buffer that grows stays charged, and copies are not charged. `BytesTracker::snapshot()` returns a `BytesTrackerSnapshot`: | Metric | Kind | Meaning | |--------|------|---------| | `live_bytes` | gauge | Memory held by the attributed allocations, including the allocation header. | | `live_allocations` | gauge | Number of attributed allocations. | The values are not part of `ServerMetrics` or `IoMetricsSnapshot`; read the trackers you created and export them next to those metrics. The two fields are read one after the other, so a snapshot taken while buffers are allocated or released is only approximately consistent. `BytesTracker::ptr_eq()` tells whether two handles refer to the same tracker. What is charged: * Every read buffer the connection allocates or takes from the page cache, with its full allocation size, not the bytes of data in it. A buffer that grows into a new allocation stays charged to the same tracker. * Buffers of a pending read, including io-uring provided-buffer pages once data is received into them. A buffer returned empty by a read goes back to the page cache and its charge ends. * Plaintext read buffers of filters such as TLS. * Frames split off a read buffer, until the last `Bytes` or `BytesMut` referring to the allocation is dropped. The charge ends on whichever thread drops it, also after the connection is closed. What is not charged: * Pages in the page cache. A cached page is charged again to its next user only. * Write buffers. * Copies of the data: small frames stored inline in the `Bytes` handle (up to 23 bytes on 64-bit targets), the result of `Bytes::trimdown()`, and anything the application copies into its own types, unless it charges them explicitly as described above. Cloning, splitting or freezing a buffer does not copy, so it stays charged. When `Io::set_config()` changes the configuration of a connection, for example after a protocol upgrade: * With `PerConnection` in both, the connection keeps its tracker. * Switching to `Shared(tracker)` moves the buffers the connection holds to `tracker`. Frames that already left the connection stay charged to the old tracker. * Switching from `Shared` or `Disabled` to `PerConnection` creates a new tracker. * Switching to `Disabled` stops charging new buffers; existing buffers stay charged until they are released. Cost: * Without the `tracker` feature there is no cost. With the feature, the buffer header grows from 16 to 32 bytes on 64-bit targets (20 bytes on 32-bit targets), for every heap buffer, attributed or not. * Cloning, splitting and freezing buffers do no tracker work. Allocating, growing and releasing the last handle of a buffer check for a tracker, and update its atomic counters if there is one. * A `Shared` tracker is updated by every worker thread and becomes a contended cache line under load. Use one per listener or service rather than one per server, or `PerConnection` when only per-connection limits are needed. Interpreting the values: * **Escaped input.** `live_bytes` of a shared tracker well above the sum of `read_buffer_capacity` for the same service means the application keeps input after the connection is done with it: request bodies queued for later processing, frames stored in caches, or pending io-uring reads. Storing a small part of a large buffer keeps the whole allocation; copy it, for example with `Bytes::trimdown()`, to release the buffer. * **Leaks.** `live_allocations` that keeps growing while the number of active connections is stable points to frames that are never dropped. * **Per-connection cost.** `live_bytes / live_allocations` is the average allocation size, and `live_bytes` divided by `active_connections` the average input memory per connection. A high average with little traffic points to the partial-input case described in [I/O metrics](#io-metrics). * **Abusive clients.** With `PerConnection`, check the connection's tracker before accepting more input, and close connections that exceed a limit. Unlike `set_read_backpressure()`, this also counts input the connection already handed to the application. ## Worker stalls and busy loops Each worker runs its connections on a single thread. Code that blocks that thread, such as a synchronous call, a lock held for a long time or a long computation between `.await` points, delays every connection on the worker. The runtime detects blocked workers and tasks that never wait, logs them, and reports them in the worker metrics. ### Stall detection Every worker runs a task that sleeps for the latency sampling interval and records how late it wakes up. A watchdog thread checks the workers every interval. A worker that has recorded no sample for longer than the interval plus the stall threshold is stalled. With the defaults, 100 ms and 1 s, a worker is stalled after about 1.1 s without progress. When a stall starts, the watchdog logs an error: ```text Arbiter my-server:worker:0(Id(3)) is stalled, no progress for 1.1s, running task spawned at src/main.rs:42:9 ``` On neon the message names the spawn location of the task that is running. Tasks spawned by ntex itself, such as the connection dispatchers, report a location inside ntex. A stall without a running task happens in the I/O driver. When the worker makes progress again, a warning reports the stall duration: ```text Arbiter my-server:worker:0(Id(3)) recovered after 2.4s stall ``` The same information is in `WorkerMetrics::runtime`: * `latency.stalls` and `latency.stall_duration` count finished stalls. Alert on any increase. * `latency.stalled` is set while the worker is stalled. `current_turn` shows how long the worker has been blocked and `current_task` where the blocking task was spawned (neon). * Shorter blocking that stays under the threshold shows up as high values in the `latency.histogram` and `poll_histogram` differences. New connections skip a worker whose runtime turn has lasted 100 ms or more, that has recorded no latency sample for 100 ms past its interval, or that is stalled, as long as another worker is available. Connections queued at the blocked worker are moved to other workers, see [Rescheduling](#rescheduling). Connections the worker has already taken still wait for it. Change the interval and the threshold, both in milliseconds, on the system builder. `#[ntex::main]` has no options for them, build the system directly: ```rust,no_run use ntex::rt::{DefaultRuntime, System}; fn main() -> std::io::Result<()> { System::build() .name("my-server") .signals(true) // backtraces on Linux need signal handling .latency_interval(50) // sample every 50 ms .ping_threshold(500) // stalled after 550 ms without a sample .build(DefaultRuntime) .block_on(async { // build and run the server Ok(()) }) } ``` A shorter interval detects stalls sooner and gives a finer latency histogram, at the cost of more timer wakeups on idle workers. Keep the threshold above the longest blocking you accept, so it does not log expected work. `latency_interval(0)` disables sampling and stall detection, and `WorkerMetrics::runtime.latency` is then `None`. ### Busy loop detection A task that wakes itself on every poll, for example a loop around `yield_now()` or `wake_by_ref()`, never waits for an event. It keeps the worker busy without blocking it, so it is not a stall, but it wastes CPU and delays other tasks. The neon runtime counts these self-wakes. When a task wakes itself 10 000 times in a row over at least one second, the runtime logs a warning once for that streak: ```text Task spawned at src/main.rs:42:9 woke itself 10000 times in 1.2s without waiting, possible busy loop ``` `WorkerMetrics::runtime.self_wakes` is the total count. A steadily rising rate together with high utilization (`busy_duration`) and a low connection rate points to a busy loop; replace it with code that waits for an event or a timer. The tokio and compio runtimes do not count self-wakes. The limits can be changed with `RuntimeBuilder::self_wake_threshold()` only when the runtime is built directly, see the runtime chapter. ### Backtrace capture On Linux and Windows the watchdog captures a backtrace of the stalled worker thread once, when the stall starts: * On Linux it sends `SIGUSR2` to the worker thread, whose handler records the stack. This requires ntex signal handling on the system, `signals = true` in `#[ntex::main]` or `System::build().signals(true)`. The application must not use `SIGUSR2` for anything else. * On Windows the watchdog briefly suspends the worker thread and walks its stack, on x86_64 and aarch64. * Other platforms only log the stall. Without a callback the backtrace is resolved and logged as an error together with the task location. `System::set_latency_callback()` replaces the logging, for example to send the backtrace to an error tracker. The callback is process-wide and runs on the watchdog thread, so keep it short; it receives the unresolved backtrace and the spawn location of the blocking task. The location is `None` when the task changed during the capture, because the backtrace then may not belong to it. ```rust,no_run use ntex::rt::System; fn install() { System::set_latency_callback(|bt, task| { let bt = bt.resolve(); log::error!("worker stalled, task spawned at {task:?}\n{bt}"); }); } ``` Backtraces need symbols. Release builds without debug information may show wrong or missing function names; add `debug = "line-tables-only"` to the release profile for readable backtraces with file and line numbers. ## Signals and shutdown Signal handling is enabled by default: | Event | Default behavior | |---|---| | `SIGTERM` | graceful stop | | `SIGINT` / Ctrl-C | immediate stop | | `SIGQUIT` | immediate stop, or graceful with `graceful_shutdown()` | | `SIGHUP` | ignored by the server | | fatal signal or application panic | immediate stop, or graceful with `graceful_shutdown()` | Fatal-signal notifications are installed with Unix signal handling. Application-panic notifications additionally require runtime panic handling. A fatal-signal shutdown is best effort because the operating system may terminate the process before graceful cleanup finishes. On Windows, Ctrl-C is delivered as the interrupt event. Signal delivery belongs to the ntex `System`, not to one isolated server. If several servers share a system, or the application installs its own signal policy, call `disable_signals()` consistently and stop the appropriate server controllers yourself. The graceful server timeout and connection shutdown timeout solve different problems. The server timeout bounds a worker as a whole. Each connection uses its own `IoConfig::set_shutdown_timeout()` while flushing and closing. Leave enough server-level time for those connection shutdowns to complete. ## Connection and protocol configuration The `SharedCfg` passed to `bind()` or `listen()` is cloned onto each accepted `Io`. The I/O layer and protocol services retrieve the configuration types they understand from the same container: ```rust,no_run use ntex::http::{HttpService, HttpServiceConfig, Response}; use ntex::io::IoConfig; use ntex::time::Seconds; use ntex::SharedCfg; #[ntex::main] async fn main() -> std::io::Result<()> { let cfg = SharedCfg::new("HTTP") .add( IoConfig::new() .set_shutdown_timeout(Seconds(2)), ) .add( HttpServiceConfig::new() .set_keepalive_timeout(Seconds(30)), ); ntex::server::build() .bind("http", "127.0.0.1:8080", cfg, async |_| { HttpService::new(async |_| { Ok::<_, std::io::Error>( Response::Ok().body("Hello world!"), ) }) })? .run() .await } ``` `IoConfig` controls connection-level behavior such as shutdown, memory, and read-rate limits. `HttpServiceConfig` controls HTTP behavior such as keep-alive, request-head limits, and protocol timeouts. TLS and other protocols add their own configuration types to the same `SharedCfg`. The [I/O chapter](7-io.md) explains buffer limits, read rates, write backpressure, keep-alive, and shutdown timing in detail. ## Test servers `test_server()` starts one worker on a separate thread, binds an available local port, and disables signal handling. Its service factory receives `&()`, just like the factories passed to `http::test::server` and `web::test::server`: ```rust use ntex::http::{HttpService, Response}; use ntex::{client::Client, server}; /// STATEMENT: A test server serves a successful HTTP response. /// /// HOW: /// - Start an HTTP service with a factory accepting unit state. /// - Send a request to its assigned port and check the response status. #[ntex::test] async fn test_server_response() { let server = server::test_server(async |_| { HttpService::new(async |_| { Ok::<_, std::io::Error>( Response::Ok().body("Hello world!"), ) }) }); let url = format!("http://{}/", server.addr()); let response = Client::new() .get(url.as_str()) .send() .await .unwrap(); assert!(response.status().is_success()); } ``` Use `addr()` for HTTP clients or `connect()` to open a client `Io` with the test server's client configuration. `server()` returns the underlying server controller. `TestServerBuilder` can set separate server and client `SharedCfg` values. Use `TestServerBuilder::with(cfg, async |state| service)` when service construction needs application state. The service factory receives `&Cfg::State`; the state-initialization closure remains `async || Ok(state)`. `build_test_server()` exposes the full `ServerBuilder` for custom listeners; when using it, set the address on the returned `TestServer` before calling `connect()`. Dropping the last `TestServer` clone stops its server and runtime. That drop briefly blocks the current thread while the test thread shuts down, so drop it outside timing-sensitive assertions. # Worker and request state Each ntex worker runs its own single-threaded runtime. To make use of multiple CPU cores, the server starts several workers and distributes incoming connections among them. Each worker then creates and runs its own application instance. Starting an application usually happens in two stages. First, the server loads configuration that is shared across the entire process. This normally happens on the main thread and may include reading environment variables, parsing command-line arguments, loading configuration files, or fetching settings from an external service. Next, ntex uses that configuration to initialize each worker. The `ServerAppConfig` trait controls this part of the process. You provide the server with an object that implements `ServerAppConfig`, and the server calls its `create()` method inside each worker every time that worker starts. If a worker fails and is restarted, `create()` runs again and the restarted worker gets fresh state. The configuration object must implement `Send + Sync` because the server shares it across worker threads. The state returned by `create()` is different: it belongs to one worker and stays on that worker's thread. As a result, worker state can contain single-threaded types such as `Rc` and `RefCell`. Worker state must also implement `Clone`. The server clones it for every listener, and servers such as `web::server_with_config()` clone it again for every connection. Put data that should be shared by all connections of a worker behind a shared handle, such as `Rc>` or `Rc`, so that the clones refer to the same value. Plain fields are copied, and changes to them are not visible to other clones. The `create()` method is asynchronous, so it can do more than simply construct a struct. It can open database connections, create client instances, initialize caches, or prepare any other resources the worker needs. If something goes wrong, it can return an error rather than starting the worker with invalid state. ```rust struct AppBuilder { // Configuration shared across the process } #[derive(Clone)] struct AppState { // Resources owned by one worker, usually behind `Rc` } impl ServerAppConfig for AppBuilder { type State = AppState; // Called each time a worker starts. async fn create(&self) -> io::Result { Ok(AppState { // Initialize this worker's resources. }) } } ``` A separate type is not always necessary. `ServerAppConfig` is also implemented for async closures that return `io::Result`, and `NoConfig` provides unit state for servers that do not need any: ```rust let server = ntex::server::build_with_config(async || { Ok(AppState { // ... }) }); ``` ## Creating the Worker Application Once the worker state is ready, the server calls an application factory to build the worker's application instance. The factory runs once per worker and receives a reference to the state created by `ServerAppConfig::create()`. ```rust #[ntex::main] async fn main() -> std::io::Result<()> { // Load the process-wide configuration. let builder = AppBuilder { // ... }; web::server_with_config(builder, async |_state: &AppState| { web::App::new().service( web::resource("/").to(async || { web::HttpResponse::Ok() }), ) }) .bind("127.0.0.1:8080", SharedCfg::default())? .run() .await } ``` Here, `AppBuilder` holds the process-wide configuration and implements `ServerAppConfig`. For every worker, ntex calls `AppBuilder::create()` to produce a new `AppState`. It then passes that state to the application factory, which builds the worker's `web::App` factory. The `App` service itself is created for each connection, with its own clone of the worker state. The factory can use the state while setting up the application and its services. The same state is also available to the worker's connection-handler pipelines throughout their lifecycle. ## Worker-Local and Process-Wide State A worker creates its state once and reuses it for every connection it handles. It does not call `create()` for every connection. Connections receive clones of the worker state, so values behind `Rc` are shared by every connection of that worker. This means that connections handled by the same worker see the same shared data. Connections handled by different workers, however, use different state instances. Updating local state in one worker does not automatically update the state in another worker. This separation is often useful. Because worker-local state never crosses thread boundaries, it can use simple single-threaded data structures without paying the cost of synchronization. Sometimes state really does need to be shared across all workers. In that case, it must be shared explicitly. One common approach is to wrap the shared value in an `Arc` and clone it into each worker state. If the value is mutable, it will also need a suitable synchronization mechanism, such as an atomic type, `Mutex`, or `RwLock`. The best choice depends on how the data is used. Prefer worker-local state when workers do not need to coordinate. Use process-wide shared state only when changes must be visible across workers. [Complete process/worker configuration example](https://github.com/ntex-rs/examples/tree/main/state/process) ## Service and Pipeline State ntex-service 5 adds a state parameter to the `Service` trait. This gives every service access to state while it is checking readiness, handling requests, or shutting down. A simplified version of the trait looks like this: ```rust trait Service { type Res; type Error; async fn call( &self, req: Req, ctx: Ctx<'_, Self, St>, ) -> Result; async fn ready( &self, ctx: Ctx<'_, Self, St>, ) -> Result<(), Self::Error>; async fn shutdown(&self, ctx: Ctx<'_, Self, St>); } ``` Here, `St` is the type of state the service expects. Since it is part of the `Service` definition, Rust can check that a service is always used with the right kind of state. The same state is available in all three lifecycle methods: - `ready()` can use it while checking whether the service is ready for more work. - `call()` can use it while handling a request. - `shutdown()` can use it while the service is stopping. You can think of a stateful service as an asynchronous function that receives both the state and the request: ```rust async fn my_service( state: &St, req: Req, ) -> Result { // ... } ``` The `Service` trait does not pass state as a separate argument. Instead, it provides state through `Ctx`. The context also connects the service to its pipeline, which manages readiness, request processing, and shutdown. ### Constructing a Pipeline To call a service, you first place it in a `Pipeline`. A pipeline keeps the service and its state together. It can contain a single service or a chain of services, all of which can use the same state. Here is a small example: ```rust async fn my_service( state: &AppState, req: usize, ) -> Result { // Use the pipeline state to handle the request. todo!() } #[ntex::main] async fn main() -> io::Result<()> { // Create a pipeline containing one state instance and one service. let svc = ntex::Pipeline::new(AppState::new(), my_service); // Both calls use the same AppState instance. let first = svc.call(1).await?; let second = svc.call(2).await?; println!("{first}:{second}"); Ok(()) } ``` The `AppState` is created when the pipeline is built. It stays with the pipeline and is reused for every call. Calling `svc.call()` does not create a new state instance. If the pipeline contains a chain of services, each compatible service can access the same state. This makes it easy to share resources across a service chain without adding a copy of those resources to every service. ### Worker state Earlier, we saw that ntex creates a separate state instance for each server worker. That state also becomes the state of the worker's connection-handler pipeline. When a worker starts, the server calls the application factory and passes it a reference to the worker state. The factory builds a connection-handler service, and the server places that service in a pipeline together with the worker state. ```rust /// Handles an I/O connection using worker-local state. async fn handle_io( state: &AppState, io: ntex::io::Io, ) -> Result<(), io::Error> { // Process the connection using the worker state. Ok(()) } #[ntex::main] async fn main() -> std::io::Result<()> { // Load the process-wide configuration. let builder = AppBuilder { // ... }; ntex::server::build_with_config(builder) .bind( "handler", "127.0.0.1:8080", SharedCfg::new("S"), async |_state: &AppState| { // Build the connection-handler service. ntex::service(handle_io) }, )? .run() .await } ``` The factory is called once for each worker and receives that worker's `AppState`. It can use the state to configure the service during construction. The server places the returned service and a clone of the worker state in a pipeline. When a connection is dispatched to `handle_io` service, the pipeline passes access to the same `AppState` instance alongside the connection. Each worker has its own state. Connections handled by the same worker share the data behind the state's shared handles, but connections handled by different workers do not automatically share state. ## State Accumulation The state we have discussed so far is long-lived. Worker state lives for as long as the worker, and pipeline state lives for as long as the pipeline. In both cases, the state is created once and reused across many calls. Sometimes we need state with a shorter lifetime. In particular, we may want to collect information while a request or connection moves through a service chain. Each service can inspect what has already been collected, add more information, and pass the updated state to the next service. Imagine a pipeline that accepts an incoming connection. The first service might record basic details such as the connection ID and peer address. The next service could read the TLS `ClientHello` and extract the Server Name Indication (SNI). Other services might load configuration for that server name, validate the client, calculate resource usage, apply throttling, and finally perform the TLS handshake before handing the connection to an HTTP server. The pipeline might look like this: ```text accept connection → collect connection information → read SNI → load client information → validate and throttle → negotiate TLS → handle HTTP ``` ntex-service provides the `RequestState` trait and the `State` type for this kind of state. Together, they let us pass a message and its accumulated state through a service chain. `State` and types that implement `RequestState` are different from pipeline state. Pipeline state is shared across calls, while accumulated state belongs to a single request. Every connection moving through the pipeline carries its own state. `State` is the most common implementation, but it is not the only one. A `(St, Req)` tuple also implements `RequestState`, which is convenient for small services. `Io` and `IoBoxed` implement it with `()` state, so a plain connection can be passed to any service that expects a `RequestState`. The ntex protocol servers support `RequestState`, including `ntex::http`, ntex-h2, ntex-mqtt, and ntex-amqp. ### Collecting Connection Information Let's start with a service that receives a raw I/O connection and collects some basic information about it: ```rust use std::{io, net::SocketAddr, time::Instant}; use ntex::{io::Io, service::State}; use uuid::Uuid; struct Connection { id: Uuid, created: Instant, peer_addr: SocketAddr, } async fn connect(_: &AppState, io: Io) -> io::Result> { let peer_addr = load_peer_addr(&io)?; Ok(State { req: io, state: Connection { id: Uuid::new_v4(), created: Instant::now(), peer_addr, }, }) } ``` The next service can unpack these values, perform the TLS handshake, and add more information: ```rust use ntex::service::{RequestState, State}; struct ConnectionWithTls { id: Uuid, created: Instant, peer_addr: SocketAddr, server_name: String, } async fn tls(_: &AppState, msg: State) -> io::Result> { let (connection, io) = msg.unpack(); let server_name = load_server_name(&io).await?; let io = accept_tls(io, &server_name).await?; Ok(State { req: io, state: ConnectionWithTls { id: connection.id, created: connection.created, peer_addr: connection.peer_addr, server_name, }, }) } ``` This service changes both parts of the input. It turns the raw `Io` connection into a TLS-enabled `TlsIo`, and it extends the connection state with the server name extracted during TLS negotiation. The connection ID, creation time, and peer address are carried forward. By the time the connection reaches the HTTP server, its state contains everything collected by the earlier services. ### Passing Connection State to the HTTP Service We can now connect these services into a single chain: ```rust use ntex::{server, service, SharedCfg}; use ntex::http::{self, Request, Response}; /// Handles an HTTP request using information about its connection. async fn handle_request(st: &ConnectionWithTls, _req: Request) -> io::Result { Ok(Response::Ok() .body(format!("server name: {}", st.server_name)), ) } #[ntex::main] async fn main() -> io::Result<()> { let builder = AppBuilder { // ... }; server::build_with_config(builder) .bind( "HTTP", "127.0.0.1:8080", SharedCfg::new("S"), async |_: &AppState| { // Collect the initial connection information. service(connect) // Perform the TLS handshake. .and_then(tls) // Pass the established connection to the HTTP service. .and_then(http::HttpService::new(handle_request)) }, )? .run() .await } ``` The worker's `AppState` is still available to `connect` and `tls`. It contains worker-level resources that can be reused across many connections. At the same time, each connection carries its own state. [Complete state accumulation example](https://github.com/ntex-rs/examples/tree/main/state/state-accumulation) ## Implementation Details ntex supports several network protocols, and each one handles state a little differently. The important distinction is between state that belongs to a connection and state that belongs to a single request. ### `ntex::http` `HttpService` expects a handler with roughly the following shape: ```rust async fn handler(state: &St, req: http::Request) -> Result { // ... } ``` Here, `state` is the state of the HTTP connection. It is extracted from the value passed to `HttpService` through the `RequestState` trait. For example, an earlier service can return a `State`. When `HttpService` receives that value, it separates the connection state from the I/O stream. The I/O stream is used to run the HTTP protocol, while `ConnectionWithTls` becomes the state provided to the HTTP request handler. This is the mechanism used in the previous section to make connection-specific information, such as the peer address and TLS server name, available while handling HTTP requests. ### `ntex::web` A `web::App` can act as the handler for `HttpService`. When it does, the state extracted by `HttpService` becomes the application's state. A web service looks roughly like this: ```rust async fn handler(state: &St, req: web::WebRequest) -> Result { // ... } ``` The `state` argument contains the long-lived application or connection state. The request can also carry its own state through the generic parameter of `WebRequest`: ```rust web::WebRequest ``` This gives a web application two separate kinds of state: - `St` is provided by the service pipeline and remains available across requests. - `RequestState` belongs to one request and moves through the request-processing chain. A filter or middleware can inspect a `WebRequest`, perform some work, and return a new `WebRequest` with a different state type. This makes it possible to build up request-specific information as the request moves through the application. For example, a request-processing chain might look like this: ```text validate request → load and authenticate the user → apply throttling → call the handler ``` The request starts without any additional state: ```rust web::WebRequest<()> ``` After the authentication service loads and verifies the user, it can return: ```rust web::WebRequest ``` The next service can then require `WebRequest`. This means it cannot be called until authentication has completed and the request contains an authenticated user. Another service could validate the request and return a different state type: ```rust web::WebRequest ``` In this way, the request type shows how far the request has moved through the processing chain. Each service declares the state it expects and the state it produces, and Rust checks that the services are connected in the right order. Keeping pipeline state and request state separate lets the application reuse connection-level resources while building up request-specific information as each request moves through the service chain. # I/O Abstraction Layer ntex provides an I/O abstraction layer that keeps protocol and service code independent of a specific runtime or reactor implementation, such as Tokio, Compio, or Neon. In addition to presenting a consistent interface across these backends, the I/O layer provides the building blocks needed to manage buffered reads and writes, backpressure, timeouts, and graceful shutdown. ## Socket abstractions ntex achieves independence from the underlying socket implementation through dependency inversion. The I/O subsystem does not call Tokio, Compio, or Neon socket APIs directly. Instead, a runtime adapter moves bytes between its socket and the buffers owned by [`Io`]. One way to picture it: the read half carries bytes from the transport to the application and the write half carries them back, with `Io` sitting between the two. Each half is an in-memory buffer that `Io` owns, which is what lets it apply backpressure and run filter transformations on bytes as they pass through. An active connection has three cooperating execution responsibilities: 1. A protocol or dispatcher task consumes decoded input and queues encoded output through `Io` or [`IoRef`]. This task typically runs the protocol service and, indirectly, application code. 2. A transport read task waits for read readiness, reads bytes from the socket, and submits them to the I/O subsystem. 3. A transport write task takes queued bytes from the I/O subsystem and writes them to the socket. The runtime adapter decides how to schedule these responsibilities. The built-in adapters typically use cooperating read and write tasks, but this is an implementation detail rather than part of the `IoStream` contract. The read task cooperates with ntex backpressure. It reads only while [`IoContext::poll_read_ready`] permits more input. It takes a buffer through [`IoContext::take_read_buf`] and releases it with the result through [`IoContext::release_read_buf`]. This allows the I/O subsystem to process filters, wake the dispatcher, and pause further reads when the configured high-water mark is reached. A backend that reads synchronously, without holding the buffer across an await, should use [`IoContext::with_read_buf`] instead. It combines the two steps and borrows the read buffer in place, so no buffer is detached and appended back. The write task follows the same pattern. It waits for [`IoContext::poll_write_ready`], obtains queued data with [`IoContext::with_write_dst`], and reports through [`IoContext::update_write_status`] how many bytes reached the peer. ntex can then apply write backpressure, resume waiting services, and coordinate graceful shutdown. A transport does not have to write the bytes before it reports. Completion based backends take ownership of whole pages and keep them until the operation completes. Those bytes leave the write buffer but have not reached the peer, so ntex counts them as outstanding output until the transport either reports them as written or returns them to the buffer. Flush completion, write backpressure, and the shutdown drain all account for them. Both methods return an [`IoTaskStatus`]. `Io` means work remains, not that the transport is ready, so a task re-arms transport readiness before its next operation. On the write side it is returned whenever output is still buffered, including after an attempt that made no progress. Output the transport already owns does not keep the task running, because its completion wakes the task again. Both readiness methods resolve to a [`Readiness`] value rather than a plain ready signal. `Readiness::Ready` allows the task to perform its next operation. `Readiness::Close` means the connection must be taken down: the task closes both directions and releases the socket, cancelling anything still in flight. For a socket this is `shutdown(SHUT_RDWR)` followed by `close()`. Whatever the peer still has in the receive queue should be discarded first, because closing a socket with unread input aborts the connection with an RST and loses the output that was just drained. `Readiness::Close` covers a graceful shutdown as well as a connection that ends because of an I/O failure, a filter failure, or an expired shutdown deadline. On the graceful path the I/O subsystem reports it only once buffered output has been drained, so nothing is left to flush. `Readiness::Terminate` is reported only for an explicit force close through [`IoRef::terminate`]. Whatever is still buffered is discarded on purpose, and the task must not close gracefully: no receive queue drain and no `shutdown(SHUT_RDWR)`. It aborts the connection instead, for a socket by setting `SO_LINGER` to zero before `close()`, so that the peer sees an RST and cannot mistake a truncated stream for a complete one. Teardown is reported back through two methods. A transport that fails or decides to abort calls [`IoContext::stop`] with the error, which terminates the connection without draining pending work. Once transport teardown has actually finished, and regardless of which side initiated it, the task calls [`IoContext::stopped`]. That is what completes [`Io::shutdown`], which resolves only after the backend reports the connection stopped, not merely when filter shutdown and flushing are done. Socket types integrate with the I/O subsystem by implementing [`IoStream`]. Its `start()` method receives an [`IoContext`], starts the transport-specific tasks, and returns a [`Handle`] used to control or query the transport. A simplified Tokio-style adapter looks like this: ```rust,ignore impl IoStream for TcpStream { fn start(self, ctx: IoContext) -> Box { let socket = Rc::new(self); tokio::task::spawn_local(read_task(socket.clone(), ctx.clone())); tokio::task::spawn_local(write_task(socket.clone(), ctx)); Box::new(SocketHandle(socket)) } } async fn read_task(socket: Rc, ctx: IoContext) { loop { // resolves `IoContext::poll_read_ready` match wait_for_read_readiness(&ctx).await { Readiness::Ready => {} Readiness::Close | Readiness::Terminate => break, } let mut buf = ctx.take_read_buf(); let result = read_from_socket(&socket, &mut buf); match ctx.release_read_buf(buf, result) { IoTaskStatus::Io => {} IoTaskStatus::Pause => wait_for_read_resume(&ctx).await, IoTaskStatus::Stop => break, } } } async fn write_task(socket: Rc, ctx: IoContext) { let terminate = loop { // resolves `IoContext::poll_write_ready` match wait_for_write_readiness(&ctx).await { Readiness::Ready => {} Readiness::Close => break false, Readiness::Terminate => break true, } let result = ctx.with_write_dst(|buf| { write_to_socket(&socket, buf) }); // `result` carries the number of bytes that reached the peer match ctx.update_write_status(result) { IoTaskStatus::Io => {} IoTaskStatus::Pause => wait_for_queued_output(&ctx).await, IoTaskStatus::Stop => break false, } }; if terminate { // force close, abort so that the peer sees an RST abort_socket(&socket); } else { // close both directions shutdown_socket(&socket).await; } // report teardown as complete ctx.stopped(None); } ``` The example is illustrative; runtime adapters use their native readiness and buffer APIs. The important boundary is that only the adapter accesses the socket, while the application and protocol layers interact with `Io` and `IoRef`. Buffer limits and read/write backpressure are therefore applied consistently across all supported runtimes. An `Io` object is normally passed to a protocol service such as [`ntex::http::HttpService`] or `ntex_mqtt::Server`. The protocol service decodes incoming bytes into protocol messages and encodes its responses back into the I/O write buffer without depending on the concrete socket type. [`Handle`]: https://docs.ntex.rs/api/ntex/io/trait.Handle.html [`Io`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html [`IoContext`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html [`IoContext::poll_read_ready`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.poll_read_ready [`IoContext::poll_write_ready`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.poll_write_ready [`IoContext::stop`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.stop [`IoContext::stopped`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.stopped [`IoContext::with_read_buf`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.with_read_buf [`IoContext::take_read_buf`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.take_read_buf [`IoContext::release_read_buf`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.release_read_buf [`IoContext::update_write_status`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.update_write_status [`IoContext::with_write_dst`]: https://docs.ntex.rs/api/ntex/io/struct.IoContext.html#method.with_write_dst [`IoRef`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html [`IoStream`]: https://docs.ntex.rs/api/ntex/io/trait.IoStream.html [`IoTaskStatus`]: https://docs.ntex.rs/api/ntex/io/enum.IoTaskStatus.html [`Readiness`]: https://docs.ntex.rs/api/ntex/io/enum.Readiness.html [`ntex::http::HttpService`]: https://docs.ntex.rs/api/ntex/http/struct.HttpService.html The runtime-specific implementations are provided by the [`ntex-net`] crate, which supports Tokio, Compio, and Neon backends. [`ntex-net`]: https://docs.ntex.rs/api/ntex_net/index.html ### Transport handles and metadata Because the underlying socket is hidden behind the I/O abstraction, code using `Io` cannot access transport-specific methods directly. The [`Handle`] returned by `IoStream::start()` provides the bridge to the underlying transport. It can resume the transport write operation on request and expose transport-specific metadata through typed queries. Each backend decides which query types it supports. For example, the built-in network backends expose the remote socket address as [`PeerAddr`]. [`IoRef::query`] is also available on `Io` through dereferencing and returns a [`QueryItem`], from which the value can be retrieved with `get()`: ```rust use ntex::io::{Io, types::PeerAddr}; fn log_peer_addr(io: &Io) { if let Some(addr) = io.query::().get() { println!("Peer address {:?}", addr.into_inner()); } } ``` Queries travel through the filter stack before reaching the transport handle, so filters may also expose their own typed metadata. [`IoRef::query`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.query [`PeerAddr`]: https://docs.ntex.rs/api/ntex/io/types/struct.PeerAddr.html [`QueryItem`]: https://docs.ntex.rs/api/ntex/io/types/struct.QueryItem.html ## Configuration and timeouts I/O settings are stored in [`IoConfig`] and are normally added to a [`SharedCfg`]. `Io::new()` retrieves the `IoConfig` from that shared configuration, using the default settings when it is not present. ```rust use ntex::{ SharedCfg, io::IoConfig, time::{Millis, Seconds}, util::BytePageSize, }; let cfg = SharedCfg::new("my-protocol") .add( IoConfig::new() .set_connect_timeout(Millis(5_000)) .set_keepalive_timeout(Seconds(30)) .set_shutdown_timeout(Seconds(2)) .set_frame_read_rate(Seconds(2), Seconds(10), 1_024) .set_write_timeout(Seconds(10)) .set_read_size(BytePageSize::Size4, BytePageSize::Size32) .set_read_backpressure(32 * 1024) .set_write_backpressure(32 * 1024) .set_write_buf_threshold(8 * 1024), ) .build(); ``` These settings are used by different parts of the stack: - The connection timeout is applied by `ntex-net` while resolving and opening an outgoing connection. - The keep-alive timeout and frame read-rate limits are interpreted by protocol dispatchers. A frame read-rate limit protects a decoder from peers that send one incomplete frame too slowly. It also applies to the first frame of a new connection, so a peer that connects and stays silent is closed once the limit expires. A write timeout protects against peers that stop reading: it bounds how long write backpressure may stay enabled, from the moment it is enabled until it is disabled, before the dispatcher stops with a write timeout. Keep-alive and read-rate timers do not run during that time. Output left after backpressure is disabled is bounded only by keep-alive. The HTTP/1 dispatcher does not use these settings; it is configured by [`HttpServiceConfig`], including its own `set_write_timeout()`. - The graceful-shutdown timeout bounds both phases of shutdown together: the filter shutdown and the transport drain of pending output. It cannot be disabled; a zero timeout is rejected. - The read and write high-water marks enable backpressure. Write backpressure is released after outstanding output falls to half its high-water mark, counting both buffered output and output a transport has taken ownership of but not yet written to the peer. - The min and max read page sizes bound the adaptive page size of new read buffers. Before another socket read, a buffer grows once less than [`BytePageSize::low`] of its page remains free. Output is held in [`BytePages`] of the write page size, so the write backpressure setting takes only a high-water mark. - The write buffer page size controls newly allocated [`BytePages`], while the write threshold controls when supported transports attempt an early direct write. Connection and keep-alive timeouts are disabled by default. Frame read-rate limits and the write timeout are also disabled. The default graceful-shutdown timeout is one second, and the default read and write high-water marks are approximately 32 KiB and 16 KiB. Start with those defaults. Change one setting because of an observed workload, not because larger values sound faster: - Read page sizes control allocation and read-call frequency, not the amount of unread data a connection may queue. Increase the maximum for sustained large frames or bodies; keep the minimum small when many mostly idle connections are expected. - Backpressure watermarks control when producers or transports should pause. They are not hard memory limits: one socket read or one encoded item can cross a watermark. - The write threshold is a latency hint for starting transport work earlier. It does not flush data and does not replace write backpressure. - A write timeout is especially important for untrusted peers. Without one, a peer that stops reading can keep a backpressured connection and its output alive indefinitely. An established connection can switch to another shared configuration with [`Io::set_config`]. This is useful when a protocol upgrade changes timeout or buffer requirements. The method is `unsafe`: replacing the configuration may release the allocation that [`IoRef::cfg`] hands out, so no reference obtained from it may be live across the call or used afterwards. [`Io::set_config`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.set_config [`IoRef::cfg`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.cfg [`SharedCfg`]: https://docs.ntex.rs/api/ntex/struct.SharedCfg.html ## Filter subsystem Applications often need to transform a byte stream before a protocol service processes it. TLS must decrypt incoming records and encrypt outgoing data. A protocol may also be tunneled through another framing layer, such as MQTT over WebSocket. ntex implements these transformations with [`FilterLayer`]. A filter operates on in-memory byte buffers: [`FilterLayer::process_read_buf`] transforms data received from the next inner layer, while [`FilterLayer::process_write_buf`] transforms data queued by the application before passing it toward the transport. Filters do not perform socket I/O themselves, so the same filter can be used with any supported runtime backend. Filters are composable. [`Io::add_filter`] adds a layer and allocates the intermediate read and write buffers that separate it from adjacent layers. For example, an MQTT service can receive its byte stream through either of these stacks: ```text socket <-> TLS <-> MQTT socket <-> TLS <-> WebSocket <-> MQTT ``` On reads, bytes move from the socket through the inner filters toward the application. On writes, they move in the opposite direction. In the second stack, the WebSocket filter removes and creates WebSocket framing, while the TLS filter decrypts and encrypts the resulting byte stream. The MQTT service still reads and writes MQTT bytes and does not need to know which transport filters are installed below it. The [TLS chapter](10-tls.md) applies this model to HTTP clients, WebSocket clients, and servers, including backend configuration and peer certificate authentication. Filters may maintain protocol state, expose typed metadata through `query()`, emit output while processing input, and participate in graceful shutdown. For example, a TLS filter can expose the negotiated protocol or peer certificate, and a WebSocket filter can generate a close frame during shutdown. Output a filter writes while processing a read is pushed toward the transport as soon as that processing finishes, without waiting for the application to write. [`FilterLayer`]: https://docs.ntex.rs/api/ntex/io/trait.FilterLayer.html [`FilterLayer::process_read_buf`]: https://docs.ntex.rs/api/ntex/io/trait.FilterLayer.html#tymethod.process_read_buf [`FilterLayer::process_write_buf`]: https://docs.ntex.rs/api/ntex/io/trait.FilterLayer.html#tymethod.process_write_buf [`Io::add_filter`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.add_filter Most byte transformations only need [`FilterLayer`]. Lower-level concerns that must observe or control the entire filter chain can instead wrap the current chain with [`Filter`] by using [`Io::map_filter`]. In addition to processing buffers, queries, and shutdown, `Filter` participates in read and write readiness decisions. A wrapper can therefore delay readiness to implement custom throttling and wake the I/O tasks when work may resume. It can also observe buffer processing for metrics or accounting without changing the byte stream. A custom `Filter` normally stores the filter it wraps and delegates every operation it does not intentionally override. ntex provides forwarding macros for readiness, queries, and shutdown to make this pattern less error-prone. [`Filter`]: https://docs.ntex.rs/api/ntex/io/trait.Filter.html [`Io::map_filter`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.map_filter ### Typed versus erased filter stacks The filter stack is encoded in the type parameter of [`Io`]. A new connection starts as `Io`, using the [`Base`] filter. Calling `add_filter(layer)` consumes the current value and returns `Io>`, where the [`Layer`] marker pairs the new outer layer `U` with the previous stack `F`. Keeping this concrete type provides static dispatch and allows [`Io::filter`] to return the concrete outer filter. ```rust,ignore let io: Io = create_io(); let io: Io> = io.add_filter(MyFilter::new()); ``` At service boundaries, different connections may have different concrete filter stacks. [`Io::seal`] erases the stack type and returns `Io`, using the [`Sealed`] marker, while [`Io::boxed`] returns the [`IoBoxed`] convenience wrapper. Both operations consume the original `Io` value and retain the same connection state and filter behavior behind a dynamically dispatched `Filter`. ```rust,ignore let io: IoBoxed = io.boxed(); start_protocol(io); ``` Additional typed layers can still be added to a sealed stream and the result can be erased again when necessary. Type erasure is therefore normally performed at the boundary where a protocol or service needs one uniform I/O type, rather than while constructing the filter stack. [`Base`]: https://docs.ntex.rs/api/ntex/io/struct.Base.html [`Io::boxed`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.boxed [`Io::filter`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.filter [`Io::seal`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.seal [`IoBoxed`]: https://docs.ntex.rs/api/ntex/io/struct.IoBoxed.html [`Layer`]: https://docs.ntex.rs/api/ntex/io/struct.Layer.html [`Sealed`]: https://docs.ntex.rs/api/ntex/io/struct.Sealed.html ## Read/write streams Incoming and outgoing bytes use separate buffer paths. ### Reading The transport adapter places bytes read from the socket into a [`BytesMut`]. The bytes pass through the filter chain and arrive in the application-facing read buffer, where a codec or protocol service can inspect and consume them. `BytesMut` is a contiguous, growable buffer. A decoder can split immutable [`Bytes`] values from it without copying larger payloads; short values are copied into the `Bytes` handle itself. This is useful when a decoded message must retain part of the input after the decoder continues processing later data. Read buffers are `ntex-bytes` pages. Each connection picks the page size of new read buffers between the min and max set with [`IoConfig::set_read_size`], `Size4` and `Size64` by default. It starts at the min, so idle and light connections pin small pages. Consecutive reads that fill their buffer form one batch with the read that ends it; a batch larger than the page grows the page size to fit it, up to the max. After four batches in a row that fit in half of the next smaller page, the page size shrinks by one step, down to the min. [`Io::set_config`] restarts it at the min. Equal min and max sizes fix the page size. The page size does not affect read backpressure: the read high-water mark is the `Size32` capacity, 32,752 bytes, by default; [`IoConfig::set_read_backpressure`] sets another one. An empty buffer goes back to the per-thread page cache of its size, shared with write pages and any other pooled `BytesMut`; [`set_page_cache_size`] tunes it. A frame split from the read buffer keeps the page alive, the page returns to the cache once the last frame is dropped. Before another socket read, the adapter obtains a buffer from `IoContext`. ntex calls [`BytesMut::reserve_more`] once less than [`BytePageSize::low`] of the page remains free, compacting the data within its page when it still fits there. If larger input must be buffered, the buffer moves to bigger page sizes and, past the largest one, to a plain allocation that is freed instead of cached. Read backpressure is based on the size of the application-facing read buffer. When it reaches the configured high-water mark, ntex pauses the transport read task. Consuming input through [`IoRef::decode`], [`IoRef::with_buf`], [`IoRef::with_read_src`] or [`IoRef::with_read_dst`] wakes the read task once the buffered input falls to half that mark. [`Io::recv`] and [`Io::read_exact`] wait for more input, so they release backpressure regardless of how much is still buffered. Choose the highest-level read API that matches the protocol: - Use `recv(codec)` for framed messages. It decodes buffered input and waits when the codec says the frame is incomplete. - Use `read_exact()` for a fixed-size header or field. - For a custom parser, inspect or consume the read buffer through `IoRef`, and call [`Io::read_more`] only after deciding that more bytes are required. `read_more()` is an active request: it resumes transport reads and releases read backpressure even if the buffer is still large. [`IoRef::read_dst_size`] is the passive size check. ### Writing Application output is queued in [`BytePages`], a growable collection of byte pages. Internally allocated pages use the size configured by [`IoConfig`]. Owned buffers passed to [`IoRef::encode_bytes`] can also become pages directly, avoiding a copy when their storage can be retained. Write filters consume the pages in order, transform their contents, and place the result into the next buffer toward the transport. [`IoRef::encode`], [`IoRef::encode_slice`], and [`IoRef::encode_bytes`] queue output but do not wait for every byte to reach the socket. Queueing makes the data available to the transport write task and schedules that task when it needs to be resumed. On transports that support direct writes, the configured write threshold can trigger an earlier write while the application is still producing output, reducing latency for large responses. Use [`Io::flush`] to apply the configured write wait policy. `flush(false)` is a backpressure wait, not a request to drain everything: it returns immediately while the outstanding output is below the high-water mark. If the high-water mark has been reached, it waits until the outstanding output falls to half that mark. `flush(true)` waits until all queued data has reached the peer, including data a transport has taken ownership of but not yet written. [`Io::send`] combines codec encoding with a full flush. Write backpressure is advisory. The `encode` methods still accept more output after the high-water mark is reached, because refusing half of a protocol message would be difficult to recover from. A producer that is not managed by a dispatcher should await [`IoRef::write_ready`] before queueing each next chunk, or use `flush(false)` between batches: ```rust use ntex::{ io::Io, util::Bytes, }; async fn queue_chunk(io: &Io, chunk: Bytes) -> std::io::Result<()> { io.write_ready().await?; io.encode_bytes(chunk) } ``` This bounds normal streaming output around the configured watermark. It does not split a single large chunk or turn the watermark into a strict per-connection memory cap. This separation allows codecs and application services to work with bytes without depending on socket readiness, while the I/O subsystem coordinates buffering and backpressure. [`BytePages`]: https://docs.ntex.rs/api/ntex/util/struct.BytePages.html [`BytePageSize::low`]: https://docs.ntex.rs/api/ntex/util/enum.BytePageSize.html#method.low [`Bytes`]: https://docs.ntex.rs/api/ntex/util/struct.Bytes.html [`BytesMut`]: https://docs.ntex.rs/api/ntex/util/struct.BytesMut.html [`BytesMut::reserve_more`]: https://docs.ntex.rs/api/ntex/util/struct.BytesMut.html#method.reserve_more [`Io::flush`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.flush [`Io::read_exact`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.read_exact [`Io::read_more`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.read_more [`Io::recv`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.recv [`Io::send`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.send [`IoConfig`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html [`IoConfig::set_read_backpressure`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html#method.set_read_backpressure [`IoConfig::set_read_size`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html#method.set_read_size [`HttpServiceConfig`]: https://docs.ntex.rs/api/ntex/http/struct.HttpServiceConfig.html [`IoRef::decode`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.decode [`IoRef::encode`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.encode [`IoRef::encode_bytes`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.encode_bytes [`IoRef::encode_slice`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.encode_slice [`IoRef::read_dst_size`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.read_dst_size [`IoRef::write_ready`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.write_ready [`IoRef::with_read_dst`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.with_read_dst [`IoRef::with_read_src`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.with_read_src [`IoRef::with_buf`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.with_buf [`set_page_cache_size`]: https://docs.ntex.rs/api/ntex_bytes/fn.set_page_cache_size.html ## Connection lifecycle and shutdown A connection stays usable until the application closes it, the peer disconnects, or the transport reports an error. A service that is waiting on something other than input, such as an unready dependency or a slow response, still needs to notice that the connection requires attention. [`Io::poll_status_update`] reports the next status as an [`IoStatusUpdate`] value: - `Timeout` when the dispatcher timer has expired, for example the configured keep-alive timeout. - `WriteBackpressure` when outstanding output has reached the write high-water mark, so the producer should stop and flush. - `PeerGone` once the connection has closed, whether the peer disconnected, the transport failed, or the shutdown was started locally. It carries the transport error if one occurred, and `None` after a clean close. Use [`Io::poll_read_pause`] instead when the service should also stop accepting more transport input while it waits. The pause is cooperative: decoding, touching the application read buffer, or explicitly asking for more input resumes reads. This makes it suitable for a dispatcher waiting on readiness, not for permanently disabling the read half. The same conditions reach a codec-driven service as [`RecvError`] from [`Io::poll_recv`], which additionally reports decoder failures. Code that only needs to be woken when the connection goes away can await the [`Waiter`] future returned by [`IoRef::on_disconnect`]. [`IoRef::is_active`] becomes false as soon as closing starts; [`IoRef::is_closed`] becomes true only after the backend has released the transport and teardown has finished. A clean EOF from the peer ends the read direction but leaves the write half open, so a service can still finish encoding and flushing its response before closing. [`IoRef::close`] requests a graceful shutdown and returns immediately. [`Io::shutdown`] drives that shutdown to completion, in two phases. In the first phase both directions stay open and every filter gets a chance to emit its own closing data through [`FilterLayer::shutdown`], such as a TLS `close_notify` or a WebSocket close frame. Reads keep running and are processed normally, so closing data sent by the peer still reaches the filters. A read failure does not abort the shutdown; it only means the filter handshake cannot finish. The second phase belongs to the transport. It drains the remaining output into the connection and then closes both directions. The read side is paused: the filters are done, so no further input can be used, and the transport discards whatever is left in the receive queue just before it closes. Input is no longer delivered to the application in this phase. A single deadline bounds both phases. If the filters finish early, the transport drain gets the remaining time. If the deadline expires while filters are still pending, ntex ends that phase and still runs transport shutdown with the already-expired deadline; when the transport phase observes it, undrained output is discarded. [`Io::shutdown`] reports the timed-out error after transport teardown. [`IoRef::terminate`] skips the process entirely and drops the connection without flushing pending output. A graceful close, including one forced forward by a timeout or failure, reaches the transport as [`Readiness::Close`]. An explicit [`IoRef::terminate`] reaches it as `Readiness::Terminate`, so a socket backend can abort rather than send a clean end-of-stream. The transport reports either teardown through [`IoContext::stopped`], which is what allows `Io::shutdown` to resolve. Dropping the owning `Io` value is not a substitute for awaiting shutdown. If no output would be lost, dropping it starts a normal close. If buffered output can no longer be delivered because the filter stack is being dropped, ntex aborts the transport so the peer does not mistake a truncated stream for a complete one. Call `shutdown().await` when delivery and a clean close matter. [`FilterLayer::shutdown`]: https://docs.ntex.rs/api/ntex/io/trait.FilterLayer.html#method.shutdown [`Io::poll_recv`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.poll_recv [`Io::poll_read_pause`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.poll_read_pause [`Io::poll_status_update`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.poll_status_update [`Io::shutdown`]: https://docs.ntex.rs/api/ntex/io/struct.Io.html#method.shutdown [`IoRef::close`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.close [`IoRef::is_active`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.is_active [`IoRef::is_closed`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.is_closed [`IoRef::on_disconnect`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.on_disconnect [`IoRef::terminate`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.terminate [`IoStatusUpdate`]: https://docs.ntex.rs/api/ntex/io/enum.IoStatusUpdate.html [`Readiness::Close`]: https://docs.ntex.rs/api/ntex/io/enum.Readiness.html#variant.Close [`RecvError`]: https://docs.ntex.rs/api/ntex/io/enum.RecvError.html [`Waiter`]: https://docs.ntex.rs/api/ntex/io/struct.Waiter.html ## Testing [`IoTest`] provides a pair of interconnected in-memory transports for testing codecs, filters, and protocol services without opening sockets. Each endpoint implements `IoStream` and can be wrapped in `Io`. Writing to one `IoTest` endpoint supplies input to the other endpoint, while `read()` collects bytes written back by the peer. ```rust use ntex::codec::BytesCodec; use ntex::io::{Io, testing::IoTest}; use ntex::util::Bytes; #[ntex::test] async fn protocol_io() { let (client, server) = IoTest::create(); // Allow the server transport to write to the client. client.remote_buffer_cap(1024); let io = Io::from(server); client.write(b"request"); let request = io.recv(&BytesCodec).await.unwrap().unwrap(); assert_eq!(request, Bytes::from_static(b"request")); io.send(Bytes::from_static(b"response"), &BytesCodec) .await .unwrap(); assert_eq!(client.read().await.unwrap(), b"response"[..]); } ``` Tests can also force pending reads, inject read or write errors, close either side, constrain write capacity to exercise backpressure, and attach a `PeerAddr` value for transport-query tests. [`IoTest`]: https://docs.ntex.rs/api/ntex/io/testing/struct.IoTest.html # Web Applications and Routing The [`ntex::web`] module provides everything needed to build an HTTP application: routing, request extractors, responses, middleware, and testing tools. [`web::App`] is where an application comes together. You add routes, resources, scopes, middleware, filters, and fallback services, then ntex builds them into the service that handles incoming requests. The server evaluates the application factory independently in each worker: ```rust,no_run use ntex::web::{self, HttpResponse}; #[ntex::main] async fn main() -> std::io::Result<()> { web::server(async |_| { web::App::new() .route( "/", web::get().to(async || HttpResponse::Ok().body("Hello")), ) }) .bind("127.0.0.1:8080", ntex::SharedCfg::default())? .run() .await } ``` The factory returns an application factory rather than a running service. `HttpService` uses it to create the application service for each connection, so application services are never shared between workers. A worker can invoke the outer factory again if its service has to be recreated, so keep factory setup repeatable rather than relying on it to run exactly once. To learn more about workers, server configuration, and application state, see [Server](./5-server.md) and [Worker and request state](./6-server-app.md). ## Handlers The quickest way to register a handler is `App::route()`: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new() .route("/users", web::get().to(async || HttpResponse::Ok())) .route( "/users", web::post().to(async || HttpResponse::Created()), ); ``` A handler takes request extractors that implement [`FromRequest`] and returns a value that implements [`Responder`]. Each call to `App::route()` creates a separate [`Resource`] containing one [`Route`]. The route's method and custom guards become resource guards. This means you can register the same path several times with different guards. It also means that when those guards fail, ntex skips that resource and uses the application fallback, which returns `404 Not Found` by default. When several routes belong to the same path, group them with `App::service()` and `web::resource()`. A resource can also have its own name, middleware, filters, and fallback: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new().service( web::resource("/users/{id}") .name("user") .route(web::get().to(async || HttpResponse::Ok())) .route(web::delete().to(async || HttpResponse::NoContent())), ); ``` `Resource::to()` adds a route without a method guard, so it accepts every HTTP method: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new().service( web::resource("/health").to(async || HttpResponse::Ok()), ); ``` ntex also provides attribute macros for common routes. The generated service can be passed directly to `App::service()`: ```rust use ntex::web::{self, HttpResponse}; #[web::get("/health")] async fn health() -> HttpResponse { HttpResponse::Ok().into() } let app = web::App::<()>::new().service(health); ``` Method-specific route helpers include `get`, `post`, `put`, `delete`, `patch`, `head`, and `query`. Routes for other methods are created with `web::method()`, for example `web::method(Method::OPTIONS)`. Method matching is exact. A `GET` route does not also register `HEAD`, and ntex does not add an `OPTIONS` route for you. Register those methods explicitly when clients need them. ### Choosing a Registration Style For a small endpoint, `App::route()` is usually the clearest choice. It keeps the path, method, and handler together: ```rust use ntex::web::{self, App}; let app = App::default() .route("/health", web::get().to(async || "ready")); ``` Use an explicit `Resource` when one path has several methods, needs a name, or has its own middleware, filter, or fallback. Use a `Scope` when several resources share a path prefix or policy. This distinction is more than style: an `App::route()` method mismatch continues through the application router, while a route mismatch inside a selected `Resource` reaches that resource's fallback. `web::service()` is the lower-level option for code that already implements an ntex service over `WebRequest`. Most handler-based applications do not need it. ### Builder Order `App` is configured in two phases. Application-wide settings come first: `middleware()`, `filter()`, `case_insensitive_routing()`, `with_config()`, and `external_resource()`. The first call to `route()`, `service()`, `configure()`, or `default_service()` switches the builder to service registration. After that, only `route()`, `service()`, and `default_service()` are available. ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new() // Application-wide settings .case_insensitive_routing() .external_resource("docs", "https://docs.example.com/{page}") // Service registration .route("/", web::get().to(async || HttpResponse::Ok())) .default_service(web::to(async || HttpResponse::NotFound())); ``` Calling a setting such as `external_resource()` after `route()` does not compile. `Scope` follows the same rule for its settings: `guard()`, `middleware()`, `filter()`, and `case_insensitive_routing()`. ## How Routing Works ### Request-target validation and normalization Request-target validation and normalization are **both disabled by default** for HTTP/1 and HTTP/2. Routing sees the original path structure: `/a/../users` does not automatically match `/users`. The router still decodes individual segments for matching and parameters, so `/%75sers` can match `/users`, but the request URI retains its original percent-encoding and query spelling. Malformed targets can reach application code in this mode. HTTP/1 Host-header validation is separately disabled by default; enable it with `HttpServiceConfig::set_host_validation(true)`. HTTP message framing, mandatory HTTP/2 pseudo-header checks, UTF-8 and representation limits remain enforced. Enable validation and normalization together in the HTTP service configuration passed to the listener (for example, the configuration argument to `web::server(...).bind(...)`): ```rust use ntex::{SharedCfg, http::HttpServiceConfig}; let config = SharedCfg::new("HTTP") .add(HttpServiceConfig::new().set_request_target_normalization(true)); ``` When enabled, ntex validates target syntax and method-specific forms, then normalizes percent-encoding, dot segments, scheme, host and port during HTTP/1 and HTTP/2 decoding. Application middleware, filters, guards, nested scopes, handlers and the application fallback all see the normalized target. Failure to validate or normalize within the URL length limit returns `400 Bad Request`, before invoking application middleware or filters. The same setting applies to lower-level `HttpService` handlers, without an App. It does not redirect clients or merge duplicate slashes. Application rewrites are not automatically validated or normalized again before routing. The public `urly::Url::validate_target()` and `urly::Url::normalize()` APIs remain available for explicit processing; callers must also enforce method-specific forms. Test request builders bypass transport decoding and preserve raw targets. ### Routing stages A request moves through the application in several stages: 1. Application middleware wraps the application service. 2. Application filters process the incoming [`WebRequest`]. 3. The application router finds a [`Resource`] whose path and resource guards match. 4. Resource middleware wraps the selected resource's filter and routes. 5. The resource filter processes the request. 6. The resource checks its routes in registration order. A route matches only if all its method and custom guards pass. 7. The first matching route calls its handler. Middleware can return a response without calling the service it wraps. When that happens, ntex skips every later stage. Where routing stops determines which fallback runs: - If no resource path-and-guard combination matches, the application default service runs. The built-in application default returns `404 Not Found`. - If a resource path matches but none of its routes match, the resource default service runs. The built-in resource default returns `405 Method Not Allowed`. - Resource fallbacks are independent of application and scope fallbacks. ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new() .service( web::resource("/reports") .route(web::get().to(async || HttpResponse::Ok())) .default_service( web::to(async || HttpResponse::MethodNotAllowed()), ), ) .default_service( web::to(async || HttpResponse::NotFound().body("Unknown path")), ); ``` Resource guards help the application router choose a resource. Route guards are checked later, after a resource has been selected. Guards can check methods, headers, or custom conditions: ```rust use ntex::http::Method; use ntex::web::{self, guard, HttpResponse}; let app = web::App::<()>::new().service( web::resource("/events").route( web::route() .method(Method::POST) .guard(guard::Header("content-type", "application/json")) .to(async || HttpResponse::Accepted()), ), ); ``` Guards only receive the request head. They are a good fit for methods, headers, and other request metadata, but they cannot inspect the request body or values produced by extractors. Use a filter, extractor, or handler for those checks. This matters when a request uses the wrong method. `App::route("/reports", web::get().to(handler))` uses the application fallback for a non-`GET` request because the method guard belongs to the generated resource. In contrast, `web::resource("/reports").route(web::get().to(handler))` first selects the resource by path, then uses its default `405 Method Not Allowed` response when the method guard fails. `Scope::route()` behaves like `App::route()`. ### Order Matters Resources and scopes are considered in registration order, and the first path whose guards pass wins. Routes inside a resource follow the same rule. Put specific patterns before broad dynamic or remainder patterns: ```rust use ntex::web::{self, App}; let app = App::default() .service(web::resource("/files/status").to(async || "ready")) .service(web::resource("/files/{path}*").to(async || "file")); ``` Reversing these registrations would let the remainder resource handle `/files/status`. For the same reason, add an unguarded route with `Resource::to()` after guarded routes; once it matches, later routes cannot be reached. Use `App::middleware()` to wrap the whole application. Use `App::filter()` to transform every incoming `WebRequest` before routing. Resources and scopes offer the same methods when the behavior should apply to only part of the application. See [Web Application Filters and Middleware](./8.3-web-app.md) for more information. ## Route Path Format Route patterns contain static and dynamic segments separated by `/`. ntex adds a leading `/` to resource and scope paths when needed, but including it makes the full route easier to read. ### Static Paths A static pattern matches the same path: ```text /users /users/profile ``` Trailing slashes matter: `/users` and `/users/` are different paths. Query strings are not included in path matching. Routing is case-sensitive by default. `App::case_insensitive_routing()` makes static path segments ASCII case-insensitive, but does not change how dynamic segments are matched. Scopes can enable the same behavior for their own nested routes. ### Dynamic Segments Write a dynamic segment as `{name}`. It matches one non-empty path segment: ```text /users/{id} /teams/{team}/users/{user} ``` You can combine variables with static text or use several variables in one segment: ```text /releases/v{major}.{minor} /files/{name}.{extension} ``` Add a regular expression as `{name:regex}` to restrict what a variable accepts: ```text /users/{id:[0-9]+} /releases/{version:v[0-9]+\.[0-9]+} ``` The regular expression must match the whole segment. Invalid route patterns and regular expressions panic while the application is built, so define routes as trusted application configuration rather than from user input. ### Remainder Matches Add `*` after a dynamic variable to capture the rest of the path, including `/` separators: ```text /files/{path}* ``` This matches both `/files/readme.txt` and `/files/images/logo.svg`. The `path` value is `readme.txt` in the first case and `images/logo.svg` in the second. The default expression is `.*`, so the captured value may be empty. Remainder matches cannot use a custom regular expression. Unlike normal dynamic segments, a remainder value is not percent-decoded. A request for `/files/my%20file.txt` captures `my%20file.txt`. Decode the value yourself if the handler needs the decoded path, and validate it before using it as a file system path. Static tail patterns such as `/files/*` also work, but a named remainder is usually clearer and works naturally with typed path extraction. ### Multiple Patterns A resource or scope can use an array or vector to accept several patterns: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new() .service( web::resource(["/health", "/status"]) .to(async || HttpResponse::Ok()), ); ``` Every pattern points to the same service. When a multi-pattern resource is named, URL generation uses its first pattern. Put the canonical path first and treat the remaining patterns as aliases. ## Accessing Path Variables ntex percent-decodes variables matched by normal dynamic segments and stores them in the request's match information. Remainder values are stored as they appear in the request path. The easiest way to read them is with the typed [`Path`] extractor: ```rust use ntex::web::{self, HttpResponse}; async fn user(path: web::types::Path<(u32,)>) -> HttpResponse { let user_id = path.0; HttpResponse::Ok().body(format!("user {user_id}")) } let app = web::App::<()>::new().route( "/users/{id:[0-9]+}", web::get().to(user), ); ``` Tuples deserialize variables in path order. To access variables by name, use a struct that derives `serde::Deserialize`: ```rust use ntex::web::{self, HttpResponse}; #[derive(serde::Deserialize)] struct UserPath { organization: String, user_id: u32, } async fn user(path: web::types::Path) -> HttpResponse { HttpResponse::Ok().body(format!( "organization: {}, user: {}", path.organization, path.user_id, )) } let app = web::App::<()>::new().route( "/organizations/{organization}/users/{user_id:[0-9]+}", web::get().to(user), ); ``` The struct fields must have the same names as the route variables. This example requires the `derive` feature from `serde`. You can also read variables directly from [`HttpRequest::match_info()`]: ```rust use ntex::web::{self, HttpRequest}; async fn file(req: HttpRequest) -> String { req.match_info() .get("path") .unwrap_or_default() .to_owned() } let app = web::App::<()>::new().route( "/files/{path}*", web::get().to(file), ); ``` ntex splits path segments before percent-decoding them. As a result, an encoded slash such as `%2F` can appear inside a normal dynamic variable after decoding. Escapes that are not valid UTF-8 remain percent-encoded. Path values are still untrusted input. In particular, a variable that looked like one URL segment can decode to `../` or contain `/`. Validate it before using it as a file name, database key, or other identifier with stricter rules. ## Scopes A [`Scope`] groups related services under a shared path prefix: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new().service( web::scope("/api") .service( web::resource("/users") .route(web::get().to(async || HttpResponse::Ok())), ) .service( web::resource("/users/{id}") .route(web::get().to(async || HttpResponse::Ok())), ), ); ``` These resources match `/api/users` and `/api/users/{id}`. You can nest scopes and use variables in their prefixes. Nested handlers can access those variables: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new().service( web::scope("/organizations/{org}") .route( "/users/{user}", web::get().to( async |path: web::types::Path<(String, String)>| { HttpResponse::Ok() .body(format!("{}:{}", path.0, path.1)) }, ), ), ); ``` A scope prefix matches complete path segments rather than arbitrary text. `scope("/api")` can contain routes below `/api/`, but it does not by itself provide a handler for the bare `/api` path and it never matches `/apix`. Inside the scope, an empty resource pattern (`""`) matches `/api`, while `"/"` matches `/api/`: ```rust use ntex::web::{self, HttpResponse}; let app = web::App::<()>::new().service( web::scope("/api") .route("", web::get().to(async || HttpResponse::Ok().body("/api"))) .route("/", web::get().to(async || HttpResponse::Ok().body("/api/"))), ); ``` A scope can have its own guards, middleware, filters, case-sensitivity setting, and fallback service. Once a scope matches, unmatched paths inside it use the scope fallback. Without a custom scope fallback, ntex returns its built-in `404 Not Found`; it does not use the application fallback. ## Named and External Resources Give a resource a name when handlers need to link to it without hard-coding its path: ```rust use ntex::web::{self, HttpRequest, HttpResponse, error::UrlGenerationError}; async fn index(req: HttpRequest) -> Result { let url = req.url_for("user", ["42"])?; Ok(HttpResponse::Ok().body(url.to_string())) } let app = web::App::<()>::new() .service( web::resource("/users/{id}") .name("user") .route(web::get().to(async || HttpResponse::Ok())), ) .route("/", web::get().to(index)); ``` Here, the `index` handler generates an absolute URL such as `http://some-host-name/users/42`. [`HttpRequest::url_for()`] returns a `ntex::url::Url`. Values fill dynamic segments in pattern order. For a resource inside a scope, values for dynamic scope segments come before values for the resource itself. Use [`HttpRequest::url_for_static()`] when the pattern has no dynamic segments. Each value is percent-encoded as one literal segment, so URL delimiters in a value cannot change the generated URL's structure. Keep resource names unique across the application. Names are application-wide lookup keys, not names local to a scope, and duplicate names make generated links depend on which definition is found. Generation can fail if the name is unknown, too few values are supplied, or the completed URL is invalid. Returning [`UrlGenerationError`] from a handler, as above, avoids panicking on those errors. For an application resource, the scheme and host come from the request's [`ConnectionInfo`], while the path comes from the resource and its containing scopes. `ConnectionInfo` can use `Forwarded`, `X-Forwarded-Proto`, `X-Forwarded-Host`, or `Host`, so a front-end proxy should replace untrusted forwarding headers before generated URLs are exposed to clients. Use `App::external_resource()` to register a named URL that is not handled by the application. Register it before the first route or service, or register it inside `App::configure()` with `ServiceConfig::external_resource()`: ```rust use ntex::web::{self, HttpRequest, HttpResponse, error::UrlGenerationError}; async fn docs(req: HttpRequest) -> Result { let url = req.url_for("documentation", ["page1.html"])?; Ok(HttpResponse::Ok().body(url.to_string())) } let app = web::App::<()>::new() .external_resource( "documentation", "https://docs.example.com/{page}", ) .route("/docs", web::get().to(docs)); ``` The `docs` handler generates `https://docs.example.com/page1.html`. Because the external pattern already contains a scheme and host, it does not use the request's connection information. External resources are URL templates only; they do not participate in request routing. ## Modular Configuration As an application grows, keeping every route in one builder chain becomes hard to read. `App::configure()` provides a [`ServiceConfig`] that lets another function register part of the application: ```rust use ntex::web::{self, HttpResponse}; fn configure_api(cfg: &mut web::ServiceConfig) { cfg.route( "/health", web::get().to(async || HttpResponse::Ok()), ); cfg.service( web::resource("/version") .to(async || HttpResponse::Ok().body("4")), ); } let app = web::App::default() .configure(configure_api) .route("/", web::get().to(async || HttpResponse::Ok())); ``` `ServiceConfig` can register routes, services, and external resources. It does not create another routing boundary. The registered items join the application at the point where `configure()` is called. Both applications and scopes support modular configuration. ## Testing Routes The `web::test` module lets you initialize an `App`, send requests to it, and inspect responses without opening a network socket: ```rust use ntex::http::StatusCode; use ntex::web::{self, HttpResponse}; use ntex::web::test::{TestRequest, call_service, init_service}; #[ntex::test] async fn user_route() { let service = init_service( web::App::new().route( "/users/{id}", web::get().to(async || HttpResponse::Ok()), ), ) .await; let request = TestRequest::get() .uri("/users/42") .to_request(); let response = call_service(&service, request).await; assert_eq!(response.status(), StatusCode::OK); } ``` It is worth testing method mismatches, trailing slashes, dynamic-variable constraints, scope boundaries, and fallback services. Each one exercises a different part of route selection. ## Lower-Level HTTP Service For most applications, `web::server(factory)` is the simplest way to start a server. It is equivalent to `web::HttpServer::new(factory)`. After you register routes and services, the application builder can become a service factory that converts `http::Request` values into `http::Response` values. [`HttpService`] uses this factory to handle HTTP/1.1 and HTTP/2 connections. Use the lower-level server builder when you need to add connection-level services before `HttpService`. These services can inspect or transform the `Io` object and map the state passed to the HTTP and application services. The same server can be built at the lower level like this: ```rust,no_run use ntex::{http, server, web, SharedCfg}; use ntex::web::HttpResponse; #[ntex::main] async fn main() -> std::io::Result<()> { server::build() .bind("http", "127.0.0.1:8080", SharedCfg::new("S"), async |_| { http::HttpService::new( web::App::new() .route("/", web::get().to(async || { HttpResponse::Ok().body("Hello") })) .build(), ) })? .run() .await } ``` To give the web application its own state instead of the surrounding service pipeline's state, see [Web Application State](./8.2-web-app.md). [`HttpRequest::match_info()`]: https://docs.ntex.rs/api/ntex/web/struct.HttpRequest.html#method.match_info [`HttpRequest::url_for()`]: https://docs.ntex.rs/api/ntex/web/struct.HttpRequest.html#method.url_for [`HttpRequest::url_for_static()`]: https://docs.ntex.rs/api/ntex/web/struct.HttpRequest.html#method.url_for_static [`ConnectionInfo`]: https://docs.ntex.rs/api/ntex/web/dev/struct.ConnectionInfo.html [`FromRequest`]: https://docs.ntex.rs/api/ntex/web/trait.FromRequest.html [`HttpService`]: https://docs.ntex.rs/api/ntex/http/struct.HttpService.html [`Path`]: https://docs.ntex.rs/api/ntex/web/types/struct.Path.html [`Resource`]: https://docs.ntex.rs/api/ntex/web/struct.Resource.html [`Responder`]: https://docs.ntex.rs/api/ntex/web/trait.Responder.html [`Route`]: https://docs.ntex.rs/api/ntex/web/struct.Route.html [`Scope`]: https://docs.ntex.rs/api/ntex/web/struct.Scope.html [`ServiceConfig`]: https://docs.ntex.rs/api/ntex/web/struct.ServiceConfig.html [`WebRequest`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html [`UrlGenerationError`]: https://docs.ntex.rs/api/ntex/web/error/enum.UrlGenerationError.html [`ntex::web`]: https://docs.ntex.rs/api/ntex/web/index.html [`web::App`]: https://docs.ntex.rs/api/ntex/web/struct.App.html # Web Application State A [`web::App`] is parameterized by an application state type. It usually inherits this state from the surrounding server or service pipeline, making the same value available to handlers, extractors, filters, middleware, and error responses. There are two kinds of state to keep in mind: - **Application state** is the state supplied by the service pipeline. A worker's value is cloned for each connection, and handlers borrow that connection's clone. Data that must be shared between connections belongs behind a shared handle such as `Rc`. - **Request-local state** belongs to one request. It is owned by `WebRequest`, can change type as the request moves through filters and middleware, and can be moved into a state-aware handler. This guide explains how both kinds of state work with `web::App`. See [Worker and request state](./6-server-app.md) for worker initialization and process-wide versus worker-local data, and [Service state](./3-state.md) for the underlying service-state model. ## The `web::State` Trait To use a type as application state, implement [`web::State`]: ```rust,ignore pub trait State: 'static { type Error; } ``` The trait has no methods. Its associated `Error` type defines the application's error domain. Handlers, filters, middleware, and fallback services use it through [`WebError`] and [`WebResponseError`]. Most applications can use [`DefaultError`]: ```rust use ntex::web; #[derive(Clone)] struct ApplicationState { greeting: String, } impl web::State for ApplicationState { type Error = web::DefaultError; } ``` `web::State` itself does not require `Clone`, but in practice application state used with the built-in HTTP server must implement it. [`ServerAppConfig::State`] and state passed to `build_with()` must be `Clone`, and [`HttpService`] clones the connection state while creating each connection's application service. A non-`Clone` state type can still be used with an application factory directly, but it cannot be carried through these server paths. ## Inheriting Worker State [`web::server_with_config()`] creates one state value per worker and passes it to that worker's application factory. The returned `App` uses the same state type: ```rust,no_run use std::io; use ntex::{SharedCfg, web}; /// State created independently for each worker. #[derive(Clone)] struct ApplicationState { greeting: String, } impl web::State for ApplicationState { type Error = web::DefaultError; } /// Process-wide factory used to create worker state. struct ApplicationConfig; impl ntex::server::ServerAppConfig for ApplicationConfig { type State = ApplicationState; async fn create(&self) -> io::Result { Ok(ApplicationState { greeting: "Hello".to_owned(), }) } } async fn index(state: &ApplicationState, _request_state: ()) -> String { state.greeting.clone() } #[ntex::main] async fn main() -> io::Result<()> { web::server_with_config(ApplicationConfig, async |_| { web::App::new().route("/", web::get().to_with_state(index)) }) .bind("127.0.0.1:8080", SharedCfg::default())? .run() .await } ``` Every request handled by a worker can access that worker's state through the service context. Each connection receives its own clone of the worker state, so values that all connections of a worker should share, such as caches or counters, belong behind `Rc`. See [Worker-Local and Process-Wide State](./6-server-app.md#worker-local-and-process-wide-state) for details. ### Design State for Cheap Cloning Application state is usually a small collection of handles rather than a large data structure copied for every connection: ```rust use std::{cell::RefCell, collections::HashMap, rc::Rc, sync::Arc}; use ntex::web; #[derive(Clone)] struct ApplicationState { worker_cache: Rc>>, shared_settings: Arc, } struct Settings { service_name: String, } impl web::State for ApplicationState { type Error = web::DefaultError; } ``` Here, connections on the same worker share `worker_cache`. All workers can also share the same thread-safe `shared_settings` value when the configuration factory gives each worker a clone of the same `Arc`. Handlers receive `&ApplicationState`, so mutation normally happens through a client handle or an interior-mutable value such as `Cell`, `RefCell`, a lock, or an atomic. Keep `RefCell` borrows short and do not hold them across an `.await`, where unrelated work could try to borrow the same value and panic. ## Accessing Application State in Handlers Use [`Route::to_with_state()`] when a handler needs direct access to state. `Resource::to_with_state()` does the same for a resource route without a method guard. The handler receives the arguments in this order: 1. A shared reference to the application state. 2. The request-local state. 3. Any request extractors. ```rust use ntex::web; #[derive(Clone)] struct ApplicationState { service_name: &'static str, } impl web::State for ApplicationState { type Error = web::DefaultError; } async fn user( state: &ApplicationState, _request_state: (), user_id: web::types::Path, ) -> String { format!("{} user {}", state.service_name, user_id.into_inner()) } fn main() { let app = web::App::::new().route( "/users/{user_id}", web::get().to_with_state(user), ); } ``` Filters, middleware, and lower-level services access application state through their service [`Ctx`]. Handlers registered with [`Route::to()`] receive only request extractors. Use `to()` when the handler does not need application or request-local state. Extractors also receive a shared reference to application state through [`FromRequest`], so a custom extractor can use application services or configuration. Extractors do not receive request-local state; that value is a separate handler argument supplied only by `to_with_state()`. ## Request-Local State Application state is long-lived and shared across requests. Request-local state is different: it belongs to one request and is carried by [`WebRequest`]. Its type is the parameter in `WebRequest`. An ordinary web request starts with `()` as its request-local state. Each filter or middleware can keep that value, mutate it through `WebRequest::st_mut()`, or replace it with another type through `map_state()`. This is useful for data produced while processing a request, such as authentication details. Filters and middleware can transform the value with [`WebRequest::map_state()`], and a handler registered with `to_with_state()` receives the result as its second argument: ```rust use std::convert::Infallible; use ntex::web::{self, WebRequest}; #[derive(Clone)] struct ApplicationState { service_name: &'static str, } impl web::State for ApplicationState { type Error = web::DefaultError; } async fn index(state: &ApplicationState, user_id: usize) -> String { format!("{} user {user_id}", state.service_name) } fn main() { let app = web::App::::new() .filter(async |req: WebRequest<()>| { // Authentication could produce this request-local value. Ok::<_, Infallible>(req.map_state(|()| 42usize)) }) .route("/", web::get().to_with_state(index)); } ``` Request-local state moves through the filter and middleware chain. Changing the state also changes the `WebRequest` type expected by the next service. This lets Rust check the pipeline at compile time: a handler expecting `AuthInfo`, for example, can only follow a filter or middleware that produces `WebRequest`. Inside filters and middleware, [`WebRequest::st()`] and `WebRequest::st_mut()` return the request-local state, not the application state. Application state comes from the service context instead. `map_state()` consumes the previous value. If later stages need some of the old information, include it in the new state: ```rust struct Authenticated { user_id: usize, } struct Validated { user: Authenticated, request_id: String, } ``` Application filters affect every route. A scope or resource filter can create state required only by that part of the application, which keeps unrelated handlers from depending on it. Request-local state is not stored in `HttpRequest` extensions, and request extractors cannot read it. Use `to_with_state()` when the final handler needs the value. A handler registered with `to()` ignores and drops it. Because the request-local state is part of the request type, it also enforces handler requirements at compile time. A handler that takes an authenticated user can only be registered where a filter has produced one. Registering it anywhere else is a type error, so the check that creates the user can't be skipped. [Complete web application state example](https://github.com/ntex-rs/examples/tree/main/state/web-state) ## Typed Web Configuration Is Separate [`WebAppConfig`] also has a typed value store populated with `set_state()`. Despite the similar name, these values are not the application's service state: - `to_with_state()` does not pass them as its first argument. - They are read from [`HttpRequest::app_state()`] or `WebRequest::app_state()`. - They must be `Send + Sync`, unlike worker-local state, which may contain `Rc` and `RefCell`. - There is one value for each concrete type. This store is useful for HTTP-facing configuration, especially extractor limits and settings that are naturally read from a request: ```rust use ntex::web::{self, App, HttpRequest, WebAppConfig}; struct Limits { max_items: usize, } async fn limits(req: HttpRequest) -> String { let max_items = req .app_state::() .map_or(100, |limits| limits.max_items); format!("maximum items: {max_items}") } let config = WebAppConfig::new().set_state(Limits { max_items: 50 }); let app = App::default() .with_config(config) .route("/limits", web::get().to(limits)); ``` Use application state for services and resources that participate in the service lifecycle. Use `WebAppConfig` values for typed web configuration. ## Supplying Fixed State with `.build_with()` After the first call to `route()`, `service()`, `configure()`, or `default_service()`, the application builder becomes [`AppServices`] (see [Builder Order](./8.1-web-app.md#builder-order)). You can finish building it with either `build()` or [`AppServices::build_with()`]. `build()` uses the state from the surrounding service pipeline. `build_with(state)` gives the application its own fixed state instead, allowing the outer pipeline to use a different state type. Use `build_with()` when the HTTP or server pipeline has different state, or no state at all, but the web application needs its own: ```rust use ntex::web::{self, HttpResponse}; #[derive(Clone)] struct ApplicationState { greeting: &'static str, } impl web::State for ApplicationState { type Error = web::DefaultError; } async fn index(state: &ApplicationState, _request_state: ()) -> HttpResponse { HttpResponse::Ok().body(state.greeting) } fn main() { let app = web::App::::new() .route("/", web::get().to_with_state(index)) .build_with::<()>(ApplicationState { greeting: "Hello", }); } ``` In this standalone example, `()` is the outer pipeline's state type. It is normally inferred when the application factory is passed to [`HttpService`]. The application state must implement `Clone` because `build_with()` clones it when creating application service instances. `build_with()` replaces the state for the application; it does not add another state layer. Code outside the application continues to use the outer state, while web handlers and application middleware receive the fixed state. Use `server_with_config()` when state needs asynchronous per-worker initialization or should be recreated with a worker. Use `build_with()` when a ready value should be embedded in an application factory, especially when the surrounding HTTP pipeline uses another state type. ## The `AppState` Wrapper If the application uses `DefaultError`, [`AppState`] saves you from writing a `web::State` implementation. It works with any `'static` `T` and dereferences to the wrapped value. It also implements `Clone` and `Default` whenever `T` does: ```rust use ntex::web; #[derive(Clone)] struct Settings { service_name: &'static str, } fn main() { let state = web::AppState::new(Settings { service_name: "users", }); assert_eq!(state.service_name, "users"); assert_eq!(state.st().service_name, "users"); } ``` Use a dedicated state type and implement `web::State` directly when you need a custom error type or other state-specific behavior. `AppState` does not change how values are cloned or shared. If `T::clone()` copies a plain field, connections receive separate copies; if it clones an `Rc` or `Arc`, they share the value behind that handle. ## Testing Stateful Routes [`test::init_service_st()`] builds an application with an explicit state value. Keep a clone of a shared handle when the test needs to inspect what the handler changed: ```rust use std::{cell::Cell, rc::Rc}; use ntex::web::{self, test}; #[derive(Clone)] struct ApplicationState { hits: Rc>, } impl web::State for ApplicationState { type Error = web::DefaultError; } async fn index(state: &ApplicationState, (): ()) -> &'static str { state.hits.set(state.hits.get() + 1); "ok" } #[ntex::test] async fn state_is_used() { let state = ApplicationState { hits: Rc::new(Cell::new(0)), }; let service = test::init_service_st( state.clone(), web::App::::new() .route("/", web::get().to_with_state(index)), ) .await; let request = test::TestRequest::get().uri("/").to_request(); let _response = test::call_service(&service, request).await; assert_eq!(state.hits.get(), 1); } ``` [`AppServices`]: https://docs.ntex.rs/api/ntex/web/struct.AppServices.html [`AppServices::build_with()`]: https://docs.ntex.rs/api/ntex/web/struct.AppServices.html#method.build_with [`AppState`]: https://docs.ntex.rs/api/ntex/web/struct.AppState.html [`Ctx`]: https://docs.ntex.rs/api/ntex_service/struct.Ctx.html [`DefaultError`]: https://docs.ntex.rs/api/ntex/web/error/struct.DefaultError.html [`FromRequest`]: https://docs.ntex.rs/api/ntex/web/trait.FromRequest.html [`HttpRequest::app_state()`]: https://docs.ntex.rs/api/ntex/web/struct.HttpRequest.html#method.app_state [`HttpService`]: https://docs.ntex.rs/api/ntex/http/struct.HttpService.html [`Route::to()`]: https://docs.ntex.rs/api/ntex/web/struct.Route.html#method.to [`Route::to_with_state()`]: https://docs.ntex.rs/api/ntex/web/struct.Route.html#method.to_with_state [`ServerAppConfig::State`]: https://docs.ntex.rs/api/ntex/server/trait.ServerAppConfig.html#associatedtype.State [`test::init_service_st()`]: https://docs.ntex.rs/api/ntex/web/test/fn.init_service_st.html [`WebAppConfig`]: https://docs.ntex.rs/api/ntex/web/struct.WebAppConfig.html [`WebError`]: https://docs.ntex.rs/api/ntex/web/error/struct.WebError.html [`WebRequest`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html [`WebRequest::map_state()`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html#method.map_state [`WebRequest::st()`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html#method.st [`WebResponseError`]: https://docs.ntex.rs/api/ntex/web/error/trait.WebResponseError.html [`web::App`]: https://docs.ntex.rs/api/ntex/web/struct.App.html [`web::State`]: https://docs.ntex.rs/api/ntex/web/trait.State.html [`web::server_with_config()`]: https://docs.ntex.rs/api/ntex/web/fn.server_with_config.html # Web Application Filters and Middleware Like the rest of ntex, the web framework is built around [`Service`] and [`Middleware`]. A web application is simply a service that accepts an HTTP request, runs it through the configured pipeline, and returns a [`WebResponse`]. Filters and middleware let you customize this pipeline in different ways: - A **filter** transforms an incoming [`WebRequest`] before processing continues. - **Middleware** wraps a service, so it can run code both before and after that service. Both can use application state. Middleware services receive it through [`Ctx`], and a filter function can take it as its first argument. For the distinction between application state and request-local state, see [Web Application State](./8.2-web-app.md). ## Filters [`App::filter()`], [`Scope::filter()`], and [`Resource::filter()`] append a service to the inbound request pipeline. A filter accepts `WebRequest` and must return `WebRequest`: ```text WebRequest -> filter -> WebRequest ``` Filters run in registration order, with each filter receiving the output of the previous one. If a filter returns an error, ntex skips the remaining filters, router, and handler, then renders the error through [`WebResponseError`]. Because request state is part of the `WebRequest` type, filters can enforce requirements at compile time. For example, an authentication filter can turn `WebRequest<()>` into `WebRequest`. A handler that expects `AuthenticatedUser` can only be registered after a filter that provides it: ```rust use std::convert::Infallible; use ntex::web::{self, WebRequest}; struct AuthenticatedUser { id: u64, } async fn authenticate( req: WebRequest<()>, ) -> Result, Infallible> { Ok(req.map_state(|()| AuthenticatedUser { id: 42 })) } async fn profile(_app: &(), user: AuthenticatedUser) -> String { format!("User {}", user.id) } web::App::default() .filter(authenticate) .route("/profile", web::get().to_with_state(profile)); ``` Here the application state is `()`, so `profile` receives `&()` as its first argument and the `AuthenticatedUser` produced by the filter as its second. [`WebRequest::map_state()`] consumes the request, transforms its request-local state, and preserves the HTTP request and payload. A filter can also take the application state as its first argument and return any error that implements [`WebResponseError`]. The error stops processing and becomes the response: ```rust use ntex::web::{self, WebRequest, error}; fn main() { let app = web::App::default() .filter(async |_state: &(), req: WebRequest<()>| { if req.headers().contains_key("x-api-key") { Ok(req) } else { Err(error::ErrorForbidden("missing API key")) } }) .route("/", web::get().to(async || "Hello")); } ``` Filters only run on the way in. They cannot inspect or modify the handler's response. Use middleware when you need to wrap the complete request-response operation. Filters also cannot return a successful response directly: their successful output must be another `WebRequest`. Return an error to reject a request, or use middleware when a cache hit, maintenance page, or authorization decision should produce a response without running the inner service. ## Middleware A middleware component receives a service and wraps it with another service. The wrapper can inspect or modify the request, call the inner service through [`Ctx`], and then inspect or modify the response. It can also participate in readiness and shutdown. Middleware can change the request-state type before calling the inner service, but a filter is usually simpler when you only need to process the request. Middleware can also short-circuit the pipeline by returning a `WebResponse` without calling the inner service. `WebRequest::into_response()` is the convenient way to keep the original request attached to that response. Middleware can be installed with [`App::middleware()`], [`Scope::middleware()`], or [`Resource::middleware()`]. Built-in middleware can be installed directly: ```rust use ntex::web::{self, App, middleware}; App::default() .middleware( middleware::DefaultHeaders::new() .header("x-application", "example"), ) .middleware(middleware::Logger::default()) .route("/", web::get().to(async || "Hello")); ``` Middleware runs in registration order on the way in. In this example, `DefaultHeaders` receives the request first, followed by `Logger`. The response takes the opposite path: `Logger` processes it first, followed by `DefaultHeaders`. `DefaultHeaders` only inserts a header when the response does not already have one, so a handler can override the default. `Logger` writes through the `log` facade; an application must install and configure a logger to see its output. The simplest custom middleware is an async closure passed to [`middleware::from_fn()`]. It receives the request and a [`Next`] handle whose `call()` runs the inner service. Return a response without calling `next.call(req)` to short-circuit the pipeline. Binding, readiness, and shutdown are forwarded automatically: ```rust use ntex::http::header::{HeaderName, HeaderValue}; use ntex::web::{self, App, HttpResponse, middleware::from_fn}; web::HttpServer::new(async |_| { App::new() .middleware(from_fn(async |req, next| { if !req.headers().contains_key("authorization") { return Ok(req.into_response(HttpResponse::Unauthorized().build())); } let mut res = next.call(req).await?; res.headers_mut().insert( HeaderName::from_static("x-application"), HeaderValue::from_static("example"), ); Ok(res) })) .route("/", web::get().to(async || "Hello")) }); ``` The closure types are inferred once the application is passed to `HttpServer` or a test helper. Return `Err(web::WebError::from_err(err))` to reject a request with any [`WebResponseError`]. To read application state, annotate the state type of `next`, for example `next: Next<'_, _, AppState>`, and call `next.st()`. [`fn_layer()`] from `ntex::service` is the protocol-independent equivalent; it works for any service, but web closures then need explicit request, response, and error types. Write a middleware struct and a wrapper service when the middleware needs its own configuration type, its own readiness source, or a named service type. A custom middleware struct follows the normal ntex service model: ```rust use ntex::http::header::{HeaderName, HeaderValue}; use ntex::web::{self, WebRequest, WebResponse}; use ntex::{Ctx, Middleware, Service}; struct ResponseHeader; struct ResponseHeaderService { service: S, } impl Middleware for ResponseHeader { type Service = ResponseHeaderService; fn create(&self, _state: &St, service: S) -> Self::Service { ResponseHeaderService { service } } } impl Service> for ResponseHeaderService where S: Service, Res = WebResponse>, { type Res = WebResponse; type Error = S::Error; ntex::forward!(St, service); async fn call( &self, req: WebRequest, ctx: Ctx<'_, Self, St>, ) -> Result { let mut res = ctx.call(&self.service, req).await?; res.headers_mut().insert( HeaderName::from_static("x-application"), HeaderValue::from_static("example"), ); Ok(res) } } web::App::default() .middleware(ResponseHeader) .route("/", web::get().to(async || "Hello")); ``` `Middleware::create()` runs when the application service is constructed. It receives the application state and the service to wrap, and returns the per-service wrapper. Its `call()` method then receives a [`Ctx`] for each request. If the wrapper needs to retain something from application state, clone an owned handle during `create()` rather than trying to retain the borrowed `&St`. Always call the inner service through [`Ctx::call()`] to preserve call-time readiness checks. Separately, forward `bind()`, `ready()`, and `shutdown()` to the wrapped service. Use `forward!` for standard forwarding, or implement these methods explicitly when custom behavior is needed. Without forwarding, the default implementations report the middleware as always ready, do not register the inner service's readiness notifications, and never shut down the inner service. The `forward!` macro takes the state type and one or more named or tuple fields, with an optional trailing `map_err = f` to convert readiness errors. See [Service pipelines](./2-pipeline.md) for more on readiness and shutdown. ## Configure Layers Before Routes At each level, add filters and middleware before the first route, service, configuration callback, or default service. Those registration methods move the builder to its service-registration stage, where `filter()` and `middleware()` are no longer available: ```rust use ntex::web::{self, middleware, App, WebRequest}; App::default() .middleware(middleware::Logger::default()) .filter(async |req: WebRequest<()>| { Ok::<_, std::convert::Infallible>(req) }) .route("/", web::get().to(async || "Hello")); ``` The same rule applies to `Scope` and `Resource`. See [Builder Order](./8.1-web-app.md#builder-order). Middleware and filter calls are two separate stacks, even if their builder calls are interleaved. All middleware at a level wraps that level's complete filter-and-router service, so it runs before every filter on the inbound path. ## Where Filters and Middleware Run You can install filters and middleware on an application, scope, or resource: | Level | Runs for | |---|---| | `App` | Every request entering the application | | `Scope` | Requests whose scope prefix and scope guards match | | `Resource` | Requests whose resource path and resource guards match | For example, a request handled by a resource inside a scope follows this path: ```text application middleware -> application filters -> application routing and scope guards -> scope middleware -> scope filters -> scope routing and resource guards -> resource middleware -> resource filters -> route guards -> route handler ``` The response travels back through the middleware layers in reverse. At each level, middleware runs in registration order for the request and reverse registration order for the response. Filters are not part of the response path. The router checks a scope or resource's own guards before entering its middleware and filters. Application filters run before the application router chooses a scope or resource, and scope filters run before the scope router chooses a nested resource. Route guards are later: the resource filter runs first, then routes and their guards are checked in registration order. Fallback services run inside the layers of their level. When no resource matches, the application default service runs after application middleware and filters. When a scope matches but none of its services do, the scope default service runs after the scope's middleware and filters. When a resource matches but no route does, its default service runs after resource middleware and filters; without a custom default, it returns `405 Method Not Allowed`. See [Web Applications and Routing](./8.1-web-app.md) for guard and fallback behavior. ## Errors and the Response Path Extractor failures and errors returned by handler `Result` values are converted into `WebResponse` values inside the route handler. Response middleware can therefore inspect or modify those error responses normally. Errors returned directly by filters, middleware services, or other web services take a different path. They remain service errors while unwinding through middleware and are rendered through [`WebResponseError`] outside the application middleware stack. A middleware that propagates an inner error with `?` does not see the final rendered response. If it must process that case, it needs to catch the error and convert it into a `WebResponse` itself. See [Web Application Errors](./8.5-web-app.md) for the error model. ## Choosing Between Them A useful rule of thumb is to use a filter for request-only work and middleware when you also need the response. Choose a filter when: - Only the incoming request needs processing. - The request-local state type should change. - Failure should stop processing before routing or handler execution. Choose middleware when: - Both the request and response need to be observed or modified. - Processing may finish early with a successful response. - Logic must wrap an entire application, scope, or resource. - The component must control how the wrapped service's readiness or shutdown is reported. Filters are services too. A filter implemented as a full `Service` takes part in readiness and shutdown like any other stage of the pipeline, but it cannot see the response or the wrapped service. [Complete example with state-aware middleware and filters](https://github.com/ntex-rs/examples/tree/main/state/web-state) [`App::filter()`]: https://docs.ntex.rs/api/ntex/web/struct.App.html#method.filter [`App::middleware()`]: https://docs.ntex.rs/api/ntex/web/struct.App.html#method.middleware [`Ctx`]: https://docs.ntex.rs/api/ntex/struct.Ctx.html [`Ctx::call()`]: https://docs.ntex.rs/api/ntex/struct.Ctx.html#method.call [`fn_layer()`]: https://docs.ntex.rs/api/ntex/service/fn.fn_layer.html [`middleware::from_fn()`]: https://docs.ntex.rs/api/ntex/web/middleware/fn.from_fn.html [`Middleware`]: https://docs.ntex.rs/api/ntex/trait.Middleware.html [`Next`]: https://docs.ntex.rs/api/ntex/web/middleware/struct.Next.html [`Resource::filter()`]: https://docs.ntex.rs/api/ntex/web/struct.Resource.html#method.filter [`Resource::middleware()`]: https://docs.ntex.rs/api/ntex/web/struct.Resource.html#method.middleware [`Scope::filter()`]: https://docs.ntex.rs/api/ntex/web/struct.Scope.html#method.filter [`Scope::middleware()`]: https://docs.ntex.rs/api/ntex/web/struct.Scope.html#method.middleware [`Service`]: https://docs.ntex.rs/api/ntex/trait.Service.html [`WebRequest`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html [`WebRequest::into_response()`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html#method.into_response [`WebRequest::map_state()`]: https://docs.ntex.rs/api/ntex/web/struct.WebRequest.html#method.map_state [`WebResponse`]: https://docs.ntex.rs/api/ntex/web/struct.WebResponse.html [`WebResponseError`]: https://docs.ntex.rs/api/ntex/web/error/trait.WebResponseError.html # Web Application Extractors and Responders Most handlers do not need to parse requests or build responses by hand. ntex handles both sides of the process: - **Extractors** provide the values passed to a handler. - **Responders** turn the handler's result into an HTTP response. This leaves the handler free to focus on application logic. ## Extractors Handler arguments are extractors. Before calling a handler registered with [`Route::to()`], ntex creates each argument from the request in the order it appears in the function signature. Each argument type must implement [`FromRequest`]. A handler can take up to 16 extractor arguments. The built-in extractors cover common request data: | Extractor | Reads | |---|---| | [`HttpRequest`] | A cheap clone of the request handle | | [`Path`] | Dynamic path segments | | [`Query`] | URL query parameters | | [`Json`] | A JSON request body | | [`Form`] | A URL-encoded request body | | [`Payload`] | The streaming request body | | [`Bytes`] | The complete request body as bytes | | `String` | The complete request body as decoded text | | `Option` | `Some(value)`, or `None` if `T` fails | | `Result` | The value, or the error produced by `T` | | `(A, B, ...)` | Several extractors combined into one argument | For example, this handler reads a user ID from the path and an optional flag from the query string: ```rust use ntex::web::{self, App}; #[derive(serde::Deserialize)] struct Options { details: Option, } async fn user( user_id: web::types::Path, options: web::types::Query, ) -> String { format!( "User {}, details: {}", user_id.into_inner(), options.details.unwrap_or(false), ) } App::default().route( "/users/{user_id}", web::get().to(user), ); ``` If any extractor fails, the handler is not called. ntex turns the error into a response through [`WebResponseError`]. `Path` and `Query` both deserialize with `serde`, but they represent different shapes: - a path tuple reads captured segments by position; - a path struct reads them by route variable name; - a query string is a set of `key=value` pairs, so use a struct or map rather than a tuple. Path values are percent-decoded after the URL has been split into segments. That means an encoded slash such as `%2F` can appear inside one extracted value. Treat path values as input, not as safe file-system paths. ### Request Bodies A request has only one body stream, and all extractors share it. `Json`, `Form`, `Bytes`, and `String` consume that stream, so a handler should normally have only one body extractor. Their default in-memory limits are: | Extractor | Default limit | |---|---:| | `Json` | 32 KiB | | `Form` | 16 KiB | | `Bytes` and `String` | 256 KiB | `Json`, `Form`, and `Query` use `serde` to deserialize values. `Json` accepts JSON media types, including types with a `+json` suffix. `Form` requires `application/x-www-form-urlencoded`. `String` decodes the body according to the request charset; `Bytes` leaves it unchanged. Use [`JsonConfig`], [`FormConfig`], and [`PayloadConfig`] to change those limits. `JsonConfig` can accept additional content types. `PayloadConfig` can require a content type for `Bytes` and `String`; without that setting, those extractors accept any content type. `FormConfig` changes only the size limit. Store extractor configuration in [`WebAppConfig`] with `set_state()` and install it with [`App::with_config()`]: ```rust use ntex::web::{self, App, WebAppConfig}; fn main() { let config = WebAppConfig::new() .set_state(web::types::JsonConfig::default().limit(4096)) .set_state(web::types::PayloadConfig::new(64 * 1024)); let app = App::default() .with_config(config) .route("/", web::post().to(async |body: String| body)); } ``` Values stored with `set_state()` must be `Send + Sync`. If an application does not call `with_config()`, it uses the `WebAppConfig` from the connection's shared configuration, and extractors fall back to their defaults when no configuration of the right type is stored. Use [`Payload`] when the handler should process the body incrementally instead of buffering it in memory. It takes ownership of the remaining body stream and does not apply `PayloadConfig`; the handler decides how much data to read and how to handle it. ### Optional Extractors Sometimes invalid input should not stop the request. Wrapping an extractor in `Option` turns a successful extraction into `Some(value)` and a failure into `None`. The failure is also logged: ```rust use ntex::web::{self, App}; #[derive(serde::Deserialize)] struct Search { query: String, } async fn search(params: Option>) -> String { match params { Some(params) => format!("Searching for {}", params.query), None => "No search query".to_owned(), } } App::default().route("/search", web::get().to(search)); ``` If the handler needs the actual error, use `Result` instead. Both forms allow the handler to run when extraction fails. These wrappers do not rewind the request body. If a wrapped body extractor reads part or all of the stream before failing, a later extractor sees only what remains. In practice, keep it as the handler's only body extractor. ### Custom Extractors You can turn application-specific request data into a handler argument by implementing [`FromRequest`]. A custom extractor receives the application state, the HTTP request, and mutable access to the request body. The body argument is the raw [`http::Payload`] stream. The [`Payload`] extractor in `web::types` is a different type: a handler argument that wraps this stream. Custom extractors use `http::Payload` directly: ```rust use ntex::http::Payload; use ntex::web::{self, App, FromRequest, HttpRequest, InternalError}; struct ClientName(String); impl FromRequest for ClientName { type Error = InternalError<&'static str>; async fn from_request(_: &St, req: &HttpRequest, _: &mut Payload) -> Result { req.headers() .get("x-client-name") .and_then(|value| value.to_str().ok()) .map(|value| ClientName(value.to_owned())) .ok_or_else(|| web::error::ErrorBadRequest("Missing client name")) } } async fn hello(client: ClientName) -> String { format!("Hello, {}!", client.0) } App::default().route("/", web::get().to(hello)); ``` Handlers registered with [`Route::to_with_state()`] receive the application state and request-local state before their extractor arguments. Extractors otherwise behave in the same way. They can use application state, but they do not receive the request-local state carried by `WebRequest`. See [Web Application State](./8.2-web-app.md) for details. ## Responders Handler return values are responders. Any type that implements [`Responder`] can be returned from a handler, and ntex turns it into an HTTP response. Responder conversion happens after the handler completes and receives both the application state and the original request. Common responder types include: | Responder | Result | |---|---| | [`HttpResponse`] or `HttpResponseBuilder` | The response as configured | | `String`, `&String`, or `&'static str` | `200 OK`, `text/plain; charset=utf-8` | | `Bytes`, `BytesMut`, or `&'static [u8]` | `200 OK`, `application/octet-stream` | | `Json` | `200 OK`, `application/json` | | `Form` | `200 OK`, `application/x-www-form-urlencoded` | | `Option` | The inner response, or `404 Not Found` for `None` | | `Result` | The inner response, or the error response produced by `E` | | `(T, StatusCode)` | The inner response with a different status code | | `Either` | The response of whichever variant is returned | | `InternalError` | The error response with its configured status | | `()` | `200 OK` with an empty body, for applications with `()` state | The error type in `Result` must implement [`WebResponseError`]. See [Web Application Errors](./8.5-web-app.md) for how errors become responses. `Either` is available as `ntex::util::Either`. Use [`Responder::with_status()`] and [`Responder::with_header()`] when you only need to change the status or set a header: ```rust use ntex::http::StatusCode; use ntex::web::{self, App, Responder}; #[derive(serde::Serialize)] struct User { id: u32, } async fn create_user() -> impl Responder { web::types::Json(User { id: 42 }) .with_status(StatusCode::CREATED) .with_header("x-api-version", "1") } App::default().route( "/users", web::post().to(create_user), ); ``` These methods wrap the original responder. They keep its body and other headers, then apply the requested status and headers. A configured header replaces values with the same name that the inner responder produced. For simple handlers, returning a concrete responder such as `Json` keeps the signature clear. Use `impl Responder` when the concrete wrapper type is unimportant, and `Either` when different branches naturally produce different responder types. ### Custom Responders Implement [`Responder`] when one of your own types should be returned directly from a handler. The conversion is asynchronous and can inspect application state or request metadata: ```rust use ntex::http::Response; use ntex::web::{self, App, HttpRequest, Responder}; struct Greeting(&'static str); impl Responder for Greeting { async fn respond_to(self, _: &St, _: &HttpRequest) -> Response { Response::Ok() .content_type("text/plain; charset=utf-8") .body(self.0) } } async fn hello() -> Greeting { Greeting("Hello!") } App::default().route("/", web::get().to(hello)); ``` [`App::with_config()`]: https://docs.ntex.rs/api/ntex/web/struct.App.html#method.with_config [`Bytes`]: https://docs.ntex.rs/api/ntex/util/struct.Bytes.html [`Form`]: https://docs.ntex.rs/api/ntex/web/types/struct.Form.html [`FormConfig`]: https://docs.ntex.rs/api/ntex/web/types/struct.FormConfig.html [`FromRequest`]: https://docs.ntex.rs/api/ntex/web/trait.FromRequest.html [`HttpRequest`]: https://docs.ntex.rs/api/ntex/web/struct.HttpRequest.html [`http::Payload`]: https://docs.ntex.rs/api/ntex/http/enum.Payload.html [`HttpResponse`]: https://docs.ntex.rs/api/ntex/http/struct.Response.html [`Json`]: https://docs.ntex.rs/api/ntex/web/types/struct.Json.html [`JsonConfig`]: https://docs.ntex.rs/api/ntex/web/types/struct.JsonConfig.html [`Path`]: https://docs.ntex.rs/api/ntex/web/types/struct.Path.html [`Payload`]: https://docs.ntex.rs/api/ntex/web/types/struct.Payload.html [`PayloadConfig`]: https://docs.ntex.rs/api/ntex/web/types/struct.PayloadConfig.html [`Query`]: https://docs.ntex.rs/api/ntex/web/types/struct.Query.html [`Responder`]: https://docs.ntex.rs/api/ntex/web/trait.Responder.html [`Responder::with_header()`]: https://docs.ntex.rs/api/ntex/web/trait.Responder.html#method.with_header [`Responder::with_status()`]: https://docs.ntex.rs/api/ntex/web/trait.Responder.html#method.with_status [`Route::to()`]: https://docs.ntex.rs/api/ntex/web/struct.Route.html#method.to [`Route::to_with_state()`]: https://docs.ntex.rs/api/ntex/web/struct.Route.html#method.to_with_state [`WebAppConfig`]: https://docs.ntex.rs/api/ntex/web/struct.WebAppConfig.html [`WebResponseError`]: https://docs.ntex.rs/api/ntex/web/error/trait.WebResponseError.html # Web Application Errors Web applications often need a consistent error format, even when failures come from reusable components or external libraries. ntex handles this through an **error domain**. Each application state selects its domain with the `State::Error` associated type, and every error that can reach the application must implement [`WebResponseError`] for that domain. The domain type is a marker: it names a set of rendering rules, and it does not have to be an error that handlers return. It can be the application's own error enum, as in the example below, or a separate empty type. This means handlers can return ordinary `Result` values instead of building error responses by hand. The exact conversion point depends on where the error comes from: extractor and handler errors become responses inside the route, while errors returned by filters, middleware, and lower-level services remain service errors until they leave the application stack. [`DefaultError`] provides plain-text responses for common ntex errors. If your API needs JSON bodies, extra headers, error codes, or other metadata, define a custom error domain and choose how each error is rendered. The compiler checks that every error used by the application has a renderer for the selected domain, so a missing implementation prevents the application from compiling. ## Default Error Handling Every application state has an error type. You choose it through the [`State`] trait: ```rust use ntex::web; #[derive(Clone)] struct AppState; impl web::State for AppState { type Error = web::DefaultError; } ``` Here, the application uses [`DefaultError`]. It already knows how to handle common ntex errors. The response body is the error's `Display` text, sent as `text/plain`: - malformed request data, such as invalid query strings, JSON or form bodies, and a wrong `Content-Type`, returns `400 Bad Request`; - JSON and form bodies that exceed their configured limit return `413 Payload Too Large`; - a malformed form `Content-Length` returns `411 Length Required`; - path segments that fail to deserialize return `404 Not Found`; - response serialization failures, such as a `Json` responder failing to encode its value, return `500 Internal Server Error`. The `Bytes` and `String` extractors use their own `PayloadError`; its default mapping is `400 Bad Request`, including when the configured body limit is exceeded. If your application uses `()` or [`AppState`] as its state, `DefaultError` is selected automatically. In that case, there is usually nothing else to configure. `DefaultError` is only a marker selecting these implementations; it is not constructed at runtime, and it does not make every Rust error renderable. A custom error returned by your code still needs a `WebResponseError` implementation or an `InternalError` wrapper. ## Returning Errors from Handlers A handler can return `Result` just like any other async Rust function. ntex uses the successful value as the response, or turns the error into an error response. For simple cases, the helpers in [`web::error`] let you choose an HTTP status without defining a new error type: ```rust use ntex::web::{self, App, InternalError}; async fn create_user( ) -> Result<&'static str, InternalError<&'static str>> { Err(web::error::ErrorBadRequest("Invalid user")) } fn main() { App::default().route( "/users", web::post().to(create_user), ); } ``` [`web::error`] has 39 `Error*` helpers, one for each common 4xx and 5xx status, such as `ErrorBadRequest`, `ErrorUnauthorized`, `ErrorForbidden`, `ErrorNotFound`, `ErrorConflict`, and `ErrorInternalServerError`. Each returns an `InternalError`, which renders the given status with `T`'s `Display` text as the body. These helpers are convenient at an application boundary, but their body is public. Avoid wrapping database errors, tokens, internal paths, or other sensitive details directly. Log the original error and return a message that is safe for the client. Under the hood, the successful value must implement [`Responder`], while the error must implement [`WebResponseError`] for the application's domain. The error is rendered as soon as the handler returns, inside the `Result` responder. Some error types implement [`WebResponseError`] for every domain, so they work with `DefaultError` and custom domains alike: - `InternalError`, including every `Error*` helper; - `WebError`, an error that has already been converted for the domain; - `ntex::util::Either`, when both `A` and `B` implement it; - `std::convert::Infallible`. `InternalError::from_response()` is useful when a call site needs a completely prepared response rather than a status and plain-text body: ```rust use ntex::web::{self, HttpResponse, InternalError}; async fn create_user( ) -> Result<&'static str, InternalError<&'static str>> { Err(InternalError::from_response( "duplicate user", HttpResponse::Conflict() .header("x-error-code", "user-exists") .body("A user with that name already exists"), )) } ``` ## Custom Error Domains For a larger application, you may want all errors to follow rules of your own. Set `State::Error` to an application-specific type, then implement [`WebResponseError`] for each error that the application can produce. This gives you full control over how errors are rendered, including errors from external components. `WebResponseError` requires `std::error::Error + 'static`, so every error type must implement the standard `Error` trait. The example below derives it with the [`thiserror`] crate, which must be added to your `Cargo.toml`. The following example returns `409 Conflict` when a user already exists and `400 Bad Request` when the request contains invalid JSON: ```rust use ntex::web::{self, App, HttpResponse, WebResponseError, error::JsonPayloadError}; #[derive(Clone)] struct AppState; #[derive(Debug, thiserror::Error)] enum ApiError { #[error("User already exists")] UserExists, } impl web::State for AppState { type Error = ApiError; } impl WebResponseError for ApiError { fn error_response(&self, _state: &AppState) -> HttpResponse { HttpResponse::Conflict().body(self.to_string()) } } impl WebResponseError for JsonPayloadError { fn error_response(&self, _state: &AppState) -> HttpResponse { match self { JsonPayloadError::Overflow => { HttpResponse::PayloadTooLarge().body("Request body is too large") } JsonPayloadError::ContentType => { HttpResponse::UnsupportedMediaType().body("Expected a JSON request") } _ => HttpResponse::BadRequest().body("Invalid JSON"), } } } #[derive(serde::Deserialize)] struct User { name: String, } async fn create_user( user: web::types::Json, ) -> Result { if user.name == "admin" { Err(ApiError::UserExists) } else { Ok(format!("Created {}", user.name)) } } fn main() { App::::new().route( "/users", web::post().to(create_user), ); } ``` There are two possible errors to account for: - the handler can return `ApiError`; - the `Json` extractor can return `JsonPayloadError` before the handler is called. `JsonPayloadError` is defined by ntex, yet the application can still implement `WebResponseError` for it. Rust's orphan rules allow this because `AppState` and `ApiError` are local types. Selecting a custom domain replaces the `DefaultError` implementations for that application. Add mappings for every extractor, responder, filter, middleware, or service error the application uses. This is deliberate: it prevents an API from silently falling back to a response format or status it did not choose. Errors can also happen while ntex is creating the response. For example, if a handler returns `Json`, your error domain must know how to handle `serde_json::Error`. A `Form` responder similarly requires a mapping for `serde_urlencoded::ser::Error`. The same rule applies outside handlers. Errors from filters and middleware, at the application, scope, or resource level, must also implement `WebResponseError` for the domain. Unlike handler errors, they are wrapped in a `WebError` and rendered when they reach the application service. If a required conversion is missing, the route does not compile. This catches unhandled error cases while you are building the application rather than when a request arrives. The `error_response()` method receives the application state, so it can use shared configuration while building the response. Its default response is `500 Internal Server Error` with the error's `Display` text as a plain-text body; override it to choose the status, headers, and body that make sense for the error. For a JSON API, a response builder's `json()` method can serialize a small error-body struct here. ## Where Error Responses Travel Extractor errors, handler `Result` errors, and responder serialization errors are converted while the route is running. They return through resource, scope, and application middleware as ordinary responses, so response middleware can inspect or modify them. Filters, middleware services, and custom web services return errors through the service layer. ntex type-erases those errors into [`WebError`] while they unwind, then the outer application service calls `error_response()`. As a result, middleware using `ctx.call(...).await?` sees an error, not the final rendered response. If it must add headers or otherwise inspect that response, it needs to catch the error and convert it itself. For normal handlers, return `Result` and let the responder handle the conversion. Construct `WebError` directly only when writing lower-level web services that use the service error path. ## Test the Rendered Response Error handling is part of the HTTP contract, so test the status, headers, and body rather than only testing the Rust error value: ```rust use ntex::http::StatusCode; use ntex::web::{self, App}; #[derive(Clone)] struct AppState; impl web::State for AppState { type Error = web::DefaultError; } async fn create_user( ) -> Result<&'static str, web::InternalError<&'static str>> { Err(web::error::ErrorConflict("User already exists")) } #[ntex::test] async fn duplicate_user_response() { let app = web::test::init_service_st( AppState, App::::new().route( "/users", web::post().to(create_user), ), ) .await; let request = web::test::TestRequest::post() .uri("/users") .to_request(); let response = web::test::call_service(&app, request).await; assert_eq!(response.status(), StatusCode::CONFLICT); } ``` [`AppState`]: https://docs.ntex.rs/api/ntex/web/struct.AppState.html [`DefaultError`]: https://docs.ntex.rs/api/ntex/web/error/struct.DefaultError.html [`Responder`]: https://docs.ntex.rs/api/ntex/web/trait.Responder.html [`State`]: https://docs.ntex.rs/api/ntex/web/trait.State.html [`WebError`]: https://docs.ntex.rs/api/ntex/web/error/struct.WebError.html [`WebResponseError`]: https://docs.ntex.rs/api/ntex/web/error/trait.WebResponseError.html [`web::error`]: https://docs.ntex.rs/api/ntex/web/error/index.html [`thiserror`]: https://docs.rs/thiserror # WebSocket Support HTTP works in one direction: the client asks, and the server answers. WebSocket removes that limit. Once a connection is open, either side can send a message whenever it likes. That makes it a natural fit for chat, live notifications, and other interactive applications. It can also carry a different protocol, such as MQTT, over an ordinary web connection. ## WebSocket support in ntex Every WebSocket connection starts life as an HTTP/1.1 request. The client asks to upgrade the connection, and the server agrees by replying with `101 Switching Protocols`. From then on, the two sides stop speaking HTTP and exchange WebSocket frames instead. A `ws://` URL means a plain connection; `wss://` means the same thing over TLS. The APIs in this chapter work with HTTP/1 upgrades, not HTTP/2 extended CONNECT. ntex gives you four ways to work with WebSocket. Most web applications only need the first: | API | Use it to | | --- | --- | | [`web::ws`] | Add a WebSocket endpoint to a routed web application | | [`HttpService`] and HTTP/1 control services | Take over an upgraded connection and drive it yourself | | [`ws::WsClient`] | Connect to a WebSocket server | | [`ws::WsTransport`] | Run a byte-oriented protocol over binary WebSocket messages | All four share the codec and protocol types in [`ntex::ws`]. WebSocket support is on by default through the `ws` Cargo feature. For `wss://`, you'll also need a configured TLS connector, using a backend such as OpenSSL, rustls, or Windows Schannel. Changing the URL alone does not enable TLS on `WsClient`; see [WebSocket TLS configuration](10-tls.md#websocket-client). The optional `compress` feature adds `permessage-deflate` support to the codec and the client. Note that `web::ws::start()` doesn't negotiate compression on its own. ### Frames, messages, and fragmentation You receive data as a [`Frame`] and send data as a [`Message`]. Both have variants for text, binary, ping, pong, close, and continuation frames. A sender can split one message into several frames. When that happens, you'll first see `Frame::Continuation(Item::FirstText(_))` or `FirstBinary(_)`, then zero or more `Continue(_)` fragments, and finally `Last(_)`. The codec makes sure the fragments arrive in a valid order, but it doesn't join them for you. If your handler needs whole messages, it has to collect the fragments itself or reject fragmented input. Two more things to watch for: - Text frames are delivered as raw bytes. Check that the text is valid UTF-8 before treating it as a string, and do it on the complete message. A multi-byte character can be split between two fragments, so a single fragment may look invalid even when the whole message is fine. - The codec's default 64 KiB limit applies to each frame, not to a whole message. If you assemble fragments, enforce your own limit on the total size. Otherwise a peer can build an arbitrarily large message out of small frames. ## Servers with `web::ws` A WebSocket endpoint starts out as an ordinary route with a GET handler. Inside the handler, call [`web::ws::start`] with the request and a service for incoming frames. For each frame, the service returns `Some(Message)` to send a reply, or `None` to send nothing. Here is a complete echo server. It handles text and binary messages that arrive in a single frame: ```rust,no_run use std::io; use ntex::web::{self, HttpRequest, ws}; async fn echo(frame: ws::Frame) -> Result, io::Error> { let message = match frame { ws::Frame::Text(bytes) => match String::from_utf8(bytes.to_vec()) { Ok(text) => ws::Message::Text(text.into()), Err(_) => ws::Message::Close(Some(ws::CloseCode::Invalid.into())), }, ws::Frame::Binary(bytes) => ws::Message::Binary(bytes), ws::Frame::Ping(bytes) => ws::Message::Pong(bytes), ws::Frame::Pong(_) => return Ok(None), ws::Frame::Close(reason) => ws::Message::Close(reason), ws::Frame::Continuation(_) => { ws::Message::Close(Some(ws::CloseCode::Unsupported.into())) } }; Ok(Some(message)) } async fn socket(req: HttpRequest) { if let Err(err) = ws::start(&req, None, echo).await { eprintln!("WebSocket connection failed: {err:?}"); } } #[ntex::main] async fn main() -> io::Result<()> { web::server(async |_| { web::App::new().route("/ws/", web::get().to(socket)) }) .bind("127.0.0.1:8080", ntex::SharedCfg::default())? .run() .await } ``` Notice that the handler doesn't build an HTTP response. `start()` validates the handshake, sends the `101` response, and takes the connection away from the HTTP layer. It then runs the WebSocket dispatcher until the connection closes, so the `await` lasts as long as the connection does, not just until it's set up. During the handoff, `start()` also cancels the HTTP layer's read timer and switches to an I/O configuration without idle keepalive. If the handshake is invalid, `start()` sends the HTTP error response itself. Once the connection has been taken over, returning an `HttpResponse` from your handler won't send anything else. If the connection fails, `start()` returns a [`WsError`]. That covers errors from your frame service, protocol errors, read and write timeouts, and I/O errors. ### Sending outside the frame handler Returning a message is the easiest way to reply to a frame. Sometimes, though, you need to send something unprompted, such as a notification produced by another task. That's what a [`WsSink`] is for. The frame service receives the sink as its state. Use `fn_service_st`, or implement `Service` yourself, to get at it. A `WsSink` is cheap to clone, and every clone sends to the same connection. Give a clone to any task that needs to push messages to the client, such as a timer, a pub/sub subscription, or a chat room broadcaster. Each task can then call `sink.send(...)` on its own, independently of the frame handler and of each other. The clones also share the connection's state: once any of them sends a close message, no more messages can be sent through any of them. The sink is reference-counted with `Rc`, so it stays on the connection's worker thread. Start those tasks with `ntex::rt::spawn` from the same worker, not on a separate thread. This service answers pings through the sink instead of returning the pong: ```rust use std::io; use ntex::service::fn_service_st; use ntex::web::ws; let service = fn_service_st(async |sink: &ws::WsSink, frame: ws::Frame| { if let ws::Frame::Ping(bytes) = frame { sink.send(ws::Message::Pong(bytes)) .await .map_err(io::Error::other)?; } Ok::, io::Error>(None) }); ``` The pong has already been sent, so the handler returns `None`. Returning it as well would send it twice. `send()` also waits while the connection is under write backpressure. If your application keeps its own queue of outgoing messages, give that queue a limit so a slow client can't make it grow forever. The [basic WebSocket example] uses a sink for its heartbeat. It comes with a browser page, a Rust client, and the server. The server's readiness hook starts the heartbeat task once, and its shutdown hook stops it. The task sends pings and keeps track of when the client last sent a ping or pong. If the client goes quiet for too long, the task closes the connection. ### Choosing a subprotocol A subprotocol tells both peers what the connection will carry. `mqtt`, for example, means the connection carries MQTT. The client lists the protocols it supports, and the server picks one. Read the client's offers with [`web::ws::subprotocols`] and pass your choice to `start()`: ```rust let chosen = ntex::web::ws::subprotocols(&req) .find(|protocol| *protocol == "my-application"); ntex::web::ws::start(&req, chosen, service).await?; ``` Whatever you pass as `Some(protocol)` must be a valid token that the client actually offered. `None` accepts the connection without a subprotocol. That's fine if your application doesn't need one. If it does, reject clients that don't offer a protocol you support. Keep in mind that the handshake only checks the WebSocket protocol. It says nothing about who is allowed to connect. Authenticate the client, and check the `Origin` header for browser clients, before you call `start()`. ### Heartbeats and closing the connection Ping, pong, and close frames reach your frame service just like data frames. `start()` takes care of dispatcher control and shutdown events, but it does **not** answer pings. Your service has to return or send the pong, as the echo handler does. When the peer sends a close frame, reply with a close of your own unless you've already sent one. Once your service has handled the peer's close, `start()` closes the I/O stream. To close the connection from your side, return or send `Message::Close`. That starts the closing-handshake timeout. You can set it with `WsClientConfig::set_close_timeout()` in the connection's shared configuration; despite the name, it applies to server-side sinks too. Background tasks such as a heartbeat should stop when the connection ends, as they do in the basic example. If a peer stops responding, or the connection has to go for some other reason, call `sink.io().close()` instead of waiting for the application to exchange more messages. If you want to handle dispatcher events yourself, use [`web::ws::start_with`]. It performs the same upgrade, but your service receives `DispatchItem` instead of `Frame`. That makes it responsible for control and stop events too, including errors and shutdown. ## Low-level servers with an HTTP control service You don't need a web application to accept WebSocket connections. If you're building a lower-level server, sharing one listener between several protocols, or running your own I/O loop, you can handle the upgrade in an HTTP control service. Attach the control service with `HttpService::h1(...).control(...)`, or with `HttpService::new(...).h1_control(...)` if you're using the combined HTTP service. Upgrade requests reach the control service as [`h1::Control::Upgrade`]. You can look at the request before deciding what to do with it, so this is the place to check the path, authentication, `Origin`, and any subprotocol you require. [`ws::handshake`] validates the WebSocket handshake and returns a response builder. Authorization and subprotocol selection are still up to you. You have three choices: - Reject the request with `upgrade.fail(error)` or `upgrade.fail_with(response)`. - Pass it on to the regular application service with `upgrade.ack()`. This is the path a `web::ws` handler relies on. - Take over the connection with `upgrade.handle()`. `handle()` gives you the acknowledgement to return, the I/O stream, the request, and the HTTP/1 codec. This excerpt from the [low-level WebSocket example] passes everything except the acknowledgement to a separate `ws_service`: ```rust .h1_control(async move |control: h1::Control<_, _>| { let ack = if let h1::Control::Upgrade(upgrade) = control { let (ack, io, request, http_codec) = upgrade.handle(); ws_service.call((request, io, http_codec)).await?; ack } else { control.ack() }; Ok::<_, io::Error>(ack) }) ``` `ws_service` checks the handshake and writes the HTTP response with the codec it received. Only then does it switch to `ws::Codec::new()` for WebSocket frames. A codec created this way follows the server's masking rules; a client would need the client rules instead. `handle()` only transfers ownership. It doesn't send a response, so your code must send either an HTTP rejection or the `101` response. Keep using the I/O stream you were given: it may already hold the start of the first WebSocket frame, buffered right after the HTTP headers. At this level, the connection's lifetime is your responsibility too: - Stop the HTTP timer with `io.stop_timer()` and choose I/O settings for the new protocol. Otherwise the old header-read timer can fire in the middle of the WebSocket session. - Drive the connection with `io.recv()` / `io.send()` or an I/O dispatcher. Your loop must handle ping/pong, fragmented messages, close frames, and protocol and transport errors. - Finish the closing handshake, shut down the I/O stream, and stop any background tasks that belong to the connection. The example changes the I/O settings with `Io::set_config()`. This call is unsafe: no reference borrowed from the old configuration may still be alive when you make it. The [I/O chapter](./7-io.md) explains why. The complete example puts all of this together with an OpenSSL acceptor and a web application that serves static files for ordinary HTTP requests. Its WebSocket loop echoes text and binary frames, answers pings, and uses pongs for its heartbeat. It rejects fragmented messages instead of assembling them. The example's README lists the server and client commands and explains how to set up the test certificate. ## Clients with `ntex::ws` To connect to a WebSocket server, create a [`WsClient`] and call `connect().await`. Client settings go in a [`WsClientConfig`] inside a `SharedCfg`. When the handshake succeeds, you get a [`WsConnection`], which holds the server's response and a sink for sending messages. This client connects to the echo server above, sends `"hello"`, prints the reply, and closes the connection: ```rust,no_run use std::io; use futures::StreamExt; use ntex::{SharedCfg, time::Seconds, ws}; #[ntex::main] async fn main() -> io::Result<()> { let cfg = SharedCfg::new("WS-CLIENT").add( ws::WsClientConfig::new() .set_max_frame_size(128 * 1024) .set_handshake_timeout(Seconds(5)) .set_close_timeout(Seconds(5)), ); let connection = ws::WsClient::new("ws://127.0.0.1:8080/ws/", cfg) .connect() .await .map_err(io::Error::other)?; println!("Handshake: {}", connection.response().status()); let sink = connection.sink(); let mut frames = connection.seal().receiver(); sink.send(ws::Message::Text("hello".into())) .await .map_err(io::Error::other)?; while let Some(frame) = frames.next().await { match frame.map_err(|err| io::Error::other(format!("{err:?}")))? { ws::Frame::Text(bytes) => { println!("Echo: {bytes:?}"); sink.send(ws::Message::Close(Some(ws::CloseCode::Normal.into()))) .await .map_err(io::Error::other)?; } ws::Frame::Ping(bytes) => { sink.send(ws::Message::Pong(bytes)) .await .map_err(io::Error::other)?; } ws::Frame::Close(_) => break, _ => {} } } Ok(()) } ``` The client reads frames through `receiver()`, using `futures::StreamExt` just like the [basic WebSocket client]. `seal()` erases the concrete I/O filter type. `receiver()` then starts the dispatcher in a background task and returns a channel of frames and errors. The receiver replies to the server's close frame for you, and dropping the receiver starts the closing handshake. Pings are different: they still show up in your loop, and you have to send the pong yourself. If you'd rather use a frame service than a channel, call `connection.seal().start(service).await`. The service works like the server-side one, except that its state is `()` rather than a sink. It still returns `Option`, and it has to reply to both pings and close frames. Be sure to await or spawn `start()`; holding on to the connection or a sink doesn't run the dispatcher. ### Configuration and TLS `WsClientConfig` is where you offer subprotocols, add headers such as `Origin` or authentication, limit frame sizes, and set the handshake and close timeouts. The handshake timeout starts after the connection is established and covers only the HTTP exchange. If you need a deadline for the whole operation, wrap it in your own timeout. If the server selects a subprotocol, the client checks that it's valid and was one of the offers. A server may also accept without selecting one. If your application depends on a particular subprotocol, check the response before sending any data. For `wss://`, enable the matching Cargo feature and configure a TLS connector with `WsClient::openssl()` or `WsClient::rustls()`. The [low-level TLS client] shows an OpenSSL setup that works with a local self-signed certificate. It turns off certificate verification for the demo; a real client must verify the server's certificate and hostname. With the `compress` feature, `set_deflate(DeflateConfig::new())` offers `permessage-deflate` compression. The server can accept or decline the offer, so turning it on doesn't guarantee a compressed connection. ## WebSocket as a byte transport Sometimes you don't care about WebSocket messages at all; you just want to run another protocol over the connection. MQTT, for example, already knows how to decode its packets, acknowledge messages, and keep the connection alive. It shouldn't also have to deal with WebSocket frames. [`WsTransport`] keeps the two apart. It's an I/O filter that strips binary WebSocket framing from incoming data and adds it to outgoing data: ```text socket <-> optional TLS <-> WebSocket filter <-> MQTT or another byte protocol ``` The protocol on top sees the bytes from binary messages and their continuation fragments. Text messages are rejected. Unlike the frame services above, the filter answers pings itself, ignores pongs, and takes part in the closing handshake when the I/O stream shuts down. **The result is a byte stream, not a stream of messages.** One protocol packet can be spread over several WebSocket messages, and one WebSocket message can contain several packets. The protocol's own decoder sorts that out. If you need to know where each WebSocket message begins and ends, use the frame API instead. ### Installing the filter On the server, send the `101` upgrade response first, then add the filter: ```rust let io = ntex::ws::WsTransport::create(io, ntex::ws::Codec::new()); ``` You can pass the resulting `Io>` straight to a protocol service, or call `io.boxed()` if the service expects type-erased I/O. To add the filter as a step in a service pipeline, use [`WsTransportService`] instead: `WsTransportService::new(ws::Codec::new())`. Either way, the HTTP handshake must already be done. On the client, convert the connection returned by `WsClient`: ```rust let io = connection.into_transport(); ``` This keeps the client's codec, along with its masking mode and any compression negotiated during the handshake. A fresh codec would use the server's masking rules, which is wrong for a client. From here on, the inner protocol owns the stream. Don't keep using an old frame sink alongside it; that would mix two different ways of writing to the same connection. The [I/O filter subsystem](./7-io.md#filter-subsystem) section explains how filters work with TLS, buffering, backpressure, and shutdown. ### MQTT over WebSocket The [MQTT-over-WebSocket example] shows all of this in practice, with a server and a client built on `ntex-mqtt`. Both speak MQTT 3.1.1 and use the standard `mqtt` subprotocol. The server accepts upgrades only on `/mqtt`, and only from clients that offer `mqtt`. After checking the handshake, it selects `mqtt` in its `101` response, stops the HTTP timer, and installs `WsTransport`. It then hands the boxed stream to `v3::MqttServer`, which handles CONNECT and incoming publishes. The server is deliberately minimal: it receives publishes, but it isn't a broker, so it doesn't manage subscriptions or forward messages. The client offers `mqtt` with `WsClientConfig::set_protocols()` and checks that the server picked it. A custom connector for `v3::client::MqttConnector` returns `connection.into_transport()` instead of a plain socket. As far as the MQTT client is concerned, this is just another stream: it sends CONNECT and exchanges packets as usual. When you run the client, it sends a QoS 1 publish and waits for the acknowledgement. The MQTT dispatcher runs at the same time so it can read that acknowledgement. The client then closes the MQTT session and lets the transport shut down. Keepalive is still MQTT's job; WebSocket ping/pong is a separate mechanism and doesn't replace it. ### AMQP over WebSocket The same approach works for AMQP 1.0, whether you're serving `ntex-amqp` or connecting through a custom client connector. The [AMQP WebSocket binding] uses `amqp` as its subprotocol, and both sides should check that it was negotiated. The binding has one rule to watch for at the start of the connection: each opening AMQP protocol header must travel in a WebSocket message of its own. That includes the SASL header, if SASL is used. After the headers, AMQP frames can span messages or share them freely. `WsTransport` has no idea where an AMQP protocol header begins or ends, so the filter alone doesn't enforce this rule. Your AMQP handshake integration has to keep each header in its own message, and you should test it against the peer you plan to use. Everything else, including authentication, sessions, links, idle timeouts, and orderly shutdown, is still handled by the AMQP implementation. If you need TLS, use `wss://` for the WebSocket connection rather than negotiating TLS inside the AMQP stream. [`web::ws`]: https://docs.ntex.rs/api/ntex/web/ws/index.html [`HttpService`]: https://docs.ntex.rs/api/ntex/http/struct.HttpService.html [`ntex::ws`]: https://docs.ntex.rs/api/ntex/ws/index.html [`ws::WsClient`]: https://docs.ntex.rs/api/ntex/ws/struct.WsClient.html [`ws::WsTransport`]: https://docs.ntex.rs/api/ntex/ws/struct.WsTransport.html [`Frame`]: https://docs.ntex.rs/api/ntex/ws/enum.Frame.html [`Message`]: https://docs.ntex.rs/api/ntex/ws/enum.Message.html [`web::ws::start`]: https://docs.ntex.rs/api/ntex/web/ws/fn.start.html [`web::ws::start_with`]: https://docs.ntex.rs/api/ntex/web/ws/fn.start_with.html [`web::ws::subprotocols`]: https://docs.ntex.rs/api/ntex/web/ws/fn.subprotocols.html [`WsSink`]: https://docs.ntex.rs/api/ntex/ws/struct.WsSink.html [`WsError`]: https://docs.ntex.rs/api/ntex/ws/error/enum.WsError.html [`h1::Control::Upgrade`]: https://docs.ntex.rs/api/ntex/http/h1/control/enum.Control.html#variant.Upgrade [`ws::handshake`]: https://docs.ntex.rs/api/ntex/ws/fn.handshake.html [`WsClient`]: https://docs.ntex.rs/api/ntex/ws/struct.WsClient.html [`WsClientConfig`]: https://docs.ntex.rs/api/ntex/ws/struct.WsClientConfig.html [`WsConnection`]: https://docs.ntex.rs/api/ntex/ws/struct.WsConnection.html [`WsTransport`]: https://docs.ntex.rs/api/ntex/ws/struct.WsTransport.html [`WsTransportService`]: https://docs.ntex.rs/api/ntex/ws/struct.WsTransportService.html [basic WebSocket example]: https://github.com/ntex-rs/examples/tree/main/websockets/basic [basic WebSocket client]: https://github.com/ntex-rs/examples/blob/main/websockets/basic/src/client.rs [low-level WebSocket example]: https://github.com/ntex-rs/examples/tree/main/websockets/lowlevel [low-level TLS client]: https://github.com/ntex-rs/examples/blob/main/websockets/lowlevel/src/client.rs [MQTT-over-WebSocket example]: https://github.com/ntex-rs/examples/tree/main/protocols/mqtt [AMQP WebSocket binding]: https://docs.oasis-open.org/amqp-bindmap/amqp-wsb/v1.0/cs01/amqp-wsb-v1.0-cs01.html # Making HTTP Requests Very few services live alone. Sooner or later yours will need to talk to a payment provider, an internal API, an object store, or someone's webhook. `ntex::client` is the tool for that job. If you have built a server with ntex, the client will feel familiar. It is made of the same pieces: [services](1-service.md), the [I/O layer](7-io.md), and shared configuration. It speaks HTTP/1.1 and HTTP/2, keeps connections open and reuses them, and supports TLS through OpenSSL or rustls. Optional features add response decompression and cookies, and middleware lets you hook your own logic into every request. Making a request comes down to three steps: build it, send it, and read what comes back. ```rust,no_run use ntex::client::Client; #[ntex::main] async fn main() -> Result<(), Box> { let client = Client::new(); let response = client .get("http://127.0.0.1:8080/") .header("user-agent", "ntex-guide") .send() .await?; println!("status: {}", response.status()); let body = response.body().await?; println!("body: {}", String::from_utf8_lossy(&body)); Ok(()) } ``` That is all you need for simple cases. The rest of this chapter looks at each step more closely, starting with what happens inside the client. ## How the client works A `Client` is not meant to be created per request. It owns connection pools and configuration, and its value comes from reusing them. An application usually creates one client for each kind of outgoing traffic, such as one for the payment provider and another for an internal API, each with its own timeouts and limits. It then keeps those clients for its whole lifetime. A `Client` is a lightweight handle. Cloning it is cheap, and all clones share the same connection pools, so you can hand copies to every part of your code that needs one. The handle is not `Send`, though: it belongs to the thread that created it. In a server, the usual pattern is to create the clients once per worker, for example in the application factory, and keep them in application state. When you send a request, it passes through a short chain of services: ```text ClientRequest ─► middleware ─► Sender ─► Connector ─┬─► plain pool ─► HTTP/1.1 connections └─► TLS pool ─► HTTP/1.1 or HTTP/2 connections ``` It starts as a `ClientRequest`, the builder you get from `client.get(...)` and friends, which collects the method, URL, headers, and body. Next comes any [middleware](#middleware) you installed; it can inspect or change the request on its way out and the response on its way back. Then the `Sender` takes over. It asks the `Connector` for a connection, writes the request, and waits for the response head. The `Connector` chooses a pool based on the URL scheme: `http` goes to the plain pool, `https` to the TLS pool. Each pool keeps connections grouped by host. An idle HTTP/1.1 connection is reused for the next request to the same host. An HTTP/2 connection can carry many requests at once, so concurrent requests to the same host share it. You don't pick the HTTP version yourself. For `https` URLs, client and server settle it during the TLS handshake using ALPN: the client offers `h2` and `http/1.1`, and the server chooses. Plain `http` URLs always use HTTP/1.1. One detail is worth remembering. An HTTP/1.1 connection goes back to the pool only after its response body has been read to the end. If you drop a response without reading the body, the client can't tell what is left on the wire, so it closes the connection instead. If you want connections reused, read the body, even when you don't care what's in it. You don't have to shut the client down explicitly. Once the last clone and every request made with it are dropped, the pools close idle connections in the background, and HTTP/2 connections close gracefully after their in-flight requests finish. The client is for requests and responses. WebSockets have their own client, `ntex::ws::WsClient`, covered in [WebSocket Support](8.6-web-app.md#clients-with-ntexws). ## Configuration `Client::new()` uses sensible defaults, and for a quick script they are fine. For a real service you will want to tune at least the timeouts. Configuration works the same way as on an ntex server. You put configuration objects into a [`SharedCfg`] and build the client from it. A single `SharedCfg` can hold several configuration types, and each part of the client picks out the one it cares about: | Configuration | Read by | What it controls | |---|---|---| | [`ClientConfig`] | The client | Timeouts, payload limits, pool sizes, default headers | | [`IoConfig`] | Connector and connections | Connect timeout, buffer sizes, backpressure, shutdown timeout | | [`TlsConfig`] | TLS connectors | TLS handshake timeout | | [`HttpServiceConfig`] | HTTP/1.1 response parser | Maximum number of headers and size of the response head | | [`Http2ServiceConfig`] | HTTP/2 connections | Window sizes, frame size, ping and handshake timeouts | Anything you leave out keeps its defaults. The name you give the `SharedCfg` shows up in logs, so choose one that tells you which client is talking. ### Client settings Most of the settings you will reach for live in `ClientConfig`: ```rust use ntex::client::{Client, ClientConfig}; use ntex::http::error::HttpError; use ntex::{SharedCfg, time::Seconds}; fn api_client(token: &str) -> Result { let cfg = SharedCfg::new("API-CLIENT").add( ClientConfig::new() .set_response_timeout(Seconds(10)) .set_response_payload_limit(4 * 1024 * 1024) .set_h1_connection_limit(32) .set_header("user-agent", "my-service/1.0")? .set_bearer_auth(token)?, ); Ok(Client::with_config(cfg)) } ``` Here is the full list, with defaults: | Setting | Default | Meaning | |---|---|---| | `set_response_timeout` | 5 s | Time to wait for the response head after the request is sent | | `set_response_payload_limit` | 256 KiB | Largest body that `body()` and `json()` will buffer; 0 means no limit | | `set_response_payload_timeout` | 10 s | Time allowed for `body()` and `json()` to read the whole body | | `set_h1_connection_limit` | 8 | HTTP/1.1 connections per pool, shared by all hosts | | `set_h1_keepalive` | 15 s | How long an idle HTTP/1.1 connection may wait in the pool | | `set_h1_lifetime` | 75 s | Maximum age of a pooled HTTP/1.1 connection | | `set_h2_connection_limit` | 16 | HTTP/2 connections per host | | `set_h2_max_streams` | 100 | Concurrent requests per HTTP/2 connection | | `set_h2_keepalive` | 60 s | How long an idle HTTP/2 connection is kept | | `set_h2_lifetime` | 1 h | Maximum age of an HTTP/2 connection | Connection limits don't make requests fail. When every connection is busy, new requests simply wait their turn. The plain and TLS pools count their connections separately. `set_header`, `set_basic_auth`, and `set_bearer_auth` add headers to every request the client sends. If a request sets a header with the same name, the request wins. These methods return a `Result` because a header name or value can be invalid, which is why the example uses `?`. ### I/O settings Client connections are ordinary ntex I/O objects, so everything in [`IoConfig`] from the [I/O chapter](7-io.md) applies to them. The setting you are most likely to need is the connect timeout. It is off by default, which means a request to an unreachable host waits for as long as the operating system is willing to try. In production, set one: ```rust use ntex::client::{Client, ClientConfig}; use ntex::io::IoConfig; use ntex::{SharedCfg, time::Seconds}; fn client() -> Client { let cfg = SharedCfg::new("API-CLIENT") .add(ClientConfig::new().set_response_timeout(Seconds(10))) .add(IoConfig::new().set_connect_timeout(Seconds(3))); Client::with_config(cfg) } ``` With that, every stage of a request is covered by a timeout. A request moves through four stages, and each one has its own limit: 1. **Connecting**, including the DNS lookup: `IoConfig::set_connect_timeout`, off by default. 2. **The TLS handshake**: `TlsConfig::set_handshake_timeout`, 5 seconds. 3. **Waiting for the response head**: `ClientConfig::set_response_timeout`, 5 seconds, or `timeout()` on an individual request. 4. **Reading the body** with `body()` or `json()`: `ClientConfig::set_response_payload_timeout`, 10 seconds, or `timeout()` on the body future. ### Secure connections The client keeps two connectors, one for each kind of URL. `http` URLs go through the plain connector, which opens a TCP connection. `https` URLs go through the secure connector, which opens a TCP connection and then performs the TLS handshake on it. The builder lets you replace each one separately: `connector()` sets the plain connector and `secure_connector()` the secure one. You rarely need to set the secure connector by hand. TLS comes from the `openssl` or `rustls` feature of `ntex`. Enable one, and `Client::new()` and `Client::builder()` install a secure connector for you. What that connector does depends on the backend: - **rustls** checks server certificates against the webpki root certificates, as you would expect. - **OpenSSL** does **not** verify certificates by default. That is handy when testing against a server with a self-signed certificate, but it is not safe in production. Give the client your own configuration instead. If both features are enabled, OpenSSL wins. If neither is, `https` requests fail with `ConnectError::SslIsNotSupported`. To use your own TLS configuration, hand it to the builder; `.rustls()` and `.openssl()` build a secure connector from it. The builder uses it exactly as given, so if you want HTTP/2, set the ALPN protocols yourself: ```rust use ntex::{SharedCfg, client::Client}; use rustls::{ClientConfig, RootCertStore}; fn https_client() -> Client { let roots = RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned()); let mut tls = ClientConfig::builder() .with_root_certificates(roots) .with_no_client_auth(); tls.alpn_protocols = vec![b"h2".to_vec(), b"http/1.1".to_vec()]; Client::builder().rustls(tls).build(SharedCfg::new("HTTPS-CLIENT")) } ``` OpenSSL works the same way: build an `openssl::ssl::SslConnector`, set ALPN with `set_alpn_protos(b"\x02h2\x08http/1.1")`, and pass it to `.openssl()`. `.rustls()` and `.openssl()` are shortcuts. Each wraps your configuration in one of ntex's TLS connectors, `ntex::connect::rustls::TlsConnector` or `ntex::connect::openssl::SslConnector`, and passes it to `secure_connector()`. You can call `secure_connector()` directly with any connector you like. It might be an OpenSSL or rustls connector set up differently, or one built on another TLS implementation entirely, such as Schannel on Windows or Secure Transport on macOS. The client doesn't care how the connection is secured. A connector is a service that takes a `ntex::connect::Connect` holding the target URL and returns an I/O object. If that I/O object reports HTTP/2 as its negotiated protocol, the client speaks HTTP/2 over it. `connector()` takes the same kind of service for plain `http` URLs. Use it when connections need something other than a direct TCP connection, for example a tunnel through a proxy or a custom transport. For certificate verification, private roots, and client certificates, see the [TLS chapter](10-tls.md#http-client). ## Sending requests `Client` has a method for each common HTTP verb: `get`, `post`, `put`, `patch`, `delete`, `head`, `options`, and `query`. For anything else there is `request(method, url)`. Each returns a [`ClientRequest`] builder. URLs must be absolute: an `http` or `https` scheme plus a host. You may notice that the builder methods never return errors. If something is wrong, such as a bad URL, an invalid header, or a query that won't serialize, the builder remembers the error and returns it when you send the request. That keeps the builder chain free of `?`. ### Building the request Here is a request that uses a few of the builder's features: ```rust,no_run use serde::Serialize; use ntex::client::Client; use ntex::time::Seconds; #[derive(Serialize)] struct Search<'a> { q: &'a str, page: u32, } async fn search(client: &Client, token: Option<&str>) -> Result<(), Box> { let response = client .get("https://api.example.com/search") .query(&Search { q: "ntex", page: 2 }) .header("accept", "application/json") .if_some(token, |token, req| req.bearer_auth(token)) .timeout(Seconds(2)) .send() .await?; println!("{}", response.status()); Ok(()) } ``` `query()` serializes any `serde::Serialize` value into the query string, replacing whatever was there. `if_some()` applies a change only when an `Option` has a value, here adding a token only when there is one. Its sibling `if_true()` does the same for a `bool`. Both let you make a request conditional without breaking up the chain. `timeout()` overrides the client's response timeout for this one request; a zero duration turns the timeout off. For headers you have three choices. `header()` appends a value, `set_header()` replaces any existing value, and `set_header_if_none()` sets the header only if it isn't there yet. `content_type()` and `content_length()` are shortcuts for the two most common ones. For authentication there are `basic_auth(user, password)` and `bearer_auth(token)`, and with the `cookie` feature, `cookie()` attaches a cookie. A few methods control the connection itself. `force_close()` closes an HTTP/1.1 connection after the response instead of returning it to the pool. `address()` connects to a specific socket address and skips DNS. The URL still decides the `Host` header and the TLS server name, which makes this useful for testing a particular backend behind a load balancer. With the `compress` feature, the client asks servers for compressed responses and decompresses them for you. `no_decompress()` turns that off for a single request, which is usually what a proxy wants. Proxies will also like `client.request_from(url, head)`, which starts a request with the method and headers copied from an incoming one. ### Sending a body You choose how to send the body by choosing which `send` method to call: | Method | Body | |---|---| | `send()` | No body | | `send_body(body)` | Anything that converts to `Body`: `&'static str`, `String`, `Bytes`, `Vec`, ... | | `send_json(&value)` | `value` serialized as JSON; sets `Content-Type: application/json` unless you set one | | `send_form(&value)` | `value` serialized as a URL-encoded form; sets the matching content type | | `send_stream(stream)` | A stream of `Result` chunks, sent as they arrive | The first four cover most needs: ```rust,no_run use serde::Serialize; use ntex::client::Client; #[derive(Serialize)] struct NewUser<'a> { name: &'a str, email: &'a str, } async fn send_bodies(client: &Client) -> Result<(), Box> { // Raw bytes or text client.post("http://127.0.0.1:8080/notes") .content_type("text/plain") .send_body("remember the milk") .await?; // JSON let user = NewUser { name: "ferris", email: "ferris@example.com" }; client.post("http://127.0.0.1:8080/users").send_json(&user).await?; // A URL-encoded form client.post("http://127.0.0.1:8080/login") .send_form(&[("user", "ferris"), ("password", "secret")]) .await?; Ok(()) } ``` `send_stream()` is for bodies that are large, or that are produced while the request is already underway. The body never has to sit in memory all at once. Any `Stream` of `Result` will do. In this example, a separate task produces the chunks and passes them through a channel: ```rust,no_run use ntex::{channel::mpsc, client::Client, util::Bytes}; async fn upload(client: &Client) -> Result<(), Box> { let (tx, rx) = mpsc::channel::>(); ntex::rt::spawn(async move { for i in 0..10 { let chunk = Bytes::from(format!("line {i}\n")); if tx.send(Ok(chunk)).is_err() { break; // the request was dropped } } }); let response = client .post("http://127.0.0.1:8080/upload") .send_stream(rx) .await?; println!("{}", response.status()); Ok(()) } ``` Because the total length isn't known up front, HTTP/1.1 sends the body with chunked transfer encoding. If the stream yields an error, the request fails. ### When something goes wrong Every `send` method returns `Result>`. `Error` dereferences to [`ClientError`], so you can match on what went wrong and react accordingly: ```rust,no_run use ntex::client::{Client, error::ClientError}; async fn fetch(client: &Client) { match client.get("http://127.0.0.1:8080/").send().await { Ok(response) => println!("status: {}", response.status()), Err(err) => match &*err { ClientError::Timeout => eprintln!("the server took too long to answer"), ClientError::Connect(e) => eprintln!("could not connect: {e}"), _ => eprintln!("request failed: {err}"), }, } } ``` Note that a 404 or a 500 is not an error here. From the client's point of view, the request worked: the server received it and answered. It's up to you to check `response.status()` and decide what that answer means. ## Reading responses `send()` completes as soon as the response head arrives. You can look at the status and headers right away, while the body may still be on its way. ```rust,no_run use ntex::client::ClientResponse; use ntex::http::{StatusCode, header}; fn inspect(response: &ClientResponse) { if response.status() == StatusCode::NOT_FOUND { println!("no such resource"); } println!("version: {:?}", response.version()); if let Some(value) = response.header(header::CONTENT_TYPE) { println!("content type: {value:?}"); } for (name, value) in response.headers() { println!("{name}: {value:?}"); } } ``` `ClientResponse` also implements `HttpMessage`, which brings helpers such as `content_type()` and, with the `cookie` feature, `cookies()`. How you read the body depends on how big it is and what you plan to do with it. ### Buffering the body If the body is reasonably small, read it all at once. `body()` collects it into `Bytes`, and `json::()` deserializes it with serde (it refuses responses whose content type isn't JSON). Both enforce the payload limit and payload timeout from `ClientConfig`, and you can adjust either for a single response: ```rust,no_run use serde::Deserialize; use ntex::client::Client; use ntex::time::Seconds; #[derive(Deserialize)] struct User { id: u64, name: String, } async fn read(client: &Client) -> Result<(), Box> { // Up to 10 MB, within 30 seconds let response = client.get("http://127.0.0.1:8080/report.csv").send().await?; let report = response.body().limit(10 * 1024 * 1024).timeout(Seconds(30)).await?; println!("report: {} bytes", report.len()); // A JSON document let response = client.get("http://127.0.0.1:8080/users/1").send().await?; let user: User = response.json().await?; println!("user {}: {}", user.id, user.name); Ok(()) } ``` The limit is there to protect you. A misbehaving server could otherwise send you gigabytes, and you'd buffer every byte. If the response declares a `Content-Length` above the limit, the read fails right away without reading any of the body. ### Streaming the body For large downloads, or bodies with no fixed end, read the response chunk by chunk. `ClientResponse` is itself a `Stream`, and `ntex::util::stream_recv` waits for the next chunk: ```rust,no_run use ntex::{client::Client, util::stream_recv}; async fn download(client: &Client) -> Result> { let mut response = client.get("http://127.0.0.1:8080/large-file").send().await?; let mut total = 0; while let Some(chunk) = stream_recv(&mut response).await { let chunk = chunk?; total += chunk.len(); // write the chunk to a file, a socket, ... } Ok(total) } ``` Since you only hold one chunk at a time, the payload limit and payload timeout don't apply here. If you need them, enforce them in the loop. To pass the body on to something else, such as a response in a proxy, take it with `take_payload()`. ## Middleware Middleware wraps the client's service chain. Each request passes through it on the way out, and each response on the way back. It is the same idea as [web middleware](8.3-web-app.md#middleware), pointed in the other direction. ### Why use middleware Some things should happen on every request, no matter where in your code the request is made. Without middleware, you'd have to remember them at every call site. With middleware, you write them once: - **Authentication** that changes over time, such as an access token that a background task refreshes. - **Tracing**: adding a request ID or trace context to every call. - **Logging and metrics**: recording method, URL, status, and latency. - **Policy**: refusing to call hosts the service shouldn't talk to. - **Rewriting**: sending requests to a different host in tests, or adjusting response headers. You don't need middleware for headers that never change, such as a fixed `User-Agent` or a static API key. The default headers in `ClientConfig` handle those. ### How middleware works Middleware is installed with `ClientBuilder::middleware()`. The quickest way to write one is `fn_layer` from `ntex::service`, which turns an async closure into middleware. The closure gets the request and the next service in the chain, and returns whatever that service returns. This one adds an `Authorization` header, reading the token fresh for each request so that a refreshed token is picked up immediately: ```rust,no_run use std::{cell::RefCell, rc::Rc}; use ntex::client::{Client, ServiceRequest}; use ntex::http::header::{AUTHORIZATION, HeaderValue}; use ntex::{SharedCfg, service::fn_layer}; fn client(token: Rc>) -> Client { Client::builder() .middleware(fn_layer(async move |mut req: ServiceRequest, svc| { // Read the current token for every request let value = token.borrow().clone(); req.headers().insert(AUTHORIZATION, value); svc.call(req).await })) .build(SharedCfg::new("API-CLIENT")) } ``` Middleware doesn't see the friendly `ClientRequest` builder. It works with a [`ServiceRequest`], the request as it is about to be sent. `head()` gives you the method, URL, and headers. `headers()` is a separate set of headers that take precedence over the ones in the head, which makes it a convenient place to add or override a header. `body()` and `address()` give you the body and the target address. On the way back, middleware gets a [`ServiceResponse`]: the response before it becomes a `ClientResponse`. `head()` holds the status and headers, and `payload()` the body stream. Middleware sees the response as soon as its head arrives, before the body is read, so timing a request in middleware measures the time until the response head arrives, not the full download. Middleware can stop a request by returning an error instead of calling the next service. It can't invent a response, though; every successful response comes from a real server. You can stack several middleware. The first one you add is the outermost: it sees the request first and the response last. For middleware you want to reuse, or that needs its own readiness or shutdown logic, implement the [`Middleware`] trait and a service, just as you would on a server. The only differences are the types: the configuration is `SharedCfg` and the request is `ServiceRequest`. Here is middleware that logs how each request went: ```rust use std::time::Instant; use ntex::client::{ServiceRequest, ServiceResponse, error::ClientError}; use ntex::{Ctx, Middleware, Service, SharedCfg, error::Error}; /// Logs the method, URL, status, and latency of every request. pub struct Timing; pub struct TimingService { service: S, } impl Middleware for Timing { type Service = TimingService; fn create(&self, _: &SharedCfg, service: S) -> Self::Service { TimingService { service } } } impl Service for TimingService where S: Service>, { type Res = ServiceResponse; type Error = Error; ntex::forward!(SharedCfg, service); async fn call( &self, mut req: ServiceRequest, ctx: Ctx<'_, Self, SharedCfg>, ) -> Result { let method = req.head().method.clone(); let url = req.head().uri.to_string(); let start = Instant::now(); let mut result = ctx.call(&self.service, req).await; match &mut result { Ok(res) => log::info!("{method} {url} -> {} in {:?}", res.head().status, start.elapsed()), Err(err) => log::warn!("{method} {url} failed after {:?}: {err}", start.elapsed()), } result } } ``` Installing it looks just like the closure version: ```rust # use ntex::client::{ServiceRequest, ServiceResponse, error::ClientError}; # use ntex::{Ctx, Middleware, Service, SharedCfg, error::Error}; # pub struct Timing; # pub struct TimingService { service: S } # impl Middleware for Timing { # type Service = TimingService; # fn create(&self, _: &SharedCfg, service: S) -> Self::Service { TimingService { service } } # } # impl Service for TimingService # where S: Service>, # { # type Res = ServiceResponse; # type Error = Error; # ntex::forward!(SharedCfg, service); # async fn call(&self, req: ServiceRequest, ctx: Ctx<'_, Self, SharedCfg>) -> Result { # ctx.call(&self.service, req).await # } # } let client = ntex::client::Client::builder() .middleware(Timing) .build(SharedCfg::new("API-CLIENT")); ``` A word of caution about retries. It is tempting to write middleware that resends a failed request, but a request is consumed when it is sent, and a streaming body can only be sent once. There is no copy of `ServiceRequest` left to resend. Retries are usually easier in application code, where you can simply build the request again. ## Summary - Create one `Client` per kind of outgoing traffic, once per thread, and reuse it for the life of the application. Clones are cheap and share connection pools. - Tune the client with `ClientConfig`, `IoConfig`, and friends in a `SharedCfg`. Always set a connect timeout. - With OpenSSL, provide your own TLS configuration so certificates are verified, and set ALPN if you want HTTP/2. - Pick the body format by picking the method: `send`, `send_body`, `send_json`, `send_form`, or `send_stream`. - Buffer small bodies with `body()` or `json()`, and stream large ones. Read bodies to the end so connections can be reused. - Reach for middleware when something must happen on every request: authentication, tracing, logging, metrics, or policy. To see a complete program, look at the [HTTP client example] and the [HTTP proxy example]. [`SharedCfg`]: https://docs.ntex.rs/api/ntex/service/cfg/struct.SharedCfg.html [`ClientConfig`]: https://docs.ntex.rs/api/ntex/client/struct.ClientConfig.html [`IoConfig`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html [`TlsConfig`]: https://docs.ntex.rs/api/ntex/tls/struct.TlsConfig.html [`HttpServiceConfig`]: https://docs.ntex.rs/api/ntex/http/struct.HttpServiceConfig.html [`Http2ServiceConfig`]: https://docs.ntex.rs/api/ntex/http/h2/struct.Http2ServiceConfig.html [`ClientRequest`]: https://docs.ntex.rs/api/ntex/client/struct.ClientRequest.html [`ClientError`]: https://docs.ntex.rs/api/ntex/client/error/enum.ClientError.html [`ServiceRequest`]: https://docs.ntex.rs/api/ntex/client/struct.ServiceRequest.html [`ServiceResponse`]: https://docs.ntex.rs/api/ntex/client/struct.ServiceResponse.html [`Middleware`]: https://docs.ntex.rs/api/ntex/service/trait.Middleware.html [HTTP client example]: https://github.com/ntex-rs/examples/tree/main/extras/http-client [HTTP proxy example]: https://github.com/ntex-rs/examples/tree/main/extras/http-proxy # HTTP/2 with ntex-h2 HTTP/2 in ntex lives in a separate crate, [ntex-h2]. `ntex::http` and `ntex::web` use it behind the scenes: when a connection negotiates HTTP/2, ntex hands it to ntex-h2, which takes care of frames, header compression, streams and flow control, and ntex turns each stream back into the familiar requests and responses. Most applications never notice the crate is there. Sometimes, though, you want HTTP/2 itself rather than the general HTTP layer on top of it. Perhaps you only ever speak HTTP/2, you need trailers or long-lived bidirectional streams, or you want to control flow control yourself. gRPC is the classic case: [ntex-grpc](11-grpc.md) uses ntex-h2 directly for both its client and its server, and you can do the same. ntex-h2 gives you three building blocks: - **`client::Client`**, a connection pool that opens and reuses connections to one server. - **`client::SimpleClient`**, a client for a single connection that you manage yourself. - **`server::Server`**, an ntex service that handles HTTP/2 connections. ntex-h2 speaks only HTTP/2 and does not negotiate the protocol, so any transport you give it must already be committed to HTTP/2. Over TLS, that means ALPN has selected `h2`. Over plain TCP (h2c), both sides must simply know in advance. The HTTP/1.1 `Upgrade` mechanism and server push are not supported. Here is a client that sends a `GET` request over plain TCP and prints the response: ```rust,no_run use ntex::SharedCfg; use ntex::http::{HeaderMap, Method}; use ntex_h2::{MessageKind, StreamEof, client::Client}; #[ntex::main] async fn main() -> Result<(), Box> { let client = Client::builder("127.0.0.1:8080").build(SharedCfg::new("H2-CLIENT")); let (_send, recv) = client .send(Method::GET, "/index.html".into(), HeaderMap::new(), true) .await?; while let Some(msg) = recv.recv().await { match msg.kind { MessageKind::Headers { pseudo, .. } => println!("status: {:?}", pseudo.status), MessageKind::Data(data, _) => println!("data: {data:?}"), MessageKind::Eof(StreamEof::Data(data, _)) => println!("last data: {data:?}"), MessageKind::Eof(_) | MessageKind::Disconnect(_) => break, } } Ok(()) } ``` This is a lower-level API than [`ntex::client`](13-client.md). There are no URLs, body helpers or `Response` type, only HTTP/2 streams and what travels on them. The rest of this chapter explains that model first, then covers the client and the server. ## Streams and messages An HTTP/2 connection carries many independent streams at once, and each request shares a stream with its response. The client opens a stream by sending request headers. Both sides then send data on it, and each closes its own half when it is done. Either side can also cut a stream short with a reset, which leaves the connection and its other streams untouched. In ntex-h2, a stream is represented by a `StreamRef`: a cheap, cloneable handle that identifies the stream and lets you send on it. Everything that arrives on a stream reaches you as a `Message`: ```rust,ignore pub struct Message { pub stream: StreamRef, pub kind: MessageKind, } ``` A client and a server see the same kinds of messages: | `MessageKind` | Meaning | |---|---| | `Headers { pseudo, headers, eof }` | A request or response head. `pseudo` holds the pseudo-headers: `method`, `scheme`, `authority`, and `path` on requests, and `status` on responses. If `eof` is `true`, there is no body. | | `Data(Bytes, Capacity)` | A chunk of the body. `Capacity` is the flow-control credit the chunk used, see [Flow control](#flow-control). | | `Eof(StreamEof)` | The peer's half of the stream is finished. It ends with `StreamEof::Data` (a last chunk), `StreamEof::Trailers` (trailing headers), or `StreamEof::Error` (the stream failed or was reset). | | `Disconnect(err)` | The connection failed while the stream was still open. | Messages always arrive in the same order: one `Headers`, then any number of `Data`, then exactly one `Eof` or `Disconnect`. If `Headers` has `eof` set, it is the only message. Sending works the same way. The headers go first, sent with `send` on a client or `send_response` on a server. Any number of `send_payload` calls follow, and the stream ends either with `eof` set on the last one or with `send_trailers`. ## Configuration Protocol settings live in `ServiceConfig`. As with other ntex configuration, you add it to a [`SharedCfg`], and every connection created with that `SharedCfg` picks it up: ```rust,no_run use ntex::{SharedCfg, time::Seconds}; use ntex_h2::ServiceConfig; fn config() -> SharedCfg { SharedCfg::new("H2") .add( ServiceConfig::new() .set_max_concurrent_streams(128) .set_initial_window_size(256 * 1024) .set_initial_connection_window_size(4 * 1024 * 1024) .set_ping_timeout(Seconds(30)), ) .into() } ``` `ServiceConfig` is the same type that `ntex::http` exposes as [`Http2ServiceConfig`], so the same settings apply to HTTP/2 in an ntex web server or HTTP client. ntex-h2 connections are ordinary ntex I/O objects, so an [`IoConfig`] in the same `SharedCfg` controls their buffers and backpressure, as described in the [I/O chapter](7-io.md). The settings fall into three groups. The first covers flow control and frames: | Setting | Default | Meaning | |---|---|---| | `set_initial_window_size` | 64 KiB − 1 | Receive window of each stream: how much body data the peer may send on a stream before it must wait | | `set_initial_connection_window_size` | 4 MiB | Receive window shared by all streams of a connection | | `set_max_frame_size` | 16 KiB | Largest frame payload the peer may send. Allowed range is 16 KiB to 16 MiB − 1 | The second limits what a peer can make you do: | Setting | Default | Meaning | |---|---|---| | `set_max_concurrent_streams` | 256 | Streams the peer may have open at once. Further streams are refused with `REFUSED_STREAM` | | `set_max_headers` | 96 | Header fields allowed in one header block. A stream with more is refused | | `set_max_header_list_size` | 48 KiB | Total size of a header block | | `set_max_header_continuation_frames` | 5 | `CONTINUATION` frames allowed for one header block | | `set_max_concurrent_reset_streams` | 32 | Recently reset streams remembered so late frames for them are ignored; 0 disables the history | | `set_reset_stream_duration` | 30 s | How long a reset stream is remembered | | `set_max_inflight_messages` | 8192 | Messages waiting for your code. When the limit is reached, the connection stops reading until some are handled | The third sets timeouts: | Setting | Default | Meaning | |---|---|---| | `set_handshake_timeout` | 5 s | Time allowed for the connection preface | | `set_settings_timeout` | 5 s | Time the peer has to acknowledge a `SETTINGS` frame | | `set_ping_timeout` | 10 s | Keep-alive interval. If a `PING` is not answered in time, the connection is closed | | `set_capacity_timeout` | 5 s | Longest a sender waits for flow-control window before the stream is reset; 0 disables it | Stream limits apply only to streams the peer opens. A server uses `set_max_concurrent_streams` to cap how many requests a client may run at once, while a client is bound by whatever limit the server announces. ntex-h2 also guards against rapid-reset attacks. If at least ten streams have been opened on a connection and half of them were reset, the connection is closed with `GOAWAY`. ## The client ### Connecting with a pool For most programs, `client::Client` is the right choice. It is a pool of connections to one server: each request goes to a connection that can open another stream, and a new connection is opened only when none can. Like the [HTTP client](13-client.md), create it once and keep it for the life of the application. Clones are cheap and share the same connections. You create a pool with a builder, and `build` takes the `SharedCfg` for every connection in the pool: ```rust,no_run use ntex::{SharedCfg, time::Seconds}; use ntex_h2::{ServiceConfig, client::Client}; fn backend() -> Client { let cfg = SharedCfg::new("BACKEND").add(ServiceConfig::new().set_ping_timeout(Seconds(30))); Client::builder("backend.internal:8080") .connect_timeout(Seconds(3)) .connection_limit(4) .lifetime(Seconds(600)) .build(cfg) } ``` The builder has these options: | Option | Default | Meaning | |---|---|---| | `connector` | TCP | The transport connector, for example a TLS connector | | `secure` | `false` | Sets the `:scheme` of requests to `https` instead of `http` | | `connect_timeout` | 1 s | Time allowed to connect, including the TLS handshake | | `minconn` | 1 | Below this many connections, the pool opens a new one for a request even if an existing connection has room | | `connection_limit` | 16 | Maximum number of connections | | `max_streams` | 100 | Streams a connection is assumed to accept until the server announces its own limit | | `lifetime` | not set | Age after which a connection is closed gracefully and replaced | | `disconnect_timeout` | 15 s | Time a closing connection has to finish its streams | | `skip_unknown_streams` | off | Ignore frames for streams the client no longer knows instead of treating them as errors | The pool opens new connections one at a time and spreads requests across them, preferring connections that use less than half of their stream limit. When every connection is full and the pool is at its limit, `send` waits for a stream to free up instead of failing. Errors that happen before a stream is opened, such as a failed connect or a timeout, are reported as `ClientError`. ### Secure connections Over TLS, the pool needs two things: a TLS connector that offers `h2` through ALPN, and `secure(true)` so that requests carry the `https` scheme. With OpenSSL: ```rust,no_run use ntex::SharedCfg; use ntex_h2::client::Client; use openssl::ssl::{SslConnector, SslMethod}; fn secure_client() -> Result { let mut builder = SslConnector::builder(SslMethod::tls())?; builder.set_alpn_protos(b"\x02h2")?; Ok(Client::builder("api.example.com:443") .connector(ntex::tls::openssl::SslConnector::new(builder.build())) .secure(true) .build(SharedCfg::new("API"))) } ``` The ALPN list `b"\x02h2"` is the protocol name `h2` prefixed with its length. Unlike `ntex::client`, ntex-h2 cannot fall back to HTTP/1.1, so if you also offer `http/1.1`, make sure the server will pick `h2`. With rustls, set `alpn_protocols` to `vec![b"h2".to_vec()]` in its `ClientConfig`. ### A single connection Sometimes you want to manage connections yourself, for example to pin a session to one connection, or to run HTTP/2 over a transport you created. `client::SimpleClient` wraps a single, already established connection: ```rust,no_run use ntex::{SharedCfg, io::Io, util::ByteString}; use ntex_h2::client::SimpleClient; async fn connect() -> Result> { let cfg: SharedCfg = SharedCfg::new("H2").into(); let io: Io = ntex::connect::connect_with("127.0.0.1:8080", &cfg).await?; Ok(SimpleClient::new(io, false, ByteString::from("127.0.0.1"))) } ``` The arguments are the transport, whether requests use the `https` scheme, and the `:authority` sent with each request. The connection reads its `ServiceConfig` from the transport's `SharedCfg`. If you'd rather not do these steps by hand, `client::Connector` is an ntex service that takes an address and returns a `SimpleClient`. `SimpleClient` is cheap to clone. The connection stays open while any clone exists and closes gracefully when the last one is dropped. You can also close it earlier: - `close()` starts a graceful shutdown. Open streams may finish, but no new ones can start. - `force_close()` drops the connection immediately. - `disconnect().await` closes gracefully and waits until the connection is gone. Use `.disconnect_timeout(..)` on the returned future to bound the wait. `is_closed()` and `on_disconnect()` tell you when the connection goes away, whether you closed it or the server did. A pool handles all of this for you, but you can still reach a pooled connection: `Client::client()` returns a `SimpleClient` from the pool that can open another stream. ### Sending a request `send` starts a request. It takes the method, the `:path` (including any query string), the headers and an `eof` flag, and returns the two halves of the new stream: a `SendStream` for the request body and a `RecvStream` for the response. ```rust,no_run use ntex::http::{HeaderMap, Method, header}; use ntex::util::Bytes; use ntex_h2::client::Client; async fn upload(client: &Client, body: Bytes) -> Result<(), Box> { let mut headers = HeaderMap::new(); headers.insert(header::CONTENT_TYPE, header::HeaderValue::from_static("application/octet-stream")); let (send, recv) = client.send(Method::POST, "/upload".into(), headers, false).await?; send.send_payload(body, true).await?; // ... read the response from `recv` # drop(recv); Ok(()) } ``` Set `eof` to `true` when the request has no body, as with most `GET` requests. Otherwise, send the body with `send_payload`, which accepts anything that converts to `Bytes`. Pass `eof: true` on the last call, or finish with `send_trailers(headers)` if the request has trailers. Since you can call `send_payload` as often as you like, streaming a large body is just a loop: ```rust,no_run # use ntex::util::Bytes; # use ntex_h2::client::SendStream; async fn stream_body( send: &SendStream, chunks: Vec, ) -> Result<(), ntex::error::Error> { for chunk in chunks { send.send_payload(chunk, false).await?; } send.send_payload(Bytes::new(), true).await } ``` `send_payload` returns once the data has been handed to the connection. It splits the data into frames that fit both the peer's maximum frame size and the flow-control window. If the window is closed, it waits for the peer to reopen it (see [Flow control](#flow-control)). It also waits while the connection's write buffer is full, so a fast sender cannot outrun the network. The request body and the response are independent: you can read the response while still sending, which is how bidirectional streaming protocols work. Just make sure only one task sends on a stream at a time. ### Reading the response `RecvStream` delivers the response as the messages described [earlier](#streams-and-messages). `recv().await` returns the next message, or `None` once the response is complete. It also implements `Stream`, if you prefer stream combinators. This function collects a whole response: ```rust,no_run use ntex::http::{HeaderMap, StatusCode}; use ntex::util::{Bytes, BytesMut}; use ntex_h2::{MessageKind, StreamEof, client::RecvStream}; async fn read_response( recv: RecvStream, ) -> Result<(StatusCode, HeaderMap, Bytes), Box> { let mut status = StatusCode::OK; let mut headers = HeaderMap::new(); let mut body = BytesMut::new(); while let Some(msg) = recv.recv().await { match msg.kind { MessageKind::Headers { pseudo, headers: hdrs, .. } => { status = pseudo.status.unwrap_or(StatusCode::OK); headers = hdrs; } MessageKind::Data(chunk, _capacity) => body.extend_from_slice(&chunk), MessageKind::Eof(StreamEof::Data(chunk, _capacity)) => { body.extend_from_slice(&chunk); break; } MessageKind::Eof(StreamEof::Trailers(_)) => break, MessageKind::Eof(StreamEof::Error(err)) => return Err(err.into()), MessageKind::Disconnect(err) => return Err(err.into()), } } Ok((status, headers, body.freeze())) } ``` Each `Data` message comes with a `Capacity`. Here it is dropped right away, which lets the server keep sending. Holding on to it is how you slow the server down, as [Flow control](#flow-control) explains. ### Resets and cancellation A stream ends early when either side resets it. A reset carries a `Reason` code from `ntex_h2::frame::Reason`, such as `CANCEL` or `INTERNAL_ERROR`. To give up on a request, just drop it. Dropping the `RecvStream` before the response is complete, or the `SendStream` before the request body is complete, resets the stream with `CANCEL`, and the server can stop working on it. As a result, a timeout around a request needs no extra code: ```rust,no_run # use ntex_h2::client::RecvStream; # async fn read_response(_: RecvStream) -> Result<(), Box> { Ok(()) } # async fn example(recv: RecvStream) { use ntex::time::{Seconds, timeout}; // If the timeout fires, `recv` is dropped and the stream is reset. let result = timeout(Seconds(10), read_response(recv)).await; # } ``` To reset with a specific reason, call `reset(reason)` on the `SendStream` or the `StreamRef`. `is_reset()` and `on_reset()` tell you whether either side has reset the stream. When the server resets a stream, you receive `Eof(StreamEof::Error(..))` and further sends fail. When the whole connection fails, every open stream gets a `Disconnect` message instead. ### Stream limits A server announces how many streams it accepts at once on each connection. When that limit is reached, `SimpleClient::send` waits until a stream finishes, while the pool uses another connection or opens a new one. If you want to check before sending, `SimpleClient` gives you the numbers: `max_streams()` is the server's limit and `active_streams()` is how many are in use. `is_ready()` tells you whether another stream can be opened, and `ready().await` waits until one can. `on_capacity(callback)` registers a function that runs whenever a stream is released. Between checking and sending, another task may grab the free stream. `reserve()` closes that gap: it claims a stream slot immediately and returns a `StreamReservation`, or `None` if the connection is full. The reservation's `send` then opens the stream without waiting. Dropping an unused reservation releases the slot. ## Flow control HTTP/2 runs many streams over one TCP connection, so TCP's own flow control is not enough: one slow consumer must not stall every other stream. HTTP/2 adds credit-based flow control on two levels. Every stream has a window, and the connection has one more, shared by all streams. A window is how many bytes of body data the sender may still send. Each `DATA` frame uses up part of both windows, and when either reaches zero, the sender must stop until the receiver grants more credit with a `WINDOW_UPDATE` frame. Only body data counts, not headers. ### Receiving data In ntex-h2, every chunk of body data you receive comes with a `Capacity`, which stands for the window credit that chunk used up. As long as you hold the `Capacity`, the credit stays used. When you drop it, or call `consume(n)` to release part of it, the credit goes back to both the stream's and the connection's window. ntex-h2 does not send a `WINDOW_UPDATE` for every released byte. It waits until enough credit has built up: a third of `initial_window_size` for a stream, and a quarter of `initial_connection_window_size` for the connection. With the default stream window of 64 KiB, it goes like this: 1. The server sends 64 KiB on a stream. The stream window drops to zero, and the server must wait. 2. Your code receives the data as several `Data` messages, each carrying its own `Capacity`. 3. You process the chunks and drop their capacities. Once about 22 KiB has been released, ntex-h2 sends a `WINDOW_UPDATE` granting that credit back, and the server resumes. 4. If you hold on to the capacities, for example because the data is waiting in a queue for a slow consumer, no update is sent and the server stays paused. That last step is what makes `Capacity` useful. It ties the sender's speed to how fast your code actually processes data, rather than how fast bytes arrive from the network. Keep the `Capacity` next to the data while it waits, and drop it only once the data has been handled: ```rust,no_run use ntex::channel::mpsc; use ntex::util::Bytes; use ntex_h2::{Capacity, MessageKind, StreamEof, client::RecvStream}; // Forward the body to a consumer task. The server can only send as fast // as the consumer drops the capacities. async fn forward(recv: RecvStream, tx: mpsc::Sender<(Bytes, Capacity)>) { while let Some(msg) = recv.recv().await { match msg.kind { MessageKind::Data(data, cap) => { if tx.send((data, cap)).is_err() { break; // consumer is gone, dropping `recv` resets the stream } } MessageKind::Eof(StreamEof::Data(data, cap)) => { let _ = tx.send((data, cap)); break; } MessageKind::Headers { .. } => {} MessageKind::Eof(_) | MessageKind::Disconnect(_) => break, } } } ``` Unreleased data also counts against the connection window, which every stream shares, so a stream that buffers a lot without releasing it can slow down the others. That is why the connection window is much larger by default (4 MiB against 64 KiB per stream). Size the two together: the stream window limits how much a single request can buffer, and the connection window limits the total across all requests. Two smaller details: - Capacities from the same stream can be combined with `+` or `+=`, which is handy when you merge several chunks into one buffer. Adding capacities from different streams panics. - `StreamRef::empty_capacity()` creates a zero-sized `Capacity` for a stream, a convenient starting value to add chunks to. If a peer ignores flow control and sends more than the window allows, the stream fails with `StreamError::RecvWindowExceeded`. ### Sending data When sending, ntex-h2 tracks the window the peer has granted. `send_payload` sends as much as the window allows, waits for more credit, and carries on until all the data is out. Usually that is all you need. If you produce data on demand, though, it is better to ask first how much the peer will accept, so you don't generate data that would only sit in memory: - `available_send_capacity()` returns the current window: the smaller of the stream's and the connection's window. - `send_capacity().await` waits until the window is open and returns its size. ```rust,no_run # use ntex::util::Bytes; # use ntex_h2::client::SendStream; # fn produce(_: usize) -> Option { None } async fn send_generated(send: &SendStream) -> Result<(), ntex::error::Error> { loop { let window = send.send_capacity().await? as usize; match produce(window) { Some(chunk) => send.send_payload(chunk, false).await?, None => return send.send_payload(Bytes::new(), true).await, } } } ``` A peer that never opens its window would leave a sender waiting forever. `set_capacity_timeout` prevents that: if a sender waits for capacity longer than the timeout (5 seconds by default), the stream is reset with `CANCEL` and the send fails with `StreamError::CapacityTimeout`. ## The server ### Running a server `server::Server` is an ntex service that takes an I/O object and serves HTTP/2 on it until the connection closes. You run it like any other ntex service, with `ntex::server::build()` (see the [server chapter](5-server.md)): ```rust,no_run use ntex::http::{HeaderMap, StatusCode}; use ntex::{SharedCfg, error::Error}; use ntex_h2::{Message, MessageKind, OperationError, ServiceConfig, server}; #[ntex::main] async fn main() -> std::io::Result<()> { let cfg = SharedCfg::new("H2-SRV").add(ServiceConfig::new().set_max_concurrent_streams(100)); ntex::server::build() .bind("h2", "127.0.0.1:8080", cfg, async |_| { server::Server::new(async |msg: Message| { if let MessageKind::Headers { pseudo, .. } = msg.kind { println!("{:?} {:?}", pseudo.method, pseudo.path); msg.stream.send_response(StatusCode::OK, HeaderMap::new(), false)?; msg.stream.send_payload("hello world", true).await?; } Ok::<_, Error>(()) }) })? .run() .await } ``` The connections take their `ServiceConfig` from the `SharedCfg` passed to `bind`. Since this listener accepts plain TCP, clients must use prior knowledge, for example `curl --http2-prior-knowledge http://127.0.0.1:8080/`. For TLS, put a TLS acceptor in front of the server and have it select `h2` during ALPN: ```rust,no_run use ntex::{Service, SharedCfg, error::Error}; use ntex::tls::openssl::SslAcceptor; use ntex_h2::{Message, OperationError, server}; use openssl::ssl::{self, AlpnError, SslFiletype, SslMethod}; #[ntex::main] async fn main() -> std::io::Result<()> { let mut builder = ssl::SslAcceptor::mozilla_intermediate(SslMethod::tls()).unwrap(); builder.set_private_key_file("key.pem", SslFiletype::PEM).unwrap(); builder.set_certificate_chain_file("cert.pem").unwrap(); builder.set_alpn_select_callback(|_, protos| { if protos.windows(3).any(|w| w == b"\x02h2") { Ok(b"h2") } else { Err(AlpnError::NOACK) } }); let acceptor = builder.build(); ntex::server::build() .bind("h2", "127.0.0.1:8443", SharedCfg::new("H2-TLS"), async move |_| { SslAcceptor::new(acceptor.clone()) .map_err(|e| server::ServerError::from(std::io::Error::other(e))) .and_then(server::Server::new(async |_msg: Message| { // handle messages Ok::<_, Error>(()) })) })? .run() .await } ``` If you already have an I/O object, say from your own acceptor or from a connection that switched protocols, `server::handle_one(io, publish, control)` serves it directly with services you have already created. That is how ntex-grpc runs its server. ### Handling streams The function or service you pass to `Server::new` is the *publish* service, and a new one is created for every connection. ntex-h2 calls it for every message on every stream the client opens: once for the request headers, once for each chunk of body data, and once for the end of the stream. These calls run concurrently; ntex-h2 does not wait for one to finish before starting the next. That keeps a slow request from holding up the others, but it changes how you write the service. You cannot `await` the request body in the call that received the headers, because the body arrives in calls of its own. Instead, the publish service keeps per-stream state and routes each message to where it is needed. A simple way to do that is a map keyed by `StreamRef`, which implements `Hash` and `Eq`. The service below collects each request body, up to a size limit, and echoes it back once the request is complete: ```rust,no_run use std::{cell::RefCell, collections::HashMap}; use ntex::http::{HeaderMap, StatusCode}; use ntex::service::{Ctx, Service}; use ntex::util::BytesMut; use ntex_h2::{Message, MessageKind, OperationError, StreamEof, StreamRef, frame::Reason}; const MAX_BODY: usize = 1024 * 1024; #[derive(Default)] struct Echo { bodies: RefCell>, } impl Service<(), Message> for Echo { type Res = (); type Error = ntex::error::Error; async fn call(&self, msg: Message, _: Ctx<'_, Self, ()>) -> Result<(), Self::Error> { let Message { stream, kind } = msg; let body = match kind { MessageKind::Headers { eof: true, .. } => BytesMut::new(), MessageKind::Headers { .. } => { self.bodies.borrow_mut().insert(stream, BytesMut::new()); return Ok(()); } MessageKind::Data(data, _) => { let mut bodies = self.bodies.borrow_mut(); if let Some(body) = bodies.get_mut(&stream) { body.extend_from_slice(&data); if body.len() > MAX_BODY { bodies.remove(&stream); stream.reset(Reason::CANCEL); } } return Ok(()); } MessageKind::Eof(StreamEof::Data(data, _)) => { let Some(mut body) = self.bodies.borrow_mut().remove(&stream) else { return Ok(()); }; body.extend_from_slice(&data); body } MessageKind::Eof(StreamEof::Trailers(_)) => { let Some(body) = self.bodies.borrow_mut().remove(&stream) else { return Ok(()); }; body } MessageKind::Eof(StreamEof::Error(_)) | MessageKind::Disconnect(_) => { self.bodies.borrow_mut().remove(&stream); return Ok(()); } }; stream.send_response(StatusCode::OK, HeaderMap::new(), false)?; stream.send_payload(body.freeze(), true).await } } ``` The `RefCell` is never borrowed across an `await`, because other calls on the same service run in between. Removing the entry on `Eof(StreamEof::Error(..))` and `Disconnect` keeps the map from leaking when a client goes away mid-request. To use a service type like this, wrap it in a factory. `fn_factory` takes an async function and calls it for each new connection: ```rust,no_run # use ntex::service::{Ctx, Service}; # use ntex_h2::{Message, OperationError}; # #[derive(Default)] struct Echo; # impl Service<(), Message> for Echo { # type Res = (); # type Error = ntex::error::Error; # async fn call(&self, _: Message, _: Ctx<'_, Self, ()>) -> Result<(), Self::Error> { Ok(()) } # } use ntex::service::fn_factory; use ntex_h2::server; # fn server() -> server::Server, std::convert::Infallible> { server::Server::new(fn_factory(async |_: &()| { Ok::<_, std::convert::Infallible>(Echo::default()) })) # } ``` Buffering works for small bodies. For large or streaming bodies, store a channel sender in the map instead and pass each chunk to a task that handles the request. Send the `Capacity` along with the data, as in the [flow control](#flow-control) example, so the client slows down to that task's pace. This is how `ntex::http` builds request payloads on top of ntex-h2. ### Sending the response You respond through the request's `StreamRef`: - `send_response(status, headers, eof)` sends the response head. Set `eof` to `true` if there is no body. - `send_payload(data, eof)` sends body data, exactly as on the client. - `send_trailers(headers)` ends the stream with trailing headers. gRPC uses these to carry its status. - `send_informational(status, headers)` sends a 1xx response, such as `103 Early Hints`, before the final one. You don't have to respond from the call that received the request. `StreamRef` is cloneable, so you can store it or move it into another task and respond later. The client's flow-control helpers, `available_send_capacity` and `send_capacity`, are available on `StreamRef` too. ### Errors and resets An error can affect one stream or the whole connection, so choose carefully. To fail a single request, send an error response or reset its stream with `stream.reset(reason)`. The connection and its other streams carry on. `REFUSED_STREAM` tells the client it may safely retry, while `CANCEL` or `INTERNAL_ERROR` tell it the request failed. Returning an error from the publish service is far more drastic. ntex-h2 treats it as a fault in the connection itself: it sends `GOAWAY` with `INTERNAL_ERROR` and closes the connection, failing every stream on it. It is still fine to use `?` on `send_response` and `send_payload`, because they fail only when the stream or connection is already gone. For problems with a request, reset the stream instead. If the client resets a stream while one of your calls for it is still running, that call is cancelled. You still receive an `Eof` message with the error, so you can clean up per-stream state as shown above. ### The control service Connection-level events go to a second service, the *control* service. The default accepts everything, and most servers never replace it. To log or react to connection events, pass your own to `.control(..)`: ```rust,no_run use ntex::error::Error; use ntex_h2::{Control, Message, OperationError, server}; # fn server() -> server::Server, std::io::Error> { server::Server::new(async |_msg: Message| Ok::<_, Error>(())) .control(async |msg: Control>| { if let Control::Disconnect(reason) = &msg { log::info!("connection closed: {reason:?}"); } Ok::<_, std::io::Error>(msg.ack()) }) # } ``` `Control::Disconnect` tells you why a connection ended: a protocol error, a `GOAWAY` from the client, the client going away, or an error returned by the publish service. Answer every control message with `msg.ack()`. ### Server state The publish and control services run in pipelines whose state is the *connection state*: a value that travels with the I/O object, as described in [Carrying state with a request](3-state.md#carrying-state-with-a-request). A plain `Io` carries no state, so the state is `()`. That is why `Echo` above implements `Service<(), Message>` and its factory takes `&()`. The worker state, created by `build_with_config()` (see [Worker and request state](6-server-app.md)), is not passed to these services. To make it available, put a service in front of `Server` that reads the worker state and returns the connection wrapped in a `State`, together with whatever the handlers need. That service can also record details about the connection, such as the peer address. ntex-h2 clones the connection state for the publish and control factories, so it must implement `Clone`. Keep shared resources behind an `Rc`. Handlers can then be [state-aware functions](3-state.md#state-aware-functions) that take the connection state as their first argument, and service factories receive it in `create`: ```rust,no_run use std::{cell::Cell, convert::Infallible, net::SocketAddr, rc::Rc}; use ntex::http::{HeaderMap, StatusCode}; use ntex::io::{Io, types::PeerAddr}; use ntex::service::{State, service}; use ntex::{SharedCfg, error::Error}; use ntex_h2::{Message, MessageKind, OperationError, server}; /// Worker state, created once per worker. #[derive(Clone)] struct AppState { requests: Rc>, } /// Connection state, shared by all streams of one connection. #[derive(Clone)] struct Conn { app: AppState, peer: Option, } async fn accept( app: &AppState, io: Io, ) -> Result, server::ServerError> { let peer = io.query::().get().map(|addr| addr.into_inner()); let conn = Conn { app: app.clone(), peer }; Ok(State { req: io, state: conn }) } async fn publish(conn: &Conn, msg: Message) -> Result<(), Error> { if let MessageKind::Headers { pseudo, .. } = msg.kind { let requests = &conn.app.requests; requests.set(requests.get() + 1); println!("{:?} from {:?}, request #{}", pseudo.path, conn.peer, requests.get()); msg.stream.send_response(StatusCode::OK, HeaderMap::new(), false)?; msg.stream.send_payload("hello world", true).await?; } Ok(()) } #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build_with_config(async || { Ok(AppState { requests: Rc::new(Cell::new(0)), }) }) .bind("h2", "127.0.0.1:8080", SharedCfg::default(), async |_: &AppState| { service(accept).and_then(server::Server::new(publish)) })? .run() .await } ``` Each worker has its own `AppState`, so `requests` counts only the requests handled by that worker. Use an `Arc` for values that must be shared across workers. Per-stream state, such as a request body that is still arriving, belongs in the publish service itself, as in the `Echo` example. With `handle_one`, you create the publish and control pipelines yourself, so they hold whatever state you gave them. ### Shutting down When the ntex server stops, its workers wait for connections to finish, as described in the [server chapter](5-server.md). To wind down HTTP/2 connections in an orderly way, call `ServiceConfig::shutdown()`. From then on, servers on the current thread refuse new streams, let active ones finish, and close their connections. ### What you take on `ntex::http` does more than relay frames. It checks that a body matches its `content-length`, enforces body size limits, guards against floods of empty `DATA` frames, and maps everything onto requests and responses. With ntex-h2 on its own, those policies are up to you. That freedom is the reason to use it directly, but plan for the extra work. ## Summary - ntex-h2 is the HTTP/2 implementation behind `ntex::http` and `ntex::web`. Use it directly, as ntex-grpc does, when you want HTTP/2 only, with full control over streams, trailers and flow control. - The protocol must already be negotiated: ALPN `h2` over TLS, or prior knowledge over plain TCP. - Configure connections with `ServiceConfig` in a `SharedCfg`. It is the same type as `ntex::http`'s `Http2ServiceConfig`. - On the client, keep one `client::Client` pool per server, or use `SimpleClient` to manage a single connection. `send` returns a `SendStream` for the request body and a `RecvStream` for the response. - Dropping either half of an unfinished request resets the stream with `CANCEL`, so timeouts and cancellation need no extra code. - Received data comes with a `Capacity`. Holding it keeps the peer waiting, and dropping it lets the peer send more. Pass it along with the data to apply backpressure. - On the server, the publish service is called concurrently for every message, so keep per-stream state keyed by `StreamRef`. Reset a stream to fail one request. Return an error only to close the whole connection. For a complete program, see the [HTTP/2 example]: a server with buffered and streaming request bodies, backpressure and trailers, plus a client that calls it. The [ntex-h2 examples] and the [gRPC chapter](11-grpc.md) have more. [ntex-h2]: https://crates.io/crates/ntex-h2 [HTTP/2 example]: https://github.com/ntex-rs/examples/tree/main/protocols/http2 [ntex-h2 examples]: https://github.com/ntex-rs/ntex-h2/tree/master/examples [`SharedCfg`]: https://docs.ntex.rs/api/ntex/service/cfg/struct.SharedCfg.html [`IoConfig`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html [`Http2ServiceConfig`]: https://docs.ntex.rs/api/ntex/http/h2/struct.Http2ServiceConfig.html # Building gRPC Services and Clients Once a system grows past a single process, its parts need a way to call each other. [gRPC] is one of the most common answers. You describe an API once, in a `.proto` file, and clients and servers written in any language can talk to each other through it. Messages are compact Protocol Buffers, calls travel over HTTP/2, and one connection carries many calls at the same time. [ntex-grpc] brings gRPC to ntex. It is a separate crate, built on the [service model](1-service.md) you already know and on [`ntex-h2`], the ntex HTTP/2 implementation. A gRPC client is a service you call, and a gRPC server is a service the ntex [server](5-server.md) runs for every connection. ntex-grpc takes a different route from most gRPC libraries: **it does not use derive macros or build scripts.** The protocol is described with ordinary traits, and Rust's type system does the wiring. A message knows how to encode itself because each of its fields does. A method is a type that knows its request path and its input and output messages. A client is generic over the transport that carries its calls. What remains is boilerplate: one struct per message, one type per method, a typed client per service. A code generator, `ntex-grpc-codegen`, writes that for you once. You commit its output and read it like the rest of your code. Nothing is generated during the build. The only macro you will meet is the `#[server]` attribute, a thin layer that connects the methods of a `.proto` service to the methods of your type. Because every piece is a trait, every piece can be replaced. You can use your own Rust types for message fields, put your own transport between a client and the network, or handle raw, undecoded calls on the server. This chapter starts with a running example, then follows a call from the `.proto` file through the client to the server. Depending on where you are starting from: - **New to ntex-grpc?** Run the [quick start](#quick-start-hello-grpc), skim [gRPC in a nutshell](#grpc-in-a-nutshell), then read [from .proto to Rust](#from-proto-to-rust) and [implementing a service](#implementing-a-service). - **Want stricter types in your messages?** Read [using your own field types](#using-your-own-field-types). - **Calling gRPC services from ntex?** Start with [calling services](#calling-services) and [errors, deadlines and retries](#errors-deadlines-and-retries). - **Running a server in production?** Read [tuning the server](#tuning-the-server) and [running over TLS](#running-over-tls), then go through the [checklist](#production-checklist). - **Something does not work?** See [troubleshooting](#troubleshooting). The snippets come from the [gRPC examples][examples]. Each example is a complete program, and its README has the run commands and a `grpcurl` session to try: | Example | What it shows | | --- | --- | | [grpc-helloworld][helloworld-example] | the smallest service, server and client | | [grpc-custom-types][custom-types-example] | your own Rust types for message fields | | [grpc-routing-client][routing-example] | a custom client transport that routes calls to different servers | | [grpc-server][server-example] | all four kinds of methods, several services, a fallback, server tuning | | [grpc-tls][tls-example] | gRPC over TLS with rustls | ## Quick start: hello, gRPC The helloworld example is a greeting service with a single method. Clone the examples repository and start the server: ```sh git clone https://github.com/ntex-rs/examples cd examples/protocols/grpc-helloworld cargo run --bin grpc-helloworld-server ``` In a second terminal, call it: ```sh cargo run --bin grpc-helloworld-client -- ntex ``` The client prints `Hello ntex!`. The server speaks standard gRPC, so other tools can call it too. With [grpcurl] it looks like this: ```sh grpcurl -plaintext -proto helloworld.proto -d '{"name": "ntex"}' \ 127.0.0.1:50051 helloworld.Greeter/SayHello ``` The whole service is a dozen lines: ```rust,no_run #[server(crate::helloworld::Greeter)] impl GreeterServer { #[method(SayHello)] async fn say_hello(&self, req: HelloRequest) -> HelloReply { let name = if req.name.is_empty() { "world" } else { &req.name }; HelloReply { message: format!("Hello {name}!").into(), } } } ``` and the client call is a few more: ```rust,no_run let client = connect(ADDR); let msg = HelloRequest { name: "ntex".into() }; let mut req = client.say_hello(&msg); req.timeout(Duration::from_secs(3)); let reply = req.send().await?; println!("{}", reply.message); ``` `HelloRequest`, `HelloReply`, the `Greeter` service definition and the `say_hello` client method all come from the generated file `src/helloworld.rs`. The rest of the chapter explains where they come from and how each part works. To use ntex-grpc in your own project, add these dependencies: ```toml [dependencies] ntex = "4" ntex-grpc = "4" ntex-h2 = "4" ``` ## gRPC in a nutshell You can use ntex-grpc without knowing the wire protocol. A little of it, though, explains many of the choices you will make later, especially about errors and retries. A gRPC call is an HTTP/2 request: - It is always a `POST` to `/{package}.{Service}/{Method}`, for example `/helloworld.Greeter/SayHello`. The path is how a server finds the method. - The request headers say that this is gRPC (`content-type: application/grpc`) and may carry a deadline (`grpc-timeout`). Your own headers, called *metadata* in gRPC, travel next to them. - Each message in the body has a 5-byte prefix: a flag that says whether it is compressed, and its length. - The response sends headers, the reply messages, and then *trailers*. The trailers carry the result of the call in `grpc-status` (`0` means `OK`) and an optional `grpc-message`. The result lives in the trailers, so a failed call still has HTTP status 200. A client knows how a call ended only after it has read the whole response. All gRPC methods use the same request. They differ only in how many messages each side sends: | Kind | `.proto` | Client sends | Server sends | | --- | --- | --- | --- | | unary | `rpc M(Req) returns (Res)` | one message | one message | | server streaming | `rpc M(Req) returns (stream Res)` | one message | any number | | client streaming | `rpc M(stream Req) returns (Res)` | any number | one message | | bidirectional | `rpc M(stream Req) returns (stream Res)` | any number | any number | Every call is one HTTP/2 stream, so a single connection runs many calls in parallel, and a slow call does not block the others. ## How ntex-grpc fits together Here is the path of a call through ntex-grpc: ```text .proto ── ntex-grpc-codegen ──▶ src/helloworld.rs (messages, MethodDefs, typed client) client: GreeterClient ─▶ Request ─▶ Transport ─▶ ntex-h2 pool ─▶ network server: network ─▶ ntex Server ─▶ GrpcServer ─▶ #[server] type ─▶ your method ``` A handful of traits hold it together: - [`NativeType`] reads and writes a single field value: an integer, a string, a nested message, or a type of your own. - [`Message`] encodes and decodes a whole message. The generated implementation calls `NativeType` for each field. - [`MethodDef`] describes one `rpc`: its request path and its input and output types. Every method is a zero-sized type that implements it. - [`ServiceDef`] describes a `service`: its full name and its methods. - [`Transport`] sends a call. A generated client is generic over it, and `send()` hands the request to it. Each of these is a seam you can use. The rest of the chapter goes through them in the order a call does. ## From .proto to Rust ### Running the code generator The code generator installs a command named `ntex-grpc`. It uses `protoc`, the Protocol Buffers compiler, to parse `.proto` files, so install `protoc` from your package manager first. If it is not in your `PATH`, point the `PROTOC` environment variable at it. ```sh cargo install ntex-grpc-codegen ``` Here is the `.proto` file of the helloworld example: ```protobuf syntax = "proto3"; package helloworld; // The greeting service. service Greeter { rpc SayHello (HelloRequest) returns (HelloReply); } // The request message with the user's name. message HelloRequest { string name = 1; } // The reply message with the greeting. message HelloReply { string message = 1; } ``` You will run the generator every time the `.proto` file changes, so the examples keep the command in a small `generate.sh` script next to it: ```sh ntex-grpc helloworld.proto helloworld.rs --out-dir ./src --include-dir ./ \ --rustfmt-path ../../rustfmt.toml ``` The output file is named after the proto `package`, not after the second argument. If you generate code for the same package twice, for example with different options, use two output directories. | Option | Effect | | --- | --- | | `--out-dir DIR` | where to write the generated file | | `--include-dir DIR` | where to look for imported `.proto` files; can be repeated | | `--map NAME=TYPE` | use your own Rust type for a field, see [using your own field types](#using-your-own-field-types) | | `--rustfmt-path FILE` | rustfmt configuration used to format the output | | `--optional-messages` | generate singular message fields as `Option`, so you can tell an unset field from an empty one | | `--open-enums` | generate enum fields as `i32`, so values your code does not know yet are kept | | `--well-known-types` | generate the `google.protobuf` types instead of using `ntex_grpc::google_types` | Commit the generated file. Reviews then show exactly how a protocol change affects your Rust code, and your build does not depend on `protoc`. ### What gets generated Add the generated file as a module, `pub mod helloworld;`. Each `message` becomes a plain struct: ```rust,no_run /// The request message with the user's name. #[derive(Clone, PartialEq, Debug)] pub struct HelloRequest { pub name: ::ntex_grpc::ByteString, } ``` Each `service` becomes three things: - a unit struct, `Greeter`, that implements [`ServiceDef`] with the full service name, `helloworld.Greeter`; - a unit struct for each `rpc` that implements [`MethodDef`]; - a typed client, `GreeterClient`, generic over its transport, with one method per `rpc`. The method type is where the type system takes over: ```rust,no_run #[derive(Debug, Copy, Clone, PartialEq, Eq)] pub struct GreeterSayHelloMethod; impl ::ntex_grpc::MethodDef for GreeterSayHelloMethod { const NAME: &'static str = "SayHello"; const PATH: ::ntex_grpc::ByteString = ::ntex_grpc::ByteString::from_static("/helloworld.Greeter/SayHello"); type Input = HelloRequest; type Output = HelloReply; const CLIENT_STREAMING: bool = false; const SERVER_STREAMING: bool = false; } ``` `MethodDef` ties a method to its request path and its message types at compile time. The client uses it to know where to send a request and what to decode from the reply; the server uses it to decode requests and encode replies. Passing the wrong message to a method is a compile error, not a failed call. Each struct also gets a [`Message`] implementation. It is a list of `NativeType` calls, one per field, each with the field's tag: ```rust,no_run impl ::ntex_grpc::Message for HelloRequest { #[inline] fn write(&self, dst: &mut ::ntex_grpc::BytePages) { ::ntex_grpc::NativeType::serialize( &self.name, 1, ::ntex_grpc::types::DefaultValue::Default, dst, ); } // `read`, `merge_from` and `encoded_len` follow the same pattern } ``` There is no reflection and no table of field descriptors at run time, only straight-line code the compiler can inline. It is also the hook for custom field types: any type that implements `NativeType` fits into the same code. ### How protobuf types map to Rust Fields get ntex types where ntex has one, so strings and byte fields share memory with the received buffer instead of being copied: | Protobuf | Rust | | --- | --- | | `double`, `float` | `f64`, `f32` | | `int32`, `sint32`, `sfixed32` | `i32` | | `int64`, `sint64`, `sfixed64` | `i64` | | `uint32`, `fixed32` | `u32` | | `uint64`, `fixed64` | `u64` | | `bool` | `bool` | | `string` | `ByteString` | | `bytes` | `Bytes` | | `enum E` | `E`, a Rust enum (`i32` with `--open-enums`) | | `M`, a message field | `M` (`Option` with `--optional-messages`) | | `optional T` | `Option` | | `repeated T` | `Vec` | | `map` | `ntex_grpc::HashMap` | | `oneof name { .. }` | `Option`, an enum with one variant per field | | `google.protobuf.Timestamp`, `Duration`, wrappers | the types in `ntex_grpc::google_types` | | `google.protobuf.Empty` | `()` | Here is a message that uses several of them: ```protobuf enum Color { COLOR_UNSPECIFIED = 0; RED = 1; GREEN = 2; } message Shape { Color color = 1; optional uint32 weight = 2; map labels = 3; oneof kind { double radius = 4; Side side = 5; } } ``` and the generated Rust: ```rust,no_run pub struct Shape { pub color: Color, pub weight: Option, pub labels: ::ntex_grpc::HashMap<::ntex_grpc::ByteString, u64>, pub kind: Option, } /// Nested message and enum types in `Shape`. pub mod shape { pub enum Kind { Radius(f64), Side(super::Side), } } #[repr(i32)] pub enum Color { Unspecified = 0, Red = 1, Green = 2, } ``` Two defaults are worth knowing about: - **Enums are closed.** A value that the generated enum does not have, for example from a newer version of the `.proto` file, decodes as the enum's default variant. If your code must keep such values, generate with `--open-enums` and convert with `Color::from_i32()` where you need to. - **Message fields are always present.** An unset message field decodes as the default message, as in proto3. If you need to tell "unset" from "empty", generate with `--optional-messages`. ### Using your own field types Protobuf has few types, and a field often means more than its wire type: a `bytes` field that always holds a 16-byte UUID, or a `string` that only ever holds product codes. With generated types, every handler has to check those rules again. The `--map` option replaces the Rust type of such fields, so the check happens once, while decoding: ```protobuf message ReserveRequest { // Client-chosen request id, a 16-byte UUID. bytes request_id = 1; // Stock keeping unit, e.g. "BOOK-42". string sku = 2; uint32 quantity = 3; } message ReserveReply { bytes request_id = 1; string sku = 2; bool reserved = 3; uint32 remaining = 4; } ``` ```sh ntex-grpc inventory.proto inventory.rs --out-dir ./src --include-dir ./ \ --map request_id=crate::types::RequestId \ --map sku=crate::types::Sku ``` The left side of `--map` is a field path. A path with a leading dot is fully qualified, such as `.inventory.ReserveRequest.sku`. A path without one matches by suffix: `sku` covers the field in both messages, while `ReserveRequest.sku` covers only one. The right side is a Rust path, copied into the generated code as is: ```rust,no_run pub struct ReserveRequest { pub request_id: crate::types::RequestId, pub sku: crate::types::Sku, pub quantity: u32, } ``` The wire format does not change, so other gRPC clients and servers still send plain `bytes` and `string` fields. What changes is the Rust side: the type decides which values are valid, and decoding fails for anything else. Your handler never sees a request with a malformed id or SKU, because the server answers `INVALID_ARGUMENT` before calling it. A `ReserveRequest` in your code is valid by construction. ### Writing a custom field type A field type implements [`NativeType`]. The generated structs derive a few traits, so the type also needs `Default`, `PartialEq` and `Debug`. | Item | What it does | | --- | --- | | `const TYPE: WireType` | the protobuf wire type: `LengthDelimited` for `bytes`, `string` and messages, `Varint` for integers and `bool`, `ThirtyTwoBit` or `SixtyFourBit` for fixed-width numbers | | `value_len(&self)` | the encoded length of the value, without the tag and length prefix | | `merge(&mut self, src)` | decode a value; for length-delimited types `src` holds exactly the value, the length prefix is already removed | | `encode_value(&self, dst)` | write the value, without the tag and length prefix | | `is_default(&self)` | `true` if the value is the default, so the field is left out of the message | Here is the request id from the custom-types example. A nil UUID plays the role of the default value, so it is left out of the message, just like an empty `bytes` field: ```rust,no_run use ntex_grpc::{BytePages, Bytes, DecodeError, NativeType, WireType}; use uuid::Uuid; #[derive(Clone, Copy, PartialEq, Eq, Hash, Debug, Default)] pub struct RequestId(Uuid); impl NativeType for RequestId { const TYPE: WireType = WireType::LengthDelimited; fn value_len(&self) -> usize { 16 } /// `src` holds exactly the field value, the length prefix is already /// removed. fn merge(&mut self, src: &mut Bytes) -> Result<(), DecodeError> { let id = Uuid::from_slice(src).map_err(|_| DecodeError::new("request id must be 16 bytes"))?; *self = Self(id); Ok(()) } fn encode_value(&self, dst: &mut BytePages) { dst.extend_from_slice(self.0.as_bytes()); } fn is_default(&self) -> bool { self.0.is_nil() } } ``` You rarely need to handle the wire format yourself. A type that wraps a built-in one can delegate to it and add its own rules. `Sku` lets `ByteString` check that the bytes are valid UTF-8, then validates the string: ```rust,no_run impl NativeType for Sku { const TYPE: WireType = WireType::LengthDelimited; fn value_len(&self) -> usize { self.0.value_len() } fn merge(&mut self, src: &mut Bytes) -> Result<(), DecodeError> { let mut s = ByteString::default(); s.merge(src)?; *self = Self::try_from(s).map_err(|e| DecodeError::new(e.to_string()))?; Ok(()) } fn encode_value(&self, dst: &mut BytePages) { self.0.encode_value(dst); } fn is_default(&self) -> bool { self.0.is_empty() } } ``` A few rules keep a custom type compatible with other gRPC implementations: - **Encode exactly `value_len()` bytes.** For length-delimited types, the length prefix is written from `value_len()` before `encode_value()` runs. If the two disagree, the message is corrupt. - **Return an error from `merge`, never panic.** The error becomes `INVALID_ARGUMENT` on the server and `InvalidResponse` on the client. - **Keep `is_default` consistent with protobuf.** Other implementations treat a missing field as the protobuf default (empty, zero or `false`), so your default must mean the same thing. - **Override `encoded_len` for types that are not length-delimited.** The provided `encoded_len(tag)` adds a length prefix, which is only correct for `LengthDelimited`. A `Varint` newtype, such as an id stored as a `u64`, must return `key_len(tag) + value_len()` itself, as the built-in integer types do; otherwise encoded messages get the wrong size. The custom-types example tests all of this. It generates the same `.proto` twice, once with `--map` and once without, and checks that the plain client and the typed server understand each other. ## Calling services A generated client has one method per `rpc`. Calling that method does not send anything yet. It returns a request that you can configure, and then you send it: ```rust,no_run use std::time::Duration; use grpc_helloworld::{ADDR, connect, helloworld::HelloRequest}; #[ntex::main] async fn main() { let client = connect(ADDR); let msg = HelloRequest { name: "ntex".into() }; let mut req = client.say_hello(&msg); req.timeout(Duration::from_secs(3)); match req.send().await { // `Response` derefs to the reply message Ok(res) => println!("{}", res.message), Err(err) => eprintln!("call failed with {:?}: {err}", err.status()), } } ``` The request borrows the client and the message, so nothing is copied until the message is encoded. Everything you set on it, such as the timeout, headers or compression, applies to this call only. ### Connecting to a server The usual transport is [`ntex_grpc::client::Client`]. It is a thin gRPC layer over an [`ntex-h2`] connection pool: ```rust,no_run use ntex::SharedCfg; use ntex_grpc::client::Client; use ntex_h2::client as h2; pub fn connect(addr: impl Into) -> GreeterClient { let pool = h2::Client::builder(addr.into()).build(SharedCfg::new("GRPC-CLIENT")); GreeterClient::new(Client::new(pool)) } ``` The pool connects lazily. Building it does not touch the network: the first call opens a connection, and later calls share it, each on its own HTTP/2 stream. When a connection closes, the pool opens a new one for the next call that needs it. Your code does not have to manage connections at all. Cloning a `Client` is cheap, and the clones share the pool. Build one `Client` per server and clone it into every typed client that talks to that server: ```rust,no_run let pool = h2::Client::builder(addr.into()).build(SharedCfg::new("GRPC-CLIENT")); let client = ntex_grpc::client::Client::new(pool); let echo = EchoClient::new(client.clone()); let stats = StatsClient::new(client); ``` The pool builder controls the connections: | Method | Effect | | --- | --- | | `connector(..)` | how to connect, e.g. through a TLS connector, see [running over TLS](#running-over-tls) | | `secure(true)` | send the `https` scheme; use it together with TLS | | `connect_timeout(..)` | time limit for the TCP connect and the TLS handshake | | `minconn(..)` / `connection_limit(..)` | how many connections to keep open and how many to allow | | `max_streams(..)` | how many calls can share one connection | | `lifetime(..)` | retire connections after a while, e.g. to pick up DNS changes | HTTP/2 settings, such as window sizes and the ping interval, come from an `ntex_h2::ServiceConfig` added to the `SharedCfg` that you pass to `build()`. It works the same way as on the [server](#tuning-the-server). ### What happens during a call A unary call has three parts: - **the request**, [`Request`], which holds the transport, the input message and a [`RequestContext`] with the headers, the timeout and the limits; - **the transport**, which turns the request into an HTTP/2 stream; - **the response**, [`Response`], which holds the decoded reply, the headers and the trailers. `send()` hands the request to the transport through the [`Transport`] trait: ```rust,no_run pub trait Transport { type Error; async fn request( &self, args: &T::Input, ctx: &mut RequestContext, ) -> Result, Self::Error>; } ``` The trait is generic over the method `T`, so a transport knows the method's `PATH`, `Input` and `Output` at compile time. The built-in transports (`Client`, `ntex_h2::client::Client` and `ntex_h2::client::SimpleClient`) send a call in six steps: 1. If a header set on the request was invalid, fail with `InvalidRequest` without sending anything. 2. Encode the input message with its `Message` implementation, compress it if requested, and add the 5-byte message prefix. 3. Take a connection from the pool and open a stream with `POST T::PATH`, the gRPC headers (`content-type: application/grpc`, `te: trailers`, and `grpc-timeout` if a timeout is set) and the headers of the request. 4. Send the message and close the sending side of the stream. 5. Read the response headers, the message and the trailers. Check the HTTP status, the content type, the message size and `grpc-status`. 6. Decode the message as `T::Output` and return a `Response`. The timeout covers the whole call, including the wait for a connection. When it runs out, the stream is reset with `CANCEL` and the call fails with `DeadlineExceeded`. A transport can also receive per-call state. `send_with(&state)` passes a reference to the transport, which reads it with `ctx.request_state()`. `send()` is simply `send_with(&())`. ### Headers, compression and message limits Request headers carry metadata such as authentication tokens or trace ids. Set them on the request before sending it: ```rust,no_run use ntex_grpc::encode_binary_header; let mut req = client.say_hello(&msg); req.header("x-request-id", "42") .header("trace-bin", encode_binary_header(&trace_id)) .timeout(Duration::from_secs(3)); let (reply, headers, trailers) = req.send().await?.into_parts(); ``` `header()` replaces a header and `append_header()` adds another value. By gRPC convention, header names ending in `-bin` carry binary values; encode them with [`encode_binary_header()`] and decode received ones with `decode_binary_header()`. The client sets the protocol headers itself (`content-type`, `te`, `grpc-timeout`, `grpc-encoding` and the others) and ignores your values for them. An invalid header name or value is not sent; the call fails with `InvalidRequest` instead. A successful [`Response`] derefs to the reply message. `headers()` and `trailers()` give the response metadata, and `into_parts()` takes all three apart. **Compression** is optional and needs the `compression` feature: ```toml ntex-grpc = { version = "4", features = ["compression"] } ``` ```rust,no_run use ntex_grpc::Compression; req.compression(Compression::Gzip); // or Compression::Zstd ``` Compression costs CPU on both sides, so turn it on for large, repetitive messages and leave it off for small ones. Empty messages are always sent uncompressed. A server that does not support the encoding fails the call with `UNIMPLEMENTED`. **Message limits** protect the client from huge replies. On `Client`, `max_message_size()` limits received messages (4 MiB by default), and `max_send_message_size()` limits sent ones. A request can override both for a single call. ### Streaming calls Streaming methods return a `StreamRequest` instead. You configure it the same way and open it through the [`StreamTransport`] trait. The messages flow through a [`SendStream`] and a [`RecvStream`]: ```rust,no_run // server streaming: one request, a stream of replies let mut rx = echo.server_streaming_echo(&msg).open().await?; while let Some(reply) = rx.recv().await { println!("{}", reply?.message); } // client streaming: send while `send()` waits for the reply let tx = SendStream::new(); let (res, sent) = join(echo.client_streaming_echo(&tx).send(), async { for word in ["one", "two", "three"] { tx.send(&echo_request(word)).await?; } // the end of the stream, the server answers now tx.close(); Ok::<_, StreamError>(()) }) .await; sent?; println!("{}", res?.message); // bidirectional streaming: send and receive at the same time let (tx, mut rx) = echo.bidi_streaming_echo().open().await?; ``` `join` is `ntex::util::join`. A client-streaming call has to send and wait for the reply at the same time. If you await `send()` before sending the messages, the call waits forever: the server answers only after the stream ends. After a stream ends, `rx.status()` returns its final `grpc-status`. ### Writing your own transport Generated clients are generic over their transport, so anything that implements `Transport` (and `StreamTransport`, for streaming methods) can take the place of `Client`. A transport is the natural home for concerns that apply to every call: authentication headers, metrics, logging, retries or routing. A custom transport usually wraps a built-in one. It adjusts the request context, then delegates: ```rust,no_run use ntex::Error; use ntex_grpc::MethodDef; use ntex_grpc::client::{Client, ClientError, RequestContext, Response, Transport}; /// Adds an `authorization` header to every call. #[derive(Clone)] pub struct WithToken { inner: Client, token: String, } impl Transport for WithToken { type Error = Error; async fn request( &self, args: &M::Input, ctx: &mut RequestContext<'_, St>, ) -> Result, Self::Error> { ctx.header("authorization", self.token.as_str()); Transport::::request(&self.inner, args, ctx).await } } ``` `GreeterClient::new(WithToken { .. })` now sends the token with every call. Keep the implementation generic over the request state `St`, so both `send()` and `send_with()` keep working. For streaming calls, implement `StreamTransport` the same way. Its `open()` returns a [`StreamCall`], the HTTP/2 stream of the call, and only the built-in transports can create one. A custom streaming transport therefore always ends by calling `open()` on a built-in transport. Generated clients own their transport. To share one transport between several typed clients, make it cheap to clone and keep the shared parts behind an `Rc` or in the pool, as `Client` does. ### Routing calls to different backends In larger systems, different services often live on different servers. The routing example puts `shop.Catalog` and `shop.Orders` on separate backends. A `Router` transport picks the backend for each call from the service name in `M::PATH`: ```rust,no_run use std::{collections::HashMap, rc::Rc}; use ntex::{Error, SharedCfg}; use ntex_grpc::client::{ CallFailure, Client, ClientError, RequestContext, Response, StreamCall, StreamTransport, Transport, }; use ntex_grpc::{GrpcStatus, MethodDef}; use ntex_h2::client as h2; /// Routes calls by service name, e.g. `shop.Catalog`, to a backend client. #[derive(Clone, Default)] pub struct Router { routes: Rc>, } impl Router { /// Sends the calls of `service` to the server at `addr`. pub fn route_to(mut self, service: &'static str, addr: impl Into) -> Self { let pool = h2::Client::builder(addr.into()).build(SharedCfg::new("GRPC-ROUTER")); Rc::make_mut(&mut self.routes).insert(service, Client::new(pool)); self } /// The backend for the service of method `M`. fn backend(&self) -> Result<&Client, Error> { let path = M::PATH; let service = service_name(&path); self.routes.get(service).ok_or_else(|| { // Nothing is sent and trying again fails the same way. Error::from(ClientError::InvalidRequest(CallFailure::new( GrpcStatus::Unimplemented, format!("no backend for {service}"), ))) }) } } /// The service part of a request path: `/shop.Catalog/GetItem` gives /// `shop.Catalog`. pub fn service_name(path: &str) -> &str { path.trim_start_matches('/') .split_once('/') .map_or("", |(service, _)| service) } impl Transport for Router { type Error = Error; async fn request( &self, args: &M::Input, ctx: &mut RequestContext<'_, St>, ) -> Result, Self::Error> { Transport::::request(self.backend::()?, args, ctx).await } } /// Only the built-in transports can open a `StreamCall`, so pick a backend /// and let it open the stream. impl StreamTransport for Router { async fn open( &self, ctx: &mut RequestContext<'_, St>, ) -> Result> { StreamTransport::::open(self.backend::()?, ctx).await } } ``` Both typed clients share one router, and the clones share the routes and the connection pools. Use the generated `ServiceDef::NAME` constants as route names. If a service is renamed in the `.proto` file, the code stops compiling instead of failing at run time: ```rust,no_run let router = Router::default() .route_to(Catalog::NAME, "127.0.0.1:50053") .route_to(Orders::NAME, "127.0.0.1:50054"); let catalog = CatalogClient::new(router.clone()); let orders = OrdersClient::new(router); ``` The router returns the same error type as the built-in transports, so callers handle a missing route like any other failed call. ### Errors, deadlines and retries Calls fail. Networks drop connections, servers restart, and requests run into bugs. What matters is how much your code can learn from a failure, and the most important question is usually: *did the server see my request?* The answer decides whether a retry is safe. The built-in transports return `ntex::Error`. It derefs to [`ClientError`], so match on `&*err`. `err.status()` gives a gRPC status code for every variant, and `err.message()` describes the cause. Each variant answers that question: | Variant | Did the server see the request? | What you can do | | --- | --- | --- | | `Status` | yes | apply the rules for the status code | | `InvalidRequest` | no | fail the call; retrying fails the same way | | `NotProcessed` | no | retry at once, for any method | | `Connection` | maybe, see `request_sent` | retry, ideally on another connection | | `StreamFailed` | maybe | retry if the method is idempotent | | `NotGrpc` | not by a gRPC server | check the address, or back off on HTTP 429, 502, 503 and 504 | | `InvalidResponse` | yes | fail the call; asking again gets the same answer | | `DeadlineExceeded` | maybe | the time for the call is up | `Status` is the only variant whose status came from the server, and it also carries the server's trailers. `NotProcessed` covers a connection that was already closing when the call started and streams that the server refused, so the request never reached your service. With that, a retry loop for an idempotent call is short: ```rust,no_run use ntex_grpc::client::ClientError; let mut attempts = 0; let reply = loop { let mut req = client.say_hello(&msg); req.timeout(Duration::from_secs(3)); match req.send().await { Ok(res) => break res.into_inner(), Err(err) => { let retry = match &*err { ClientError::NotProcessed(_) => true, ClientError::Connection { request_sent, .. } => !request_sent, _ => false, }; attempts += 1; if !retry || attempts == 3 { return Err(err); } } } }; ``` **Deadlines** follow the gRPC spec. The client sends `timeout()` to the server in the `grpc-timeout` header, rounded up so the server never gives up before the client does. When the deadline passes, the server stops the call and answers `DEADLINE_EXCEEDED`, and the client stops waiting at the same moment. A zero timeout fails at once, without sending anything. Without a timeout, a call waits as long as its connection lives. A server that hangs then holds your call, and whatever is waiting for it, forever. Set a timeout on every call. Streaming calls fail in the same ways. `open()` returns the errors of opening the stream, `recv()` returns a [`StreamError`] if the stream fails later, and `rx.status()` gives the final status. ## Serving gRPC A gRPC server has three layers: - **your type**, with one async method per `rpc`, turned into an ntex service by `#[server]`; - **[`GrpcServer`]**, which speaks the gRPC protocol and routes each call to a service by its service name; - **an ntex [`Server`](5-server.md)**, which accepts connections, optionally runs TLS, and hands each connection to `GrpcServer`. ### Implementing a service Here is the complete helloworld server: ```rust,no_run use std::convert::Infallible; use ntex::{ServiceFactory, SharedCfg, server::Server}; use ntex_grpc::server; #[derive(Clone, Debug, Default)] pub struct GreeterServer; #[server(crate::helloworld::Greeter)] impl GreeterServer { #[method(SayHello)] async fn say_hello(&self, req: HelloRequest) -> HelloReply { let name = if req.name.is_empty() { "world" } else { &req.name }; HelloReply { message: format!("Hello {name}!").into(), } } } /// `GrpcServer` creates the service for every connection. impl ServiceFactory<(), server::ServerRequest> for GreeterServer { type Res = server::ServerResponse; type Error = server::ServerError; type Service = GreeterServer; type InitError = Infallible; async fn create(&self, _: &()) -> Result { Ok(self.clone()) } } #[ntex::main] async fn main() -> std::io::Result<()> { Server::builder() .bind("grpc", "127.0.0.1:50051", SharedCfg::new("GRPC"), async |_| { server::GrpcServer::new(GreeterServer) })? .run() .await } ``` `#[server(path)]` takes the generated service definition. Each `#[method(Name)]` uses the method name from the `.proto` file, and the attribute checks the argument and return types against the generated `MethodDef` at compile time. A typo in a method name or a wrong message type is a compile error. The attribute implements `Service<(), ServerRequest>` and `NamedService` for your type; everything else is ordinary Rust. Import the attribute with `use ntex_grpc::server;`. That brings in both the `server` module and the `server` attribute. `use ntex_grpc::server::{self, ..}` imports only the module, and the confusing error you get is that `NamedService` is not implemented. The signature of a method depends on its kind: | Kind | Arguments | Returns | | --- | --- | --- | | unary | `req: Input` or `req: Request` | `Output`, `Response`, `Result` with `E: Into`, or `Result, E>` with `E: Into` | | server streaming | `req`, `tx: SendStream` | `()` or `Result<(), E>` with `E: Into` | | client streaming | `rx: RecvStream` | as unary | | bidirectional | `rx: RecvStream`, `tx: SendStream` | as server streaming | Take `Request` when you need the request headers. It derefs to the message, so the fields are still at hand: ```rust,no_run #[method(UnaryEcho)] async fn unary_echo(&self, req: Request) -> EchoReply { let prefix = req .headers .get(PREFIX_HEADER) .and_then(|v| v.to_str().ok()) .unwrap_or(""); EchoReply { message: format!("{prefix}{}", req.message.message).into(), } } ``` `GrpcServer` creates a service instance for every connection by calling `create()`. Where state lives depends on who should see it: - State for one connection goes into the service that `create()` returns. - State for one worker can be created in the closure passed to `bind()`, which is called for every worker; an `Rc` is enough there. The closure may run again when a worker restarts, so keep that setup safe to repeat. - State for the whole server, such as a counter or a store, goes into an `Arc` that all clones share, because workers run on different threads. ### Streaming methods Streaming methods receive their messages from a `RecvStream` and send them through a `SendStream`. A server-streaming method gets the request and a sender: ```rust,no_run #[method(ServerStreamingEcho)] async fn server_streaming_echo( &self, req: EchoRequest, tx: SendStream, ) -> Result<(), ServerError> { if req.repeat > MAX_REPEAT { return Err(ServerError::new( GrpcStatus::InvalidArgument, encode_grpc_message(&format!("repeat must be at most {MAX_REPEAT}")), None, )); } let reply = EchoReply { message: req.message }; for _ in 0..req.repeat { tx.send(&reply).await?; } Ok(()) } ``` A client-streaming method reads until the client closes its side, then answers once: ```rust,no_run #[method(ClientStreamingEcho)] async fn client_streaming_echo(&self, mut rx: RecvStream) -> EchoReply { let mut messages = Vec::new(); while let Some(Ok(req)) = rx.recv().await { messages.push(req.message); } EchoReply { message: messages.join(" ").into(), } } ``` A bidirectional method gets both and can answer each message as it arrives: ```rust,no_run #[method(BidiStreamingEcho)] async fn bidi_streaming_echo( &self, mut rx: RecvStream, tx: SendStream, ) -> Result<(), ServerError> { while let Some(req) = rx.recv().await { let req = req?; tx.send(&EchoReply { message: req.message }).await?; } Ok(()) } ``` `?` works on failed sends and receives. If the client goes away, the next `send()` fails and the method returns early. When the method returns, the call ends and its result becomes the `grpc-status` of the call. ### Reporting errors to clients Some errors are handled for you, before your method runs: | Situation | Status | | --- | --- | | unknown service or method | `UNIMPLEMENTED` | | request message that cannot be decoded, including a custom field type that rejects its value | `INVALID_ARGUMENT` | | request message larger than `max_message_size` | `RESOURCE_EXHAUSTED` | | compressed request in an encoding the server does not support | `UNIMPLEMENTED` | | deadline from `grpc-timeout` runs out | `DEADLINE_EXCEEDED` | Your methods report their own errors through their return value. A `ServerError` becomes the `grpc-status` and `grpc-message` of the call, and its optional `HeaderMap` adds more trailers. Build it with `ServerError::new()`, and pass the text through [`encode_grpc_message()`], because gRPC expects `grpc-message` to be percent-encoded. Streaming methods return `Result<(), ServerError>`, as shown above. Unary and client-streaming methods return a reply, so they need a way to say whether the `Err` side is a failure. Return `Result, ServerError>` to fail the call with a status. `Response` also carries extra trailers for a successful reply, and a plain message turns into one with `.into()`: ```rust,no_run use ntex::http::header::HeaderValue; use ntex_grpc::server::{Response, ServerError}; #[method(Reserve)] async fn reserve( &self, req: ReserveRequest, ) -> Result, ServerError> { let Some(seats) = self.store.free_seats(&req.show) else { return Err(ServerError::new( GrpcStatus::NotFound, encode_grpc_message(&format!("no show {}", req.show)), None, )); }; let reply = self.store.reserve(&req)?; // a store error converts into ServerError Ok(Response { message: reply, headers: vec![(SEATS_LEFT, HeaderValue::from(seats - 1))], }) } ``` The trailers in `Response::headers` follow `grpc-status: 0`. A `grpc-status` or `grpc-message` among them is ignored, so a `Response` is always a successful call. The client reads them from `trailers()` on its response. A method that returns `Result` works differently: it **always ends with `OK`**. Its error type is converted into a reply message (`E: Into`), so the error has to be part of the reply. That suits APIs that model errors in the protocol anyway, where a `oneof` gives the reply an explicit success or failure: ```protobuf message ReserveReply { oneof outcome { Reservation reserved = 1; Problem problem = 2; } } ``` ```rust,no_run impl From for ReserveReply { fn from(problem: Problem) -> Self { ReserveReply { outcome: Some(reserve_reply::Outcome::Problem(problem)), } } } #[method(Reserve)] async fn reserve(&self, req: ReserveRequest) -> Result { let reservation = self.store.reserve(&req)?; Ok(ReserveReply { outcome: Some(reserve_reply::Outcome::Reserved(reservation)), }) } ``` Errors in the reply are part of your API: clients get a typed `Problem` instead of a status code and a string. Pick a gRPC status when the failure is generic, such as a missing resource or a denied permission, and a `oneof` when clients need to act on the details. ### Hosting several services One `GrpcServer` can serve several services, and it routes calls by the service name in the request path: ```rust,no_run pub fn grpc_server(calls: Arc) -> GrpcServer { GrpcServer::new(EchoServer { calls: calls.clone(), }) .service(StatsServer { calls }) .default_service(Fallback) .max_message_size(MAX_MESSAGE_SIZE) } ``` | Method | Effect | | --- | --- | | `new(service)` / `service(service)` | add a `#[server]` service under its `NamedService::NAME` | | `service_named(name, service)` | add a service under another name, e.g. a hand-written one | | `default_service(service)` | take the calls for every service that is not added | Without a default service, a call to an unknown service fails with `UNIMPLEMENTED`. Adding the same name twice panics, so the mistake shows up when the server starts, not when the first call arrives. The default service and `service_named` services are plain ntex services of `ServerRequest`, written by hand. They see the call before anything is decoded: the service and method names, the request headers and the encoded payload. They answer with a `ServerResponse` holding the encoded reply, or with a `ServerError` holding any status. They handle only unary calls. ```rust,no_run #[derive(Clone, Copy, Debug, Default)] pub struct Fallback; impl Service<(), server::ServerRequest> for Fallback { type Res = server::ServerResponse; type Error = ServerError; async fn call( &self, req: server::ServerRequest, _: Ctx<'_, Self, ()>, ) -> Result { let msg = format!( "unknown service {}, this server has echo.Echo and echo.Stats", req.service ); Err(ServerError::new( GrpcStatus::Unimplemented, encode_grpc_message(&msg), None, )) } } ``` `Fallback` also needs a `ServiceFactory` implementation, just like the `#[server]` types. ### Tuning the server `GrpcServer` has two gRPC limits: - `max_message_size(size)` is the largest request message the server accepts, checked again after decompression. A larger one fails the call with `RESOURCE_EXHAUSTED`. The default is 4 MiB. - `max_send_message_size(size)` is the largest reply message the server sends. Everything below gRPC is HTTP/2. `GrpcServer` reads its HTTP/2 settings from the configuration of the listener it is bound to, so add an `ntex_h2::ServiceConfig` to the `SharedCfg` passed to `bind()`: ```rust,no_run use ntex::{SharedCfg, time::Seconds}; use ntex_h2 as h2; pub fn server_cfg() -> SharedCfg { SharedCfg::new("GRPC") .add( h2::ServiceConfig::new() // parallel calls per connection .set_max_concurrent_streams(256) // per-call and per-connection receive windows .set_initial_window_size(256 * 1024) .set_initial_connection_window_size(1024 * 1024) // close connections that stop answering pings .set_ping_timeout(Seconds(20)) .set_handshake_timeout(Seconds(5)), ) .into() } Server::builder() .bind("grpc", ADDR, server_cfg(), async move |_| grpc_server(calls.clone()))? .run() .await ``` The defaults suit most services. Change them when your traffic is unusual: | Setting | Default | Why change it | | --- | --- | --- | | `set_max_concurrent_streams` | 256 | the number of calls a client can run in parallel on one connection | | `set_initial_window_size` | 65,535 | larger windows speed up large messages and streams | | `set_initial_connection_window_size` | 4 MiB | the receive window shared by all calls on a connection | | `set_ping_timeout` | 10 seconds | detect dead connections; zero disables pings | | `set_handshake_timeout` | 5 seconds | how long a new connection has to complete the HTTP/2 handshake | With the `compression` feature, the server also accepts gzip and zstd requests and compresses each reply like its request, unless the client's `grpc-accept-encoding` header rules that out. ### Running over TLS gRPC needs HTTP/2, and `GrpcServer` speaks only HTTP/2. It serves any [`Io`](7-io.md), so TLS is simply one more service in front of it. On the server, the TLS acceptor runs first and passes the decrypted stream on: ```rust,no_run use ntex::io::Io; use ntex::tls::rustls::TlsAcceptor; use ntex::Service; pub fn grpc_tls_service( config: Arc, ) -> impl Service<(), Io, Res = (), Error = Box> { ntex::service(TlsAcceptor::new(config)) .map_err(Box::::from) .and_then(grpc_server()) } ``` On the client, give the pool a TLS connector and turn on `secure(true)`, so requests carry the `https` scheme: ```rust,no_run use ntex::tls::rustls::TlsConnector; pub fn connect(addr: impl Into, config: ClientConfig) -> GreeterClient { let pool = h2::Client::builder(addr.into()) .connector(TlsConnector::new(config)) .secure(true) .build(SharedCfg::new("GRPC-TLS-CLIENT")); GreeterClient::new(Client::new(pool)) } ``` Both sides must offer `h2` in ALPN (`config.alpn_protocols = vec![b"h2".to_vec()]`); many gRPC clients refuse a server that does not select it. The client must connect to a host name that matches the server certificate, such as `localhost:50056`. Certificates, trust roots and the other TLS backends are covered in the [TLS chapter](10-tls.md). Without TLS, ntex-grpc uses HTTP/2 with *prior knowledge*: the client starts with the HTTP/2 preface right away, with no HTTP/1.1 upgrade. That is what gRPC clients expect for plaintext connections. Tools such as `grpcurl` need the `-plaintext` flag to talk to such a server. ## Testing your services Clients and servers are ordinary ntex services, so tests use the ntex test server. Start the gRPC server on a random port, connect a generated client to it, and make real calls: ```rust,no_run use ntex::server::TestServerBuilder; /// STATEMENT: A unary handler that takes `Request<_>` reads the request headers. /// /// HOW: /// - Call `UnaryEcho` with the `x-echo-prefix` header set to `> `. /// - Check the reply is the message with the prefix in front. #[ntex::test] async fn unary_echo_reads_request_headers() { let calls = Arc::new(AtomicU64::new(0)); let srv = TestServerBuilder::new(async move |_| grpc_server(calls.clone())) .config(server_cfg()) .start(); let (echo, _) = connect(srv.addr().to_string()); let msg = echo_request("hello"); let mut req = echo.unary_echo(&msg); req.header(PREFIX_HEADER, "> ").timeout(Seconds(3)); assert_eq!(req.send().await.unwrap().message, "> hello"); } ``` `TestServerBuilder::config()` takes the same `SharedCfg` as the binary, so tests run with the production HTTP/2 settings. A few more patterns from the examples: - **Test unknown methods.** Implement `MethodDef` by hand with a `PATH` the server does not have, send it with `ntex_grpc::client::Request::new`, and check the `UNIMPLEMENTED` or fallback answer. - **Test wire compatibility.** Generate the same `.proto` twice, once with `--map` and once without, and call the server that uses custom types with the plain client. That is exactly what other implementations see. - **Unit-test custom field types.** Encode a message with `Message::write`, decode it with `Message::read`, compare the two, and check that `merge` rejects invalid input with an error. - **Set a timeout on every call in tests**, so a broken server fails the test instead of hanging it. ## Troubleshooting | Symptom | Likely cause | Fix | | --- | --- | --- | | The code generator cannot find `protoc` | `protoc` is not installed or not in `PATH` | install it, or set `PROTOC` to its path | | The generated file has an unexpected name | the file is named after the proto `package` | use the package name in `mod` declarations | | "the trait `NamedService` is not implemented" | the `server` attribute is not imported | use `use ntex_grpc::server;`, not `use ntex_grpc::server::{self}` | | A client-streaming call never returns | `send()` was awaited before the messages were sent | send and await the reply at the same time, with `join` | | Every call fails with `UNIMPLEMENTED` | the client and the server disagree on the package or service name | compare the request path with `ServiceDef::NAME` on both sides | | Calls fail with `INVALID_ARGUMENT` "Cannot decode grpc message" | the `.proto` files differ, or a custom field type rejected a value | regenerate from the same `.proto`; check your `merge` | | Calls fail with `RESOURCE_EXHAUSTED` | a message is larger than `max_message_size` | raise the limit on the side that receives the message | | Calls fail with `NotGrpc` | the address points at something that is not a gRPC server, or a proxy answered | check the address and the HTTP status in the error | | `grpcurl` reports a TLS error against a plaintext server | `grpcurl` uses TLS by default | add `-plaintext` | | `grpcurl` cannot list services | the server does not offer reflection | pass the `.proto` file with `-proto` | | A TLS client refuses to talk to the server | ALPN does not select `h2` | add `h2` to `alpn_protocols` on both sides | | A call hangs forever | no timeout and a stalled server | set `timeout()` on every call | ## Production checklist - **The generated code is up to date.** `generate.sh` lives next to the `.proto` files and its output is committed, so reviews show protocol changes. - **Custom field types match the wire.** `value_len` is exact, `is_default` matches the protobuf default, `merge` returns errors instead of panicking, and varint types override `encoded_len`. - **Every call has a deadline.** Clients set `timeout()`, so a stalled server cannot hold a call forever. - **Retries look at the error.** Only `NotProcessed`, `Connection` with `request_sent == false`, or failures of idempotent methods are retried. - **Clients share pools.** There is one `Client` per backend, cloned into the typed clients and transports that use it. - **Shared state is in an `Arc`.** Services are created per connection, on several worker threads. - **Errors reach clients with the right status.** Methods return `ServerError`s for failed calls, or model errors in the reply with a `oneof`. - **Limits are set.** `max_message_size` fits the largest expected message on both sides, and the HTTP/2 stream and window settings fit the load. - **Compression is a deliberate choice.** If clients compress, the server is built with the `compression` feature. - **TLS offers `h2`.** Both sides use ALPN `h2`, and clients use `secure(true)` and a host name from the certificate. [gRPC]: https://grpc.io [grpcurl]: https://github.com/fullstorydev/grpcurl [ntex-grpc]: https://github.com/ntex-rs/ntex-grpc [`ntex-h2`]: https://docs.rs/ntex-h2 [`NativeType`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/trait.NativeType.html [`Message`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/trait.Message.html [`MethodDef`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/trait.MethodDef.html [`ServiceDef`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/trait.ServiceDef.html [`Transport`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/trait.Transport.html [`StreamTransport`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/trait.StreamTransport.html [`StreamCall`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/struct.StreamCall.html [`Request`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/struct.Request.html [`RequestContext`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/struct.RequestContext.html [`Response`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/struct.Response.html [`ClientError`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/enum.ClientError.html [`ntex_grpc::client::Client`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/client/struct.Client.html [`SendStream`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/struct.SendStream.html [`RecvStream`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/struct.RecvStream.html [`StreamError`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/enum.StreamError.html [`GrpcServer`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/server/struct.GrpcServer.html [`encode_binary_header()`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/fn.encode_binary_header.html [`encode_grpc_message()`]: https://docs.rs/ntex-grpc/latest/ntex_grpc/fn.encode_grpc_message.html [examples]: https://github.com/ntex-rs/examples/tree/main/protocols [helloworld-example]: https://github.com/ntex-rs/examples/tree/main/protocols/grpc-helloworld [custom-types-example]: https://github.com/ntex-rs/examples/tree/main/protocols/grpc-custom-types [routing-example]: https://github.com/ntex-rs/examples/tree/main/protocols/grpc-routing-client [server-example]: https://github.com/ntex-rs/examples/tree/main/protocols/grpc-server [tls-example]: https://github.com/ntex-rs/examples/tree/main/protocols/grpc-tls # MQTT with ntex-mqtt [MQTT] is a lightweight publish/subscribe protocol. It is common in IoT, telemetry and messaging systems, where many small devices keep long-lived connections to a broker. [ntex-mqtt] brings MQTT to ntex. It provides both a client and a server, and both speak the two versions of the protocol still in use, MQTT 3.1.1 and MQTT 5.0. A single server can even accept clients of both versions on the same port. ntex-mqtt is built from the same pieces as the rest of ntex. Connections are ntex I/O objects, handlers are ntex services, settings travel in a [`SharedCfg`], and a server runs inside the ordinary [ntex server](5-server.md). If you know how to write an ntex web service, most of this chapter will feel familiar. Add the crate next to ntex: ```toml [dependencies] ntex = { version = "4", features = ["tokio"] } ntex-mqtt = "9" ``` Here is a server that accepts both protocol versions and prints every message it receives: ```rust,no_run use ntex::SharedCfg; use ntex_mqtt::{MqttServer, v3, v5}; #[derive(Debug)] struct ServerError; impl TryFrom for v5::PublishAck { type Error = ServerError; fn try_from(err: ServerError) -> Result { Err(err) } } #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build() .bind("mqtt", "127.0.0.1:1883", SharedCfg::default(), async |_| { MqttServer::new() .v3( v3::MqttServer::new(async |publish: v3::Publish| { println!("v3 publish to {}", publish.publish_topic()); Ok::<_, ServerError>(()) }) .build(async |connect: v3::Connect| { Ok::<_, ServerError>(connect.ack((), false)) }), ) .v5( v5::MqttServer::new(async |publish: v5::Publish| { println!("v5 publish to {}", publish.publish_topic()); Ok::<_, ServerError>(publish.ack()) }) .build(async |connect: v5::Connect| { Ok::<_, ServerError>(connect.ack(())) }), ) })? .run() .await } ``` Most server examples in this chapter use an error type like `ServerError`. Its `TryFrom` implementation answers one question: can this error be turned into a `PUBACK` that refuses the message, or must it close the connection? Returning `Err` means "close". [Handling publishes](#handling-publishes) shows the other choice. The rest of the chapter explains how the crate is organized, how to configure it, and then covers the client and the server in turn. ## How ntex-mqtt is organized The two protocol versions live in two modules, `ntex_mqtt::v3` and `ntex_mqtt::v5`. They have the same shape: each has a server builder (`MqttServer`), a client connector (`client::MqttConnector`), a `Publish` message, a `Router`, a `MqttSink` and a `codec` module with the packet types. Code written for one version translates to the other almost line by line. MQTT 5 adds reason codes, properties and a few new packets, so its types carry more information. The crate root holds what the versions share: the combined `MqttServer`, the `MqttServiceConfig` settings, and the `Control` messages that report what happens to a connection. Every connection, on either side, is handled by up to three services: - The **publish service** receives `PUBLISH` packets from the peer. This is where your application logic lives. On the client, this role is filled by router handlers and the protocol service. - The **protocol service** receives the other packets that need an answer: `SUBSCRIBE`, `UNSUBSCRIBE`, `PINGREQ`, `DISCONNECT`, `PUBREL` and, in MQTT 5, `AUTH`. It returns the response to send back. - The **control service** is told about events in the connection's life, such as write backpressure or the connection stopping, and why it stopped. The protocol and control services have sensible defaults, so you only replace them when you need to. Two more types show up everywhere. A **sink** (`MqttSink`) is a cheap, cloneable handle for sending packets to the peer: publishes, subscriptions, and the final `DISCONNECT`. A **session** (`v3::Session` or `v5::Session`) bundles the sink with your own per-connection state `St`. It dereferences to `St`, and its `sink()` method returns the sink. ## Configuration Protocol settings live in `MqttServiceConfig`. As with other ntex configuration, you add it to a [`SharedCfg`], and every connection created with that `SharedCfg`, client or server, picks it up: ```rust,no_run use ntex::{SharedCfg, time::Seconds}; use ntex_mqtt::{MqttServiceConfig, QoS}; fn config() -> SharedCfg { SharedCfg::new("MQTT") .add( MqttServiceConfig::new() .set_max_qos(QoS::ExactlyOnce) .set_max_size(1024 * 1024) .set_max_receive(32) .set_connect_timeout(Seconds(10)), ) .into() } ``` MQTT connections are ordinary ntex I/O objects, so an [`IoConfig`] in the same `SharedCfg` controls their buffers, write backpressure and timeouts, as described in the [I/O chapter](7-io.md). The settings fall into three groups. The first covers what a peer may send you: | Setting | Default | Meaning | |---|---|---| | `set_max_qos` | `AtLeastOnce` | Highest QoS a peer may publish with. A higher QoS is a protocol violation and closes the connection. An MQTT 5 server announces it in `CONNACK` | | `set_max_size` | 256 KiB | Largest packet the peer may send; 0 means the protocol maximum of 256 MiB. An MQTT 5 server announces it in `CONNACK` | | `set_max_topic_alias` | 32 | Topic aliases an MQTT 5 client may use when publishing to the server | | `set_check_subs_availability` | `true` | MQTT 5 server only. Close the connection when a client subscribes to a shared or wildcard filter that `CONNACK` declared unavailable. When disabled, the protocol service decides | The second controls flow control, explained in detail [below](#flow-control): | Setting | Default | Meaning | |---|---|---| | `set_max_receive` | 16 | Incoming QoS 1 and 2 publishes being handled at once. MQTT 5 announces it to the peer as Receive Maximum; 0 means 65535. For MQTT 3.1.1 it queues extra publishes; 0 disables the limit | | `set_max_receive_size` | 64 KiB | Total size of incoming publishes being handled. When it is reached, the connection stops reading | | `set_max_queue` | 64 | Responses waiting to be sent. Acks go out in the order the publishes arrived; when the queue is full, the connection stops reading | | `set_max_send` | 16 | Outgoing QoS 1 and 2 publishes waiting for an ack. For MQTT 5 the peer's Receive Maximum can lower it | | `set_min_chunk_size` | 32 KiB | How much of a large payload to collect before handing a chunk to your handler | | `set_max_payload_buffer_size` | 32 KiB | Unread payload to buffer for a streaming publish before the connection stops reading | | `set_handle_qos_after_disconnect` | `None` | Highest QoS of received publishes that are still handed to the publish service after the connection starts closing. By default they are dropped | The third sets timeouts: | Setting | Default | Meaning | |---|---|---| | `protocol_version_timeout` | 5 s | Combined server only. Time a client has to send enough of `CONNECT` to reveal its protocol version | | `set_connect_timeout` | 5 s | Server only. Time a client has to send its `CONNECT` packet | | `set_handshake_timeout` | disabled | Client only. Time allowed for connecting, sending `CONNECT` and receiving `CONNACK` | Keep-alive is not part of `MqttServiceConfig`. A client chooses it in its `CONNECT` packet, and a server may override it while accepting the connection. ## The client The client is a service that opens a connection, performs the MQTT handshake and returns a `Client`. The examples here use MQTT 5. The MQTT 3.1.1 client in `v3::client` works the same way, and the differences are noted as they come up. ### Connecting `v5::client::MqttConnector` is an ntex service. Wrap it in a [`Pipeline`] together with the `SharedCfg` it should use, and call it with a `v5::client::Connect` that describes the connection: ```rust,no_run use ntex::{Pipeline, SharedCfg, time::Seconds}; use ntex_mqtt::v5; #[ntex::main] async fn main() -> Result<(), Box> { let connector = Pipeline::new(SharedCfg::default(), v5::client::MqttConnector::new()); let client = connector .call( v5::client::Connect::new("127.0.0.1:1883") .client_id("sensor-17") .keep_alive(Seconds(30)) .max_packet_size(64 * 1024) .max_receive(32), ) .await?; println!("session present: {}", client.session_present()); Ok(()) } ``` `Connect` builds the `CONNECT` packet. Besides the address, it sets: - `client_id`, the identity the server uses to find your session; - `keep_alive`, how often the client pings an idle connection. It is disabled by default; - `clean_start`, which asks the server to discard any stored session (`clean_session` in MQTT 3.1.1); - `last_will`, a message the server publishes if the client disappears without disconnecting; - `username` and `password`, and in MQTT 5 `auth` for enhanced authentication; - `max_packet_size` and `max_receive`, the client's own limits, announced to the server (MQTT 5 only); - `properties` and `packet`, for direct access to the rest of the packet. The connector returns a `Client` once the server accepts the connection. If the server rejects it, the error carries the server's `CONNACK`, including its reason code. The server's `CONNACK` also shapes the connection. Its Receive Maximum limits how many publishes the client may have unacknowledged, its Maximum Packet Size limits what the client may send, and a Server Keep Alive replaces the client's own keep-alive. `client.packet()` gives access to the full `CONNACK`. ### Secure connections By default the connector opens a plain TCP connection. To use TLS, give it a different transport connector with `connector()`. Any ntex connector works, for example openssl: ```rust,no_run use ntex::{Pipeline, SharedCfg, connect::openssl::SslConnector}; use ntex_mqtt::v5; use openssl::ssl; #[ntex::main] async fn main() -> Result<(), Box> { let mut tls = ssl::SslConnector::builder(ssl::SslMethod::tls())?; tls.set_alpn_protos(b"\x04mqtt")?; let connector = Pipeline::new( SharedCfg::default(), v5::client::MqttConnector::new().connector(SslConnector::new(tls.build())), ); let client = connector .call(v5::client::Connect::new("broker.example.com:8883").client_id("sensor-17")) .await?; # drop(client); Ok(()) } ``` `ntex_mqtt::TLS_PORT` (8883) and `TCP_PORT` (1883) hold the standard ports. The same mechanism carries MQTT over WebSockets; the `mqtt-ws-client` example in the ntex-mqtt repository shows how, and the [MQTT over WebSocket example] has a complete client and server. ### Running the connection A `Client` does nothing on its own. Something has to read packets from the server, answer pings and deliver acks to the publishes you are waiting on. That is the job of the client's dispatcher, which you start by consuming the `Client`: - `start_default()` runs the connection with default handlers. Any publish from the server closes the connection, so this suits a client that only sends. - `start(service)` runs it with your own protocol service, which sees every publish and every other packet the server sends. - `start_with_control(service, control)` also replaces the control service. Each of these returns a future that completes when the connection closes. Take the sink first, then spawn the dispatcher on the current thread: ```rust,no_run # use ntex::{Pipeline, SharedCfg}; # use ntex_mqtt::v5; # #[ntex::main] # async fn main() -> Result<(), Box> { # let connector = Pipeline::new(SharedCfg::default(), v5::client::MqttConnector::new()); let client = connector.call(v5::client::Connect::new("127.0.0.1:1883")).await?; let sink = client.sink(); ntex::rt::spawn(client.start_default()); let ack = sink.publish("sensors/17/temperature").send_at_least_once("21.5".into()).await?; println!("reason: {:?}", ack.reason_code); sink.close(); # Ok(()) # } ``` The client sends keep-alive pings while the dispatcher runs. Acks are read by the dispatcher too, so a publish that waits for its ack waits forever if the dispatcher was never started. ### Publishing `sink.publish(topic)` returns a `PublishBuilder`. Set the packet options you need, then send the payload with the QoS you want: | Method | QoS | Completes when | Returns | |---|---|---|---| | `send_at_most_once` | 0 | The packet is written | `()` | | `send_at_least_once` | 1 | The server sends `PUBACK` | `PublishAck` (`()` in MQTT 3.1.1) | | `send_exactly_once` | 2 | The server sends `PUBREC` | `PublishReceived` | A QoS 1 publish completes with the server's `PUBACK`. In MQTT 5 that ack carries a reason code, and the server can use it to refuse a message without closing the connection, for example with `NotAuthorized` or `QuotaExceeded`. Check `reason_code` when it matters. An error from the call itself means the publish could not be sent or acknowledged at all, usually because the connection closed. QoS 2 takes two round trips. `send_exactly_once` returns once the server has received the message; call `release()` on the result to send `PUBREL` and wait for `PUBCOMP`. If you drop the `PublishReceived` instead, ntex-mqtt releases the publish for you. MQTT 5 publishes can carry properties: user properties, content type, message expiry, response topic and correlation data for request/response, and so on. Set them with `properties()`: ```rust # use ntex_mqtt::v5; # async fn example(sink: v5::MqttSink) -> Result<(), v5::error::SendPacketError> { let ack = sink .publish("sensors/17/temperature") .retain(true) .properties(|props| { props.content_type = Some("text/plain".into()); props.message_expiry_interval = Some(60); props.user_properties.push(("unit".into(), "celsius".into())); }) .send_at_least_once("21.5".into()) .await?; # drop(ack); # Ok(()) # } ``` Packet ids are assigned automatically. You can set one yourself with `packet_id()`, but do not mix both styles on one connection, or ids will collide. ### Publishing without waiting Awaiting each ack limits a task to one message per round trip. When you want to keep many publishes in flight from one place, register an ack callback and send with `send_at_least_once_no_block()`: ```rust # use ntex_mqtt::v5; # fn example(sink: v5::MqttSink) -> Result<(), v5::error::SendPacketError> { sink.publish_ack_cb(|ack, disconnected| { if disconnected { println!("publish {} was not acknowledged", ack.packet_id); } }); if sink.is_ready() { sink.publish("events").send_at_least_once_no_block("started".into())?; } # Ok(()) # } ``` The callback runs for every ack, in the order they arrive. When the connection closes, it runs once more for each publish still in flight, with `disconnected` set to `true`. It runs while the sink is borrowed, so it must not call any sink method; hand the work to a channel or a task instead. `send_at_least_once_no_block()` panics if the sink is not ready, so always check `is_ready()` first, or wait for `ready()` as described below. ### Streaming large payloads A publish normally carries its whole payload in one `Bytes`. For large messages you can send the payload in pieces instead. Tell the builder the total size, and you get a `StreamingPayload` to feed it through: ```rust # use ntex::util::Bytes; # use ntex_mqtt::v5; # async fn example(sink: v5::MqttSink, chunks: Vec) -> Result<(), v5::error::SendPacketError> { let size = chunks.iter().map(|c| c.len() as u32).sum(); let (ack, payload) = sink.publish("files/report.csv").stream_at_least_once(size); let send = async { for chunk in chunks { payload.send(chunk).await?; } Ok::<_, v5::error::SendPacketError>(()) }; let (ack, sent) = ntex::util::join(ack, send).await; sent?; println!("reason: {:?}", ack?.reason_code); # Ok(()) # } ``` The two halves must make progress together: `payload.send()` waits until the publish header has been written, and the ack only arrives after the last chunk. That is why the example joins them. The total of all chunks must match the declared size exactly. `stream_at_most_once(size)` does the same for QoS 0. While a streaming publish is in progress, other publishes on the same connection wait, because MQTT cannot interleave packets inside a payload. Incoming publishes can be streamed as well; see [Reading payloads](#reading-payloads). ### Flow control The server tells the client how many QoS 1 and 2 publishes it may have unacknowledged at once, its Receive Maximum. ntex-mqtt tracks this as **send credit**. Each such publish takes one unit of credit when it is sent and returns it when the ack arrives. When no credit is left, the next `send_at_least_once()` or `send_exactly_once()` simply waits until an ack frees some. Waiting publishes get credit in the order they asked for it. QoS 0 publishes do not use credit. The sink also follows the connection's write backpressure. When the write buffer grows past its high watermark (see the [I/O chapter](7-io.md)), sends wait until it drains. The sink exposes both so you can plan ahead: - `credit()` returns the credit left; - `is_ready()` is `true` when there is credit and no write backpressure; - `ready().await` waits until the sink is ready, and returns `false` if the connection has closed. A producer that should slow down rather than queue up can loop on `ready()`: ```rust # use ntex::util::Bytes; # use ntex_mqtt::v5; # async fn next_reading() -> Option { None } # async fn example(sink: v5::MqttSink) -> Result<(), v5::error::SendPacketError> { while let Some(reading) = next_reading().await { if !sink.ready().await { break; // connection closed } sink.publish("sensors/17").send_at_least_once_no_block(reading)?; } # Ok(()) # } ``` `ready()` does not reserve credit. When several tasks share a sink, another task may use up the credit between your `ready()` and your send. An awaiting send simply waits again, but `send_at_least_once_no_block()` would panic, so in that case check `is_ready()` immediately before it. In the other direction, the client tells the server its own Receive Maximum with `Connect::max_receive()`, and the settings from [Configuration](#configuration) limit how much incoming data it buffers. ### Subscribing To receive messages, subscribe to one or more topic filters. In MQTT 5 each filter comes with options: ```rust # use ntex_mqtt::v5::{self, codec}; # async fn example(sink: v5::MqttSink) -> Result<(), v5::error::SendPacketError> { let ack = sink .subscribe(None) .topic_filter( "commands/sensor-17/#".into(), codec::SubscriptionOptions { qos: codec::QoS::AtLeastOnce, no_local: false, retain_as_published: false, retain_handling: codec::RetainHandling::AtSubscribe, }, ) .send() .await?; for status in &ack.status { println!("subscription: {status:?}"); } # Ok(()) # } ``` The argument to `subscribe()` is an optional subscription identifier. The server attaches it to every publish delivered through this subscription. The `SUBACK` holds one status per filter, in order: the granted QoS, or a reason the server refused that filter. `sink.unsubscribe()` works the same way. In MQTT 3.1.1, `subscribe()` takes no identifier and `topic_filter()` takes just a QoS. The ack is a list of `SubscribeReturnCode`s. Subscribing does not use send credit, but the server's answer is read by the dispatcher, so the dispatcher must be running. ### Receiving publishes Messages from your subscriptions arrive as publishes from the server. To handle them, attach handlers to topics with `resource()`. This turns the `Client` into a `ClientRouter`, which you start like a client: ```rust,no_run use ntex::{Pipeline, SharedCfg}; use ntex_mqtt::v5; #[derive(Debug)] struct Error; impl TryFrom for v5::PublishAck { type Error = Error; fn try_from(err: Error) -> Result { Err(err) } } async fn command(publish: v5::Publish) -> Result { let id = publish.topic().get("id").unwrap_or_default().to_string(); let payload = publish.read_all().await.map_err(|_| Error)?; println!("command for {id}: {payload:?}"); Ok(publish.ack()) } async fn status(publish: v5::Publish) -> Result { println!("status: {:?}", publish.read_all().await); Ok(publish.ack()) } #[ntex::main] async fn main() -> Result<(), Box> { let connector = Pipeline::new(SharedCfg::default(), v5::client::MqttConnector::new()); let client = connector.call(v5::client::Connect::new("127.0.0.1:1883")).await?; let sink = client.sink(); let router = client .resource("commands/{id}/#", command) .resource("status", status); ntex::rt::spawn(router.start_default()); sink.subscribe(None) .topic_filter("commands/sensor-17/#".into(), Default::default()) .topic_filter("status".into(), Default::default()) .send() .await?; ntex::time::sleep(ntex::time::Seconds(60)).await; Ok(()) } ``` Resource patterns match topic levels. A `{name}` segment matches one level and captures it, which you read back with `publish.topic().get("name")`; a trailing `#` matches the rest of the topic. A handler is any service that takes a `Publish` and returns a `PublishAck`. It can also take `&v5::Session<()>` as its first argument. In MQTT 3.1.1, handlers return `()`. The router acknowledges each publish with what the handler returns: `PUBACK` for QoS 1, `PUBREC` for QoS 2. In MQTT 5 a handler can refuse a message with `PublishAck::new(reason)`. A handler error closes the connection, unless the error converts to a `PublishAck`. The `TryFrom` implementation above decides that; returning `Ok(ack)` from it turns the error into a negative ack instead. With `start_default()`, a publish whose topic matches no resource closes the connection. To handle those, and every other packet from the server, use `start()` with a protocol service. ### The protocol service A client protocol service receives `v5::client::ProtocolMessage` values: - `Publish`: a publish that no resource matched; - `PublishRelease`: the server's `PUBREL` for a QoS 2 publish; - `Ping`: a `PINGRESP` from the server; - `Disconnect`: a `DISCONNECT` sent by the server, MQTT 5 only. Each message has an `ack()` method that produces the normal response, and `disconnect()` closes the connection instead. Unrouted publishes can be read and acknowledged here: ```rust,no_run # use ntex::{Pipeline, SharedCfg}; use ntex_mqtt::v5::{self, client::ProtocolMessage, client::ProtocolMessageAck, codec}; async fn protocol(msg: ProtocolMessage) -> Result { Ok(match msg { ProtocolMessage::Publish(publish) => { println!("unrouted {:?}: {:?}", publish.packet().topic, publish.read_all().await); publish.ack(codec::PublishAckReason::Success) } msg => msg.ack(), }) } # #[ntex::main] # async fn main() -> Result<(), Box> { # let connector = Pipeline::new(SharedCfg::default(), v5::client::MqttConnector::new()); # let client = connector.call(v5::client::Connect::new("127.0.0.1:1883")).await?; ntex::rt::spawn(client.start(protocol)); # Ok(()) # } ``` A `ClientRouter` accepts a protocol service the same way, with `start()`. ### The control service The control service hears about the connection's life rather than its packets. It receives `ntex_mqtt::Control` values: - `WrBackpressure` reports when write backpressure starts and stops. - `Stop(reason)` is sent once, as the connection shuts down. The reason is `Error` for an error from one of your services, `Protocol` when the peer broke the protocol or a keep-alive timed out, and `PeerGone` when the connection closed underneath. The service returns an optional packet to send before closing, usually a `DISCONNECT` with a reason code. The default control service does nothing. On the client, set it with `start_with_control()`; the [server section](#the-control-service-1) shows a full example. ### Closing `sink.close()` sends `DISCONNECT` and closes the connection gracefully, flushing what is already buffered. In MQTT 5, `close_with_reason()` sends a `DISCONNECT` of your choice, for example with a session expiry interval or a reason code. `force_close()` drops the connection immediately, discarding buffered data and failing any publishes still in flight. `sink.is_open()` turns `false` as soon as the connection starts closing, whichever side closed it. ## The server An MQTT server is a service factory that plugs into the [ntex server](5-server.md). You can build one for a single protocol version with `v3::MqttServer` or `v5::MqttServer`, or combine the two with `ntex_mqtt::MqttServer`. ### A server for one version A version-specific server needs two things: a publish service, given to `new()`, and a handshake service, given to `build()`. The handshake service receives the client's `CONNECT` and decides whether to accept it: ```rust,no_run use ntex::SharedCfg; use ntex_mqtt::v5; #[derive(Debug)] struct ServerError; impl TryFrom for v5::PublishAck { type Error = ServerError; fn try_from(err: ServerError) -> Result { Err(err) } } #[derive(Debug)] struct Device { id: String, } async fn handshake(connect: v5::Connect) -> Result, ServerError> { let packet = connect.packet(); if packet.username.as_deref() != Some("device") { return Ok(connect.failed(v5::codec::ConnectAckReason::BadUserNameOrPassword)); } let device = Device { id: packet.client_id.to_string(), }; Ok(connect.ack(device)) } async fn publish( session: &v5::Session, publish: v5::Publish, ) -> Result { println!("{} published to {}", session.id, publish.publish_topic()); Ok(publish.ack()) } #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build() .bind("mqtt", "127.0.0.1:1883", SharedCfg::default(), async |_| { v5::MqttServer::new(publish).build(handshake) })? .run() .await } ``` The value you pass to `ack()` becomes the connection's session. Every publish, protocol and control handler can take a `&Session` as its first argument and read it, as `publish` does above. [The handshake](#the-handshake) covers this step in detail. A v3 server looks the same, with two differences: the publish service returns `()`, and `ack()` takes a second argument, the "session present" flag, which tells the client whether you already hold state for it: ```rust,no_run # use ntex::SharedCfg; # use ntex_mqtt::v3; # #[derive(Debug)] # struct ServerError; # #[ntex::main] # async fn main() -> std::io::Result<()> { ntex::server::build() .bind("mqtt", "127.0.0.1:1883", SharedCfg::default(), async |_| { v3::MqttServer::new(async |publish: v3::Publish| { println!("publish to {}", publish.publish_topic()); Ok::<_, ServerError>(()) }) .build(async |connect: v3::Connect| Ok::<_, ServerError>(connect.ack((), false))) })? .run() .await # } ``` ### The handshake Every connection starts with a handshake. The server reads the client's first packet, which must be a `CONNECT` and must arrive within the connect timeout (see [Configuration](#configuration)). Otherwise the server closes the connection, and the handshake service is never called. The handshake service receives the packet wrapped in a `Connect`, which gives it everything it needs to decide about the client: - `packet()` is the decoded `CONNECT`: client id, username and password, keep-alive, last will and, in MQTT 5, its properties. `packet_mut()` lets you take fields out of it instead of copying them. - `io()` is the connection itself, for querying the peer address or TLS details. - `st()` is the connection state collected before the handshake (see [Server state](#server-state)). - `sink()` is the client's sink, which you can keep for sending to the client later. The service answers with a `ConnectAck`, which is both the `CONNACK` packet and the outcome of the handshake: - `ack()` accepts the client and gives the connection its session state. - `failed()` refuses the client with a reason code. The server sends that `CONNACK` and closes the connection. In MQTT 5, `fail_with()` takes a complete `CONNACK` packet, so you can add a reason string or other properties. MQTT 3.1.1 has fewer codes and a helper for each: `identifier_rejected()`, `bad_username_or_pwd()`, `not_authorized()` and `service_unavailable()`. - An error from the handshake service closes the connection without sending a `CONNACK` at all. An accepted `ConnectAck` has a few more settings for the connection. In MQTT 5, `keep_alive()` sets the Server Keep Alive, and `with()` gives direct access to the `CONNACK` packet: assigned client id, session expiry, response information, limits and other properties. In MQTT 3.1.1, `idle_timeout()` sets how long the connection may stay silent, and `max_packet_size()` limits incoming packets. Both versions have `max_send()`, which sets the [send credit](#flow-control-1) for this connection. #### From acknowledgement to session `ack()` wraps your value in a `Session`. A session is a cheap, reference-counted handle that holds three things: your value, which the session dereferences to; the client's sink, available as `session.sink()`; and the connection's configuration, available as `session.cfg()`. Since the session is shared, any part of your value that changes during the connection needs interior mutability, such as a `Cell` or a `RefCell`. When the handshake service returns an accepted `ConnectAck`, the server finishes setting up the connection: 1. It applies the acknowledgement's settings. In MQTT 5, the limits you advertise in the `CONNACK`, such as maximum QoS, receive maximum, topic alias maximum and maximum packet size, are the limits the server then enforces on the client. The send credit never exceeds the receive maximum the client announced in its `CONNECT`. 2. It sends the `CONNACK`. 3. It creates the connection's publish, protocol and control services from their factories, passing the session to `create()`. These services are created once per connection, so a factory you write yourself can prepare per-connection resources from the session. If a factory fails, the connection is closed. 4. It starts reading packets. The session becomes the state of the connection's pipeline, which is why every handler receives it as `&Session`. The session lives as long as something holds a clone of it. Normally that is the connection's services, so it is dropped when the connection closes. With `ack()`, only the connection's own services hold the session. If other parts of your application need to reach the connection, use `ack_and_session()`, which returns the session alongside the acknowledgement. Here the handshake registers every device in a shared map, and the control service removes it again when its connection stops: ```rust,no_run use std::{cell::RefCell, collections::HashMap, rc::Rc}; use ntex::SharedCfg; use ntex_mqtt::{Control, v5}; # #[derive(Debug)] # struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { # Err(err) # } # } type Registry = Rc>>>; struct Device { id: String, registry: Registry, } async fn control( session: &v5::Session, msg: Control, ) -> Result, ServerError> { if let Control::Stop(_) = msg { session.registry.borrow_mut().remove(&session.id); } Ok(None) } #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build() .bind("mqtt", "127.0.0.1:1883", SharedCfg::default(), async |_| { let registry = Registry::default(); v5::MqttServer::new(async |p: v5::Publish| Ok::<_, ServerError>(p.ack())) .control(control) .build(async move |connect: v5::Connect| { let device = Device { id: connect.packet().client_id.to_string(), registry: registry.clone(), }; let (ack, session) = connect.ack_and_session(device); registry.borrow_mut().insert(session.id.clone(), session); Ok::<_, ServerError>(ack) }) })? .run() .await } ``` Any handler can now look up another device and publish to it through its `sink()`. The server calls the factory closure once per worker, so each worker has its own registry and it never needs locking. Removing the entry when the connection stops matters: the registry holds a clone of the session, so without the cleanup the session would never be dropped. ### Serving both versions Clients state their protocol version in the `CONNECT` packet. The combined `ntex_mqtt::MqttServer` reads it and hands the connection to a v3 or a v5 server, as the quick start at the top of the chapter shows. Build each version's server as usual and pass it to `v3()` and `v5()`. A version you do not configure is refused. The two servers must use the same error type, but each has its own handlers and session type. A client has `protocol_version_timeout` from `MqttServiceConfig` to send enough of its `CONNECT` for the combined server to read the version. ### Server state An MQTT server works with three kinds of state, each with its own lifetime. They all build on the general mechanisms described in the [state chapter](3-state.md). **Worker state** lives as long as a server worker. When you start the server with `build_with_config()`, each worker creates its own state, and the MQTT server runs in a pipeline that holds it (see [Worker and request state](6-server-app.md)). The handshake service can read it if you write the handshake as a [state-aware function](3-state.md#state-aware-functions) that takes the worker state as its first argument. Worker state is the place for resources shared by all connections of a worker, such as a database pool or a device registry. **Connection state** is collected before the MQTT protocol starts. A service in front of the MQTT server can return the connection wrapped in a [`State`](3-state.md#carrying-state-with-a-request) together with any data it gathered, for example the peer address or the TLS server name. Create the server with `MqttServer::with()` instead of `new()` to accept that state; the handshake then receives a `Connect`, and `connect.st()` returns the collected value. **Session state** lives as long as one MQTT connection. It is the value you pass to `ack()`, and the publish, protocol and control services see it as `&Session`. The other two kinds of state are not passed to these services directly, so copy whatever they need into the session during the handshake. Cloning an `Rc` from the worker state is cheap and gives every handler access to the worker's shared resources. The example below uses all three. The worker state holds a list of allowed devices and a message counter, `accept` records the peer address, and the handshake combines both into the session: ```rust,no_run use std::{cell::Cell, collections::HashSet, net::SocketAddr, rc::Rc}; use ntex::io::{Io, types::PeerAddr}; use ntex::service::{State, service}; use ntex::SharedCfg; use ntex_mqtt::{MqttError, v5}; # #[derive(Debug)] # struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { # Err(err) # } # } /// Worker state, created once per worker. #[derive(Clone)] struct AppState { allowed: Rc>, received: Rc>, } /// Connection state, collected before the MQTT handshake. struct Peer { addr: Option, } /// Session state, one per MQTT connection. struct Device { id: String, peer: Option, app: AppState, } async fn accept( _: &AppState, io: Io, ) -> Result, MqttError> { let addr = io.query::().get().map(|addr| addr.into_inner()); Ok(State { req: io, state: Peer { addr } }) } async fn handshake( app: &AppState, connect: v5::Connect, ) -> Result, ServerError> { let id = connect.packet().client_id.to_string(); if !app.allowed.contains(&id) { return Ok(connect.failed(v5::codec::ConnectAckReason::NotAuthorized)); } let device = Device { id, peer: connect.st().addr, app: app.clone(), }; Ok(connect.ack(device)) } async fn publish( session: &v5::Session, publish: v5::Publish, ) -> Result { let received = &session.app.received; received.set(received.get() + 1); println!( "{} ({:?}) published to {}, {} messages so far", session.id, session.peer, publish.publish_topic(), received.get(), ); Ok(publish.ack()) } #[ntex::main] async fn main() -> std::io::Result<()> { ntex::server::build_with_config(async || { Ok(AppState { allowed: Rc::new(HashSet::from(["sensor-1".to_string()])), received: Rc::new(Cell::new(0)), }) }) .bind("mqtt", "127.0.0.1:1883", SharedCfg::default(), async |_: &AppState| { service(accept).and_then(v5::MqttServer::with(publish).build(handshake)) })? .run() .await } ``` Each worker has its own `AppState`, so `received` counts only the messages handled by that worker. Share a value through an `Arc` if it must be visible across workers. ### Configuring the server The server takes its settings from the `SharedCfg` passed to `bind()`: ```rust,no_run # use ntex::{SharedCfg, time::Seconds}; # use ntex_mqtt::{MqttServiceConfig, QoS, v5}; # #[derive(Debug)] struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { Err(err) } # } # #[ntex::main] # async fn main() -> std::io::Result<()> { let cfg = SharedCfg::new("MQTT").add( MqttServiceConfig::new() .set_max_qos(QoS::ExactlyOnce) .set_max_size(1024 * 1024) .set_max_receive(64) .set_connect_timeout(Seconds(10)), ); ntex::server::build() .bind("mqtt", "0.0.0.0:1883", cfg, async |_| { v5::MqttServer::new(async |p: v5::Publish| Ok::<_, ServerError>(p.ack())) .build(async |c: v5::Connect| Ok::<_, ServerError>(c.ack(()))) })? .run() .await # } ``` An MQTT 5 server announces several of these settings to clients in `CONNACK`: the maximum QoS, the maximum packet size, the Receive Maximum and the topic alias limit. MQTT 3.1.1 has no way to announce them, so a client that exceeds the limits is simply disconnected. For TLS, bind the server with an ntex TLS acceptor in front of it; the `openssl` example in the ntex-mqtt repository shows how. To serve MQTT over WebSockets, upgrade an HTTP/1 connection and hand the WebSocket transport to the MQTT server, as the [MQTT over WebSocket example] does. ### Handling publishes The publish service receives a `Publish` for every `PUBLISH` packet. It gives you the packet's details: `publish_topic()`, `qos()`, `retain()`, `dup()`, `id()`, and in MQTT 5 the properties through `packet().properties`. If the client used a topic alias, ntex-mqtt has already resolved it, so the topic is always complete. The service's response is the acknowledgement: - For QoS 0 nothing is sent. - For QoS 1, ntex-mqtt sends `PUBACK` when the service returns. - For QoS 2, it sends `PUBREC` when the service returns. The client's `PUBREL` then goes to the protocol service, whose default answers with `PUBCOMP`. Because the ack is sent only after your service returns, returning means "I have taken responsibility for this message". Store it, forward it, or reject it before you return. In MQTT 5, `publish.ack()` creates a successful `PublishAck`. Build one with `PublishAck::new(reason)` to refuse a message without closing the connection, and add a reason string or user properties if you like: ```rust # use ntex_mqtt::v5::{self, codec::PublishAckReason}; # #[derive(Debug)] struct ServerError; # fn allowed(_: &str) -> bool { true } async fn publish(publish: v5::Publish) -> Result { if !allowed(publish.publish_topic()) { return Ok(v5::PublishAck::new(PublishAckReason::NotAuthorized) .reason("topic is read-only".into())); } Ok(publish.ack()) } ``` A service error closes the connection, unless it can become a negative ack. That is what the `TryFrom for v5::PublishAck` implementation in the examples decides: returning `Err` keeps the error, returning `Ok(ack)` sends the ack and keeps the connection open. MQTT 3.1.1 has no negative acks, so there a service error always closes the connection. Acks go out in the order publishes arrived, even when the services finish in a different order. An ack that is ready early waits in the response queue, whose size `set_max_queue` limits. ### Reading payloads `publish.read_all().await` returns the whole payload. Small payloads arrive together with the packet, so this returns at once. Large payloads are different. When a publish is bigger than what has arrived so far, ntex-mqtt does not wait for all of it. It calls your service as soon as the header is in, and streams the rest of the payload to you in chunks of at least `set_min_chunk_size` bytes. You can still call `read_all()`, or process the chunks as they come and never hold the whole message in memory: ```rust # use ntex_mqtt::v5; # #[derive(Debug)] struct ServerError; async fn upload(publish: v5::Publish) -> Result { let mut total = 0; while let Some(chunk) = publish.read().await.map_err(|_| ServerError)? { total += chunk.len(); } println!("received {total} of {} bytes", publish.payload_size()); Ok(publish.ack()) } ``` `payload_size()` tells you the full size up front. If you use `take_payload()` to move the payload to another task, keep reading it there: dropping a payload before it has been read to the end closes the connection, because the rest of the stream has nowhere to go. ### Routing Rather than one publish service with a big `match` on topics, you can give `new()` a `Router`. It dispatches publishes to handlers by topic, with a default handler for anything that does not match: ```rust # use ntex_mqtt::v5; # #[derive(Debug)] struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { Err(err) } # } # #[derive(Debug)] struct Device; # fn run() -> std::io::Result<()> { # let _ = ntex::server::build().bind("mqtt", "127.0.0.1:1883", ntex::SharedCfg::default(), async |_| { let server = v5::MqttServer::new( v5::Router::new(async |publish: v5::Publish| { println!("unrouted publish to {}", publish.publish_topic()); Ok::<_, ServerError>(publish.ack()) }) .resource(["alerts", "alarms"], async |publish: v5::Publish| { Ok(publish.ack()) }) .resource( "devices/{id}/telemetry", async |session: &v5::Session, publish: v5::Publish| { let id = publish.topic().get("id").unwrap_or_default(); println!("telemetry from {id} via {session:?}"); Ok(publish.ack()) }, ), ) .build(async |connect: v5::Connect| Ok::<_, ServerError>(connect.ack(Device))); # server # })?; # Ok(()) # } ``` A resource takes one pattern or a list of them. Patterns match the topic level by level, `{name}` segments capture a level, and a trailing `#` matches the rest. Each handler is a full service with its own readiness, so a busy handler slows down only the topics it serves. `v3::Router` works the same way. ### Sending to clients A server publishes to a client through that client's sink. Get it from `connect.sink()` during the handshake, keep it in your session state or a shared registry, or reach it with `session.sink()` inside a handler: ```rust # use ntex_mqtt::v5; # #[derive(Debug)] struct ServerError; async fn handshake(connect: v5::Connect) -> Result, ServerError> { let sink = connect.sink(); ntex::rt::spawn(async move { let _ = sink .publish("welcome") .send_at_least_once("hello".into()) .await; }); Ok(connect.ack(())) } ``` A server sink has the same publishing API as a client sink: QoS levels, properties, streaming, ack callbacks, credit and `ready()`. Only `subscribe()` and `unsubscribe()` are refused, since only clients send those packets. The sink also closes the connection: `close()` sends `DISCONNECT`, which in MQTT 5 can carry a reason code via `close_with_reason()`. There is one trap to avoid. Acks from the client are read by the same connection that delivers its publishes, and the connection stops reading while it is busy (see the next section). A publish handler that awaits the ack of a message it sent back to *the same client* can therefore wait forever: the ack cannot be read until the handler returns. Spawn such sends, as the handshake above does, or use `send_at_least_once_no_block()` with an ack callback. ### Flow control ntex-mqtt protects the server from fast or misbehaving clients in a few layers. **Incoming publishes.** `set_max_receive` limits how many QoS 1 and 2 publishes a connection handles at once. An MQTT 5 server announces it as Receive Maximum, and a client that sends more is breaking the protocol. MQTT 3.1.1 has no such announcement, so extra publishes wait in order for a free slot. **Read pause.** The connection stops reading new packets while: - the publish or protocol service is not ready; - the response queue is full (`set_max_queue`); - the publishes being handled add up to `set_max_receive_size` bytes; - a streaming payload has `set_max_payload_buffer_size` bytes your handler has not read yet. Reading resumes once the handler has read the buffer down to half that size. While reading is paused, the client's data waits in its TCP buffers, so TCP flow control eventually slows the client down. No read timeout runs during a pause, because the delay is the server's own doing, not the client's. **Outgoing publishes.** Publishes to a client use send credit, just as on the client side. The credit is `set_max_send`, or `max_send()` on the `ConnectAck`, and in MQTT 5 at most the client's Receive Maximum. Sends wait for credit and for write backpressure, so a slow client slows its senders down instead of filling server memory. Your own services take part through readiness. If a publish handler reports that it is not ready, for example because a downstream queue is full, the connection stops reading until it is. Services whose readiness can change must notify ntex-mqtt through `ReadyNotifier` in `bind()`; the [pipeline chapter](2-pipeline.md) explains how. ### The protocol service The protocol service handles everything a client sends apart from publishes. Without one, the server answers pings, `DISCONNECT` and `PUBREL`, and closes the connection on anything else, including subscriptions. A broker therefore always needs one. It receives a `ProtocolMessage` and returns a `ProtocolMessageAck`: ```rust # use ntex::util::ByteString; # use ntex_mqtt::v5::{self, ProtocolMessage, ProtocolMessageAck, QoS}; # use std::cell::RefCell; # #[derive(Debug)] struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { Err(err) } # } #[derive(Default)] struct Client { topics: RefCell>, } async fn protocol( session: &v5::Session, msg: ProtocolMessage, ) -> Result { Ok(match msg { ProtocolMessage::Subscribe(mut subscribe) => { for mut sub in subscribe.iter_mut() { if sub.topic().starts_with("$SYS") { sub.fail(v5::codec::SubscribeAckReason::NotAuthorized); } else { session.topics.borrow_mut().push(sub.topic().clone()); sub.confirm(QoS::AtLeastOnce); } } subscribe.ack() } ProtocolMessage::Unsubscribe(unsubscribe) => { let mut topics = session.topics.borrow_mut(); topics.retain(|t| unsubscribe.iter().all(|u| u != t)); unsubscribe.ack() } msg => msg.ack(), }) } # fn run() -> std::io::Result<()> { # let _ = ntex::server::build().bind("mqtt", "127.0.0.1:1883", ntex::SharedCfg::default(), async |_| { let server = v5::MqttServer::new(async |p: v5::Publish| Ok::<_, ServerError>(p.ack())) .protocol(protocol) .build(async |c: v5::Connect| Ok::<_, ServerError>(c.ack(Client::default()))); # server # })?; # Ok(()) # } ``` For a `SUBSCRIBE`, visit each topic filter and either `confirm()` it with the QoS you grant or `fail()` it. Filters you leave alone are refused. In MQTT 5 each one has `options()` with the client's QoS, no-local, retain-as-published and retain-handling choices, and the whole packet is available through `packet()`. Unsubscribe works the same way, with `success()` and `fail()`. The other messages are: - `Ping`, answered with `PINGRESP` by `ack()`; - `Disconnect`, the client's `DISCONNECT`. In MQTT 5 it carries a reason code and may update the session expiry interval; - `PublishRelease`, the `PUBREL` that finishes a QoS 2 publish; - `Auth`, MQTT 5 only, a step of an enhanced authentication exchange. Answer with `auth.ack(packet)`. Any message can be turned into a disconnect with `disconnect()` (in MQTT 5 `disconnect_with()` picks the reason code). Protocol messages are handled one at a time, in order, with up to 16 buffered. ### The control service The control service is where you observe connections coming and going. It receives the same `Control` messages as on the client, and can return a packet to send before the connection closes: ```rust # use ntex_mqtt::{Control, Reason, v5}; # #[derive(Debug)] struct ServerError; # impl TryFrom for v5::PublishAck { # type Error = ServerError; # fn try_from(err: ServerError) -> Result { Err(err) } # } # #[derive(Debug)] struct Device; async fn control( session: &v5::Session, msg: Control, ) -> Result, ServerError> { match msg { Control::Stop(Reason::Error(err)) => { println!("{session:?}: service error {:?}", err.get_ref()); let disconnect = v5::codec::Disconnect::new( v5::codec::DisconnectReasonCode::ImplementationSpecificError, ); Ok(Some(v5::codec::Packet::from(disconnect).into())) } Control::Stop(Reason::Protocol(err)) => { println!("{session:?}: protocol error {:?}", err.get_ref()); Ok(None) } Control::Stop(Reason::PeerGone(_)) => { println!("{session:?}: connection lost"); Ok(None) } Control::WrBackpressure(bp) => { println!("{session:?}: write backpressure {}", bp.enabled()); Ok(None) } } } # fn run() -> std::io::Result<()> { # let _ = ntex::server::build().bind("mqtt", "127.0.0.1:1883", ntex::SharedCfg::default(), async |_| { let server = v5::MqttServer::new(async |p: v5::Publish| Ok::<_, ServerError>(p.ack())) .control(control) .build(async |c: v5::Connect| Ok::<_, ServerError>(c.ack(Device))); # server # })?; # Ok(()) # } ``` `Stop` arrives exactly once per accepted connection, which makes it the place to clean up: remove the client from a registry, publish its presence, release resources. A graceful `DISCONNECT` from the client goes to the protocol service first, then the connection stops. When MQTT 5 shuts a connection down because of a protocol error, ntex-mqtt sends the matching `DISCONNECT` reason code itself, so returning `None` is usually right. For MQTT 3.1.1, which has no server `DISCONNECT`, return `None`. ### MQTT 5 features Most MQTT 5 additions are packet fields and properties, and ntex-mqtt gives you direct access to them rather than hiding them behind policy: - **Reason codes** appear in `CONNACK`, `PUBACK`, `SUBACK`, `UNSUBACK` and `DISCONNECT`, through the builders shown above. - **User properties and reason strings** can be added to acks with `properties()` and `reason()`, and read from any packet. - **Topic aliases** from clients are resolved before your publish service sees the message, up to `set_max_topic_alias` aliases per connection. To send aliases to a client, set `properties.topic_alias` on your publishes, within the limit the client gave in `CONNECT`. - **Session expiry** and **will delay** come from the `CONNECT` and `DISCONNECT` packets; storing and expiring sessions is up to your application. - **Request/response** uses the response topic and correlation data properties of a publish. - **Subscription identifiers** are in the `SUBSCRIBE` packet; attach them to matching publishes with `properties.subscription_ids`. - **Server Keep Alive** and **assigned client identifiers** are set on the `ConnectAck`. - **Enhanced authentication** runs through `Auth` protocol messages. ntex-mqtt is a protocol framework rather than a ready-made broker. It handles the protocol rules, ordering, flow control and limits; matching subscriptions to publishes, retained messages and persistent sessions are left to you, so you can store them however your application needs. ## Summary - ntex-mqtt provides an MQTT client and server, each for MQTT 3.1.1 (`v3`) and MQTT 5 (`v5`). The combined `MqttServer` accepts both on one port. - Each connection has a publish service, a protocol service for other packets and a control service for lifecycle events. A sink sends packets to the peer, and a session holds your per-connection state. - `MqttServiceConfig` in a `SharedCfg` sets limits, flow control and timeouts. - The client connects through `MqttConnector`, publishes and subscribes through its sink, and handles incoming messages with resources and a protocol service. Its dispatcher must be running for acks to arrive. - The server accepts or rejects clients in the handshake service, handles publishes directly or through a `Router`, answers subscriptions in the protocol service, and sends to clients through their sinks. - Acks are sent when your service returns, in arrival order. Send credit, write backpressure and read pause keep both sides from overrunning each other. Never await an ack from the client whose publish you are handling. For a complete program, see the [MQTT over WebSocket example]: an MQTT 3.1.1 server behind an HTTP/1 WebSocket upgrade, plus a client that publishes to it. The [ntex-mqtt examples] cover more ground, including TLS, sessions, routing and subscriptions. [MQTT]: https://mqtt.org [ntex-mqtt]: https://docs.rs/ntex-mqtt [ntex-mqtt examples]: https://github.com/ntex-rs/ntex-mqtt/tree/main/examples [MQTT over WebSocket example]: https://github.com/ntex-rs/examples/tree/main/protocols/mqtt [`SharedCfg`]: https://docs.rs/ntex/latest/ntex/struct.SharedCfg.html [`IoConfig`]: https://docs.rs/ntex/latest/ntex/io/struct.IoConfig.html [`Pipeline`]: https://docs.rs/ntex/latest/ntex/struct.Pipeline.html # Securing Connections with TLS Sooner or later, almost every service has to leave `localhost`. When it does, its traffic crosses networks you do not control, and you want two guarantees: nobody in between can read or alter the data, and each side is talking to the peer it expects. TLS provides both. The good news is that adding TLS to an ntex application changes how you set up a connection, not how you handle its messages. TLS encrypts the traffic and lets peers authenticate each other, while your HTTP handlers, WebSocket services, and custom protocols continue to work with ordinary application bytes. TLS does not depend on the runtime's I/O implementation either. All ntex TLS backends work on in-memory byte streams through the I/O filter layer; they do not read from or write to sockets themselves. You can use them with any ntex-supported I/O or runtime backend, as long as the TLS backend supports your platform. ntex also provides a platform-independent way to work with X.509 certificates and identities through its [`x509`] module. You use a common API for loading identities, inspecting certificates, and checking trust, while ntex uses the appropriate platform or backend functionality underneath. This chapter starts with a small HTTPS server you can run in a few minutes. It then walks through choosing a backend, configuring connections, and working with certificates. Depending on where you are starting from: - **New to TLS in ntex?** Read the [quick start](#quick-start-an-https-server) and the [certificate primer](#certificates-in-a-nutshell). - **Already have a TLS configuration?** Go straight to [configuring ntex components](#configuring-ntex-components). - **Need to identify your clients?** Start with [client-certificate authentication](#client-certificate-authentication). - **Preparing for production?** Finish with the [checklist](#production-checklist). The snippets focus on configuration. For complete programs, use the [ntex examples repository][examples]; each example's README includes dependencies, certificate setup, and run commands. ## Supported backends ntex supports three TLS backends. Their types are available through `ntex::tls`, or directly from `ntex_tls`. | Backend | Enable it | Platform and configuration | | --- | --- | --- | | OpenSSL | `ntex/openssl` or `ntex-tls/openssl` | Platforms with OpenSSL; requires its native libraries and headers | | rustls | `ntex/rustls` or `ntex-tls/rustls` | Configured with rustls `ClientConfig` and `ServerConfig`; choose a crypto provider | | Schannel | Available automatically on Windows | Windows-native identities, trust policy, protocol versions and cipher suites | Choose OpenSSL or rustls by enabling the corresponding feature; neither is enabled by default in `ntex-tls`. Schannel is already available on Windows, so you do not need to enable its compatibility-only `schannel` feature. For an OpenSSL application, start with: ```toml [dependencies] ntex = { version = "4", features = ["tokio", "openssl"] } openssl = "0.10" ``` For rustls, you also need a crypto provider: the implementation of the cryptographic algorithms. The examples in this chapter use ring: ```toml [dependencies] ntex = { version = "4", features = ["tokio", "rustls"] } ntex-tls = { version = "4.7", features = ["rustls-ring"] } rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] } rustls-pemfile = "2" ``` Here, `ntex-tls/rustls-ring` enables rustls and makes ring available to the X.509 helpers when no process-wide provider has been installed. Both examples choose Tokio, but you can use another ntex-supported runtime without changing the TLS configuration. ### Choosing a backend All three backends give you the same ntex integration, so the choice is mostly about your platform and operations rather than your application code: - **rustls** is a good default for new services. It is written in Rust, needs no system TLS library, and makes cross-compilation and static builds simple. You choose its crypto provider explicitly. - **OpenSSL** fits best when your organization already standardizes on it, when you need to reuse existing OpenSSL configuration, or when you need a feature only OpenSSL provides here, such as pre-shared-key (PSK) identities. - **Schannel** is the natural choice for Windows-first deployments. It uses the Windows certificate stores, trust policy, and keys that can be used without being exported. Because protocol code does not depend on the backend, switching later usually means changing only the configuration code shown in this chapter. ### TLS and X.509 backend selection are different The backend that encrypts a connection is not necessarily the one that loads or verifies certificates. You can enable both TLS features and choose a transport for each connection, but the [`x509`] helpers follow a separate rule: they always use native APIs on Windows; elsewhere, they use rustls when enabled and OpenSSL otherwise. If verification fails, they do not retry with another backend. The `x509` module is always available on Windows. On other platforms, enable OpenSSL or rustls; enabling `x509` alone is not enough. There is one extra dependency to keep in mind: on Unix, loading a PKCS#12 archive requires OpenSSL, even if rustls handles your TLS connections and certificate verification. ## Quick start: an HTTPS server Before looking at the moving parts, here is the whole thing working. This program serves one route over HTTPS using rustls and the dependencies listed above: ```rust,no_run use std::sync::Arc; use ntex::tls::{rustls::certified_key, x509::Identity}; use ntex::web::{self, App, HttpResponse}; use rustls::{ServerConfig, sign::SingleCertAndKey}; #[ntex::main] async fn main() -> Result<(), Box> { // The server's certificate chain and private key. let identity = Identity::from_pem( &std::fs::read("certs/server.pem")?, &std::fs::read("certs/server.key")?, )?; let key = certified_key(&identity)?; let config = ServerConfig::builder_with_provider(Arc::new( rustls::crypto::ring::default_provider(), )) .with_safe_default_protocol_versions()? .with_no_client_auth() .with_cert_resolver(Arc::new(SingleCertAndKey::from(key))); web::server(async |_| { App::new().service( web::resource("/").to(|| async { HttpResponse::Ok().body("Hello over TLS!") }), ) }) .bind_rustls("127.0.0.1:8443", &config, ntex::SharedCfg::new("HTTPS"))? .run() .await?; Ok(()) } ``` You need a certificate to run it. The quickest way is to clone the [examples repository][examples], run `sh tls/generate-certs.sh` from its root, and copy `server.pem`, `server.key`, and `ca.pem` from `tls/certs` into a `certs` directory next to your project. Then, in another terminal: ```sh curl --cacert certs/ca.pem https://localhost:8443/ ``` The `--cacert` option tells curl to trust the example certificate authority. Without it, curl rightly refuses the connection because nobody it trusts has vouched for the server. Notice what did *not* change: the application, its routes, and its handler are the same as for a plain HTTP server. Only the setup code loads an identity, builds a rustls `ServerConfig`, and calls `bind_rustls` instead of `bind`. The rest of this chapter explains each of those pieces, and how to do the same with the other backends and on the client side. The [rustls example][rustls-example] is a fuller version of this program. ## TLS as an I/O filter Think of TLS as another layer between the socket and your protocol. On reads, it decrypts bytes before your application sees them. On writes, it encrypts the bytes your application produces: ```text socket <-> TLS <-> HTTP socket <-> TLS <-> WebSocket <-> application protocol ``` Reads move from left to right in these diagrams; writes move in the opposite direction. The filters transform buffers, while the I/O backend handles the socket. The [I/O filter subsystem](7-io.md#filter-subsystem) chapter explains this model in more detail. **Client and server protocol code does not need to depend on a TLS backend to operate over TLS.** Write your protocol against ntex I/O rather than a bare TCP socket, and leave the transport choice to the connection setup: | Connection stage | Responsibility | | --- | --- | | Server listener | Accept a transport connection | | TLS acceptor | Upgrade existing `Io` through a server handshake | | TLS connector | Establish a transport and perform a client handshake | | Protocol service or codec | Process application bytes, independent of the installed TLS backend | Adding TLS preserves any filters already installed on the connection. The type `Io>` describes that stack. Your protocol can stay generic over `F: Filter`, or use `Io::seal` or `Io::boxed` to hide the concrete filter type. Hiding the type does not change the connection: encryption, metadata, readiness, and shutdown behavior all remain in place. For example, this byte-stream service has no TLS imports: ```rust,no_run use ntex::codec::BytesCodec; use ntex::io::IoBoxed; async fn echo(io: IoBoxed) -> std::io::Result<()> { while let Some(bytes) = io.recv(&BytesCodec).await? { io.send(bytes, &BytesCodec).await?; } Ok(()) } ``` The same function accepts plain I/O or `io.boxed()` after a TLS handshake. It echoes byte chunks, not complete application messages: if your protocol has message boundaries, use a framing codec instead of `BytesCodec`. You can see this separation in the examples. The [connection-state example][state-example] performs a handshake after accepting a connection and before constructing its HTTP state. The [WebSocket example][ws-example] uses the same frame handlers for plain and encrypted connections. There is one important exception: if your application uses certificates to decide who may connect, that part of the application needs to inspect and verify the peer's certificate. ## Certificates in a nutshell Most TLS problems are really certificate problems, so a little vocabulary goes a long way. If you already know how certificate chains work, skip ahead. A **certificate** is a signed statement: "this public key belongs to these names." The names live in the certificate's subject alternative names (SANs), for example `DNS:api.example.com`, `IP:10.0.0.5`, or a URI such as `spiffe://example.org/billing`. The certificate also carries a validity period and the identity of its **issuer**, the party that signed it. The matching **private key** never leaves its owner. During the handshake, a peer proves that it holds the key, so a stolen copy of a certificate alone is useless. In ntex, a certificate chain together with its private key is called an **identity**. Issuers are themselves identified by certificates, which forms a **chain**: ```text leaf (api.example.com) -> intermediate CA -> root CA ``` The leaf identifies the endpoint. Intermediates connect it to a root. A root is self-signed, so it proves nothing by itself; it is useful only because *you* decided to trust it. Those locally configured certificates are called **trust anchors**, and certificate authorities (CAs) are the organizations or internal services that issue certificates. To verify a peer, a TLS implementation checks that: 1. the peer proves it owns the leaf's private key; 2. the chain leads from the leaf to one of your trust anchors; 3. every certificate in the path is within its validity period and allowed for this use; 4. for a server, the leaf's names include the host the client meant to reach. Nearly every failure you will see corresponds to one of these steps: an unknown issuer, an expired certificate, or a name mismatch. Keep the list in mind when reading error messages, and fix the configuration rather than disabling the check. ## Configuring a backend A TLS configuration answers four questions: - Which identity, meaning a certificate chain and private key, do I present? - Which certificate authorities do I trust to identify the other side? - As a client, which server name do I expect? - Which application protocols can we use? TLS negotiates these through Application-Layer Protocol Negotiation (ALPN). The server name deserves particular care. The connector normally checks the host from the request, not just the address it connects to. A certificate for `localhost` does not also identify `127.0.0.1` unless it lists that IP address. Server Name Indication (SNI) helps the server choose which certificate to present; it does not replace the client's verification. To try the examples, first run [`tls/generate-certs.sh`][cert-setup] from the examples repository root: ```sh sh tls/generate-certs.sh ``` The script creates a local certificate authority (CA) and certificates with subject alternative names (SANs), which identify their hosts or application identities. It also creates unencrypted PKCS#8 private keys. These files are gitignored and meant only for local development; never deploy them. Run each program from its own example directory so the relative certificate paths resolve correctly. If a connection fails, check the trust configuration and certificate names rather than turning off verification. ### OpenSSL The [OpenSSL example][openssl-example] includes an HTTPS server and client, plus a mutual TLS (mTLS) server that authenticates clients. Start the client configuration by enabling verification and loading the CA certificates it should trust: ```rust,no_run use openssl::ssl::{SslConnector, SslMethod, SslVerifyMode}; fn client_config() -> Result { let mut config = SslConnector::builder(SslMethod::tls())?; config.set_verify(SslVerifyMode::PEER); config.set_ca_file("../certs/ca.pem")?; config.set_alpn_protos(b"\x02h2\x08http/1.1")?; Ok(config.build()) } ``` The ntex connector also checks the server name against the connection host. For production, replace the example CA with the appropriate system or organization trust roots. The server needs an identity to present to clients. Load it and add it to the acceptor configuration: ```rust,no_run use ntex::tls::{openssl::set_identity, x509::Identity}; use openssl::ssl::{SslAcceptor, SslAcceptorBuilder, SslMethod}; fn server_config() -> Result> { let identity = Identity::from_pem( &std::fs::read("../certs/server.pem")?, &std::fs::read("../certs/server.key")?, )?; let mut config = SslAcceptor::mozilla_intermediate_v5(SslMethod::tls())?; set_identity(&mut config, &identity)?; Ok(config) } ``` `set_identity` installs the private key, the endpoint's certificate (the leaf), and its intermediate certificates in chain order. It leaves out the self-signed root and does not change how peers are verified. The `mozilla_intermediate_v5` preset supports modern TLS, including TLS 1.3. You can pass these configurations to the HTTP helpers below, or build `ntex::tls::openssl::SslConnector` and `SslAcceptor` services directly. If you are building a custom protocol, set ALPN for that protocol instead of copying the HTTP protocol list. ### rustls The [rustls example][rustls-example] selects ring explicitly for TLS and enables `rustls-ring` for the portable identity and trust helpers. Loading or converting an identity does not silently install a process-wide crypto provider. A client needs both a crypto provider and a root store: ```rust,no_run use std::{fs::File, io::BufReader, sync::Arc}; use rustls::{ClientConfig, RootCertStore}; fn client_config() -> Result> { let mut roots = RootCertStore::empty(); for cert in rustls_pemfile::certs(&mut BufReader::new(File::open("../certs/ca.pem")?)) { roots.add(cert?)?; } let mut config = ClientConfig::builder_with_provider(Arc::new( rustls::crypto::ring::default_provider(), )) .with_safe_default_protocol_versions()? .with_root_certificates(roots) .with_no_client_auth(); config.alpn_protocols = vec![b"h2".to_vec(), b"http/1.1".to_vec()]; Ok(config) } ``` Here, `with_no_client_auth` means "do not send a client certificate." The client **still verifies the server**. On the server side, convert the portable identity into a rustls signing key. A certificate resolver then makes that identity available during the handshake: ```rust,no_run use std::sync::Arc; use ntex::tls::{rustls::certified_key, x509::Identity}; use rustls::{ServerConfig, sign::SingleCertAndKey}; fn server_config() -> Result> { let identity = Identity::from_pem( &std::fs::read("../certs/server.pem")?, &std::fs::read("../certs/server.key")?, )?; let key = certified_key(&identity)?; let mut config = ServerConfig::builder_with_provider(Arc::new( rustls::crypto::ring::default_provider(), )) .with_safe_default_protocol_versions()? .with_no_client_auth() .with_cert_resolver(Arc::new(SingleCertAndKey::from(key))); config.alpn_protocols = vec![b"h2".to_vec(), b"http/1.1".to_vec()]; Ok(config) } ``` On this side of the connection, `with_no_client_auth` means "do not ask for a client certificate." The server still presents its own identity. To require client certificates, configure a client verifier instead. Be careful if your dependencies enable more than one rustls crypto provider. Before using a default builder that needs a provider, explicitly install the one you intend to use process-wide. `builder_with_provider` only configures that builder; it does not select a provider for every other builder or X.509 operation. ### Windows Schannel The [Schannel example][schannel-example] uses Windows' native TLS backend. Its default client configuration verifies server certificates and names using Windows trust policy: ```rust,no_run #[cfg(windows)] fn client_config() -> ntex::tls::schannel::ClientConfig { ntex::tls::schannel::ClientConfig::new() } ``` This client offers `h2` and `http/1.1`, but sends no client certificate. To send one, call `set_client_cert` explicitly; Schannel does not choose an identity from the user's certificate stores for you. A server requires a native identity: ```rust,no_run #[cfg(windows)] fn server_config( identity: &ntex::tls::x509::Identity, ) -> std::io::Result { ntex::tls::schannel::ServerConfig::new(identity.schannel().clone()) } ``` By default, the server does not ask for client certificates. You can change ALPN with `set_alpn_protocols` on either configuration. The available TLS versions and cipher suites depend on Windows policy and OS support. This client configuration does not provide a separate root store for verifying servers. For local development, follow the example README to add and later remove the example CA in an isolated Windows account. Keep verification enabled; configure Windows trust rather than bypassing it. ## Configuring ntex components Once you have a backend configuration, attach it to the client or server that opens the connection. The APIs differ slightly between HTTP and WebSocket clients, so use the section that matches your application. ### HTTP client For HTTPS, pass your configuration to `ClientBuilder::openssl(config)` or `ClientBuilder::rustls(config)`. Both install a secure connector, and the rest of the client works as usual: ```rust,no_run async fn fetch(config: rustls::ClientConfig) -> Result<(), Box> { let client = ntex::client::Client::builder() .rustls(config) .build(ntex::SharedCfg::new("HTTPS")); let response = client.get("https://localhost:8443/").send().await?; println!("{}: {}", response.status(), String::from_utf8_lossy(&response.body().await?)); Ok(()) } ``` Build the client once and reuse it: it keeps a connection pool, so later requests to the same host can skip the TLS handshake. If you have a custom TLS connector service, including Schannel, pass it to `secure_connector`: ```rust,no_run #[cfg(windows)] fn client() -> ntex::client::Client { use ntex::tls::schannel::{ClientConfig, TlsConnector}; ntex::client::Client::builder() .secure_connector(TlsConnector::with_config(ClientConfig::new())) .build(ntex::SharedCfg::new("HTTPS")) } ``` Do not confuse this with `ClientBuilder::connector`: that method configures **unsecured** connections and leaves the HTTPS connector unchanged. > **Configure verification explicitly.** The built-in default OpenSSL HTTP > connector currently disables certificate verification. Supply a verifying > configuration, such as the OpenSSL configuration above, instead. The default rustls configuration uses webpki roots and sends no client identity. If you enable both backend features, the default HTTP client selects OpenSSL. On Windows without either feature, install Schannel explicitly through `secure_connector`. The [HTTP client example][http-client-example] shows a custom HTTPS connector. The backend examples also show how to use private trust roots and client identities. Once the client is built, request and body handling stay the same. When wrapping connection errors, keep the original error in an application error type or `Box` rather than turning it into an `io::Error` containing only a string. ### WebSocket client `WsClient` has its own connector and does not inherit settings from an HTTP `Client`. Its default connector uses plain TCP: **a `wss://` URL alone does not enable TLS.** Install a TLS connector explicitly. These WebSocket APIs use an HTTP/1.1 upgrade, not HTTP/2 extended CONNECT, so offer `http/1.1` rather than `h2`: ```rust,no_run use ntex::{SharedCfg, ws::WsClient}; use openssl::ssl::{SslConnector, SslMethod, SslVerifyMode}; #[ntex::main] async fn main() -> Result<(), Box> { let mut config = SslConnector::builder(SslMethod::tls())?; config.set_verify(SslVerifyMode::PEER); config.set_ca_file("../../tls/certs/ca.pem")?; config.set_alpn_protos(b"\x08http/1.1")?; let connection = WsClient::new("wss://localhost:8443/ws/", SharedCfg::default()) .openssl(config.build()) .connect() .await? .seal(); println!("Upgrade response: {:?}", connection.response()); Ok(()) } ``` With rustls, set `config.alpn_protocols = vec![b"http/1.1".to_vec()]` and use `.rustls(Arc::new(config))`. With Schannel, restrict ALPN in `ClientConfig` and use `.connector(TlsConnector::with_config(config))`. Unlike the HTTP client's `connector`, `WsClient::connector` replaces the whole connector for this connection, including its TLS behavior. The [secure WebSocket example][ws-example] calls `seal` for both plain and TLS connections. That lets both paths use the same message-handling loop. ### HTTP and WebSocket servers For web applications, use the backend-specific HTTP server helpers: | Method | Arguments after `self` | | --- | --- | | `bind_openssl` | address, `SslAcceptorBuilder`, shared configuration | | `bind_rustls` | address, `&ServerConfig`, shared configuration | | `listen_openssl` | existing listener, shared configuration, `SslAcceptorBuilder` | | `listen_rustls` | existing listener, shared configuration, `ServerConfig` | These helpers perform the TLS handshake before serving HTTP and configure HTTP ALPN for you. Keep your existing routes, middleware, and handlers. A WebSocket connection starts as HTTP, so the server setup is the same: use an HTTPS listener and keep your upgrade and frame handlers. TLS stays in place after the upgrade; `web::ws::start` does not remove it. Use the lower-level server builder when you need Schannel, a custom protocol, or a certificate check before serving any requests. Compose the services in the order you want them to run: ```text listener -> TLS acceptor -> optional authorization gate -> HttpService / protocol service ``` There is no `bind_schannel` web-server helper. The [Schannel server][schannel-example] and both mutual-authentication examples show this service chain. Their HTTP service does not depend on the concrete TLS filter, and they use a shared boxed error type to preserve errors from the handshake, authentication, and HTTP layers. ### Shared handshake configuration Use [`TlsConfig`] to control how long a TLS handshake may take. Attach it to the server's accepted I/O configuration or the client connector's `SharedCfg`: ```rust use ntex::{SharedCfg, time::Seconds, tls::TlsConfig}; let cfg: SharedCfg = SharedCfg::new("TLS") .add(TlsConfig::new().set_handshake_timeout(Seconds(10))) .into(); ``` Pass `cfg` to the server's bind/listen call, `ClientBuilder::build`, or `WsClient::new`. The default TLS handshake timeout is five seconds; zero disables the deadline. This timeout covers only TLS negotiation. DNS lookup, establishing a TCP connection, waiting for an HTTP response, and the WebSocket opening handshake have separate settings in their respective components. ## X.509 certificates and identity loading The [`x509`] module provides a platform- and backend-independent API for working with X.509 certificates and identities. It uses platform- and backend-specific functionality underneath, so your application can use common types instead of calling each backend directly. The API is portable, but supported features and verification behavior can still differ by backend. Before loading certificate files, it helps to separate three concepts: | Type | Meaning | | --- | --- | | `Certificate` | A parsed public certificate, without a private key | | `Identity` / `ClientIdentity` | Certificate chain plus a matching private key | | `CaCerts` | Certificates you explicitly trust when verifying clients | Loading an identity checks that its data is well-formed and the key matches the certificate. It does not tell you whether to trust that identity, grant it access, or accept it for a particular hostname. Parsing a certificate likewise reads its contents; it does not authenticate whoever sent it. The [X.509 example][x509-example] includes an inspector you can run against your files, plus the authentication helpers shared by all three backend examples. ### PEM and DER PEM is a text encoding; DER is binary. To load an identity from PEM, use `Identity::from_pem(certificates, private_key)` with the leaf certificate first in the chain and an **unencrypted PKCS#8** private key. `Identity::from_der(&[&[u8]], private_key)` takes the equivalent DER inputs. If your key is in PKCS#1, SEC1, or encrypted PEM form, convert it before using these portable loaders. `Certificate::from_pem` reads a certificate; `from_pem_multi` reads a bundle. `Certificate::from_der` parses owned DER bytes. These operations need no private key. If you are loading from files, `IdentitySource` handles the file reads for you: ```rust,no_run use ntex::tls::x509::{Identity, IdentitySource}; fn load() -> Result { IdentitySource::PemFiles { certificates: "../certs/server.pem".into(), private_key: "../certs/server.key".into(), } .load() } ``` In the examples, `load_identity("../certs/server")` is a convenience helper that loads `server.pem` and `server.key`. ### PKCS#12 and Windows stores PKCS#12 archives bundle certificates and private keys. If the archive has a password, use `Identity::from_pkcs12(bytes, password)`; `IdentitySource::Pkcs12File(path)` uses an empty password. Be explicit about the format: parsing a plain string as `IdentitySource` treats it as a PKCS#12 path, **not** a PEM path. Load passwords from a secret source or protected environment configuration, not command-line arguments, and keep them out of logs. PKCS#12 loading uses native APIs on Windows and requires the `openssl` feature on Unix. You can still use rustls for TLS; using OpenSSL to import an archive does not choose the connection's TLS backend. On Windows, you can also select an identity and its key from a certificate store: ```text cert:CurrentUser\MY\<40-hex-SHA1-thumbprint> cert:LocalMachine\MY\subject= ``` A thumbprint selector does not check whether the certificate is currently valid. A subject selector picks a currently valid match with an accessible private key, choosing the one with the latest expiry. Neither verifies trust. These selectors work only on Windows. Windows keys may be non-exportable: a backend can use the key to sign without being able to read its bytes. Key lifetime also matters when importing an identity. `Identity` keeps an isolated imported key for Schannel until its last owner is dropped, then normally deletes it. An abnormal process exit can leave that key behind. `ClientIdentity` imports use ephemeral native keys instead, which is useful when the identity is only needed by a client. ### Using an identity with each backend | Backend | Adapter | Important limitation | | --- | --- | --- | | OpenSSL | `ntex::tls::openssl::set_identity(&mut context, &identity)` | Requires an exportable PKCS#8 key; cannot use opaque Windows CNG keys through this adapter | | rustls | `certified_key(&identity)` or `client_certified_key(&client_identity)` | Install the result through a certificate resolver; native Windows keys can sign without being exported | | Schannel | `identity.schannel().clone()` | Windows-only native identity for client/server configuration | For rustls, use `SingleCertAndKey` when you have one configured identity; it works for both clients and servers. `ClientIdentity::from_identity` prepares an ordered client chain from an existing identity. You can also load a `ClientIdentity` directly from PEM, DER, or PKCS#12. The OpenSSL and rustls adapters put the chain in order for you. Underneath, `Identity::ordered_chain(false)` arranges the intermediates, leaves out the self-signed root, and drops unrelated certificates with a warning. Sending a root certificate does not make the other side trust it. Trusted certificates belong in the verifier's configuration, not in a chain supplied by the peer. ## Client-certificate authentication With ordinary HTTPS, the client authenticates the server. Mutual TLS also asks the client to present a certificate and prove that it owns the corresponding private key. Your application still needs to answer three separate questions: 1. Did the client provide a certificate? 2. Is that certificate trusted? 3. Is that identity allowed to use this application? Start by configuring the server to ask for or require a client certificate: | Backend | Requiring or requesting a client identity | | --- | --- | | OpenSSL | Set `PEER \| FAIL_IF_NO_PEER_CERT` and configure client trust anchors | | rustls | Configure `WebPkiClientVerifier` with client roots; do not enable anonymous clients when a certificate is mandatory | | Schannel | `request_client_cert(true)` requests an optional certificate and proof of key possession; the application must require and verify it | In OpenSSL, `PEER` alone does not require the client to send a certificate. In rustls, making the verifier optional is not the same as turning off client authentication altogether. **Schannel only requests a certificate; it does not enforce your authentication policy.** The handshake can succeed without a certificate, and it does not check the client's certificate trust here. Your application must reject missing or unacceptable identities before doing any protected work. The example makes this check before `HttpService`, so rejection closes the connection without sending an HTTP response. The client also needs its own identity configured: use `set_identity` with OpenSSL, a certificate resolver with rustls, or `set_client_cert` with Schannel. This is separate from the roots it uses to trust the server. Sending a client certificate does not replace checking the server's name and certificate chain. To try the OpenSSL example, run these from `tls/openssl` in separate terminals: ```sh cargo run --bin openssl-mtls-server cargo run --bin openssl-client -- https://localhost:8443/ ../certs/client ``` With the expected client identity, the response is `authorized`. Try again without an identity, with `untrusted-client`, and with `other-client`; each connection should be rejected. The last case is worth noticing: `other-client` has a valid certificate from a trusted issuer, but not the identity this application allows. The rustls and Schannel READMEs provide equivalent commands. ## Reading peer certificates When your application needs the peer's certificate, use [`PeerCertDer`] and [`PeerCertChainDer`]. These queries work across backends, unlike native certificate-chain queries whose conventions differ. This helper works with `IoRef` from any supported TLS backend: ```rust use ntex::io::IoRef; use ntex::tls::{PeerCertChainDer, PeerCertDer}; use ntex::tls::x509::{self, PeerCerts}; fn peer_certs(io: &IoRef) -> Result, x509::Error> { let Some(leaf) = io.query::().as_ref().cloned() else { return Ok(None); }; let chain = io.query::() .as_ref() .cloned() .map(|item| item.0) .unwrap_or_default(); PeerCerts::from_der(leaf.0, chain).map(Some) } ``` The query values own their DER buffers. `PeerCerts::from_der` parses them and removes a duplicated leaf if it appears at the start of the supplied chain. `PeerCerts::chain` holds the remaining chain, which may be incomplete or untrusted. On versions that provide `QueryItem::into_inner`, you can move the query value out instead of cloning it. In a web handler, use `HttpRequest::io()` to reach the connection. This example returns the leaf certificate's SHA-256 thumbprint: ```rust,no_run async fn certificate(req: ntex::web::HttpRequest) -> ntex::web::HttpResponse { use ntex::tls::{PeerCertChainDer, PeerCertDer}; use ntex::tls::x509::{PeerCerts, digest::Algorithm}; use ntex::web::HttpResponse; let Some(io) = req.io() else { return HttpResponse::Unauthorized().build(); }; let Some(leaf) = io.query::().as_ref().cloned() else { return HttpResponse::Unauthorized().build(); }; let chain = io.query::().as_ref().cloned() .map(|item| item.0).unwrap_or_default(); match PeerCerts::from_der(leaf.0, chain) { Ok(peer) => HttpResponse::Ok().body(peer.leaf().thumbprint(Algorithm::Sha256)), Err(_) => HttpResponse::Forbidden().build(), } } ``` This handler reports metadata; it does **not** decide whether the peer is allowed to use your application. It handles both a missing connection, as can happen with a synthetic request, and a connection without a peer certificate. Keep endpoints like this protected: certificate fields and stable identity identifiers should not be exposed as public diagnostics. You can also read a `Certificate`'s subject, issuer, SANs, and validity dates. Other TLS queries include `Servername`, `TlsVersion`, `TlsCipher`, and `ntex::io::types::HttpProtocol`, depending on the backend and negotiated session. `PskIdentity` is available only with OpenSSL. ## Trust and application authorization Once you have a peer certificate, checking trust and deciding access are separate steps. [`CaCerts`] handles client-certificate trust; it is not a hostname verifier for outbound HTTPS connections. Build the store from certificates your application explicitly trusts, called *trust anchors*. Never build it from the chain a client sends you. Every certificate you register becomes an anchor, even an intermediate or a pinned leaf. An empty store trusts nothing. Pay attention to import errors: `from_certs` logs and skips rejected anchors, while `from_certs_with` lets you handle those rejections. The examples reject an incomplete import rather than continuing with fewer roots than configured. First verify the peer's chain at the current time, then check the application identity. For example, this function accepts only the expected URI: ```rust use std::time::SystemTime; use ntex::tls::x509::{CaCerts, PeerCerts, TrustError, auth}; #[derive(Debug)] enum AuthError { Trust(TrustError), Name(auth::Failure), } fn authorize(peer: &PeerCerts, trust: &CaCerts) -> Result { let expires = trust.is_trusted(peer, SystemTime::now()) .map_err(AuthError::Trust)?; auth::name_matches( peer.leaf(), auth::NameField::Uri, "spiffe://ntex.example/client", ) .map_err(AuthError::Name)?; Ok(expires) } ``` The [shared example helpers][x509-example] use this policy. `AuthError` keeps trust failures separate from name mismatches so callers can tell them apart. The URI is an example application identity; matching it is not a complete implementation of SPIFFE policy. ### Expiry and long-lived connections `is_trusted` returns the earliest expiry in the verified certificate path, including the trust anchor. Treat this as a deadline: an authenticated connection must not stay open beyond it. The examples schedule the connection to close at that time and cancel the timer if it disconnects earlier. For long-lived sessions, also decide how your application will respond to configuration changes and revoked identities. ### Other authorization checks The `auth` helpers let you choose other identity checks: | Helper | Behavior | | --- | --- | | `name_matches` | Matches any SAN of the chosen kind, or the first subject common name (CN); names are compared case-insensitively | | `field` | Extracts one identity field; rejects multiple SAN values of the chosen kind | | `validity` | Checks the certificate's validity window | | `thumbprint_matches` | Matches an allow-list without checking dates | | `verify_thumbprint` | Checks both the allow-list and validity | Use SHA-256 for new thumbprint policies. SHA-1 is still useful for Windows store selectors and compatibility, but these helpers do not verify certificate chains, regardless of the digest you choose. ### Backend differences `CaCerts` does not check revocation. Also, do not assume that Crypt32, WebPKI, and OpenSSL accept exactly the same certificates. They differ in supported algorithms, Extended Key Usage (EKU) checks for client authentication, and how they handle a directly trusted leaf certificate. For example, Windows checks validity rather than performing ordinary chain validation for a pinned leaf. OpenSSL also handles EKU on intermediate certificates differently from Crypt32 and WebPKI. Check the [`CaCerts` documentation][CaCerts] when your policy needs to work consistently across platforms. ## Operational guidance ### Handshake limits Use `max_concurrent_ssl_accept` to limit how many TLS handshakes run at once per worker. Acceptors on the same thread share the limit, which defaults to 256. Set it before starting workers: changing it later does not update counters that other threads have already initialized. This is a **soft** limit. Readiness slows down acceptance but does not reserve a slot, so independently ready pipelines can exceed it. Setting the limit to zero keeps acceptor readiness pending; zero does **not** mean unlimited. Low-level handshake functions and filter constructors do not use this acceptor limit and have their own timeout behavior. In particular, OpenSSL's free `accept` and `connect` functions apply neither the service timeout nor its concurrency limit. Use the services when you need those controls. ### Shutdown TLS filters take part in normal, graceful I/O shutdown. Let the connection or server shut down normally when possible. Forcing the transport to close can lose queued bytes or prevent the TLS close notification from reaching the peer. ### Troubleshooting When a connection fails, first check whether the problem is in TLS negotiation, certificate verification, or the application protocol: | Symptom | Check | | --- | --- | | `wss://` fails before the upgrade | Install a TLS connector on `WsClient`; the URL does not choose one | | WebSocket upgrade fails after TLS | Offer HTTP/1.1, not HTTP/2 | | Unknown issuer | Configure the verifier's roots; sending a root from the peer does not establish trust | | Hostname mismatch | Check SANs and the request host separately from the destination address and SNI | | No compatible rustls provider | Select a provider and configure X.509's provider/fallback too | | Identity import fails | Check PKCS#8 encoding, encryption, key/leaf matching, and PKCS#12 backend support | | Only some clients fail verification | Check missing intermediates, validity, EKU and platform policy | | No peer certificate | Make sure the server requests one and the client explicitly supplies an identity | | Schannel accepts an anonymous TLS handshake | Expected with its request-only setting; the application must reject missing identities | | Windows store identity is unusable | Check store location, key permissions, selector and backend support for non-exportable keys | Log failures without private keys or passwords. Treat peer certificate fields as identity data, not harmless debugging output. ## Production checklist Before you ship, walk through these questions: - **Verification is on everywhere.** Every client, including the HTTP client with OpenSSL and every `WsClient`, uses a verifying configuration with the right trust roots. Nothing disables verification "temporarily." - **Names match.** Server certificates list every hostname and IP address that clients use to connect. - **Identities are real.** Example certificates and keys are not deployed. Private keys and PKCS#12 passwords come from protected storage and never appear in logs, command lines, or source control. - **Renewal is planned.** You know when each certificate expires and how a replacement reaches the running service. - **ALPN matches the protocol.** HTTP servers and clients offer the protocols they actually speak; WebSocket clients offer `http/1.1`. - **Client authentication is enforced, not just requested.** For mutual TLS, missing certificates are rejected, client trust anchors are configured explicitly, and the application checks *which* identity connected, not only that its certificate is valid. - **Long-lived connections respect expiry.** Authenticated connections close when their verified certificate path expires. - **Handshakes are bounded.** The handshake timeout and `max_concurrent_ssl_accept` suit your expected load. If every answer is yes, TLS is doing its job, and your handlers never had to know it was there. [examples]: https://github.com/ntex-rs/examples/tree/main/tls [cert-setup]: https://github.com/ntex-rs/examples/blob/main/tls/README.md#local-certificates [openssl-example]: https://github.com/ntex-rs/examples/tree/main/tls/openssl [rustls-example]: https://github.com/ntex-rs/examples/tree/main/tls/rustls [schannel-example]: https://github.com/ntex-rs/examples/tree/main/tls/schannel [x509-example]: https://github.com/ntex-rs/examples/tree/main/tls/x509 [state-example]: https://github.com/ntex-rs/examples/tree/main/state/state-accumulation [http-client-example]: https://github.com/ntex-rs/examples/tree/main/extras/http-client [ws-example]: https://github.com/ntex-rs/examples/tree/main/websockets/basic [`x509`]: https://docs.ntex.rs/api/ntex/tls/x509/index.html [`TlsConfig`]: https://docs.ntex.rs/api/ntex/tls/struct.TlsConfig.html [`PeerCertDer`]: https://docs.ntex.rs/api/ntex/tls/struct.PeerCertDer.html [`PeerCertChainDer`]: https://docs.ntex.rs/api/ntex/tls/struct.PeerCertChainDer.html [`CaCerts`]: https://docs.ntex.rs/api/ntex/tls/x509/struct.CaCerts.html [CaCerts]: https://docs.ntex.rs/api/ntex/tls/x509/struct.CaCerts.html # Byte Buffers When a socket read contains two messages, a decoder needs to hand the first message to the application while keeping the second for later. Copying each message into a new `Vec` works, but adds allocations and memory copies to every request. `ntex-bytes` lets the decoder hand out an owned view instead. The view keeps the data alive without borrowing the decoder's read buffer. The same idea helps on the write side: a small header and a large body can be queued separately, rather than copied into one growing buffer. For tiny values, sharing would cost more than copying, so the crate stores short byte strings directly in their handles. For frequently reused I/O buffers, per-thread page caches reduce trips to the allocator. The `ntex` crate re-exports the main types in [`ntex::util`]: [`Bytes`], [`BytesMut`], [`ByteString`], [`BytePages`], [`BytePage`], [`BytePageSize`], [`Buf`] and [`BufMut`]. Page cache tuning, [`set_page_cache_size`], is available from `ntex_bytes` directly. ## The mental model Start with three types. Use `BytesMut` while filling or changing a contiguous buffer, `Bytes` once data is ready to share, and `BytePages` when output can remain in separate chunks. `ByteString` is the text version of `Bytes`: it adds the guarantee that the bytes are valid UTF-8. The important distinction is between a **handle** and its **allocation**. Several handles can keep one allocation alive, but they do not all have permission to change it. A `BytesMut` owns the writable tail; `Bytes` views can own earlier, read-only portions. Splitting off a message gives it an independent lifetime, not necessarily independent storage. Here, "zero-copy" means avoiding a copy between application buffers. It does not promise that the operating system or a TLS layer will avoid copying. It is also an optimization, not a rule: copying a handful of bytes is often cheaper than maintaining another shared reference. If you remember only one rule from this chapter, make it this: > Share data while it is moving through the pipeline; copy or trim the small > part that needs to live much longer than the buffer it came from. If you are unsure where to start, use a `BytesMut` for input and message construction, hand completed data out as `Bytes`, and reach for `BytePages` only when the consumer can accept several chunks. The rest of this chapter explains when those defaults are worth changing. ## Types at a glance | Type | Mutability | Clone cost | Typical use | |------------------|------------|------------------------|-----------------------------------------------| | [`Bytes`] | Immutable | Usually shares storage or copies inline data; external storage decides | Decoded frames, payloads, header values | | [`BytesMut`] | Unique | Copies the data | Read buffers, building output | | [`ByteString`] | Immutable | Same as `Bytes` | UTF-8 text: header names, paths, topics | | [`BytePages`] | Unique | Shares or copies according to page storage and append rules | Write queues | | [`BytePage`] | Immutable | Shares data, `Vec` pages copy | One chunk of a `BytePages` queue | | [`BytePageSize`] | - | - | Allocation size classes | ## `Bytes` [`Bytes`] is an immutable view into contiguous memory. Heap-backed values normally share a reference-counted allocation when cloned, sliced or split. Small values and static data use simpler representations, described below. `Bytes` is `Send + Sync`, so it can be passed between threads. ```rust use ntex_bytes::Bytes; let mut msg = Bytes::copy_from_slice(&[1u8; 1024]); // These two views share the original allocation. let first = msg.split_to(256); assert_eq!(first.len(), 256); assert_eq!(msg.len(), 768); let part = msg.slice(100..200); // shares the allocation too let cloned = msg.clone(); // another handle, not another 768-byte buffer drop(msg); assert_eq!(part.len(), 100); // the other handles still own their data assert_eq!(cloned.len(), 768); ``` A `Bytes` value uses one of four storage kinds: | Kind | Created by | Clone | |----------|----------------------------------------------------|-------------------------------| | Inline | Data of at most 23 bytes (11 on 32-bit targets) | Copies the bytes, no allocation | | Static | [`Bytes::from_static`], `From<&'static [u8]>`, `From<&'static str>` | Copies the pointer | | Shared | A heap buffer, usually from [`BytesMut`] | Increments the reference count | | External | [`Bytes::from_ext`], `Arc` via [`ByteString`] | Delegated to the owner's vtable | Inline storage keeps small values inside the `Bytes` struct itself. Operations such as [`Bytes::slice`], [`Bytes::split_to`], [`BytesMut::split_to`] and [`BytesMut::freeze`] copy small results inline instead of taking another reference to the heap buffer. A short header name can therefore outlive a request without keeping its read buffer alive. [`Bytes::is_inline`] reports the storage kind. Static constructors keep their static representation even for short strings. The examples below assume a 64-bit target, where the inline limit is 23 bytes. On 32-bit targets it is 11 bytes. These limits and the layouts later in the chapter describe the current implementation, not a stable memory-layout API. ### When sharing retains too much memory A view that is too large to inline keeps the whole allocation alive, even if it uses only a small part. Keeping a 100-byte token from a 16 KiB read buffer in a long-lived cache can therefore retain far more than 100 bytes. That is not a leak: the allocation is still owned. It is a lifetime tradeoff. [`Bytes::trimdown`] helps at this boundary. It moves data inline when it fits, or copies it into an exactly sized buffer when at least 64 bytes of the backing capacity are unused. Inline and static values are left alone. ```rust use ntex_bytes::Bytes; let packet = Bytes::copy_from_slice(&[b'x'; 16 * 1024]); let mut token = packet.slice(100..200); token.trimdown(); // copy 100 bytes so `token` no longer retains the packet drop(packet); assert_eq!(token.len(), 100); ``` Trimming one view does not release the original allocation if other handles still own it. Use this deliberately for long-lived values, rather than trimming every frame and losing the benefit of sharing. ### Externally owned data External storage lets `Bytes` share memory owned by something else. A type implements the unsafe [`StorageExt`] trait and provides a static vtable with `as_ptr`, `len`, `clone` and `drop` functions. A `clone` that returns `None` makes clones copy the data into native storage instead. `Arc` implements it, so `ByteString::from(Arc)` shares the string without copying it. This is an integration API, not something a normal decoder needs to implement. The unsafe contract requires the exposed bytes to remain valid and immutable for every live handle, including across threads; the vtable must clone and release ownership correctly. ## `BytesMut` [`BytesMut`] is a unique, growable view into a heap buffer, similar to `Vec`. Only the `BytesMut` handle can write into its part of the buffer, but the same allocation can be shared with `Bytes` values split off its front. Think of the allocation as a line moving through a decoder: ```text | completed frame | completed frame | unread data | spare capacity | |<------ immutable Bytes views ---->|<--------- BytesMut -------->| ``` Only one `BytesMut` can own the writable part of a given allocation. Once a prefix is handed out as `Bytes`, the mutable view advances past it and cannot overwrite it. The decoder can continue filling the tail while the application reads earlier frames, without locks around the data. ```rust use ntex_bytes::{BufMut, BytesMut}; let mut buf = BytesMut::with_capacity(1024); buf.put_slice(b"hello "); buf.put_u16(0x1234); buf.extend_from_slice(b"world"); // 6 bytes fit inline, so `head` is a copy and `buf` stays unique let head = buf.split_to(6); assert!(head.is_inline()); assert!(buf.is_unique()); buf.extend_from_slice(&[0u8; 100]); let body = buf.take(); // shares the allocation with `buf` assert!(!buf.is_unique()); drop(body); assert!(buf.is_unique()); // a unique buffer gets its whole capacity back buf.clear(); assert_eq!(buf.capacity(), 1024); ``` The main operations: | Method | Effect | |--------------------------------|------------------------------------------------------------| | [`BytesMut::split_to`] | Returns the first `at` bytes as `Bytes`; shares large results and copies small ones inline | | [`BytesMut::take`] | Returns all data as `Bytes`, keeping the spare capacity for `self` | | [`BytesMut::freeze`] | Consumes the mutable handle; reuses heap storage or copies a small result inline | | [`BytesMut::advance_to`] | Drops the first `cnt` bytes, `O(1)` | | [`BytesMut::clear`] | Empties the buffer, reclaims the full capacity if unique | | [`BytesMut::reserve`] | Ensures spare capacity, see [Growth](#growth) | | [`BytesMut::is_unique`] | `true` if no `Bytes` shares the allocation | | `BytesMut::from(Bytes)` | Reuses a unique shared heap allocation; copies inline, static, external or still-shared data | Three similar-looking operations cover most handoff points: - Use `split_to(n)` after decoding a complete prefix and keep reading from the remainder. - Use `take()` when all current bytes form one message but the mutable handle should stay with the connection and reuse its remaining capacity. - Use `freeze()` when the mutable buffer itself is finished. It consumes the handle and turns its current data into `Bytes`. None is universally faster. Small results are copied inline; larger results usually share the allocation. Choose the operation that matches ownership first, then measure if that path matters. [`BytesMut::new`] currently starts with 112 bytes of data capacity (a 128-byte allocation including its header), [`BytesMut::with_capacity`] allocates exactly the requested capacity, and [`BytesMut::with_page_size`] takes a pooled page, see [Pooled buffers](#pooled-buffers). Cloning a `BytesMut` always copies the data. ### Length, capacity and spare space `len()` counts initialized bytes. `capacity()` counts what the current mutable view can hold, including those bytes, and the difference is spare space at the end. Capacity is not necessarily the size of the whole allocation: splitting or advancing the front shrinks the mutable view. Dropping a frame makes its space eligible for reuse, but does not immediately move the read cursor backward. A later `clear()` reclaims the whole allocation if it is unique; `reserve()` can move unread data to the front when more space is needed. If another handle is still alive, that memory cannot be overwritten. `clear()` means "discard my data", not "free my allocation". This is useful for a buffer that will be filled again. Drop the buffer when it is no longer needed; whether its memory is freed or cached depends on how it was allocated. ### A buffer's typical journey In a protocol decoder, one allocation often goes through the same cycle many times: 1. The transport writes socket data into the spare tail of a `BytesMut`. 2. The decoder examines the initialized prefix. 3. A complete frame is removed with `split_to`. 4. The returned `Bytes` travels through services and application code. 5. The `BytesMut` stays with the connection and receives more input. 6. When all shared frames are dropped, `reserve` or `clear` can reclaim the space at the front. This is why the mutable buffer can remain useful even while immutable frames are alive: they own disjoint views. It is also why holding one frame for a long time can change allocation behavior for that connection. The decoder remains correct, but it may need another allocation when the tail fills. `BytesMut` implements both the ntex [`BufMut`] trait and the `BufMut` trait of the `bytes` crate. They differ in one detail: ntex's `BufMut::remaining_mut()` reports the current spare capacity, while the `bytes` version reports `usize::MAX - len` because the buffer grows on demand. All `put_*` methods reserve capacity as needed in both cases. ### Zero-copy decoding Splitting frames off the read buffer is the core pattern of ntex codecs. The decoder below parses a two-byte, big-endian length followed by a payload. Large frames share the read buffer; tiny ones are copied inline: ```rust use ntex_bytes::{Bytes, BytesMut}; fn decode(src: &mut BytesMut) -> Option { if src.len() < 2 { return None; } let len = u16::from_be_bytes([src[0], src[1]]) as usize; if src.len() < 2 + len { // make room for the rest of the frame src.reserve(2 + len - src.len()); return None; } src.advance_to(2); Some(src.split_to(len)) } let mut src = BytesMut::copy_from_slice(b"\0\x03one"); assert_eq!(decode(&mut src).unwrap(), b"one"[..]); assert!(src.is_empty()); ``` The length field is treated as untrusted input. The decoder checks that the complete frame is present before calling `advance_to` and `split_to`, because those methods panic when asked to move past the end. If an index is not already validated, use [`BytesMut::split_to_checked`] instead: ```rust use ntex_bytes::{Bytes, BytesMut}; fn take_prefix(src: &mut BytesMut, len: usize) -> Option { src.split_to_checked(len) } let mut src = BytesMut::copy_from_slice(b"hello"); assert_eq!(take_prefix(&mut src, 2).unwrap(), b"he"[..]); assert_eq!(src, b"llo"[..]); ``` Bounds checks are not a size limit. A real decoder should reject a declared length above its protocol or application limit before calling `reserve`; otherwise a peer can make the connection retain a very large buffer without ever sending the promised frame. While a frame is alive, the read buffer is not unique, and the frame keeps the whole allocation alive, unless the frame was small enough to inline. Once all shared frames are dropped, the read buffer is unique again, and a later `clear()` or `reserve()` can reuse the space in front of its data. If it must grow before then, it copies only its remaining data into another buffer; existing frames remain valid in the old one. ## `ByteString` [`ByteString`] is an immutable UTF-8 string backed by `Bytes`. It has the same storage kinds and cost model, and dereferences to `&str`. ```rust use ntex_bytes::{ByteString, Bytes}; let header = ByteString::from("content-type: text/plain"); let name = header.slice(0..12); // copies this short result inline on 64-bit targets assert_eq!(name, "content-type"); // validates UTF-8, takes over the bytes without copying let s = ByteString::try_from(Bytes::from_static(b"utf-8 text")).unwrap(); assert_eq!(s.as_str(), "utf-8 text"); ``` Indices are byte offsets, not character positions. `slice`, `split_to` and `split_off` panic if an index is not on a UTF-8 character boundary, so take care when working with non-ASCII text. [`ByteString::from_bytes_unchecked`] is unsafe: it skips validation, and the caller must guarantee valid UTF-8. Use the checked conversion unless that guarantee is already established. With the `simd` feature, UTF-8 validation uses SIMD instructions. Converting an owned `Bytes` or `BytesMut` into `ByteString` validates UTF-8 and then reuses the bytes, apart from the usual small-value inlining. Converting back with [`ByteString::into_bytes`] is free. Converting a `String` or `Vec` copies into native byte storage; `ByteString::from(Arc)` is the exception and shares the `Arc`. ```rust use ntex_bytes::{ByteString, BytesMut}; let bytes = BytesMut::copy_from_slice("hello".as_bytes()).freeze(); let text = ByteString::try_from(bytes).unwrap(); assert_eq!(text, "hello"); let bytes = text.into_bytes(); assert_eq!(bytes, b"hello"[..]); ``` `ByteString` can retain a larger allocation for the same reason as `Bytes`. Call [`ByteString::trimdown`] when a short string is being promoted from a request buffer into long-lived state. ## `Buf` and `BufMut` [`Buf`] reads from a buffer with a cursor, [`BufMut`] writes into one. They provide the `get_*` and `put_*` helpers for integers and floats in big- and little-endian order. The unsuffixed integer methods use big-endian order; methods ending in `_le` use little-endian order. ```rust use ntex_bytes::{Buf, BufMut, BytesMut}; let mut buf = BytesMut::with_capacity(16); buf.put_u32(0xDEAD_BEEF); buf.put_u8(1); let mut b = buf.freeze(); assert_eq!(b.get_u32(), 0xDEAD_BEEF); assert_eq!(b.get_u8(), 1); assert!(!b.has_remaining()); ``` Reading advances the cursor: after `get_u32()`, those four bytes are no longer part of the remaining input. Reads panic if there are not enough bytes, so a decoder should check `remaining()` or `len()` before reading a field. The traits work with more than these owned buffer types: a `&[u8]` can be a read cursor too, and a generic `Buf` need not store all its data contiguously. [`Bytes`] and [`BytesMut`] also implement the `Buf` trait of the `bytes` crate, and `BytesMut` its `BufMut` trait, so they work with libraries built on `bytes`. `BytesMut` and `BytePages` implement `std::io::Write`, and `BytesMut` implements `std::fmt::Write`, so `write!` can format into them. `Bytes` and `ByteString` implement `serde`'s `Serialize` and `Deserialize`. ### Panicking and checked operations Methods such as `slice`, `split_to`, `split_off`, `advance_to` and the `get_*` family assume their indices or lengths have already been validated. They panic on an invalid range or insufficient input. This keeps a codec's hot path simple after it has performed its bounds checks. For values derived directly from input, prefer the checked variants where available: [`Bytes::slice_checked`], [`Bytes::split_to_checked`], [`Bytes::split_off_checked`] and [`BytesMut::split_to_checked`]. A failed check then becomes `None` instead of a process-level panic. ## Internal organization You do not need the layout details to use the types, but they explain two design choices: why tiny values are copied and why a mutable buffer can share an allocation without sharing its writable bytes. ### Handles `Bytes` is three machine words: a data pointer, a length, and an `offset` word. The two low bits of `offset` select the storage kind, the remaining bits hold the distance from the start of the heap buffer to the data pointer. Inline storage reuses all three words for data and keeps the kind and length in one byte, which is where the 23-byte inline capacity comes from. `BytesMut` is a single pointer to its heap buffer. Its view, the start and length of its data, is stored in the buffer header, so `BytesMut` is one word in size. ### Heap buffers Each native shared heap buffer is one allocation: a 16-byte header followed by the data. External storage and `Vec`-backed pages have their own layouts. ```text header (16 bytes) data +--------+-----+----------+-----------+---------+---------+------------+-------+ | offset | len | capacity | ref_count | Bytes 1 | Bytes 2 | BytesMut | spare | +--------+-----+----------+-----------+---------+---------+------------+-------+ ^ ^ ^ ^ allocation data start offset offset + len ``` | Field | Meaning | |-------------|-------------------------------------------------------------------------| | `offset` | Start of the `BytesMut` view, from the beginning of the allocation | | `len` | Length of the `BytesMut` view | | `capacity` | Data capacity of the whole allocation, never changes while it is shared | | `ref_count` | Number of handles: the `BytesMut` plus every `Bytes` view | Everything else is derived from these fields. The spare capacity after the `BytesMut` view is `16 + capacity - offset - len`, and the [`BytePageSize`] class follows from the allocation size `16 + capacity`, see [Pooled buffers](#pooled-buffers). Lengths, capacities and offsets use `u32`; the reference count is an `AtomicU32`. A native heap buffer holds at most `u32::MAX - 16` bytes of data, just under 4 GiB. Requests for larger capacities panic. Each `Bytes` view stores its own pointer and length, and computes the header address from its `offset`. Views split off a `BytesMut` cover memory in front of the `BytesMut` view, so the `BytesMut` never writes to memory that a `Bytes` can read. ### Reference counting and threads The reference count is atomic and follows the same memory ordering as `std::sync::Arc`: a `Release` decrement, and an `Acquire` fence before the last handle frees the buffer. [`BytesMut::is_unique`] uses an `Acquire` load, so after it returns `true`, all accesses through dropped handles, including handles dropped on other threads, happen before the buffer is reused. Apart from the reference count, `Bytes` handles read only the `capacity` field of the header, which does not change while the buffer is shared, so the `BytesMut` handle can update its view without synchronization. ## Allocation strategy ### Regular buffers [`BytesMut::with_capacity`] makes one allocation of `16 + capacity` bytes, and [`BytesMut::capacity`] is exactly the requested capacity. Buffers created this way, by `copy_from_slice`, and by conversions such as `Bytes::from(Vec)` have no page size, unless the requested capacity is exactly a page capacity. They are freed when their last handle is dropped. In particular, `Bytes::from(Vec)` copies the data into the crate's storage (or inline for small values); it does not adopt the vector allocation. The shared-buffer header needs space before the data. If you are building data specifically to share as `Bytes`, starting with a `BytesMut` avoids that conversion copy. ### Growth Appending data reserves capacity as needed. `reserve(additional)` asks for room for that many bytes **after the current data**, not for a total capacity. [`BytesMut::reserve`] tries these options in order: 1. If the buffer has enough spare capacity, nothing happens. 2. If the buffer is unique and the whole allocation is large enough, the data is moved to the front of the allocation, reusing the space of dropped `Bytes` views. 3. If the buffer is unique and has no page size, the allocation is grown with `realloc`, which can often extend it in place. 4. If the buffer has a page size, its data moves to a pooled page, see [Pooled buffers](#pooled-buffers). 5. Otherwise, the data is copied into a new allocation without a page size. `Bytes` views split off the buffer keep the old allocation alive. When growth needs more storage, the target capacity is at least twice the current length, so repeatedly appending small amounts does not allocate on every append. `realloc` may extend an allocation in place, but it may also move it; the buffer makes no promise that its address stays the same. The three reservation methods serve different purposes: | Method | Use it when | |--------|-------------| | [`BytesMut::reserve`] | You want room for more bytes and expect the buffer to keep growing. | | [`BytesMut::reserve_exact`] | You know how many more bytes you need and want to avoid the doubling policy. The allocation is exactly the new capacity plus the 16-byte header; pooled buffers are not rounded up to a page class and keep one only if the new capacity is a page capacity. | | [`BytesMut::reserve_more`] | You don't know how much more is coming, for example the next read. If less than `BytePageSize::low` remains (1 KiB for `Size16`, 4 KiB without a page size), the allocation is reused when it holds the data plus `half_capacity()`: a unique buffer is compacted in place, a shared page moves to a page of the same class. Otherwise a pooled buffer moves to the next page class and any other buffer grows by its capacity, by at least 112 bytes and at most 64 KiB. | Neither `reserve` nor `reserve_exact` is a general-purpose shrinking operation: if there is already enough spare capacity, it leaves the buffer alone. ### Page sizes [`BytePageSize`] defines the size classes for pooled buffers. A page allocation requests exactly the class size, including the header, rather than a class-sized payload plus extra metadata. This avoids overshooting those useful size boundaries, though an allocator's actual size classes are its own implementation detail. Data capacity is the class size minus the 16-byte header. These are allocation categories, not operating-system virtual-memory pages. | Class | Allocation | Capacity | `half_capacity()` | `low()` | Cached pages by default | |-----------|------------|---------------|-------------------|-----------|-------------------------| | `Size4` | 4 KiB | 4,080 bytes | 2 KiB | 256 bytes | 128 | | `Size8` | 8 KiB | 8,176 bytes | 4 KiB | 512 bytes | 64 | | `Size16` | 16 KiB | 16,368 bytes | 8 KiB | 1 KiB | 64 | | `Size24` | 24 KiB | 24,560 bytes | 12 KiB | 1.5 KiB | 32 | | `Size32` | 32 KiB | 32,752 bytes | 16 KiB | 2 KiB | 16 | | `Size48` | 48 KiB | 49,136 bytes | 16 KiB | 3 KiB | 8 | | `Size64` | 64 KiB | 65,520 bytes | 16 KiB | 4 KiB | 16 | | `Size128` | 128 KiB | 131,056 bytes | 16 KiB | 8 KiB | 2 | | `Size256` | 256 KiB | 262,128 bytes | 16 KiB | 16 KiB | 1 | | `Unset` | - | 65,520 bytes | 16 KiB | 4 KiB | never cached | `Size16` is the default class. [`BytePageSize::for_capacity`] returns the smallest class that holds a given capacity, or `Unset` above the largest data capacity (262,128 bytes). [`BytePageSize::next`] and [`BytePageSize::prev`] step between classes. `half_capacity()` is the recommended write-buffer threshold for a page size, `low()`, 2/32 of the class size, is the free-capacity threshold of [`BytesMut::reserve_more`]. The enum is `#[non_exhaustive]`, so more classes may be added. ```rust use ntex_bytes::BytePageSize; assert_eq!(BytePageSize::for_capacity(100), BytePageSize::Size4); assert_eq!(BytePageSize::for_capacity(20_000), BytePageSize::Size24); assert_eq!(BytePageSize::Size16.next(), BytePageSize::Size24); assert_eq!(BytePageSize::Size256.next(), BytePageSize::Unset); ``` ### Pooled buffers [`BytesMut::with_page_size`] takes a page of the given class from the current thread's page cache, or allocates a new one if the cache is empty. `with_page_size(BytePageSize::Unset)` takes a `Size64` page, the two have the same allocation size. [`BytesMut::page_size`] reports the class of a buffer. The header does not store the class. A buffer belongs to a class when its allocation, header included, is exactly the class size, so the class is looked up from the `capacity` field. Any buffer of a page size is pooled, including one created by `with_capacity` or `copy_from_slice` with exactly a page capacity, or grown into one by `realloc`. When the last handle to a page is dropped, whether it is the `BytesMut` or a `Bytes` view split off it, the page returns to the cache of its class on the thread that drops it. If that cache is full, the page is freed. Only buffers with a page size are cached. Buffers with `Unset` are always freed. A pooled buffer that grows moves to a page of the smallest class that fits the new capacity, but never to a smaller class than its current one. The data is copied, and the old page returns to the cache once its `Bytes` views are dropped. If the growth target exceeds the largest page's data capacity, the buffer becomes a regular buffer without a page size. From then on it grows like any regular buffer and is freed when its last handle drops. ```rust use ntex_bytes::{BytePageSize, BytesMut}; let mut buf = BytesMut::with_page_size(BytePageSize::Size4); assert_eq!(buf.capacity(), 4096 - 16); buf.extend_from_slice(&[0; 5000]); assert_eq!(buf.page_size(), BytePageSize::Size8); buf.extend_from_slice(&[0; 10_000]); assert_eq!(buf.page_size(), BytePageSize::Size16); // beyond the largest class, a regular buffer buf.reserve(300 * 1024); assert_eq!(buf.page_size(), BytePageSize::Unset); ``` Small results are inlined, which affects pooled buffers too: freezing a pooled buffer with at most 23 bytes of data copies them inline. If no earlier `Bytes` views still share the allocation, the page can return to the cache right away. ```rust use ntex_bytes::{BytePageSize, BytesMut}; let mut buf = BytesMut::with_page_size(BytePageSize::Size32); buf.extend_from_slice(b"small"); let small = buf.freeze(); assert!(small.is_inline()); assert_eq!(small, b"small"[..]); // `small` no longer needs the page; a later allocation can reuse it. ``` Pages handed to another thread return to that thread's cache. A thread that only receives data, for example a thread that writes data produced on other threads, fills its cache up to the limits and frees the rest. While a thread's thread-local storage is being destroyed, the cache is unavailable, and pages released then are freed. ### Tuning the page cache Caching trades retained memory for fewer allocations. An idle worker can still hold cached pages, but those pages are ready for reuse; they are not live messages. [`set_page_cache_size`] sets the number of cached pages of one class for the current thread. The defaults, listed in the table above, cache more pages of small classes and fewer of large ones. The largest shares go to the classes that I/O uses most: 4 to 16 KiB pages, where connection reads start and the default write page; and 64 KiB pages, where reads of busy connections grow to. With all caches full, a thread retains about 5 MiB. ```rust use ntex_bytes::{BytePageSize, set_page_cache_size}; // retain more large pages for a thread that handles large messages set_page_cache_size(BytePageSize::Size256, 4); // stop returning 4 KiB pages to this thread's cache set_page_cache_size(BytePageSize::Size4, 0); ``` The setting affects only the calling thread, so call it on every thread that uses pooled buffers, for example at the start of each worker thread. A smaller limit does not free pages already in the cache. They are reused, and the limit applies when pages are released. The older `set_pages_cache()`, which sets one limit for all classes, is deprecated. These limits count **cached** pages, not pages in use. They do not bound the memory held by live requests or slow consumers. Start with the defaults, then adjust them using the sizes and concurrency of your actual workload. More workers also means more independent caches. For example, the default cache can retain roughly 5 MiB per worker when every size class is full. Eight otherwise idle workers could therefore retain about 41 MiB in page caches. That may be a good trade when traffic returns quickly; for sparse workloads or many workers, smaller limits may be better. ## `BytePages` Suppose a response contains a short header, a large existing body and a trailer. A contiguous buffer may have to move earlier output as it grows, and copying the body into it would be wasteful. [`BytePages`] keeps a queue of chunks instead: new bytes fill a current writable page, while owned payloads can be queued separately. When the current page fills, another page is started. Existing output stays where it is. ```rust use ntex_bytes::{BufMut, BytePageSize, BytePages, Bytes}; let mut pages = BytePages::new(BytePageSize::Size4); pages.extend_from_slice(b"HTTP/1.1 200 OK\r\n\r\n"); let body = Bytes::copy_from_slice(&[b'x'; 8192]); pages.append(body); // moves the existing body into the queue, no extra copy pages.put_slice(b"trailer"); assert_eq!(pages.num_pages(), 3); // the transport consumes pages in order while let Some(page) = pages.take() { // write `page` to the socket assert!(!page.is_empty()); } ``` How data enters the queue: - `put_slice()`, `extend_from_slice()` and the other `BufMut` methods copy data into the current page. New pages are taken from the page cache with the queue's page size. - [`BytePages::append`] adds an owned buffer. If the current page holds data, buffers of up to 4 KiB are copied into it. Larger buffers become a separate page without copying, and the current page keeps its spare capacity for later writes. If the current page is empty, the buffer is added without copying, and a unique `BytesMut` with spare capacity becomes the new current page. - [`BytePages::prepend`] inserts a page at the front. How data leaves it: - [`BytePages::take`] returns the next [`BytePage`], with the current page last. - [`BytePages::split_to`] and [`BytePages::split_into`] move a byte prefix to another queue, splitting a page if needed. - [`BytePages::freeze`] returns all data as one `Bytes`. A single page is converted according to its storage kind: native heap storage can be reused, tiny data may be inlined, and `Vec` storage is copied. Several pages are copied into one buffer. - [`BytePages::copy_to`] leaves the source unchanged and appends cloned pages to another queue. [`BytePages::move_to`] leaves the source empty and appends its pages instead. Both follow `append`'s small-buffer copy rules; `copy_to` also copies `Vec`-backed pages when cloning them. A [`BytePage`] holds a `Bytes`, the storage of a `BytesMut`, or a `Vec`. Splitting or advancing a `Vec` page copies its data, so prefer the other kinds when output will be split or shared. Queuing an owned `Vec` directly can still avoid an initial copy; that is different from converting the vector to `Bytes` first. Keep data paged for as long as the consumer accepts chunks. Calling `freeze()` just before passing it to a chunk-aware transport would undo the main benefit by combining all pages into one allocation. Paging avoids copies; it does not bound queued output. A producer can still outpace the transport and retain many pages, including large buffers appended without copying. Protocol code should stop producing when its I/O layer reports write backpressure instead of treating `BytePages` as an unlimited queue. The page size of a `BytePages` cannot be `Unset`, [`BytePages::new`] and [`BytePages::set_page_size`] panic on it. `BytePages::default()` uses `Size16`. The page list itself is also reused: each thread keeps up to 128 empty page lists for new `BytePages` values. Changing the page size affects future page allocations, not data already queued or the current page. Page buffers are allocated lazily, so creating an empty queue does not immediately allocate a payload page. ## Buffers in the I/O layer The [I/O Abstraction Layer](./7-io.md) builds on these types. Reads and writes share the same per-thread page cache: - **Reads.** The transport reads into a pooled `BytesMut` read buffer. Codecs split frames off it with `split_to`, so large decoded messages share the read buffer while tiny ones are copied inline. New read buffers use a per-connection page size that adapts to the read load between the min and max read sizes of [`IoConfig`], `Size4` and `Size64` by default. A buffer grows once less than `BytePageSize::low()` of its page remains free, larger input moves it to bigger page sizes, beyond `Size256` it becomes an unpooled buffer that is freed when empty. - **Writes.** Encoders write into `BytePages` with the page size of [`IoConfig::write_size`], `Size16` by default. `IoRef::encode_bytes` appends owned buffers, so large payloads are not copied. The write threshold, `half_capacity()` of the page size by default, controls when a transport may start writing while output is still being produced. An empty read buffer is released immediately, but its page returns to the cache only when the last frame split from it is dropped. Holding a whole request or frame therefore keeps the full page alive; copy the field instead when only a small part needs to survive. [`set_page_cache_size`] tunes the cache for read and write pages alike. ### What memory usage means in practice When investigating memory growth, separate three categories: - **Live data:** bytes still owned by requests, responses, transports or application state. - **Retained allocations:** a small live view keeps a larger shared allocation alive. `trimdown` can help when the view is intentionally long-lived. - **Cached allocations:** empty pages or read buffers kept for reuse. Cache limits control this category, not live data. Only the first category is directly proportional to pending work. The other two are performance tradeoffs and can make resident memory stay high after a traffic spike without indicating an ownership leak. ## Tracking buffer memory Shared buffers make one question hard to answer: who is holding this memory? A large buffer can stay alive because of a single frame sitting in a cache, long after the code that allocated it has moved on. The allocator can tell you the memory is in use, but not why. The `tracker` feature answers that question. Any heap allocation can be charged to a [`BytesTracker`], and the charge follows the allocation, not a particular handle. When the last handle is dropped, the charge ends. You decide what a tracker stands for: a cache, a tenant, a subsystem, or a single connection. The I/O layer is one user of this mechanism. It can charge connection read buffers to trackers, as described at the [end of this section](#example-read-buffers-in-the-io-layer). The feature is off by default. Enabling it on `ntex` also turns it on in the I/O and network layers, and re-exports `BytesTracker` and [`BytesTrackerSnapshot`] in `ntex::util`: ```toml ntex = { version = "4", features = ["tracker"] } ``` If you use `ntex-bytes` on its own, enable `ntex-bytes/tracker` instead. The feature has a small cost even for buffers that are never charged. The header in front of every heap buffer grows from 16 to 32 bytes (20 bytes on 32-bit targets) to make room for the tracker pointer. Cloning, splitting and freezing do no tracker work. Only allocating, growing, and releasing the last handle of a charged buffer touch the tracker's atomic counters. ### How charges work A tracker is a set of atomic counters shared by all of its handles. `snapshot()` returns two numbers: - `live_bytes`: the total size of the charged allocations, header included. This is the allocation size, not the amount of data in it, so a `Size4` buffer counts as 4096 bytes whether it is empty or full. - `live_allocations`: how many allocations are charged. Charges follow the allocation, in line with the [handle and allocation](#the-mental-model) distinction: - An allocation stays charged while any `BytesMut` or `Bytes` handle refers to it. A frame split off a buffer keeps the whole allocation charged, just as it keeps it alive. - A buffer that grows into a new allocation charges the new one to the same tracker. - The charge ends when the last handle is dropped, on whichever thread drops it. The page then goes back to the cache uncharged, and is charged again only by its next user. - Copies are not charged: a cloned `BytesMut`, small data stored inline in a `Bytes` handle, the result of `trimdown`, or data converted into other types. ```rust use ntex_bytes::{BytePageSize, BytesMut, BytesTracker}; let tracker = BytesTracker::new(); let mut buf = BytesMut::with_page_size_in(BytePageSize::Size4, &tracker); buf.extend_from_slice(&[b'x'; 1000]); let frame = buf.split_to(600); // shares the allocation let tag = frame.slice(..8); // short enough to be copied inline drop(buf); // `frame` alone keeps the whole page charged let snap = tracker.snapshot(); assert_eq!(snap.live_allocations, 1); assert_eq!(snap.live_bytes, 4096); // `tag` is a copy, so dropping `frame` ends the charge drop(frame); assert_eq!(tracker.snapshot().live_bytes, 0); assert_eq!(&tag[..], b"xxxxxxxx"); ``` Compared with the categories in [What memory usage means in practice](#what-memory-usage-means-in-practice), a tracker counts live data and retained allocations, never cached ones. That makes it a good leak detector: if tracked memory drops after a traffic spike but resident memory stays high, the difference is cache, not a leak. ### Building your own tracker Create a tracker for any part of the application whose memory you want to see, such as a response cache, a per-tenant queue, or a decompression stage. There are two ways to charge a buffer: - [`BytesMut::with_page_size_in`] takes a new buffer from the page cache and charges it. Like `with_page_size`, `BytePageSize::Unset` means `Size64`. - [`BytesMut::set_tracker`] charges an existing buffer. If the buffer is already charged to another tracker, the charge moves, even while `Bytes` frames split from it are still alive. Setting the same tracker again does nothing. [`BytesMut::tracker`] tells you which tracker a buffer is charged to. Only a `BytesMut` can change the charge, so do it before you freeze the buffer and hand out `Bytes`. Here is a cache that accounts for everything it stores: ```rust use std::collections::HashMap; use ntex_bytes::{Bytes, BytesMut, BytesTracker, BytesTrackerSnapshot}; #[derive(Default)] struct BodyCache { items: HashMap, tracker: BytesTracker, } impl BodyCache { fn insert(&mut self, key: String, mut body: BytesMut) { // moves the charge from any previous tracker to the cache body.set_tracker(&self.tracker); self.items.insert(key, body.freeze()); } fn usage(&self) -> BytesTrackerSnapshot { self.tracker.snapshot() } } ``` The cache is charged for the whole allocation, spare capacity included, because that is what it keeps alive. If entries are small slices of large buffers, the numbers will show it. That is your cue to `trimdown` before storing. A few properties make trackers easy to plug in: - Clones share the same counters, and `BytesTracker` is `Send` and `Sync`. A metrics task on any thread can hold a clone and call `snapshot()`. `ptr_eq` tells whether two handles refer to the same tracker. - A tracker lives while any handle or charged allocation exists. Dropping the cache's handle does not affect buffers that are still alive elsewhere. - Trackers report usage; they do not enforce limits. To enforce a budget, check `snapshot()` before accepting more data. - The two counters are read one after the other, so a snapshot taken while buffers are being allocated or released is only approximately consistent. ### Example: read buffers in the I/O layer `ntex-io` builds on the same API to account for connection input. Instead of charging buffers yourself, you choose a tracker policy per [`IoConfig`] with [`IoConfig::set_buf_tracker`], and every read buffer a connection allocates is charged according to it: | [`BufTracker`] | Behavior | |-------------------|----------| | `Disabled` | Read buffers are not charged. This is the default. | | `PerConnection` | Each connection gets its own tracker, created on first use. | | `Shared(tracker)` | All connections of the configuration, on every worker, charge `tracker`. | ```rust use ntex::{SharedCfg, io::{BufTracker, IoConfig}, util::BytesTracker}; let api_input = BytesTracker::new(); let cfg = SharedCfg::new("API").add( IoConfig::new().set_buf_tracker(BufTracker::Shared(api_input.clone())), ); ``` Keep the clone and read `api_input.snapshot()` from a metrics task. Because frames split off a read buffer keep it charged, the number includes every request body or message the application still holds, even after its connection has closed. That is input the I/O metrics no longer see. Only read buffers are charged; write buffers are not. With `PerConnection`, [`IoRef::buf_tracker`] returns the connection's tracker (`IoContext::buf_tracker` does the same inside filters). You can use it to close a client that holds too much input. The reference must not be used after `Io::set_config`; clone the tracker if you need it for longer. A clone keeps reporting buffers that outlive the connection, but it does not keep the connection alive. The connection's tracker is an ordinary `BytesTracker`, so the two approaches combine. To attribute your own buffers to a connection, such as a body assembled from many frames or a decompressed payload, pass the tracker from `io.buf_tracker()` to `with_page_size_in`. That memory then shows up in the same numbers as the connection's read buffers. A `Shared` tracker is a single cache line updated by every worker thread, so it becomes contended under load. Prefer one tracker per listener or service over one for the whole server. The [server chapter](./5-server.md#io-buffer-attribution) covers the rest: what TLS and io-uring buffers count toward, how `Io::set_config` moves charges, and how to read the numbers next to the I/O metrics. ## Performance guidelines The fastest choice depends on how long the data lives, not just how large it is. A useful starting point is: | Situation | Start with | |-----------|------------| | Hand a decoded frame to its immediate consumer | `split_to` or `take`, so large payloads can share the input allocation. | | Keep one field in a long-lived cache | A slice followed by `trimdown`, or an explicit copy, to avoid retaining an entire input buffer. | | Build a contiguous message of a known size | `BytesMut::with_capacity`, or one `reserve` for the bytes still to be written. | | Repeatedly build short-lived I/O buffers | `with_page_size` with a class that fits the usual size; cache hits avoid a fresh payload allocation. | | Send an existing large body | `BytePages::append`, rather than copying it with `put_slice`. | | Queue an owned `Vec` for output | Append the `Vec` directly to `BytePages`; converting it to `Bytes` first copies it. | | Use constant protocol bytes | `from_static`, which neither allocates nor copies the bytes. | Drop shared frames when they are no longer needed, and do not equate "cheap to clone" with "cheap to retain". A clone may cost only an atomic increment yet extend the lifetime of a large allocation. Conversely, an inline clone copies a few bytes but needs neither allocation nor reference counting. Measure allocation volume and retained memory alongside throughput when deciding whether to share, copy or cache. [`ntex::util`]: https://docs.ntex.rs/api/ntex/util/index.html [`Buf`]: https://docs.ntex.rs/api/ntex_bytes/buf/trait.Buf.html [`BufMut`]: https://docs.ntex.rs/api/ntex_bytes/buf/trait.BufMut.html [`BufTracker`]: https://docs.ntex.rs/api/ntex/io/enum.BufTracker.html [`BytePage`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePage.html [`BytePageSize`]: https://docs.ntex.rs/api/ntex_bytes/enum.BytePageSize.html [`BytePageSize::for_capacity`]: https://docs.ntex.rs/api/ntex_bytes/enum.BytePageSize.html#method.for_capacity [`BytePageSize::next`]: https://docs.ntex.rs/api/ntex_bytes/enum.BytePageSize.html#method.next [`BytePageSize::prev`]: https://docs.ntex.rs/api/ntex_bytes/enum.BytePageSize.html#method.prev [`BytePages`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html [`BytePages::append`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.append [`BytePages::copy_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.copy_to [`BytePages::freeze`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.freeze [`BytePages::move_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.move_to [`BytePages::new`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.new [`BytePages::prepend`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.prepend [`BytePages::set_page_size`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.set_page_size [`BytePages::split_into`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.split_into [`BytePages::split_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.split_to [`BytePages::take`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytePages.html#method.take [`ByteString`]: https://docs.ntex.rs/api/ntex_bytes/struct.ByteString.html [`ByteString::from_bytes_unchecked`]: https://docs.ntex.rs/api/ntex_bytes/struct.ByteString.html#method.from_bytes_unchecked [`ByteString::into_bytes`]: https://docs.ntex.rs/api/ntex_bytes/struct.ByteString.html#method.into_bytes [`ByteString::trimdown`]: https://docs.ntex.rs/api/ntex_bytes/struct.ByteString.html#method.trimdown [`Bytes`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html [`Bytes::from_ext`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.from_ext [`Bytes::from_static`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.from_static [`Bytes::is_inline`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.is_inline [`Bytes::slice`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.slice [`Bytes::slice_checked`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.slice_checked [`Bytes::split_off_checked`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.split_off_checked [`Bytes::split_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.split_to [`Bytes::split_to_checked`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.split_to_checked [`Bytes::trimdown`]: https://docs.ntex.rs/api/ntex_bytes/struct.Bytes.html#method.trimdown [`BytesMut`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html [`BytesMut::advance_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.advance_to [`BytesMut::capacity`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.capacity [`BytesMut::clear`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.clear [`BytesMut::freeze`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.freeze [`BytesMut::is_unique`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.is_unique [`BytesMut::new`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.new [`BytesMut::page_size`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.page_size [`BytesMut::reserve`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.reserve [`BytesMut::reserve_exact`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.reserve_exact [`BytesMut::reserve_more`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.reserve_more [`BytesMut::set_tracker`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.set_tracker [`BytesMut::split_to`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.split_to [`BytesMut::split_to_checked`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.split_to_checked [`BytesMut::take`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.take [`BytesMut::tracker`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.tracker [`BytesMut::with_capacity`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.with_capacity [`BytesMut::with_page_size`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.with_page_size [`BytesMut::with_page_size_in`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesMut.html#method.with_page_size_in [`BytesTracker`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesTracker.html [`BytesTrackerSnapshot`]: https://docs.ntex.rs/api/ntex_bytes/struct.BytesTrackerSnapshot.html [`IoConfig`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html [`IoConfig::set_buf_tracker`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html#method.set_buf_tracker [`IoConfig::write_size`]: https://docs.ntex.rs/api/ntex/io/struct.IoConfig.html#method.write_size [`IoRef::buf_tracker`]: https://docs.ntex.rs/api/ntex/io/struct.IoRef.html#method.buf_tracker [`set_page_cache_size`]: https://docs.ntex.rs/api/ntex_bytes/fn.set_page_cache_size.html [`StorageExt`]: https://docs.ntex.rs/api/ntex_bytes/trait.StorageExt.html # Timers and Time Utilities A network server is full of deadlines. A connection must finish its TLS handshake in time, a client that stops sending must eventually be dropped, an idle keep-alive connection must be closed, and an upstream request must not wait forever. Almost none of these timers ever fire: the handshake completes, the next request arrives, the upstream answers. Each one is armed, then cancelled or pushed further into the future, thousands of times per second. `ntex-util` provides the timer functionality for ntex. The `ntex` crate re-exports it as [`ntex::time`], and it works the same with every runtime backend. It contains: - [`Millis`] and [`Seconds`], compact duration types used throughout the ntex configuration APIs; - asynchronous utilities: [`sleep`], [`deadline`], [`timeout`], [`timeout_checked`] and [`interval`]; - [`CallbackTimer`], which runs a short closure on expiry without a task; - [`now`] and [`system_time`], cheap cached clocks. ## How the timer works The ntex timer implementation is ported from the Linux kernel [timer wheel](https://github.com/torvalds/linux/blob/master/kernel/time/timer.c). The kernel faces the same problem as a network server: most of its timers are timeouts for events that are expected to arrive, such as a network packet or an I/O completion, and are cancelled long before they expire. [Reinventing the timer wheel](https://lwn.net/Articles/646950/) on LWN describes the design and the reasoning behind it. ### A wheel of buckets Keeping timers in a sorted list or a binary heap gives exact ordering, but every insert and cancel costs `O(log n)` or more. A **timer wheel** works more like a clock face. It is an array of buckets, and each bucket collects all timers that expire during the same slice of time. Adding a timer means computing a bucket index and pushing an entry; cancelling it means removing that entry. Both are `O(1)`, however many timers there are. The wheel doesn't count time in milliseconds but in *units*, and by default **one unit is 16 milliseconds**. Every delay is converted to units before it is stored, so one unit is the finest resolution an ntex timer can have. The unit can be made smaller, see [Choosing the resolution](#choosing-the-resolution). Each bucket on the first level covers one unit, so 64 buckets reach only about one second ahead. To cover longer delays, ntex stacks 8 levels of 64 buckets, and each level is 8 times coarser than the one below it. With the default 16 ms unit: | Level | Granularity | Range | |-------|-------------|------------------| | 0 | 16 ms | 0 .. ~1 s | | 1 | 128 ms | ~1 s .. ~8 s | | 2 | ~1 s | ~8 s .. ~64 s | | 3 | ~8 s | ~64 s .. ~8.6 m | | 4 | ~65 s | ~8.6 m .. ~69 m | | 5 | ~8.7 m | ~69 m .. ~9.2 h | | 6 | ~70 m | ~9.2 h .. ~3 d | | 7 | ~9.3 h | ~3 d .. ~24.5 d | A timer goes on the first level that can hold its delay: a 100 ms timeout lands on level 0, a 30-second keep-alive on level 2. Anything longer than the whole wheel, about 24 days, is clamped to that maximum. ### No cascading In a classic hierarchical wheel, timers *cascade*: when a coarse bucket comes due, its timers are moved down into finer levels so that each one fires at its exact time. That work comes in bursts, and most of it is wasted, because by then most of those timers would have been cancelled anyway. The modified wheel, added to Linux in 2016 and used by ntex, skips cascading entirely. A timer stays in the bucket where it was placed, and its expiry is **rounded up** to that level's granularity. As a result: - **A timer never fires early.** The delay counts from the moment the timer is armed, and is then rounded up. - **A timer may fire a little late**, by at most one granularity of its level. For delays over one second this is less than about 13%: a 5-second timer can be up to 128 ms late, a 30-second timer up to about a second. - **Every operation is `O(1)`**, including expiry processing. ### Finding the next expiry Each level keeps a 64-bit bitmap with one bit per non-empty bucket. To find the next timer that is due, the wheel rotates the bitmap to the current position and counts the trailing zeros. Empty buckets are never scanned. ### One driver per thread Every runtime thread, such as each server worker, has its own wheel. Timer operations therefore need no locks or atomics. This is also why timer handles are `!Send`: create a timer on the thread that awaits it. Each thread runs a single driver task. It sleeps until the earliest non-empty bucket is due, then wakes every timer in that bucket. The driver moves the wheel clock to the time the bucket was *scheduled* for, not the time it actually woke up, so one late wakeup doesn't delay the timers that come after it. The driver starts with the first timer, so a thread that never uses timers never wakes up for them. When the runtime stops, every pending timer is marked as elapsed. ### Low precision by design ntex timers are made for network work: timeouts, keep-alives, retries and periodic housekeeping. They give up precision in exchange for constant-time operations. Don't use them to measure time, to rate-limit at millisecond resolution, or to drive animations. Their resolution is one unit (16 ms by default), and even a zero-duration [`sleep`] waits for the next timer tick. To measure how long something took, use [`std::time::Instant`]. The ntex I/O layer goes a step further. Per-connection dispatcher timeouts, such as keep-alive and frame read rate, are grouped into one-second slots, and all connections on a thread share a single timer. That's one reason most I/O configuration methods take [`Seconds`]. ### Choosing the resolution The unit is a per-system setting. It must be a power of two between 1 and 16 milliseconds; other values panic. Set it with [`Builder::timer_resolution`] when building the system yourself, or with the `timer_resolution` argument of `#[ntex::main]`: ```rust #[ntex::main(timer_resolution = 2)] async fn main() { // timers on this system tick every 2 ms assert_eq!(ntex::rt::System::current().timer_resolution(), 2); } ``` ```rust let sys = ntex::rt::System::build() .name("server") .timer_resolution(2) .build(ntex::rt::DefaultRuntime); ``` Every thread of the system, including server workers, reads the setting when its timer driver starts, and keeps it until the runtime stops and the wheel is empty. A thread without a running system uses 16 ms. The level layout stays the same; only the unit changes. All granularities and ranges in the table above scale with it. With a 2 ms unit, level 0 has 2 ms buckets and covers about 128 ms, and level 1 has 16 ms buckets. The whole wheel shrinks in the same proportion: | Unit | Level 0 range | Whole wheel | |-------|---------------|-------------| | 16 ms | ~1 s | ~24.5 d | | 8 ms | ~512 ms | ~12 d | | 4 ms | ~256 ms | ~6 d | | 2 ms | ~128 ms | ~3 d | | 1 ms | ~64 ms | ~1.5 d | A smaller unit helps services that wait on short delays, for example a handler that sleeps for 10 ms, which otherwise always waits a full 16 ms tick. Longer delays gain little, since they land on a higher level with about the same granularity as before. The cost is more frequent driver wakeups while short timers are pending, and a shorter maximum delay: longer timers are clamped to the end of the wheel. Keep the default unless short sleeps or timeouts show up in your latency. ## `Millis` and `Seconds` [`std::time::Duration`] is 16 bytes and can represent billions of years. Network timeouts need neither. ntex uses two small `Copy` types instead: | Type | Inner value | Size | Longest span | |-----------|-------------|---------|---------------| | `Millis` | `u32` | 4 bytes | about 49 days | | `Seconds` | `u16` | 2 bytes | about 18 hours | Both are plain tuple structs, so they can be written directly and used in `const` items. They implement `Eq`, `Ord` and `Hash`, and their default value is zero: ```rust use std::time::Duration; use ntex::time::{Millis, Seconds}; const CONNECT_TIMEOUT: Millis = Millis(5_000); const KEEP_ALIVE: Seconds = Seconds(30); fn conversions() { assert_eq!(Millis::from_secs(2), Millis(2_000)); assert_eq!(Millis::from(Seconds(2)), Millis(2_000)); assert_eq!(Millis::from(Duration::from_millis(1_500)), Millis(1_500)); let d: Duration = KEEP_ALIVE.into(); assert_eq!(d, Duration::from_secs(30)); let d: Duration = CONNECT_TIMEOUT.into(); assert_eq!(d, Duration::from_secs(5)); // Mixed arithmetic produces `Millis`. assert_eq!(Millis(250) + Seconds(1), Millis(1_250)); assert_eq!(Duration::from_secs(1) + Millis(250), Duration::from_millis(1_250)); } ``` Conversions and arithmetic **saturate** instead of overflowing or panicking. A negative `i32` becomes zero, and a `Duration` or `usize` that does not fit becomes the maximum value; the oversized cases also log an error: ```rust use std::time::Duration; use ntex::time::{Millis, Seconds}; fn saturation() { assert_eq!(Millis(u32::MAX) + Millis(1), Millis(u32::MAX)); assert_eq!(Millis::from(-5), Millis::ZERO); assert_eq!(Seconds::checked_new(100_000), Seconds(u16::MAX)); assert_eq!(Millis::from(Duration::from_secs(u64::MAX)), Millis(u32::MAX)); } ``` [`Millis`] converts from [`Seconds`], `Duration`, `u32`, `i32` and `usize`; [`Seconds`] converts from `u16`, `i32` and `usize`. Both convert into `Duration`. There is no conversion from `Duration` to `Seconds`, because it would silently drop the sub-second part. ### Zero means "disabled" In ntex configuration, a zero timeout usually turns the feature off: for example `Seconds::ZERO` as a keep-alive timeout disables keep-alive. The `is_zero`, `non_zero` and `map` helpers make that convention easy to follow in your own code. `map` calls the closure only for a non-zero value; on `Seconds`, the closure receives the value as `Millis`: ```rust use ntex::time::{Millis, Seconds, Sleep, sleep}; fn idle_timer(timeout: Seconds) -> Option { timeout.map(sleep) } fn retry_delay(base: Millis, attempt: u32) -> Millis { if base.is_zero() { Millis::ZERO } else { Millis(base.0.saturating_mul(attempt)) } } ``` ### Passing durations to timers Every timer constructor accepts `impl Into`, so the same function takes any of these types: ```rust use std::time::Duration; use ntex::time::{Millis, Seconds, sleep}; async fn waits() { sleep(Millis(100)).await; sleep(Seconds(1)).await; sleep(Duration::from_millis(100)).await; } ``` ## Reading the clock Reading the system clock is cheap, but not free, and a busy server reads it for every request: to check deadlines, write a `Date` header or log access times. `ntex::time` offers two cached clocks: - [`now`] returns an [`Instant`] for the start of the current runtime turn. On the neon and compio backends, all tasks polled in the same turn see the same value; on tokio it is [`Instant::now`]. - [`system_time`] returns a cached [`SystemTime`] that is refreshed roughly every 150 ms. Use it for wall-clock values that tolerate that lag, such as log lines and HTTP `Date` headers. Both are approximations. When exact elapsed time matters, call [`Instant::now`] yourself. ## Asynchronous timers All asynchronous timers are futures that are woken by the timer driver. No task is spawned per timer, and dropping a timer cancels it. ### `sleep` [`sleep`] returns a [`Sleep`] future that completes after the given delay: ```rust use ntex::time::{Millis, sleep}; #[ntex::main] async fn main() { sleep(Millis(100)).await; println!("at least 100 ms have elapsed"); } ``` A `Sleep` does not have to be consumed by `.await`. Its methods take `&self`, so one timer can be waited on, inspected and rearmed repeatedly: | Method | Effect | |------------------------|--------| | `wait().await` | waits without consuming the `Sleep` | | `reset(dur)` | rearms the timer to expire `dur` from now, before or after it has elapsed | | `elapse()` | completes the timer immediately and wakes the waiting task | | `is_elapsed()` | `true` once the timer has fired or was elapsed | | `poll_elapsed(cx)` | polls the timer from a hand-written future or service | Resetting is cheaper than creating a new timer: it moves the existing entry to another bucket and keeps the registered waker. This makes `Sleep` a good fit for an idle timeout that restarts on every message: ```rust use ntex::channel::mpsc; use ntex::time::{Seconds, sleep}; use ntex::util::{Either, select}; const IDLE: Seconds = Seconds(30); async fn read_until_idle(rx: mpsc::Receiver) { let idle = sleep(IDLE); loop { match select(rx.recv(), idle.wait()).await { Either::Left(Some(msg)) => { idle.reset(IDLE); println!("received {msg}"); } Either::Left(None) => break, // all senders are gone Either::Right(()) => { println!("no messages for {} seconds", IDLE.0); break; } } } } ``` `select` polls its first future first, so a message that arrives at the same moment as the timeout is still processed. ### `deadline` [`deadline`] works like `sleep`, but treats zero differently: a zero-duration `sleep` waits for one timer tick, while a zero-duration [`Deadline`] **never** completes. This fits the "zero means disabled" convention, because an optional timeout can be awaited without a separate code path: ```rust use ntex::channel::mpsc; use ntex::time::{Seconds, deadline}; use ntex::util::{Either, select}; /// Returns `None` if no message arrives in time. `Seconds::ZERO` waits forever. async fn recv_within(rx: &mpsc::Receiver, limit: Seconds) -> Option { match select(rx.recv(), deadline(limit)).await { Either::Left(msg) => msg, Either::Right(()) => None, } } ``` `Deadline::reset` takes `&mut self`. Resetting to zero disables the deadline, and resetting a disabled deadline to a non-zero value arms it again. `is_elapsed()` reports a disabled deadline as elapsed, meaning that there is nothing left to wait for. ### `timeout` and `timeout_checked` [`timeout`] runs a future with a time limit. It returns `Ok` with the future's output, or `Err(())` if the time ran out first, in which case the inner future is dropped: ```rust use ntex::time::{Seconds, sleep, timeout}; #[ntex::main] async fn main() { let fast = timeout(Seconds(1), async { "done" }).await; assert_eq!(fast, Ok("done")); let slow = timeout(Seconds(1), sleep(Seconds(5))).await; assert_eq!(slow, Err(())); } ``` If the future and the timer become ready during the same poll, the future wins. Like `sleep`, a zero-duration `timeout` still waits one timer tick, so it fails almost immediately unless the future is already ready. [`timeout_checked`] treats zero as "no timeout" and simply runs the future. Use it when the limit comes from configuration: ```rust use ntex::time::{Seconds, timeout_checked}; async fn call_upstream(limit: Seconds) -> Result { timeout_checked(limit, async { String::from("response") }) .await .map_err(|()| "upstream timed out") } ``` ### `interval` [`interval`] returns an [`Interval`] that ticks repeatedly. The first tick completes immediately: ```rust use ntex::time::{Seconds, interval}; async fn report_stats() { let ticks = interval(Seconds(10)); loop { ticks.tick().await; println!("collecting statistics"); } } ``` Each later tick completes one period after the previous tick was *observed*. If the task is busy and picks up a tick late, the schedule shifts instead of delivering a burst of missed ticks. That suits periodic housekeeping, but an `Interval` is not a precise metronome. `Interval` also implements [`Stream`] with `()` items, and [`poll_tick`] is available for hand-written futures. Dropping the interval stops it. ### Timers in hand-written futures `Sleep`, `Deadline` and `Interval` are `Unpin`, and each has a `poll_*` method that takes `&self`. A service or future can store a timer as a plain field and poll it from its own `poll` or readiness check: ```rust use std::{future::Future, pin::Pin, task::Context, task::Poll}; use ntex::time::{Millis, Sleep}; struct Throttle { delay: Sleep, } impl Future for Throttle { type Output = (); fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> { self.delay.poll_elapsed(cx) } } fn throttle(delay: Millis) -> Throttle { Throttle { delay: Sleep::new(delay) } } ``` ## Callback timers An asynchronous timer only does something when a task polls it. That is fine for one task waiting for one timer, but it is wasteful when the reaction to a timeout is tiny, such as setting a flag or waking someone else: you would need a task, or an extra branch in an existing `select`, just to observe the timer. [`CallbackTimer`] runs a closure directly from the timer driver instead. No task is spawned and nothing is polled. Each timer is a small wheel entry plus the closure, so it suits per-connection or per-request state. The ntex [`KeepAlive`] middleware uses one to notice idle services. ### Using a callback timer `CallbackTimer::new` arms a single invocation of the closure. The timer is **reusable**: after it fires or is cancelled, `reset` arms another invocation of the same closure. ```rust use std::{cell::Cell, rc::Rc}; use ntex::time::{CallbackTimer, Millis, sleep}; #[ntex::main] async fn main() { let fired = Rc::new(Cell::new(0)); let counter = fired.clone(); let timer = CallbackTimer::new(Millis(50), move || { counter.set(counter.get() + 1); }); sleep(Millis(100)).await; assert_eq!(fired.get(), 1); // Rearm, then change our mind. timer.reset(Millis(50)); timer.cancel(); sleep(Millis(100)).await; assert_eq!(fired.get(), 1); // Rearm again; this time it fires. timer.reset(Millis(50)); sleep(Millis(100)).await; assert_eq!(fired.get(), 2); } ``` | Method | Effect | |----------------|--------| | `new(dur, f)` | arms one invocation of `f` after `dur` | | `reset(dur)` | arms a new invocation, replacing any pending one | | `cancel()` | disarms the timer and keeps the closure for later | | `is_elapsed()` | `true` when disarmed: fired, cancelled or stopped by shutdown | | drop | cancels any pending invocation and releases the closure | The handle must be kept alive for as long as the callback should run. Dropping it is the way to stop the timer for good. ### Lazy rearming The real strength of callback timers is that a frequently extended deadline does not need a timer operation on every extension. Record the activity time in a `Cell`, and let the callback check whether the deadline really passed. If not, it rearms itself for the remaining time. This is how [`KeepAlive`] works: ```rust use std::{cell::Cell, rc::Rc, rc::Weak, time::Duration, time::Instant}; use ntex::time::{CallbackTimer, Millis, now}; struct IdleTracker { timeout: Millis, last_seen: Cell, expired: Cell, timer: CallbackTimer, } impl IdleTracker { fn new(timeout: Millis) -> Rc { Rc::new_cyclic(|this: &Weak| { let this = this.clone(); IdleTracker { timeout, last_seen: Cell::new(now()), expired: Cell::new(false), timer: CallbackTimer::new(timeout, move || { if let Some(tracker) = this.upgrade() { tracker.check(); } }), } }) } /// Called on every request. It only stores the time; the timer is untouched. fn touch(&self) { self.last_seen.set(now()); } fn check(&self) { let deadline = self.last_seen.get() + Duration::from(self.timeout); let current = now(); if deadline <= current { self.expired.set(true); } else { self.timer.reset(Millis::from(deadline - current)); } } } ``` A connection that receives a thousand requests within one timeout period costs a thousand `Cell` writes and at most one or two timer resets. The callback captures a `Weak` reference. The tracker owns the timer and the timer owns the closure, so a strong `Rc` in the closure would form a cycle and the tracker would never be dropped. ### Rules for callbacks - **Keep callbacks short, synchronous and non-blocking.** They run inside the thread's timer driver; a slow callback delays every other timer on that thread. To do asynchronous work, wake a task, notify a waiter or send to a channel from the callback. - **The closure is `Fn`, not `FnMut`,** because it can run many times. Keep changing state in `Cell`, `RefCell` or similar. - **Callbacks may use timers freely.** They run after the driver has released the wheel, so a callback can create, reset, cancel or drop timers, including its own. A callback that resets its own timer arms a new invocation. - **Order is not guaranteed** between callbacks that fire on the same tick. - **A zero duration never runs the callback inline;** it waits for the next timer tick, like `sleep`. - **Panics propagate** through the runtime and may stop the timer driver of that thread. Do not use panics for control flow in callbacks. - **Callback timers are local.** The handle is neither `Send` nor `Sync` and cannot be cloned, so the closure may capture `Rc` state. - **Shutdown cancels pending callbacks.** A surviving handle can be reset in a new runtime on the same thread, but it is not rearmed automatically. - `is_elapsed()` tells you that the timer is disarmed, not that the callback has already run. Track that in your own state if it matters. ## Summary - Use [`Millis`] and [`Seconds`] for timeouts. They are small, `Copy`, saturate instead of overflowing, and convert to and from `Duration`. - Use [`sleep`] for a delay, and `reset` a `Sleep` for a restartable idle timer. - Use [`deadline`] or [`timeout_checked`] when zero should mean "no limit", and [`timeout`] when zero should fail fast. - Use [`interval`] for periodic housekeeping, not for precise scheduling. - Use [`CallbackTimer`] when the reaction to a timeout is a short, synchronous state change, especially for per-connection timers that are extended often. - Expect timers to fire on time or a little late, never early. Measure elapsed time with `Instant`. - Keep the default 16 ms timer resolution unless short delays dominate your latency; then lower it with `timer_resolution`. [`ntex::time`]: https://docs.ntex.rs/api/ntex/time/index.html [`Builder::timer_resolution`]: https://docs.ntex.rs/api/ntex/rt/struct.Builder.html#method.timer_resolution [`CallbackTimer`]: https://docs.ntex.rs/api/ntex/time/struct.CallbackTimer.html [`Deadline`]: https://docs.ntex.rs/api/ntex/time/struct.Deadline.html [`deadline`]: https://docs.ntex.rs/api/ntex/time/fn.deadline.html [`Interval`]: https://docs.ntex.rs/api/ntex/time/struct.Interval.html [`interval`]: https://docs.ntex.rs/api/ntex/time/fn.interval.html [`KeepAlive`]: https://docs.ntex.rs/api/ntex/util/keepalive/struct.KeepAlive.html [`Millis`]: https://docs.ntex.rs/api/ntex/time/struct.Millis.html [`now`]: https://docs.ntex.rs/api/ntex/time/fn.now.html [`poll_tick`]: https://docs.ntex.rs/api/ntex/time/struct.Interval.html#method.poll_tick [`Seconds`]: https://docs.ntex.rs/api/ntex/time/struct.Seconds.html [`Sleep`]: https://docs.ntex.rs/api/ntex/time/struct.Sleep.html [`sleep`]: https://docs.ntex.rs/api/ntex/time/fn.sleep.html [`Stream`]: https://docs.ntex.rs/api/ntex/util/trait.Stream.html [`system_time`]: https://docs.ntex.rs/api/ntex/time/fn.system_time.html [`timeout`]: https://docs.ntex.rs/api/ntex/time/fn.timeout.html [`timeout_checked`]: https://docs.ntex.rs/api/ntex/time/fn.timeout_checked.html [`Instant`]: https://doc.rust-lang.org/std/time/struct.Instant.html [`Instant::now`]: https://doc.rust-lang.org/std/time/struct.Instant.html#method.now [`std::time::Instant`]: https://doc.rust-lang.org/std/time/struct.Instant.html [`std::time::Duration`]: https://doc.rust-lang.org/std/time/struct.Duration.html [`SystemTime`]: https://doc.rust-lang.org/std/time/struct.SystemTime.html # Migrating from ntex 3 to ntex 4 ntex 4 requires Rust 1.97 and upgrades the service stack to `ntex-service` 5. Most migration work involves the new service state and lifecycle APIs. ## Server Server service factories now create a `Service` directly. They no longer create a `ServiceFactory` whose initialization configuration is `SharedCfg`. The factory is called once per worker and receives a reference to that worker's application state. For a server without custom application state, continue to use `ntex::server::build()`. The factory receives `&()`: ```rust,no_run use std::io; use ntex::http::{HttpService, Response}; use ntex::SharedCfg; #[ntex::main] async fn main() -> io::Result<()> { ntex::server::build() .bind( "http", "127.0.0.1:8080", SharedCfg::default(), async |_| { HttpService::new(async |_| { Ok::<_, io::Error>(Response::Ok().body("Hello")) }) }, )? .run() .await } ``` Use `ntex::server::build_with_config()` when each worker needs application state. Its argument implements `ServerAppConfig` and creates the state for each worker. An asynchronous closure can be used directly: ```rust,no_run use std::io; use ntex::SharedCfg; #[derive(Clone)] struct WorkerState; #[ntex::main] async fn main() -> io::Result<()> { ntex::server::build_with_config( async || Ok::<_, io::Error>(WorkerState), ) .bind( "service", "127.0.0.1:8080", SharedCfg::default(), async |state: &WorkerState| { let state = state.clone(); ntex::service::fn_service(move |_| { let _state = state.clone(); async { Ok::<_, io::Error>(()) } }) }, )? .run() .await } ``` `ntex-service` 5 also removes `Service::poll()`. The `Service` trait now uses the asynchronous `ready()` and `shutdown()` lifecycle methods, while service chains provide `readiness()` and `shutdown()` callbacks. `Pipeline` and middleware APIs have been updated to bind and propagate service state. `Pipeline::call_nowait()` and `PipelineBinding::call_nowait()` have been removed; use `call()`. With notification-based readiness, successful readiness is cached until a subscribed `ReadyNotifier` invalidates it; calls do not consume that success. `Ctx::call_nowait()` was removed. Use `ctx.call(&inner, req).await` for nested dispatch, even after explicitly checking readiness. Valid cached readiness avoids another readiness check or allocation. Unlike the removed bypass, `call()` can wait or fail after readiness changes. Forward `Service::bind` through wrappers, and notify every readiness-changing transition before relying on persistent caching. Raw `Ctx::ready()` calls do not populate the upstream cache; use `ready(svc)` for independent child caching. Primitive `Service::call()` is not deprecated. ### Server builder Several `ServerBuilder` and web `HttpServer` methods have been renamed or removed: | ntex 3 | ntex 4 | |--------|--------| | `maxconn()` | `max_connections()` | | `shutdown_timeout()` | `graceful_shutdown_timeout()` | | `HttpServer::maxconnrate()` | `HttpServer::max_tls_handshakes()` | | `ServerBuilder::config()`, `HttpServer::config()` | pass `SharedCfg` to `bind()` or `listen()` | | `on_worker_start()`, `on_accept()` | `build_with_config()` or `HttpServer::with_config()` | `WorkerPool::shutdown_timeout()` is also renamed to `graceful_shutdown_timeout()`. ### Runtime features The deprecated `neon` feature has been removed from `ntex`, `ntex-rt` and `ntex-net`, the native runtime is used without it. Remove it from `ntex` and direct `ntex-rt` or `ntex-net` dependencies. The `neon-iocp` feature has been removed as well, Windows always uses IOCP. Select the `tokio`, `compio`, `neon-polling` or `neon-uring` feature when a specific runtime backend is required. ## HTTP services `HttpService::openssl()` and `HttpService::rustls()` have been replaced by the `http::openssl()` and `http::rustls()` service wrappers. OpenSSL: ```rust,ignore let service = ntex::http::openssl( acceptor, ntex::http::HttpService::new(handler), ); ``` rustls now accepts the ALPN protocol names separately: ```rust,ignore let service = ntex::http::rustls( config, &["h2", "http/1.1"], ntex::http::HttpService::new(handler), ); ``` ### HTTP configuration * `HttpServiceConfig::set_enable_headers_vec()` is replaced by `set_headers_vec(bool)`. * `h1::Codec::upgrade()` has been removed; use `Request::upgrade()` to detect upgrade and `CONNECT` requests. * New settings: `set_half_close()`, `set_host_validation()`, `set_max_start_line_size()`, and `set_write_timeout()`. ## URLs ntex 4 uses `urly::Url`, re-exported as `ntex::url`, instead of `http::Uri`: * `ntex::http::Uri` and the `ntex::http::uri` module have been removed. * `RequestHead::uri`, `Request::uri()`, `HttpRequest::uri()`, and `WebRequest::uri()` are `Url` values. `Url::path()` and `Url::query()` return `ntex::url::Path` and `ntex::url::Query`; use `as_str()` to get string slices. `HttpRequest::path()` and `HttpRequest::query_string()` still return `&str`. * `HttpRequest::url_for()` returns `ntex::url::Url` instead of `url::Url`. * The `url` feature has been removed; URL support is always available. * The HTTP and WebSocket clients accept any type that converts to `Url`, including `&str`, `String`, and `http::Uri`. Client connectors use `Connect` instead of `Connect`. * Server request targets are normalized: dot segments are removed, a fragment is dropped, and an origin-form path starting with `//` is a path, not an authority. ## HTTP client The HTTP client builder, connector, and connection pool have been redesigned: * `ClientBuilder` now owns the connector configuration; the separate client connector builder has been removed. * Custom connector factories are replaced by connector `Service` values. * `ClientBuilder::build()` is synchronous and takes a `SharedCfg`. * Client settings are stored in `ClientConfig` inside `SharedCfg`. The default connector is configured automatically: ```rust use ntex::SharedCfg; use ntex::client::{Client, ClientConfig}; fn create_client() -> Client { let cfg = SharedCfg::new("client").add( ClientConfig::new() .set_h1_connection_limit(8) .set_response_payload_limit(256 * 1024), ); Client::builder().build(cfg) } ``` Use `ClientBuilder::connector()` or `ClientBuilder::secure_connector()` to install a custom connector service. Request defaults have moved from `ClientBuilder` to `ClientConfig`: | ntex 3 `ClientBuilder` | ntex 4 `ClientConfig` | |------------------------|-----------------------| | `header()` | `set_header()` | | `basic_auth()`, `bearer_auth()` | `set_basic_auth()`, `set_bearer_auth()` | | `response_timeout()`, `disable_timeout()` | `set_response_timeout()`, `disable_timeout()` | | `response_payload_limit()` | `set_response_payload_limit()` | | `response_payload_timeout()` | `set_response_payload_timeout()` | The `ClientConfig` getters `timeout()`, `payload_limit()`, and `payload_timeout()` are now `response_timeout()`, `response_payload_limit()`, and `response_payload_timeout()`. `ClientBuilder::disable_redirects()`, `max_redirects()`, and `no_default_headers()` have been removed. The error variants `ClientError::TunnelNotSupported`, `ConnectError::Timeout`, `ConnectError::SslError`, `ConnectError::SslHandshakeError`, and `EncodeError::Fmt` have been removed. `InvalidUrl::Http` is replaced by `InvalidUrl::Parse`. ## WebSocket client WebSocket client settings have moved to `WsClientConfig`. Construct `WsClient` directly with the URI and configuration; a separate builder is no longer required: ```rust,ignore use ntex::ws::{WsClient, WsClientConfig}; let client = WsClient::new( "ws://127.0.0.1:8080/ws", WsClientConfig::new().set_max_frame_size(128 * 1024), ); let connection = client.connect().await?; ``` URL validation errors are now reported by `connect()` rather than by `WsClient::new()`, as `WsConfigError::Parse` instead of `WsConfigError::Http`. Custom connectors and TLS are still selected with `connector()`, `openssl()`, or `rustls()` on `WsClient`. `WsSink::on_disconnect()` now returns `ntex::io::Waiter<'static>`; the `OnDisconnect` future type has been removed. ## Web applications Web application state and error handling have been redesigned. ### Application state in handlers The application state type implements `web::State`. For common state that uses the default web error type, `web::AppState` provides a ready-made wrapper. State is created once per worker with `web::server_with_config()` and is passed to state-aware handlers by reference. The `web::types::State` extractor has been removed. Replace handlers that use it with `Route::to_with_state()`. A state-aware handler receives: 1. A shared reference to the application state. 2. The request-local state. 3. Any request extractors. ```rust,no_run use std::io; use ntex::{web, SharedCfg}; #[derive(Clone)] struct TestAppState { value: &'static str, } impl web::State for TestAppState { type Error = web::DefaultError; } async fn index( state: &TestAppState, _request_state: (), ) -> web::HttpResponse { web::HttpResponse::Ok().body(state.value) } #[ntex::main] async fn main() -> io::Result<()> { web::server_with_config( async || { Ok::<_, io::Error>(TestAppState { value: "Hello", }) }, async |_| { web::App::::new() .route("/", web::get().to_with_state(index)) }, ) .bind("127.0.0.1:8080", SharedCfg::default())? .run() .await } ``` The state-aware variants are available as `web::to_with_state()`, `Route::to_with_state()`, `Resource::to_with_state()`, and `ResourceServices::to_with_state()`. Continue to use `to()` when a handler only needs request extractors. The old `App::state()` model is replaced by service state for the primary application state. Additional configuration values can be stored with `WebAppConfig::set_state()` and retrieved with `HttpRequest::app_state()`. Request-local state is available through `WebRequest::st()` and `WebRequest::st_mut()`. Filters and middleware can change its type with `WebRequest::map_state()`. Custom implementations of the `Handler` trait must change `call()` from a method returning `impl Future` to an `async fn`. Ordinary `async fn` handlers do not need this change. ### Error handling The `ErrorRenderer` API has been removed. The state's associated `Error` type defines the application's error type, and errors are rendered through `WebResponseError`. Error rendering receives the application state instead of an `HttpRequest`. Service initialization errors now use `ntex::error::Failure` and `IntoFailure`. ### Middleware order Middleware registered with `App::middleware()`, `Scope::middleware()`, and `Resource::middleware()` now runs in registration order. The first registered middleware is the outermost one: it receives the request first and the response last. In ntex 3 the order was reversed, and the last registered middleware was the outermost one. ```rust use ntex::web::{self, App, middleware}; App::default() .middleware(middleware::DefaultHeaders::new().header("x-app", "example")) .middleware(middleware::Logger::default()) .route("/", web::get().to(async || "Hello")); ``` In ntex 4, `DefaultHeaders` receives the request before `Logger`, and `Logger` processes the response before `DefaultHeaders`. In ntex 3 the same code ran `Logger` first. To keep the ntex 3 behavior, reverse the order of `middleware()` calls. ## Custom codecs and I/O This section applies to code that implements codecs, filters, or dispatchers directly on top of `ntex::io` and `ntex::codec`. ### Codecs `ntex-codec` 2 writes encoded data into `BytePages`. The deprecated `encode(BytesMut)` method has been removed and `encodev()` has been renamed to `encode()`, which is now required: ```rust,ignore impl Encoder for MyCodec { type Item = Bytes; type Error = io::Error; fn encode(&self, item: Bytes, dst: &mut BytePages) -> Result<(), io::Error> { dst.append(item); Ok(()) } } ``` `Decoder::decode_eof()` is called when the peer closes the stream. Its default implementation calls `decode()`; override it to decode a final frame that has no terminator. If undecodable bytes remain after a clean EOF, `Io::recv()` and the dispatcher report an `io::ErrorKind::UnexpectedEof` error. ### `IoConfig` * `set_disconnect_timeout()` / `disconnect_timeout()` are renamed to `set_shutdown_timeout()` / `shutdown_timeout()`. A zero timeout panics. * Read and write buffers are `ntex-bytes` pages and share one page cache, configured per thread with `ntex_bytes::set_page_cache_size()`. `BufConfig` and `IoConfig::read_buf()` / `write_buf()` have been removed. * Each connection adapts its read page size to its reads, between the sizes set with `set_read_size(min, max)` (4 KiB to 64 KiB by default). * Backpressure is configured with a single high watermark and is released at half of it: | ntex 3 | ntex 4 | |--------|--------| | `set_read_buf(high, low, cache_size)` | `set_read_backpressure(high)` and `set_read_size(min, max)` | | `set_write_buf(high, low, cache_size)` | `set_write_backpressure(high)` | | `set_write_page_size()`, `write_page_size()` | `set_write_size()`, `write_size()` | * `set_write_timeout()` closes connections whose peer stops reading. ### `Io` and `IoRef` | ntex 3 | ntex 4 | |--------|--------| | `IoRef::force_close()` | `IoRef::terminate()` | | `IoRef::wants_shutdown()` | `IoRef::close()` | | `IoRef::with_read_buf()` | `IoRef::with_read_dst()` | | `IoRef::with_read_src_buf()` | `IoRef::with_read_src()` | | `IoRef::with_write_buf()` | `IoRef::with_write_src()` | | `IoRef::with_write_dst_buf()` | `IoRef::with_write_dst()` | | `IoRef::on_disconnect()` returning `OnDisconnect` | returns `Waiter<'static>` | | `Io::read_ready()`, `Io::poll_read_ready()` | `Io::read_more()`, `Io::poll_read_more()` | | `Io::poll_dispatch()` | `Io::register_dispatch()` | | `Io::pause()` | removed; reads resume via `poll_read_more()` | | `Io::set_config()` | pass the configuration to `Io::new()` | | `Io::take()` | `unsafe Io::take()` | | `IoRef::resize_read_buf()`, `IoContext::resize_read_buf()` | `BytesMut::reserve_more()` | | `IoStatusUpdate::KeepAlive` | `IoStatusUpdate::Timeout` | `IoRef::is_closed()` now reports whether closing has finished. Use the new `IoRef::is_active()` to check whether the connection is still usable. ### Dispatcher `Reason::KeepAliveTimeout` is renamed to `Reason::KeepAlive`, and the new `Reason::WriteTimeout` is reported when `IoConfig::set_write_timeout()` expires. ## Other API changes * `ntex::rt`: `System::stop_on_panic()` has been removed and `Builder::stop_on_panic()` no longer has an effect. Use `Builder::panic_handling()` or `#[ntex::main(panic_handling = true)]`. `System::set_latency_callback()` requires a `Send + Sync` callback. * `ntex::time`: `query_system_time()` has been removed; use `system_time()`. * `ntex_util::channel::bstream::Receiver::max_buffer_size()` is deprecated in favor of `set_watermarks()`. * `ntex::http::HeaderMap` no longer implements `FromIterator`; build maps with `insert()` or `append()`. * `HeaderValue::to_str()` accepts any valid UTF-8 value, not only visible ASCII. ### Router `ntex::router` (`ntex-router` 2) renames several APIs: | ntex 3 | ntex 4 | |--------|--------| | `Router::recognize_mut_checked()` | `Router::recognize_checked_mut()` | | `ResourceDef::resource_path()`, `resource_path_named()` | `ResourceDef::build_path()`, `build_path_named()` | | `ResourceDef::name_mut()` | `ResourceDef::set_name()` | | `RouterBuilder::rdef()` | `RouterBuilder::resource()` | | `Path::unprocessed()` | `Path::path()` | | `Path::query()` | `Path::get()` | | `Router::build()`, `RouterBuilder::finish()` | removed | `RouterBuilder` registration methods return `&mut RouterEntry` instead of a tuple; use `set_id()`, `set_name()`, `set_check_value()`, and `resource_mut()`. `Path::skip()` takes a `u32`. Path segments that do not decode to valid UTF-8 are kept percent-encoded. ## Connection and protocol configuration `SharedCfg` remains the container for connection and protocol configuration, but it is separate from worker application state: * Pass `SharedCfg` to server `bind()` or `listen()` methods. * Pass `SharedCfg` to `ClientBuilder::build()`. * Use `build_with_config()` or `web::server_with_config()` for per-worker application state. See [Connection and Protocol Configuration](./5-server.md#connection-and-protocol-configuration) for a complete example.