Envoy 开启 HTTP2 后偶现 404 如何解决？

在大部分基于 Envoy 实现的网关里，都存在这样一个问题，当开启 http2 时，客户端访问会出现偶发的 404，并且可以从日志注意到这些 404 的请求，:authority 头里的域名和 SNI 里的域名不一致。

• https://github.com/envoyproxy/envoy/issues/6767

• https://github.com/istio/istio/issues/13589

• https://github.com/projectcontour/contour/issues/1493

1. 域名 B 和域名 A 是解析到同一个 IP。
2. 跟域名 A 建立通信时获取的证书的 Common Name 是 wildcard，且可以匹配到域名 B；或者 SAN 中存在域名 B。

在 Envoy 的网关里，常见的 SNI 和域名路由的映射方式是一对一的，这样匹配到 SNI A 时，只会出现 A 域名的路由配置，没有 B 域名的路由，所以产生了 404。

这个方案有两处不足：

1. 如果当域名的证书发生了修改，需要重建 filter chain，就会影响 downstream 连接断开。

2. 增加了控制面的复杂度，例如使用 Gateway API 时，不能基于 filter chain 和 Gateway 中 Listener 一对一映射。

              "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"              inlineCode: |                function envoy_on_request(request_handle)                  local streamInfo = request_handle:streamInfo()                  if streamInfo:requestedServerName() ~= "" then                    if (string.sub(streamInfo:requestedServerName(), 1, 2) == "*." and not string.find(request_handle:headers():get(":authority"), string.sub(streamInfo:requestedServerName(), 2))) then                      request_handle:respond({[":status"] = "421"}, "Misdirected Request")                    end                    if (string.sub(streamInfo:requestedServerName(), 1, 2) ~= "*." and streamInfo:requestedServerName() ~= request_handle:headers():get(":authority")) then                      request_handle:respond({[":status"] = "421"}, "Misdirected Request")                    end                  end                end

1. 失去了连接复用的特性，对于一些依赖 HTTP2 连接复用实现页面加载优化的场景，可能是不接受的。
2. HTTP 421 存在版本兼容问题，特别是在中国有很多基于低版本 chromium 构建的 Hybrid Android APP，存在对不同域名复用连接，但是又不支持基于 421 重新建连的问题，返回 421 会直接造成业务报错。

1. 基于 RDS

2. 基于 VHDS

3. 基于 SRDS

1. 要支持 wildcard domain，支持前缀匹配。
2. 不同的 port 下相同域名逻辑可能不同，例如 80 下可能是强制跳转。

下面是 Higress 对 ScopedRoutes 做的扩展在配置上的体现：

// [#next-free-field: 6]message ScopedRoutes {     option (udpa.annotations.versioning).previous_message_type =        "envoy.config.filter.network.http_connection_manager.v2.ScopedRoutes";......      message HostValueExtractor {        option (udpa.annotations.versioning).previous_message_type =            "envoy.config.filter.network.http_connection_manager.v2.ScopedRoutes.ScopeKeyBuilder."            "FragmentBuilder.HostValueExtractor";
        // The maximum number of host superset recomputes. If not specified, defaults to 100.        google.protobuf.UInt32Value max_recompute_num = 1;      }
      message LocalPortValueExtractor {        option (udpa.annotations.versioning).previous_message_type =            "envoy.config.filter.network.http_connection_manager.v2.ScopedRoutes.ScopeKeyBuilder."            "FragmentBuilder.LocalPortValueExtractor";      }

      oneof type {        option (validate.required) = true;
        // Specifies how a header field's value should be extracted.        HeaderValueExtractor header_value_extractor = 1;
        // Extract the fragemnt value from the :authority header, and support recompute with the wildcard domains,        // i.e. ``www.example.com`` can be recomputed with ``*.example.com``, then ``*.com``, then ``*``.        HostValueExtractor host_value_extractor = 101;
        // Extract the fragment value from local port of the connection.        LocalPortValueExtractor local_port_value_extractor = 102;      }    }
    // The final(built) scope key consists of the ordered union of these fragments, which are compared in order with the    // fragments of a :ref:`ScopedRouteConfiguration<envoy_v3_api_msg_config.route.v3.ScopedRouteConfiguration>`.    // A missing fragment during comparison will make the key invalid, i.e., the computed key doesn't match any key.    repeated FragmentBuilder fragments = 1 [(validate.rules).repeated = {min_items: 1}];  }

在所有 filter chains 共用路由配置的情况下，因为不同的 filter chain 可能有不同的认证策略，比如常见的需要认证客户端证书（mTLS）的情况，或者基于 IP 的 RBAC 等。

// [#protodoc-title: HTTP route components]// * Routing :ref:`architecture overview <arch_overview_http_routing>`// * HTTP :ref:`router filter <config_http_filters_router>`
// The top level element in the routing configuration is a virtual host. Each virtual host has// a logical name as well as a set of domains that get routed to it based on the incoming request's// host header. This allows a single listener to service multiple top level domain path trees. Once// a virtual host is selected based on the domain, the routes are processed in order to see which// upstream cluster to route to or whether to perform a redirect.// [#next-free-field: 24]message VirtualHost {  option (udpa.annotations.versioning).previous_message_type = "envoy.api.v2.route.VirtualHost";......  // If non-empty, a list of server names (such as SNI for the TLS protocol) is used to determine  // whether this request is allowed to access this VirutalHost. If not allowed, 421 Misdirected Request will be returned.  //  // The server name can be matched whith wildcard domains, i.e. ``www.example.com`` can be matched with  // ``www.example.com``, ``*.example.com`` and ``*.com``.  //  // Note that partial wildcards are not supported, and values like ``*w.example.com`` are invalid.  //  // This is useful when expose all virtual hosts to arbitrary HCM filters (such as using SRDS), and you want to make  // mTLS-protected routes invisible to requests with different SNIs.  //  // .. attention::  //  //   See the :ref:`FAQ entry <faq_how_to_setup_sni>` on how to configure SNI for more  //   information.  repeated string allow_server_names = 101;}