概述
阿里云服务网格ASM支持Sidecar模式和Sidecarless模式。Sidecar模式，即在每个服务实例旁运行一个代理，是当前最主流、稳定的方案，但这种架构引入了显著的延迟和资源开销。为了解决Sidecar模式所带来的延迟和资源消耗，近些年出现了不同的Sidecarless模式方案，例如Istio Ambient，Istio Ambient 通过在每个节点部署ztunnel，对节点上运行的Pod进行四层流量代理，并引入了waypoint对七层流量进行代理。虽然Sidecarless模式能够降低延迟和资源消耗，但其稳定性以及功能的完善程度仍有待提升。
阿里云服务网格ASM当前支持不同的Sidecarless模式，例如Istio Ambient模式，ACMG模式以及Kmesh等。Kmesh（具体参考https://kmesh.net/）是一款基于ebpf+可编程内核实现的高性能服务网格数据面软件。通过将流量治理下沉到内核，实现网格内服务通信无需经过代理软件，大大缩减了流量转发路径，有效提升了服务访问的转发性能。
- Kmesh简介 -
Kmesh的双引擎模式（dual-engine）模式使用eBPF在内核空间拦截流量，部署Waypoint Proxy来处理复杂的L7流量管理，从而将L4和L7的治理在内核空间（eBPF）和用户空间（Waypoint）之间分离。相比Istio的Ambient Mesh，延迟减少了30%。相较于内核原生模式，双引擎模式无需对内核进行增强，具有更广泛的适用性。
图丨Kmesh双引擎模式架构图
当前阿里云服务网格ASM支持使用Kmesh双引擎模式作为服务网格的数据面之一，从而实现更加高效的服务治理。具体来说, 可以使用ASM作为控制面，并在ACK Kubernetes集群中部署Kmesh作为数据面。
在阿里云ACK集群中
部署Kmesh并连接ASM控制面
- 使用前提 -
参考阿里云服务网格ASM的官方文档, 创建一个ASM集群实例与ACK Kubernetes集群, 并将ACK Kubernetes集群添加到ASM集群实例中进行管理。具体操作步骤, 可以参考文档：
https://help.aliyun.com/zh/asm/getting-started/add-a-cluster-to-an-asm-instance-1?spm=a2c4g.11186623.help-menu-147365.d_1_2.436356ccEXYKBU
- 安装Kmesh -
执行以下命令，将Kmesh项目下载到本地。
git clone https://github.com/kmesh-net/kmesh.git && cd kmesh查看ASM控制面Service信息
下载完成后，您首先需要执行以下命令，查看当前ASM控制面在集群中的Service名称，以配置Kmesh与ASM控制面的连接。
kubectl get svc -n istio-system | grep istiod
使用kubectl安装Kmesh
您可以使用kubectl在ACK Kubernetes集群中安装 Kmesh，但在安装前，请为Kmesh DaemonSet添加ClusterId和xdsAddress环境变量，用于Kmesh和ASM控制面认证与连接。ClusterId为您部署Kmesh的ACK Kubernetes集群Id，xdsAddress为ASM控制面的Service信息。
apiVersion: apps/v1kind: DaemonSetmetadata:name: kmeshlabels:app: kmeshnamespace: kmesh-systemspec:spec:containers:- env:# 修改 xdsAddress 为 ASM 控制面 Service- name: XDS_ADDRESSvalue: "istiod-1-22-6.istio-system.svc:15012"# 添加阿里云 ACK 集群 id- name: CLUSTER_IDvalue: "cluster-id"...
修改以下命令中的Kmesh Daemonset环境变量后，执行以部署Kmesh。
kubectl apply -f -<<EOFapiVersion: apps/v1kind: DaemonSetmetadata:name: kmeshlabels:app: kmeshnamespace: kmesh-systemspec:selector:matchLabels:app: kmeshtemplate:metadata:labels:app: kmeshannotations:prometheus.io/path: "status/metric"prometheus.io/port: "15020"prometheus.io/scrape: "true"spec:tolerations:- effect: NoScheduleoperator: Exists- key: CriticalAddonsOnlyoperator: Exists- effect: NoExecuteoperator: Existsvolumes:- name: mnthostPath:path: /mnt- name: sys-fs-bpfhostPath:path: /sys/fs/bpf- name: lib-moduleshostPath:path: /lib/modules# k8s default cni conflist path- name: cnihostPath:path: /etc/cni/net.d# k8s default cni path- name: kmesh-cni-install-pathhostPath:path: /opt/cni/bin- name: host-procfshostPath:path: /proctype: Directory- name: istiod-ca-certconfigMap:defaultMode: 420name: istio-ca-root-cert- name: istio-tokenprojected:defaultMode: 420sources:- serviceAccountToken:audience: istio-caexpirationSeconds: 43200path: istio-token# 修改 xdsAddress 为 ASM 控制面 Service- name: XDS_ADDRESSvalue: "istiod-1-22-6.istio-system.svc:15012"# 添加阿里云 ACK 集群 id- name: CLUSTER_IDvalue: "cluster-id"containers:- name: kmeshimage: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/kmesh:latestimagePullPolicy: IfNotPresentcommand: ["/bin/sh", "-c"]args:["./start_kmesh.sh --mode=dual-engine --enable-bypass=false --enable-bpf-log=true",]securityContext:privileged: truecapabilities:add: ["all"]env:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: INSTANCE_IPvalueFrom:fieldRef:fieldPath: status.podIP- name: XDS_ADDRESSvalue: "istiod.istio-system.svc:15012"- name: SERVICE_ACCOUNTvalueFrom:fieldRef:fieldPath: spec.serviceAccountName- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeNamevolumeMounts:- name: mntmountPath: /mntreadOnly: false- name: sys-fs-bpfmountPath: /sys/fs/bpfreadOnly: false- name: lib-modulesmountPath: /lib/modulesreadOnly: false# k8s default cni conflist path- name: cnimountPath: /etc/cni/net.dreadOnly: false# k8s default cni path- name: kmesh-cni-install-pathmountPath: /opt/cni/binreadOnly: false- mountPath: /host/procname: host-procfsreadOnly: true- name: istiod-ca-certmountPath: /var/run/secrets/istio- name: istio-tokenmountPath: /var/run/secrets/tokensresources:limits:memory: "800Mi"cpu: "1"priorityClassName: system-node-criticalserviceAccountName: kmesh---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:name: kmeshlabels:app: kmeshrules:- apiGroups: [""]resources: ["pods","services","namespaces"]verbs: ["get", "update", "patch", "list", "watch"]- apiGroups: ["apps"]resources: ["daemonsets"]verbs: ["get"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:name: kmeshlabels:app: kmeshroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: kmeshsubjects:- kind: ServiceAccountname: kmeshnamespace: kmesh-system---apiVersion: v1kind: Namespacemetadata:name: kmesh-system---apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: add-listener-filternamespace: istio-systemlabels:asm-system: 'true'provider: asmspec:workloadSelector:labels:gateway.istio.io/managed: istio.io-mesh-controllerconfigPatches:- applyTo: LISTENERmatch:proxy:proxyVersion: .*patch:operation: ADDvalue:name: kmesh-listeneraddress:socket_address:protocol: TCPaddress: 0.0.0.0port_value: 15019additional_addresses:- address:socket_address:protocol: TCPaddress: "::"port_value: 15019default_filter_chain:filters:- name: envoy.filters.network.tcp_proxytyped_config:"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxystat_prefix: kmeshcluster: main_internalfilter_chains:- filter_chain_match:application_protocols:- "http/1.1"- "h2c"filters:- name: envoy.filters.network.http_connection_managertyped_config:"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManagerstat_prefix: kmeshroute_config:name: defaultvirtual_hosts:- name: defaultdomains:- '*'routes:- match:prefix: "/"route:cluster: main_internalhttp_filters:- name: waypoint_downstream_peer_metadatatyped_config:"@type": type.googleapis.com/udpa.type.v1.TypedStructtype_url: type.googleapis.com/io.istio.http.peer_metadata.Configvalue:downstream_discovery:- workload_discovery: {}shared_with_upstream: true- name: envoy.filters.http.routertyped_config:"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Routerlistener_filters:- name: "envoy.listener.kmesh_tlv"typed_config:"@type": "type.googleapis.com/udpa.type.v1.TypedStruct""type_url": "type.googleapis.com/envoy.listener.kmesh_tlv.config.KmeshTlv"- name: "envoy.filters.listener.http_inspector"typed_config:"@type": "type.googleapis.com/envoy.extensions.filters.listener.http_inspector.v3.HttpInspector"---apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: skip-tunnelingnamespace: istio-systemlabels:asm-system: 'true'provider: asmspec:workloadSelector:labels:gateway.istio.io/managed: istio.io-mesh-controllerconfigPatches:- applyTo: NETWORK_FILTERmatch:proxy:proxyVersion: .*listener:filterChain:filter:name: envoy.filters.network.tcp_proxypatch:operation: REPLACEvalue:name: envoy.filters.network.tcp_proxytyped_config:"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxystat_prefix: kmesh_original_dst_clustercluster: kmesh_original_dst_cluster---apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: add-original-dst-clusternamespace: istio-systemlabels:asm-system: 'true'provider: asmspec:workloadSelector:labels:gateway.istio.io/managed: istio.io-mesh-controllerconfigPatches:- applyTo: CLUSTERmatch:proxy:proxyVersion: .*context: SIDECAR_INBOUNDpatch:operation: ADDvalue:name: "kmesh_original_dst_cluster"type: ORIGINAL_DSTconnect_timeout: 2slb_policy: CLUSTER_PROVIDED---apiVersion: v1kind: ServiceAccountmetadata:name: kmeshnamespace: kmesh-systemEOF
- 检查Kmesh服务状态 -
安装完毕后，执行以下命令查看Kmesh服务启动状态。
kubectl get pods -A | grep kmesh
执行以下命令查看Kmesh服务运行状态。
kubectl logs -f -n kmesh-system kmesh-l5z2j
您可以通过执行以下命令来对指定命名空间启用Kmesh。
kubectl label namespace default istio.io/dataplane-mode=Kmesh- 流量调度示例 -
部署示例应用和流量调度规则
在为default命名空间启用Kmesh后，执行以下命令安装示例应用。
kubectl apply -f samples/fortio/fortio-route.yamlkubectl apply -f samples/fortio/netutils.yaml
执行以下命令查看实例应用运行状态。
kubectl get pod# NAME READY STATUS RESTARTS AGEkubectl describe pod netutils-575f5c569-lr98z | grep Annotations# Annotations: kmesh.net/redirection: enabled
当您看到应用Pod具有kmesh.net/redirection: enabled时，代表Kmesh转发已经对该Pod启用。
执行以下命令查看当前定义的流量调度规则，可以看出，此时定义了90%的流量流向v1版本的fortio，10%的流量流向v2版本的fortio。
kubectl get virtualservices -o yaml# apiVersion: v1# - apiVersion: networking.istio.io/v1beta1# {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"fortio","namespace":"default"},"spec":{"hosts":["fortio"],"http":[{"route":[{"destination":{"host":"fortio","subset":"v1"},"weight":90},{"destination":{"host":"fortio","subset":"v2"},"weight":10}]}]}}# creationTimestamp: "2024-07-09T09:00:36Z"# resourceVersion: "11166"# - fortio# - route:# - destination:# - destination:# resourceVersion: ""
发起测试流量
您可以通过执行以下命令发起测试流量，您应该可以看到，此时只有大约10%的流量流向v2版本的fortio。
for i in {1..20}; do kubectl exec -it $(kubectl get pod | grep netutils | awk '{print $1}') -- curl -v $(kubectl get svc -owide | grep fortio | awk '{print $3}'):80 | grep "Server:"; done# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 2# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 1# < Server: 2# < Server: 1# < Server: 1
/ END /
