Введение

В этой статье представлена ​​соответствующая документация и ссылки для драйверов Mellanox Linux.

Linux Driver

• быстрый старт
  • Getting started with ConnectX-4 100Gb/s Adapter for Linux
  • Getting Started with ConnectX-5 100Gb/s Adapters for Linux
  • Mellanox Adapters - Comparison Table Сравнительная таблица сетевых карт

  Class	Feature	ConnectX-3	ConnectX-3 Pro	ConnectX-4	ConnectX-4 Lx	ConnectX-5	ConnectX-6	References and Notes
  Interface	Port/Speed options	2 ports of 10/40/56GbE	2 ports of 10/40/56GbE	2 ports of 100/56/50/40/25/10GbE	2 ports of 10/25GbE	2 ports of 100/56/50/40/25/10GbE	2 ports of 200/100/56/50/40/25/10GbE	Note: There various of adapter cards. Some OPNs for example could be speed limited, e.g.ConnectX-4 FDR (that won’t link up EDR) or Connect-IB that is x8 only and not x16.
  					1 port of 40/50GbE
  Interface	PCIe	x8 Gen3	x8 Gen3	x8, x16 Gen3	x8 Gen3	x16 Gen3	2x Gen3 x16
  						x16 Gen4	Gen4 x16
  Interface	PCIe Switch					PCIe x16	PCIe x32
  Interface	Coherent Accelerator Processor Interface (CAPI)	-	-	-	-	Supported (v2)	Supported (v2)	IBM - Coherent Accelerator Processor Interface (CAPI)
  								http://openpowerfoundation.org/blogs/interconnect-your-future-mellanox-100gb-edr-capi-infiniband-and-interconnects/
  IB RDMA / RoCE	IB RDMA / RoCE	IB RDMA, RoCE*	IB RDMA, RoCE	IB RDMA, RoCE	RoCE	IB RDMA, RoCE	IB RDMA, RoCE	RDMA/RoCE Solutions
  RoCE	RoCE Congestion Control	-	Supported	Supported	Supported	Supported	Supported	RDMA/RoCE Solutions
  								Understanding RoCEv2 Congestion Management
  								RoCEv2 CNP Packet Format Example
  								HowTo Configure RoCE Congestion Control for Windows 2012
  HPC	CORE-Direct®	Supported	Supported	Supported	Supported	Supported	Supported	Exploited by FCA and extended by Mellanox SHARP technology.
  								http://www.mellanox.com/related-docs/whitepapers/TB_CORE-Direct.pdf
  HPC	PeerDirect®	Supported	Supported	Supported	Supported	Supported	Supported	GPU Direct Uses PeerDirect.
  								http://www.mellanox.com/page/products_dyn?product_family=116
  HPC	Dynamically Connected Transport (DCT)	-	-	Supported	Supported	Supported	Supported	http://www.mellanox.com/related-docs/applications/SB_Connect-IB.pdf
  CPU Offloads	Stateless Ethernet offload	Supported	Supported	Supported + LRO, LSOv2	Supported + LRO, LSOv2	Supported + LRO, LSOv2	Supported + LRO, LSOv2	LRO = Large Receive Offload.
  								LSO = Large Send offload.
  								See MLNX_OFED User Manual for more info.
  CPU Offloads	RSS (MAC, VLAN 5 Tuple)	Supported	Supported	Supported +	Supported +	Supported +	Supported +
  Virtualization	SR-IOV	Supported	Supported	Supported	Supported	Supported	Supported	Virtualization Solutions
  Virtualization	Multi Host	-	-	4 hosts	4 hosts	4 hosts	8 hosts	http://www.mellanox.com/page/products_dyn?product_family=210&mtag=multihost
  Open V-Switch (OVS)	OVS Offload	-	-	-	Supported	Supported +	Supported +	Describes whether the adapters can run the OVS data-plane in HW. Supported means that at least matching for a specific flow is performed in HW. Supported+ means that HW Encap/Decap is also included.
  								See http://www.mellanox.com/page/asap2.
  Overlay Network	Stateless offload	-	Supported	VXLAN / NVGRE / GENEVE	VXLAN / NVGRE / GENEVE	VXLAN / NVGRE / GENEVE and FlexParse	VXLAN / NVGRE / GENEVE and FlexParse	Virtualization Solutions
  Overlay Network	Encap/Decap (in HW)	-	-	-	Supported	Supported	Supported
  (VXLAN/NVGRE)
  Storage	Erasure Coding Offload	-	-	Supported	Supported	Supported	Supported	Reed Solomon Erasure Coding hardware offload is supported by the adapters.
  								Verbs API are available.
  								Understanding Erasure Coding Offload
  Storage	T10/DIF Signature Handover	-	-	Supported	-	Supported	Supported	HowTo Enable T10-PI (T10-DIF) Data Integrity Protection in iSER with LIO Target
  Storage	NVMe oF Target Offload	-	-	-	-	Supported	Supported +
  	(also for Burst Buffer)
  Storage	Host Chaining	-	-	-	-	Supported	Supported
  Cloud Integration	Mirantis Fuel	Supported Fuel 7.0/8.0	Supported Fuel 7.0/8.0	Supported Fuel 8.0	Supported Fuel 8.0	Supported	Supported	Cloud Solutions
  Media & Entertainment	Packet Pacing	-	-	Supported	Supported	Supported	Supported
  Security	Secure Firmware update	-	-	Supported	Supported	Supported	Supported
  Security	Secure Boot	-	-	-	-	-	Supported

* Partial support Частично поддерживается

• Mellanox Linux Driver Modules Relationship (MLNX_OFED) Mellanox Linux Взаимосвязь модулей драйвера (MLNX_OFED)

Схема программного обеспечения OFED

HowTo Install MLNX_OFED Driver Как установитьMLNX_OFEDводить машину

• Перекомпилировать OFED/перестроить ofed: https://enterprise-support.nvidia.com/s/article/howto-compile-mlnx-ofed-drivers--160---mlnx-ofa-kernel-example-x Скомпилируйте MLNX_OFED для других систем: https://enterprise-support.nvidia.com/s/article/howto-compile-mlnx-ofed-for-different-linux-kernel-distribution--160---160-x
• HowTo Install MLNX_OFED for ConnectX-4/ConnectX-5 via ISO Image
• Ethtool Commands
• How To Extract MLNX_OFED Source Files

• Port
  • HowTo Change Port Type in Mellanox ConnectX-3 Adapter
  • HowTo Find the Logical-to-Physical Port Mapping (Linux)
  • HowTo Change the Ethernet Port Speed of Mellanox Adapters (Linux)
  • HowTo Enable LLDP on Linux Servers for Link Discovery
  • HowTo Find Cable Info on Mellanox Adapters and Switches
• QoS
  • HowTo Set Egress Priority VLAN on Linux
  • HowTo Configure Packet Pacing on ConnectX-4
  • HowTo Auto-Config PFC and ETS on ConnectX-4 via LLDP DCBX
  • Default ToS to skprio mapping on Linux
  • HowTo Tune Receive Buffers on Mellanox Adapter Cards
• Maintenance and Monitoring Мониторинг и поддержание
  • HowTo Enable SNMP Server (snmpd) on Linux
  • HowTo Find Mellanox Adapter Type and Firmware/Driver version (Linux)
  • HowTo Compile MLNX_OFED for Different Linux Kernel Distribution
  • HowTo manage Mellanox Linux Driver modules and RPMs
  • HowTo Dump Driver Configuration Using Ethtool
  • How-To Dump RDMA traffic Using the Inbox tcpdump tool (ConnectX-4)
• Counters прилавок
  • Counters Troubleshooting for Linux Driver
  • MLNX_OFED ConnectX-3 RDMA Diagnostic Counters
  • Understanding mlx5 ethtool Counters Понимание счетчика эфира mlx5
  • Understanding mlx5 Linux Counters and Status Parameters
  • Understanding mlx4 Linux Counters
• Time Synchronization Синхронизация времени
  • Running TimeKeeper with ConnectX-4
  • Running Linux PTP with ConnectX-4/ConnectX-5
  • Running PTPd with ConnectX-4
  • HowTo Steer PTP traffic to single RX ring (via ethtool)
  • HowTo Test 1PPS on Mellanox Adapters
• Flow Steering контроль потока
  • Understanding Linux debug-fs Flow Steering rules
• Offload Features
  • How To Enable Large Receive Offload (LRO)
• Tools
  • ibdev2netdev Сопоставление физического интерфейса с логическим интерфейсом
  • mlx_fs_dump
  • mlnx_qos
• Other Features
  • MLNX_OFED driver configuration file: openib.conf
  • HowTo Increase Memory Size used by Mellanox Adapters
  • HowTo Configure aRFS on ConnectX-4
  • HowTo Implement PeerDirect Client using MLNX_OFED

Инструменты для прошивки Инструменты для прошивки

• Getting started with Mellanox Firmware tools (MFT) for Linux
• mlxlink

PXE (среда выполнения перед загрузкой, среда выполнения перед загрузкой)

• Expansion ROM for BIOS and UEFI on Mellanox cards
• HowTo Configure PXE Server in UEFI Mode for IPoIB Network

Другие темы, связанные с Linux, см. в других технических темах.

• Virtualization Solutions Решения для виртуализации
• RDMA/RoCE Solutions Решение RoCE
• Performance Tuning for Mellanox Adapters Настройка производительности сетевой карты MLX
• Programming Solutions Примеры программирования/ODP/односторонние/двусторонние операции и т.д.

Подвести итог

• Эта статья Подвести итоги статьи и ссылки, связанные с драйверами NvidiaMLX, Эту индексную страницу можно обновить позже.
• Для протокола RoCE можно производить захват и анализ пакетов. Например, можно перехватывать пакеты для сетевой карты серии MLX5 через ibdump в скомпилированном вами исходном коде драйвера, сделайте БЕЗ_FW_TOOLS=yes [install].

https://github.com/Mellanox/ibdump

ссылка

Решение драйвера Linux: https://enterprise-support.nvidia.com/s/article/linux-driver-solutions

Сяобин (ssbandjl)

блог: https://cloud.tencent.com/developer/user/5060293/articles | https://logread.cn | https://blog.csdn.net/ssbandjl | https://www.zhihu.com/people/ssbandjl/posts

столбец ДПУ

https://cloud.tencent.com/developer/column/101987

Технические друзья: друзья, которые интересуются DPU/интеллектуальными сетевыми картами/разгрузкой/сетями, ускорением хранения данных/изоляцией безопасности и другими технологиями, могут присоединиться к группе обмена технологиями DPU.