Printable version

Drivers & software

SGI product documentation and software downloads are currently being migrated, please visit the SGI portal for your SGI documentation and software needs during this migration period.

* RECOMMENDED * HPE Mellanox RoCE (RDMA over Converged Ethernet) Driver for Red Hat Enterprise Linux 6 Update 8 (x86_64)

By downloading, you agree to the terms and conditions of the Hewlett Packard Enterprise Software License Agreement.
Note:  Some software requires a valid warranty, current Hewlett Packard Enterprise support contract, or a license fee.

Type: Driver - Network
Version: 3.4(12 Jul 2017)
Operating System(s):
Red Hat Enterprise Linux 6 Server (x86-64)
Multi-part download
File name: kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig (2.0 KB)
File name: kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm (1.2 MB)
File name: mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig (2.0 KB)
File name: mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm (1.0 MB)
This RPM contains the HPE Tested and Approved Linux based Mellanox RoCE (RDMA over Converged Ethernet) driver for supported HPE Mellanox adapter cards.

HPE Mellanox RoCE driver version 3.4 contains the following changes and new features:

  • Added the following kernel module parameters:
    • mlx4_en_only_mode
    • udev_dev_port_dev_id

To ensure the integrity of your download, HPE recommends verifying your results with the following SHA-256 Checksum values:

69024ac98a19cfb07010896edfbd4723327e2b2d39e1bd993b5beb0c6c53b951 kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig
ad62d1e322b76aa0ba0c258e2d1f36326d0f34466447cf1a37ad10033cbf5977 kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm
9f93da7bc25143fe379a1d7f673c1a6554974e826a6232ce97fa65506be033ab mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig
90f6b4d2dc98cb31edb9fed8cf4310396186c29eec89e27341a08810adc8586e mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm

Reboot Requirement:
Reboot is required after installation for updates to take effect and hardware stability to be maintained.


Installation:

Login as root, download the RPM to a directory on your hard drive and change to that directory.

Note: Irrespective of the kernel version or type used, the "mlnx-ofa_kernel-3.4" RPM must be installed to enable the user space functionality for RoCE. The RoCE user space library RPM (mlnx-ofa_kernel) may conflict with the OpenMPI RPMs included with the OS distribution.This is a known behavior. Insure that the OpenMPI RPMs from the Linux distribution are not installed on the target node prior to the installation of "mlnx-ofa_kernel". If already installed, uninstall any OpenMPI distribution RPMs before installing HPE Mellanox RoCE driver packages.

To install or upgrade the driver:

If using kernel version 2.6.32-642.el6 or any future errata:

# rpm -Uvh kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm

Setup is now complete. Reboot your computer for the driver to take effect.


End User License Agreements:
HPE Software License Agreement v1


Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


Supported Devices and Features:

SUPPORTED KERNELS:
The kernels of Red Hat Enterprise Linux 6U8 (x86_64) supported by this binary rpm are:
2.6.32-642.el6 -  (x86_64) and future update kernels.


Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues are fixed in version 3.4:

  • "ethtool" self test used to fail on interrupt test after timeout if mlx4_ib module was not loaded.
  • On rare occassions, kernel panic occured during system reboot caused by mlx4_en_get_drvinfo() called from asynchronous event handler.
  • When attempting to disable SR-IOV while there are any VF netdevs open, the operation failed to succeed.
Version:4.2 (2 Mar 2018)
Fixes

Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues have been fixed in version 4.2:

  • RPM commands used to fail and create a core file occassionally after reboot, with messages such as “Bus error (core dumped)”, causing the openibd service to fail to start.
  • RoCEv2 multicast traffic using RDMA-CM with IPv4 address were not received by the adapter.
  • ethtool -P output was 00:00:00:00:00:00 when using old kernels.
  • Replaced a few “GPL only” legacy libibverbs function with upstream implementation that conforms with libibverbs GPL/BSD dual license model.
  • ACCESS_REG command failure used to appear upon RoCE Multihost driver restart in dmesg.
  • Concurrent client requests got corrupted when working in persistent server mode due to a race condition on the server side.
  • Client side did not exit gracefully in RTT mode when the server side was not reachable.
Enhancements

HPE Mellanox RoCE driver version 4.2 contains the following changes and new features:

  • Added a feature that allows registering a specific physical address range.
  • Added support for PTP feature over PKEY interfaces. This feature allows for accurate synchronization between the distributed entities over the network. The synchronization is based on symmetric Round Trip Time (RTT) between the master and slave devices, and is enabled by default. 
  • Support for Virtual MAC feature, which allows users to add up to 4 virtual MACs (VMACs) per Virtual Function (VF).
  • Added the option to change receive buffer size and cable length. Changing cable length will adjust the receive buffer's xon and xoff thresholds. 
  • Added support for the following GRE tunnel offloads:
    • TSO over GRE tunnels
    • Checksum offloads over GRE tunnels
    • RSS spread for GRE packets
  • Added support for the host side (RDMA initiator) in Red Hat Enterprise Linux 7 Update 2 and above.
  • Added support for the driver to notify the Firmware when Software receive queues are overloaded.
  • Added support for configuring PFC stall prevention in cases where the device unexpectedly becomes unresponsive for a long period of time. PFC stall prevention disables flow control mechanisms when the device is stalled for a period longer than the default pre-configured timeout. Users now have the ability to change the default timeout by moving to auto mode.
  • Added support for Q-in-Q VST feature in ConnectX-5 adapter cards family.
  • Added support for VGT+ in ConnectX-4/ConnectX-5 HCAs. This feature is s an advanced mode of Virtual Guest Tagging (VGT), in which a VF is allowed to tag its own packets as in VGT, but is still subject to an administrative VLAN trunk policy. The policy determines which VLAN IDs are allowed to be transmitted or received. The policy does not determine the user priority, which is left unchanged.
  • Added support for hardware Tag Matching offload with Dynamically Connected Transport (DCT).
  • Added support for the driver to take an automatic snapshot of the device’s CR-Space in cases of critical failures.

Version:3.4 (12 Jul 2017)
Fixes

Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues are fixed in version 3.4:

  • "ethtool" self test used to fail on interrupt test after timeout if mlx4_ib module was not loaded.
  • On rare occassions, kernel panic occured during system reboot caused by mlx4_en_get_drvinfo() called from asynchronous event handler.
  • When attempting to disable SR-IOV while there are any VF netdevs open, the operation failed to succeed.
Enhancements

HPE Mellanox RoCE driver version 3.4 contains the following changes and new features:

  • Added the following kernel module parameters:
    • mlx4_en_only_mode
    • udev_dev_port_dev_id

Type: Driver - Network
Version: 3.4(12 Jul 2017)
Operating System(s):
Red Hat Enterprise Linux 6 Server (x86-64)

Description

This RPM contains the HPE Tested and Approved Linux based Mellanox RoCE (RDMA over Converged Ethernet) driver for supported HPE Mellanox adapter cards.

Enhancements

HPE Mellanox RoCE driver version 3.4 contains the following changes and new features:

  • Added the following kernel module parameters:
    • mlx4_en_only_mode
    • udev_dev_port_dev_id

Installation Instructions

To ensure the integrity of your download, HPE recommends verifying your results with the following SHA-256 Checksum values:

69024ac98a19cfb07010896edfbd4723327e2b2d39e1bd993b5beb0c6c53b951 kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig
ad62d1e322b76aa0ba0c258e2d1f36326d0f34466447cf1a37ad10033cbf5977 kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm
9f93da7bc25143fe379a1d7f673c1a6554974e826a6232ce97fa65506be033ab mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.compsig
90f6b4d2dc98cb31edb9fed8cf4310396186c29eec89e27341a08810adc8586e mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm

Reboot Requirement:
Reboot is required after installation for updates to take effect and hardware stability to be maintained.


Installation:

Login as root, download the RPM to a directory on your hard drive and change to that directory.

Note: Irrespective of the kernel version or type used, the "mlnx-ofa_kernel-3.4" RPM must be installed to enable the user space functionality for RoCE. The RoCE user space library RPM (mlnx-ofa_kernel) may conflict with the OpenMPI RPMs included with the OS distribution.This is a known behavior. Insure that the OpenMPI RPMs from the Linux distribution are not installed on the target node prior to the installation of "mlnx-ofa_kernel". If already installed, uninstall any OpenMPI distribution RPMs before installing HPE Mellanox RoCE driver packages.

To install or upgrade the driver:

If using kernel version 2.6.32-642.el6 or any future errata:

# rpm -Uvh kmod-mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm mlnx-ofa_kernel-3.4-OFED.3.4.2.1.5.1.ged26eb5.rhel6u8.x86_64.rpm

Setup is now complete. Reboot your computer for the driver to take effect.


Release Notes

End User License Agreements:
HPE Software License Agreement v1


Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


Supported Devices and Features:

SUPPORTED KERNELS:
The kernels of Red Hat Enterprise Linux 6U8 (x86_64) supported by this binary rpm are:
2.6.32-642.el6 -  (x86_64) and future update kernels.


Fixes

Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues are fixed in version 3.4:

  • "ethtool" self test used to fail on interrupt test after timeout if mlx4_ib module was not loaded.
  • On rare occassions, kernel panic occured during system reboot caused by mlx4_en_get_drvinfo() called from asynchronous event handler.
  • When attempting to disable SR-IOV while there are any VF netdevs open, the operation failed to succeed.

Revision History

Version:4.2 (2 Mar 2018)
Fixes

Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues have been fixed in version 4.2:

  • RPM commands used to fail and create a core file occassionally after reboot, with messages such as “Bus error (core dumped)”, causing the openibd service to fail to start.
  • RoCEv2 multicast traffic using RDMA-CM with IPv4 address were not received by the adapter.
  • ethtool -P output was 00:00:00:00:00:00 when using old kernels.
  • Replaced a few “GPL only” legacy libibverbs function with upstream implementation that conforms with libibverbs GPL/BSD dual license model.
  • ACCESS_REG command failure used to appear upon RoCE Multihost driver restart in dmesg.
  • Concurrent client requests got corrupted when working in persistent server mode due to a race condition on the server side.
  • Client side did not exit gracefully in RTT mode when the server side was not reachable.
Enhancements

HPE Mellanox RoCE driver version 4.2 contains the following changes and new features:

  • Added a feature that allows registering a specific physical address range.
  • Added support for PTP feature over PKEY interfaces. This feature allows for accurate synchronization between the distributed entities over the network. The synchronization is based on symmetric Round Trip Time (RTT) between the master and slave devices, and is enabled by default. 
  • Support for Virtual MAC feature, which allows users to add up to 4 virtual MACs (VMACs) per Virtual Function (VF).
  • Added the option to change receive buffer size and cable length. Changing cable length will adjust the receive buffer's xon and xoff thresholds. 
  • Added support for the following GRE tunnel offloads:
    • TSO over GRE tunnels
    • Checksum offloads over GRE tunnels
    • RSS spread for GRE packets
  • Added support for the host side (RDMA initiator) in Red Hat Enterprise Linux 7 Update 2 and above.
  • Added support for the driver to notify the Firmware when Software receive queues are overloaded.
  • Added support for configuring PFC stall prevention in cases where the device unexpectedly becomes unresponsive for a long period of time. PFC stall prevention disables flow control mechanisms when the device is stalled for a period longer than the default pre-configured timeout. Users now have the ability to change the default timeout by moving to auto mode.
  • Added support for Q-in-Q VST feature in ConnectX-5 adapter cards family.
  • Added support for VGT+ in ConnectX-4/ConnectX-5 HCAs. This feature is s an advanced mode of Virtual Guest Tagging (VGT), in which a VF is allowed to tag its own packets as in VGT, but is still subject to an administrative VLAN trunk policy. The policy determines which VLAN IDs are allowed to be transmitted or received. The policy does not determine the user priority, which is left unchanged.
  • Added support for hardware Tag Matching offload with Dynamically Connected Transport (DCT).
  • Added support for the driver to take an automatic snapshot of the device’s CR-Space in cases of critical failures.

Version:3.4 (12 Jul 2017)
Fixes

Upgrade Requirement:
Recommended - HPE recommends users update to this version at their earliest convenience.


The following issues are fixed in version 3.4:

  • "ethtool" self test used to fail on interrupt test after timeout if mlx4_ib module was not loaded.
  • On rare occassions, kernel panic occured during system reboot caused by mlx4_en_get_drvinfo() called from asynchronous event handler.
  • When attempting to disable SR-IOV while there are any VF netdevs open, the operation failed to succeed.
Enhancements

HPE Mellanox RoCE driver version 3.4 contains the following changes and new features:

  • Added the following kernel module parameters:
    • mlx4_en_only_mode
    • udev_dev_port_dev_id