Cadence Macb Linux Driver for Zynq and Zynq Ultrascale+ MPSoC

Introduction

This page gives an overview of the macb driver which is available as part of the Zynq and ZynqMP Linux distribution and in the mainline.
Paths, files, links and documentation on this page are given relative to the Linux kernel source tree.

HW IP features

  • Speed support for 10/100/1000 Mbps
  • MAC loopback and PHY loopback
  • Partial store and forward option
  • Packet buffer option
  • Flow control - TX/RX pause
  • Checksum offload support, CRC checking, FCS stripping
  • Promiscuous mode, Broadcast mode
  • Collision detection and enforcement - this is an IP feature, no SW support required
  • MDIO support for PHY layer management
  • Multicasting support
  • VLAN tagged frames
  • Half duplex support
  • Programmable IPG
  • External FIFO interface
  • Wake on LAN
  • IEEE1588 support for ZynqMP
  • Jumbo frame size support for ZynqMP
  • 64 bit addressing for ZynqMP
  • Priority queue support for ZynqMP
  • PS SGMII support (hardwired to 1Gbps) is present in ZynqMP

Features supported in driver

(Functional HW IP and stack related features)
  • Speed support for 10/100/1000 Mbps with clock framework
  • Packet buffer option
  • Checksum offload support, CRC checking, FCS stripping
  • MDIO support for PHY layer management
  • Multicasting support
  • Programmable IPG
  • IEEE1588 support for ZynqMP
  • Jumbo frame size support for ZynqMP
  • 64 bit addressing for ZynqMP
  • Priority queue support for ZynqMP
  • PS SGMII support is present in ZynqMP and supported in the driver
  • This driver can be used with PL SGMII/1000BaseX driver on Zynq and ZynqMP
  • This driver can be used with gmii2rgmii converter driver
  • Support for EthTool queries
  • NAPI support
  • Clock adaptation on Zynq and ZynqMP
  • Runtime PM and suspend/resume supported on ZynqMP
  • Partial store and forward
  • Wake on LAN support using ARP on ZynqMP

Missing Features, Known Issues and Limitations

  • Linux does not support loopback
  • Flow control support is not present in the driver. RX pause frames can be received by the IP but TX pause frame support is not provided.
  • External FIFO interface is not supported by the driver - this implementation is DMA based.
  • No interrupt support for PHY events in driver. The current implementation relies on polling method for phy event
  • No IEEE 1588 support for Zynq as the timestamp implementation in IP is not accurate enough.
  • Support for single MAC managing multiple PHYs is not yet merged.
  • PS SGMII GT initialization is not supported via zynqmp_phy.c - it needs to be explicitly initialized.
  • WOL does not work on warm restart designs because GEM WOL requires an RX BD scratch area that is accessible even during suspend (OCM is used for this) and OCM is secure in this design which is a limitation for this feature.

Important AR links

  • PTP time adjustment for a large negative delta fails in 2018.1/2 - AR-71332
  • WOL does not work on warm restart designs due to some limitations - AR-71028
  • MACB MDIO bus support - Please find the patches for 2017.1, 2017.2, 2017.3 and 2017.4 at the AR - AR-69132
  • ZynqMP PS SGMII GT initialization and related - AR-68866
  • TI PHY design on ZynqMP evaluation board has incorrect straps and can be remedied with a SW workaround (already implemented in drivers) - AR-70686
  • PL PCS PMA initialization in fsbl for Zynq and ZynqMP - refer to xapp1026 and xapp130
  • For full list of ARs, search XKB


Kernel Configuration

The following config options should be enabled in order to build the macb driver
CONFIG_ETHERNET
CONFIG_NET_CADENCE
CONFIG_MACB
CONFIG_NETDEVICES
CONFIG_HAS_DMA
macb_config.png

Optional kernel configuration:
-> CONFIG_MACB_USE_HWSTAMP
macb_config_hwtstamp.png
Use IEEE 1588 hwstamp (only supported in ZynqMP) - This config option supports use of 1588 HW TSTAMP support in ZynqMP and depends on MACB.
This option enables IEEE 1588 Precision Time Protocol (PTP) support for MACB.

Devicetree

Compatible string can be:
-> "cdns,gem" for Zynq
-> "cdns,zynqmp-gem" fro ZynqMP. This compatible string enables use of jumbo frame sizes, 1588 and HW timestamping suport and any features exclusive to ZynqMP.

For more details on phy bindings please refer "Documentation/devicetree/bindings/net/macb.txt"
gem0: ethernet@e000b000 {
    compatible = "cdns,gem";
    reg = <0xe000b000 0x1000>;
    status = "disabled";
    interrupt-parent = <&gic>;
    interrupts = <0 22 4>;
    clocks = <&clkc 30>, <&clkc 30>, <&clkc 13>;
    clock-names = "pclk", "hclk", "tx_clk";
    #address-cells = <1>;
    #size-cells = <0>;
    phy-handle = <&ethernet_phy>;
    phy-mode = "rgmii-id";
    ethernet_phy: ethernet-phy@7{
        reg = <7>;
    };
 
};

Related devicetree information

For generic ethernet DT property information, refer to:
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/net/ethernet.txt
For PHY related DT information, refer to:
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/net/phy.txt
When selecting phy specific settings, make sure to mention interface type, speed (if limited/fixed) and phy address properties.
PHY/Converter devices that may be used with this MAC:
-> Xilinx GMII2RGMII converter (https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/net/xilinx_gmii2rgmii.txt)
-> Xilinx PCS PMA PHY ( https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/net/xilinx-phy.txt )

RGMII tuning is driven in phy framework using "rgmii-id", "rgmii-txid", "rgmii-rxid" properties Make sure to set phy-mode to any of these as per your board requirement.
In addition to enabling tuning, some phys also give control of tuning values via devicetree. Please refer to the devicetree bindings documentation of the phy you use in order to tune these according to your board.

Clock adaption is present by default for both Zynq and ZynqMP. For more details refer to devicetree clock bindings and respective wiki pages.
ZynqMP also has tsu-clk adaption support in addition to all the other reference clocks.

-> This driver can be used for a MAC - MAC fixed link connection. In order to do so, please update the devicetree fixed link node as per
https://github.com/Xilinx/linux-xlnx/blob/master/Documentation/devicetree/bindings/net/fixed-link.txt
and set the phy-mode to "moca" (https://github.com/Xilinx/linux-xlnx/blob/master/include/linux/phy.h)

Performance

These benchmark performance numbers were obtained by connecting Xilinx boards to Linux PCs/server machines (Ubuntu/Red Hat Enterprise).
The tool used is netperf (Refer to tool information below).
The protocol, MTU size and option to note CPU load can all be selected from netperf/netserver options

Zynq

Board: ZC706
CPU Freq: 666MHz (A9)
Link Speed: 1000Mbps, Full duplex
Linux version: 4.14

TCP (Mbps
UDP(Mbps)
MTU
RX
CPU(%)
TX
CPU(%)
RX
CPU(%)
TX
CPU(%)
1500
582.13
93.81
767.47
98.76
451.3
99.65
849.3
51.15

ZynqMP

Board: ZCU102
CPU Freq 1100MHz (A53)
Link Speed 1000Mbps, Full duplex
DDR 533MHz
CCU: No
Linux version: 4.14

TCP ( Mbps)
UDP(Mbps)
MTU
Rx
CPU(%)
Tx
CPU(%)
Rx
CPU(%)
Tx
CPU(%)
1500
941.38
49.27
941.08
10.36
961.6
27.87
961.4
11.35
8192
988.85
7.91
988.46
6.32
960
7.24
967.8
7.7

Test Procedure

Diagnostic and Protocol Tests

PING

This utility used to test the reachability of a host on an Internet Protocol(IP) network and to measure the round trip time for messages sent from the originating host to a destination computer.
How to run:
ping <Remote IP Address>

WebServer

Connect zynq board to a Linux x86 machine. Ensure that telnet server is running on the Zynq board. It tests for remote access for Zynq board on host machine
Open a web browser on host machine and enter the static IP assigned to zynq board. Webpage is expected to be displayed properly.

Telnet

telnet <Server IP Address>

FTP & TFTP

How to run:
Open a ftp client on the host with the Zynq.
x86> ftp 192.168.1.10
Transfer a big enough file (in MBs) using mput command.
x86> mput <file_name>
File transfer should be completed without any error.

Pkt Generator

Please refer to link below for how to run and various options
https://www.kernel.org/doc/Documentation/networking/pktgen.txt

Performance Tests

Netperf

How to run:
Server:
netserver
Client:
taskset 2 ./netperf -H <Server IP> -t TCP_STREAM
taskset 2 ./netperf -H <Server IP> -t UDP_STREAM
For more information please refer to the link below:
http://www.netperf.org/netperf/

Iperf

How to run:
Server:
./iperf_arm -s -u
./iperf_arm -s
Client:
./iperf_arm -c <Server IP> -u -b <banwidth>
./iperf_arm -c <Server IP>
For more information please refer to the link below:
http://en.wikipedia.org/wiki/Iperf

Stress Test

Iperf with option -d

Run iperf in dual testing mode. This will cause the server to connect back to the client on the port specified in the -L option (or defaults to the port the client connected to the server on). This is done immediately therefore running the tests simultaneously.
./iperf_arm -c <Server IP> -d

Ping flood test

Users can send hundred or more packets per second using -f option. It prints a ‘.’ when a packet is sent, and a backspace is printed when a packet is received
ping -f localhost

PTP

1588 synchronization can be tested on ZynqMP using open source linuxptp application.
http://linuxptp.sourceforge.net/
The setup requires a master with precise clock and timstamping capabilities, typically a NIC or another 1588 capable device.
How to run
master:
#ptp4l -i <interface name> -m
slave:
#ptp4l -i <interface name> -s -m

Mainline status

The macb driver is currently in sync with mainline kernel 4.14 except for the following:
-> Power management support
-> Partial store and forward support
-> Minor differences including gpio phy reset support and mdio phy node support
Any further changes will be upstreamed.
-> Fixed link, HRESP and RX multiple queue handling will be in sync with mainline 4.16 kernel.

PHY details

The following PHYs were tested with ZynqMP GEM:
-> TI DP83867IR
-> TI DP83867E (SGMII)
-> Marvell 88E1112
-> Realtek RTL8211
-> Vitesse VSC8211

Change Log

2018.2
No changes

2018.1
Summary:
  • Use mainline implementation of 64 bit addressing
  • Use mainline implementation of PTP support
  • Add clock management for tsu-clk
  • Add WOL support for ZynqMP
Commits:
Sync with 4.14 mainline kernel
9aa7608 net: macb: Add tsu_clk property and use it
e964800 net: macb: Add WOL support for ZynqMP
53ec68b net: macb: Fix GEM crash when suspend/resume plus down/up is done
2d522a5 net: macb: Remove older MACB_EXT_BD config option
617ea48 net: macb: Correct check for 64 bit addressing
fa19565 net: macb: Add phy suspend and resume
95a33ee net: macb: Set rx mode in resume
d7aa0c8 net: macb: Update macb RX tie-off descriptors
e6fca01 net: macb: Remove unnecessary DBW read back from NWCFG
23d6b42 net: macb: Cleanup empty lines

2017.4
No changes

2017.3
Summary:
  • Added support for partial store and forward
  • Pulled in minor mainline fixes and phy related issues
  • Added support for macb suspend/resume
Commits:
bf85fd4 net: macb: Add support for partial store and forward
f646336 net: macb: Fix gpio for phy reset
a29aa21 net: macb: Fix issues with FPD off
e1a214d net: macb: Misc cleanup

2017.2
Summary:
  • Pulled in a minor mainline fix for mdio bus scan error check
Commits:
4356634 macb: fix mdiobus_scan() error check

2017.1
Summary:
  • Added PM runtime support
  • Added context loss support; Cleanup around clock and suspend, resume paths. Although this support is added in macb driver, there is a know issue at the moment that GEM does not work on resume directly. It is required to bring the interface down and up again.
  • Fixed ptp time adjustment for large negative delta
  • Fix PHY reset and only call GPIOD functions when valid GPIO is present
  • Fixed spinlocks in macb_close around ptp_clock_unregister to avoid kernel panic.
  • Fixed TSU CAPS mask
Related phy driver changes:
  • DP83867: Added a SW workaround for link instability on ZCU102 board.
Commits:
afeaf15 arm64: zynqmp: macb: release spinlock before calling ptp_clock_unregister
36f7baa net: macb: Correct TSU_CAPS mask
27f1c64 macb: fix PHY reset
7613445 net: macb: Only call GPIO functions if there is a valid GPIO
2288919 net: macb: Fix ptp time adjustment for large negative delta
6cbc5cd net: cadence: macb: Fix kernel-doc format
ddd4804 net: macb: fix the clk enable and disable
1b0a659 net: macb: Add runtime support
4dc7d77 net: macb: Add context loss support
b9a2910 net: macb: Fix the double disable of clocks
756de54 net: macb: Cleanup the clock code
2f2bb37 net: macb: Fix unused warning
911b158 net: macb: Enable clocks for the mdio accesses
25f7255 net: macb: Convert the infinite wait loop to a timeout
53ac032 net: macb: Move to runtime_put to cut clocks
d415d56 net: macb: Update the phy write sequence
DP83867 phy driver:
7557928 net: macb: SW workaround for link instability on DP83867

2016.4
Summary:
  • Added support for fixed link
Commits:
59e3534 net: macb: Add support for fixed link

2016.3
Summary:
  • Added support for 64 bit addressing
  • Added support to use gmii2rgmii convertor driver
  • Handle HRESP error with SW reset and re-initialization of necessary parameters
  • The above changes are also in mainline
Commits:
b0fbcba net: macb: Handle HRESP error
ff73646 net: macb: Fixed mixed declaration and code warnings
190b6af net: macb: Update TX and RX EXT BD registers only when required
d470dfb net: macb: Correct CAPS masks
6121d00 net: macb: Add support for 64 bit addressing
f9c43e8 net: macb: add support for mdio phy nodes

Related Links