ZYNQ_PCIe_TRD_2014.3

History
ISE DS 14.7 Zynq PCIe Targeted Reference Design
ISE DS 14.6 Zynq PCIe Targeted Reference Design
ISE DS 14.5 Zynq PCIe Targeted Reference Design
ISE DS 14.4 Zynq PCIe Targeted Reference Design
ISE DS 14.3 Zynq PCIe Targeted Reference Design

1 Introduction

This page provides instructions on how to build various components of the Zynq PCIe Targeted Reference Design (TRD) and how to setup the hardware platform and run the design on the ZC706 Evaluation Kit. The ZC706 Evaluation kit is based on a XC7Z045 FFG900-2 Zynq-7000 SoC device. For additional information, refer to UG961.

1.1 About the Zynq PCIe TRD

The Zynq PCIe Targeted reference design expands the Base Targeted Reference Design (UG925) by adding PCI Express communication with a host system at PCIe x4 GEN2 speed. In the Base Targeted Reference design, the input of the video processing pipeline is generated by a test pattern generator in the FPGA fabric. In this design, the input of the video processing pipeline is generated by an application on the host computer at 1080p60 resolution and transmitted to the ZC706 board via PCIe. The data is processed by video pipeline and passed back to the host system via PCIe. As full 1080p60 video stream only take up around 4Gbps, an additional data generator and a checker are implemented and connected to channel 1 of PCIe DMA showcasing the maximum PCIe x4 GEN2 bandwidth achieved by the hardware. For additional information, please refer to UG963

1.2 Zynq PCIe TRD Package Contents

The Zynq PCIe TRD package is released with the source code, Xilinx PlanAhead and SDK projects, and an SD card image that enables the user to run the video demonstration and software
application. It also includes the binaries necessary to configure and boot the Zynq-7000 AP SoC board. The package also contains the software driver source files required to run application
software in the PCIe host machine. This wiki page assumes the user has already downloaded the PCIe TRD package and extracted its contents to the PCIe TRD home directory referred to as
ZYNQ_TRD_HOME in this wiki and to the PCIe host machine in a folder of choice.



2 Prerequisites

• The ZC706 Evaluation Kit ships with the version 14.x Device-locked to the Zynq-7000 XC7Z045 FFG900-2 device and all required licenses to build the TRD. For additional information, refer to UG973 VIVADO Design Suite 2014.3: Release Notes, Installation and Licensing Guide.
• PC with PCIe v2.0 slot. Recommended PCI Express Gen2 PC system motherboards are ASUS P5E (Intel X38), ASUS Rampage II Gene (Intel X58) and Intel DX58SO (Intel X58).
• Fedora 16 LiveCD for booting Linux on PCIe host machine.
• A Linux development PC with the ARM GNU tools installed. The ARM GNU tools are included with the Xilinx ISE Design Suite Embedded Edition or can be downloaded separately.
• A Linux development PC with the distributed version control system Git installed. For more information, refer to Using Git and to UG821: Xilinx Zynq-7000 EPP Software Developers Guide.
• Petalinux 2014.2 SDK
• Open JDK
• Other system utilities like make (3.82 or higher) and corkscrew if accessing git behind a firewall.
• A Linux development PC with QT and QWT libraries cross-compiled for Zynq platform. Set ZYNQ_QT_INSTALL environment variable by referring to Xilinx Zynq Qt/Qwt Libraries - Build Instructions

3 Building the FPGA Hardware Bitstream

This section explains how to generate the FPGA hardware bitsream using the Xilinx PlanAhead tool and how to export the hardware platform to Xilinx Software Development Kit (SDK) for software application development. Inside the PlanAhead project, a Xilinx Platform Studio (XPS) project is referenced that contains the embedded hardware design. The design top level file instantiates the embedded top level file along with the system with PCIe IP wrapper, PCIe DMA, PCIe performance monitor and hardware generator and checker blocks.

3.1 Building the Bitstream

Steps for building the FPGA hardware bitstream

Browse to $ZYNQ_TRD_HOME/hardware/vivado/scripts directory:

• On Windows ,Open Vivado 2014.3 Tcl Shell by clicking Start > All Programs > Xilinx Design Tools > Vivado 2014.3 > Vivado 2014.3 Tcl Shell
  • In the command prompt navigate to $ZYNQ_TRD_HOME/hardware/vivado/scripts and enter " vivado -source z7_pcie_trd_project.tcl " command
• On Linux, enter " vivado -source z7_pcie_trd_project.tcl " at the command prompt.

This opens the Vivado® Integrated Design Environment (IDE), loads the block diagram, and adds the required top file and XDC file to the project.




In the Flow Navigator pane on the left-hand side under Program and Debug, click Generate Bitstream.

• Note: Click on Yes if a window appears as No Implementation Results are available.

The bit stream will be generated at $ZYNQ_TRD_HOME/hardware/vivado/runs/z7_pcie_trd_2014.3.runs/impl_1/z7_pcie_trd_top.bit

3.2 Exporting the Hardware Platform to SDK

Steps for exporting the hardware platform to SDK

From the VIVADO project flow navigator click on Open Implemented Design.
From the VIVADO menu bar, select File > Export > Export Hardware.



In the Export Hardware window press OK. The hardware platform will be exported to $ZYNQ_TRD_HOME/hardware/vivado/runs/z7_pcie_trd_2014.3.sdk

4 Installation of Petalinux SDK

4.1 Prerequisites

• 2GB RAM (recommended minimum for Xilinx tools)
• Pentium 4 2GHz CPU clock or equivalent.
• 5 GB free HDD space.
• Supported OS:
  • RHEL 5 (32-bit or 64-bit)
  • RHEL 6 (32-bit or 64-bit)
  • SUSE Enterprise 11 (32-bit or 64-bit)
• Petalinux release package downloaded.
• Valid Petalinux license.
• Common system packages and libraries are installed on your workstation. The installation process will check for these. See the section Required Tools and Libraries for more details. For detailed information refer petalinux installation guide UG976 .
• Download Petalinux 2014.2 SDK software from Xilinx website download section

NOTE: The 2014.3 update is based on Petalinux 2014.2 SDK, because Petalinux SDK update is not quarterly.

4.2 Extract the Petalinux Package

Assuming all the prerequisites described in the last subsection are satisfied, Petalinux installation is very straight forward.Without any options, the installer will installl as a subdirectory of the current directory. Alternatively, an installation path may be specific . Run the downloaded petalinux installer.
Petalinux will be installed in the petalinux-v2014.2-final directory, directly underneath the working directory of this command.
So, if you install the installer into your home directory /home/user, Petalinux will be installed in /home/user/petalinux-v2014.2-final.
You may move the resulting petalinux-v2014.2-final directory to a preferred location before continuing.

4.3 Install License

PetaLinux licenses are managed using the same system as all other Xilinx Design Tools. For more details on licensing and setup of license please refer to the
Xilinx Design Tools: Installation and Licensing Guide (UG798)" section "Obtaining and Managing a License".

4.4 Setup Petalinux Working Environment

After extracting the package, the remainder of the setup is completed automatically.
1. Go to the Petalinux root directory by running this command on the command console:
cd <path-to-installed-PetaLinux>
e.g.:
2. Source the appropriate PetaLinux setup script by running this command on the command console:
For Bash:
The first time the setup script is sourced, it will perform some post installation tasks to check system dependencies and initialise the Linux kernel source tree.
Below is an example of the output from sourcing the setup script for the first time:

Petalinux environment set to ’/home/user/petalinux-v2014.2-final
INFO: Finalising Petalinux installation
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
The post-install step only occurs once. Subsequent runs of the settings script should be much quicker, and simply output a confirmation message such as that shown below:
PetaLinux environment set to ’/home/user/petalinux-v2014.2-final'

4.4 Verify Petalinux Installation

Verify that the Petalinux working environment has been set:
/home/user/petalinux-v2014.2-final
Environment variable "$PETALINUX" should point to the path to the installed Petalinux. Your echo output may be different from this example, depending upon where you installed Petalinux.

5 Zynq PCIe TRD BSP Installation

Petalinux includes reference designs for you to start working with and customise for your own projects. These are provided in the form of installable BSP (Board Support Package) files, and include all necessary design and configuration files, including pre-built and tested hardware and software images, ready for download to your board or for booting in the QEMU system simulation environment.
Run petalinux-create command on the command console:
petalinux-create -t project -s <path-to-bsp>

Example:

6 Add Linux kernel 3.15.0 support

Zynq PCIe TRD uses Xilinx Linux 3.15.0 kernel version.Petalinux provides option to add individual project
specific kernel/u-boot version. Below steps demonstrates how to add/config/build the Linux kernel.
Create a new branch named zynq_pcie_trd_2014_3 based on the xilinx-v2014.3 tag.

Apply the kernel patch that contain changes specific to Zynq PCIe TRD version 2014.3 release.


Petalinux configuration help us to select custom kernel required for Zynq PCIe TRD project.


Select kernel menu in Linux configuration. It opens Kernel selection sub-menu.



Select linux-xlnx kernel , exit and select yes for "Do you wish to save your new configuration ?" .
It updates kernel selection for petalinux build process.

7 Build Petalinux images

Finally, it’s time to build your petalinux image. TRD installable BSP auto-configures required software settings.
Below steps are categorized into two sections:
a) Petalinux images for SD BOOT
b) Petalinux images for QSPI boot

Images to be build from sources.
a) image.ub
b) BOOT.bin
c) qspi_image.ub
d) zc706_pcie_trd.bin

For the purpose of keeping the images generated in step 8 and 9 , it is recommended to create $ZYNQ_TRD_HOME/build/ready_to_test/prog_qspi directory
and also copy the init.sh script to build directory.

Update the kernel configuration to the Zynq PCIe TRD. Also create folder to place the images.

7.1 Petalinux SD BOOT images

Run ’petalinux-build’ in the petalinux SDK project directory to build the Petalinux system image:
Note: For more verbose build message use petalinux-build -v.
The console shows the compilation progress. e.g.:
INFO: Checking component...
INFO: Generating make files and build Linux
INFO: Generating make files for the subcomponents of linux
INFO: Building Linux
<snip>

NOTE:
Compilation log are stored in build.log in the $(PETALINUX)/zynq_pcie_trd_2014_3/build directory.
Build image.ub is generated in $(PETALINUX)/zynq_pcie_trd_2014_3/images/linux directory.
Copy built image.ub and BOOT.BIN to $ZYNQ_TRD_HOME build directory

7.2 Petalinux QSPI BOOT images

Since petalinux BSP default configures to SD mode , we need to update petalinux configuration
for QSPI boot mode and TRD specific device tree.

a) Update default device tree source (system.dts) with TRD device tree.
b) Change boot mode to QSPI.
Run ’petalinux-config’ in the petalinux SDK project directory to configure the PetaLinux system for QSPI boot
In the linux configuration menu, select "Subsystem AUTO Hardware Settings --> Advanced Bootable Images Storage settings --> Kernel image settings --> Primary Flash"











In the linux configuration menu , select "Subsystem AUTO Hardware Settings ---> Flash Settings --->" menu.

Select "Advanced Flash Auto Configuration" to enable QSPI boot mode.








After configuring petalinux setting finally its time to build images.
Note: For more verbose build message use petalinux-build -v.
The console shows the compilation progress. e.g.:
INFO: Checking component...
INFO: Generating make files and build Linux
INFO: Generating make files for the subcomponents of linux
INFO: Building Linux
<snip>
NOTE:
Compilation log are stored in build.log in the $(PETALINUX)/zynq_pcie_trd_2014_3/build directory.
Build image.ub is generated in $PETALINUX/zynq_pcie_trd_2014_3/images/linux directory.
Rename image.ub to qspi_image.ub and copy it to $ZYNQ_TRD_HOME build directory

8 Generate Zynq BOOT image

Petalinux SDK provides petalinux-package utility to create Zynq boot images.
Creation of Zynq boot images is categorized into two sections.

NOTE: For using prebuilt Zynq FSBL ,bistream and u-boot images.
Pre Petalinux BSP installation : $ZYNQ_TRD_HOME/software/zynq_ps/petalinux/boot_image directory
Post Petalinux BSP installation : $PETALINUX/zynq_pcie_trd_14.7/pre-built/linux/images directory
For creating Xilinx BOOT.BIN image any of the mentioned directories can be looked in case prebuilt components are required

petalinux-package --boot --fsbl <Path to FSBL image> --fpga <Path to FPGA bitstream> --uboot=<Path to uboot image> -o <output file>
Required option for boot image package:
--fsbl <FSBL_ELF> Path to FSBL ELF image location
Options for boot image package:
--force Force overwrite the boot binary image
--fpga <BITSTREAM> Path to FPGA bitstream image location
--uboot[=<UBOOT_IMG>] Path to the u-boot elf image location
(default <PROJECT>/images/linux/u-boot.elf)

Prerequisite
petalinux-package command requires bootgen utility to be present in $PATH. Refer Xilinx ISE tools installation section for further information.

9 Building the Zynq SW application (XSDK)

This section demonstrates building of TRD applications using Xilinx software development kit (XSDK) .
NOTE: It is an optional step and recommended for advance users who are not using petalinux SDK and may wish to modify application/build and deploy it on target .

Launch Xilinx SDK:

• On Linux, enter xsdk at the command prompt.

In the Workspace Launcher window, click Browse and navigate to $ZYNQ_TRD_HOME/software/zynq_ps/workspace, then click OK.
Close the welcome screen.

9.1 Import project

To import the performance library (perfmon_lib), video library (video_lib) and QT application (zynq_pcie_qt) into the SDK workspace.

• Select File > Import.
• In the Import wizard, expand the General folder, select Existing Projects into Workspace, and click Next. All projects are located at the top-level inside your SDK workspace.
• Click Browse and navigate to $ZYNQ_TRD_HOME/software/zynq_ps/workspace. Press OK.
• Make sure the perfmon_lib, video_lib and zynq_pcie_qt are checked . Press Finish.

9.2 Setup build environment

Select Window->Preferences -> C/C++ -> Build-> Environment


Add these global environment variables.

9.3 Build project

• Right click on video_cmd project ,and select Build Project. Since video command-line project adds "video_lib" as a project references , it is automatically selected and build by XSDK.

INFO: Please ensure $ZYNQ_QT_INSTALL environment variable is set , before building QT application.

• Right click on zynq_pcie_qt project ,and select Build Project. Since video QT project adds "video_lib" and "perfmon_lib" as a project references , it is automatically selected and build by XSDK.

9.4 Deploy applications

Copy video_cmd and zynq_pcie_qt application on SD partition .
SD deployment binaries:

• BOOT.BIN
• image.ub
• autostart.sh
• zynq_pcie_qt
• video_cmd

Next we need to edit autostart.sh to copy video_cmd/qt application to petalinux rootfs before launching video QT application on start-up. Assume, the SD card mount point is "/media".
Edit autostart.sh
<Add these lines before - #Execute Qt application >
cp /media/video_cmd /usr/bin
cp /media/zynq_pcie_qt /usr/bin
Copy modified binaries/scripts on SD partition and follow Step 2 for running the video demo.

NOTE: In case user wish to compile prebuilt libraries that are used while building application/libraries.

Refer README/INSTALL instructions for building/installing above libraries.
One specific point:
While configuring library use host option as --host=arm-xilinx-linux-gnueabi .
For Kernel headers installation refer - Documentation/make/headers_install.txt

10 Building the PCIe host SW application

The software application compilation procedure is provided here.
IMPORTANT: The traffic generator needs the C++ compiler which is not shipped with the Fedora 16 live
CD. Likewise, Java compilation tools are not shipped as part of the Fedora 16 live CD. Hence, GUI compilation will need additional installations.
The source code is provided for an end user to build upon this design; for TRD testing recompilation of application or GUI is not recommended.

Steps for building the PCIe host software application

The source code (threads.cpp) for the Zynq-7000 PCIe TRD is available under the directory:
$ZYNQ_TRD_HOME/software/host/gui/jnilib/src

User can add debug messages or enable log verbose to aid in debug.
IMPORTANT: Changes in data structure will also lead to changes in the GUI compilation, which is not recommended.
To compile the traffic generator application:
1. Open a terminal window.
2. Navigate to the $ZYNQ_TRD_HOME/software/host/gui/jnilib/src directory and execute ./genlib.sh
Shared object (.so) files are generated and copied into 32 or 64 directories present in the jnilib directory, according to the type of Operating System being used.
TIP: Enable verbose messages by adding the -DDEBUG_VERBOSE flag to genlib.sh.

11 Running Demo Applications

This section explains through step by step instructions how to bring up the ZC706 board for video demonstration part of the TRD and running different video demonstrations out of the box.

The ZC706 Evaluation Kit comes with an SD-MMC card pre-loaded with binaries that enable the user to run the video demonstration and software applications. It also includes the binaries necessary to configure and boot the Zynq-7000 AP SoC based ZC706 board.

Note:
If the evaluation kit design files were downloaded online, copy the entire folder ZYNQ_TRD_HOME/ready_to_test/prog_qspi from the package onto the primary partition of the SD-MMC card (which is formatted as FAT32) using a SD-MMC card reader for loading the QSPI device with boot image and Linux kernel image. Once QSPI programming is over, load ZYNQ_TRD_HOME/ready_to_test/sd_image content onto the primary partition of the SD-MMC card (which is formatted as FAT32) using a SD-MMC card reader.
Petalinux console Login Details:-
User : root
Password : root

11.1 Hardware Setup Requirements

The ZC706 board setup to run & test the video demonstration applications require the following items:
Requirements for TRD Linux application demo setup

• The ZC706 evaluation board with the XC7Z045 FFG900-2 part
• Reference Design zip file containing BOOT.BIN and Petalinux FIT image. Host system driver and GUI files
• A control PC with ISE design Suite Logic Edition Tools v2014.3
• A control PC with teraTerm pro (or similar) terminal program
• A control PC with USB-UART driver from silicon labs installed
• Mini USB cable
• USB mouse (not included with the kit)
• Class 4 equivalent SD card
• Monitor supporting 1080p
• HDMI cable
• 4-pin to 6-pin PCIe adapter cable
• Fedora 16 LiveCD
• PC with PCIe v2.0 slot. Recommended PCI Express Gen2 PC system motherboards are ASUS P5E (Intel X38), ASUS Rampage II Gene (Intel X58) and Intel DX58SO (Intel X58). Note the Intel X58 chipsets tend to show higher performance. This PC could also have Fedora Core 16 Linux OS installed on it.

Please note that two PCs are required for running this TRD. A control PC which is connected to the zc706 board via mini-USB running teraTerm and a host PC running Fedora Core 16 Linux connected to zc706 via PCIe slot on the motherboard.
Note: The example mentioned in this package has been tested with Ivy Bridge and Sandy Bridge PCIe Host machine and a Dell model #P2412H display monitor. However, the example should work well with any HDMI-compatible output device.

11.2 Board Setup

For running the Host GUI and QT-based application on Zynq PS, please refer to Zynq-7000 SoC ZC706 Evaluation Kit UG961.

11.3 Running the Host GUI and Qt-based GUI Application on Zynq PS

For running the Host GUI and QT-based application on Zynq PS, please refer to Zynq-7000 SoC ZC706 Evaluation Kit UG961.

12 References

• Documentation for Zynq-7000 AP SoC
• Documentation for ZC706 Evaluation Kit
• Main Xilinx wiki