ZYNQ_PCIe_TRD_14.7

History
ISE DS 14.6 Zynq PCIe Targeted Reference Design
ISE DS 14.5 Zynq PCIe Targeted Reference Design
ISE DS 14.4 Zynq PCIe Targeted Reference Design
ISE DS 14.3 Zynq PCIe Targeted Reference Design

1 Introduction

This page provides instructions on how to build various components of the Zynq PCIe Targeted Reference Design (TRD) and how to setup the hardware platform and run the design on the ZC706 Evaluation Kit. The ZC706 Evaluation kit is based on a XC7Z045 FFG900-2 Zynq-7000 SoC device. For additional information, refer to UG961.

1.1 About the Zynq PCIe TRD

The Zynq PCIe Targeted reference design expands the Base Targeted Reference Design (UG925) by adding PCI Express communication with a host system at PCIe x4 GEN2 speed. In the Base Targeted Reference design, the input of the video processing pipeline is generated by a test pattern generator in the FPGA fabric. In this design, the input of the video processing pipeline is generated by an application on the host computer at 1080p60 resolution and transmitted to the ZC706 board via PCIe. The data is processed by video pipeline and passed back to the host system via PCIe. As full 1080p60 video stream only take up around 4Gbps, an additional data generator and a checker are implemented and connected to channel 1 of PCIe DMA showcasing the maximum PCIe x4 GEN2 bandwidth achieved by the hardware. For additional information, please refer to UG963

1.2 Zynq PCIe TRD Package Contents

The Zynq PCIe TRD package is released with the source code, Xilinx PlanAhead and SDK projects, and an SD card image that enables the user to run the video demonstration and software
application. It also includes the binaries necessary to configure and boot the Zynq-7000 AP SoC board. The package also contains the software driver source files required to run application
software in the PCIe host machine. This wiki page assumes the user has already downloaded the PCIe TRD package and extracted its contents to the PCIe TRD home directory referred to as
ZYNQ_TRD_HOME in this wiki and to the PCIe host machine in a folder of choice.

2 Prerequisites

• The ZC706 Evaluation Kit ships with the version 14.x Device-locked to the Zynq-7000 XC7Z045 FFG900-2 device and all required licenses to build the TRD. For additional information, refer to UG631 ISE Design Suite 14: Release Notes, Installation and Licensing Guide.
• PC with PCIe v2.0 slot. Recommended PCI Express Gen2 PC system motherboards are ASUS P5E (Intel X38), ASUS Rampage II Gene (Intel X58) and Intel DX58SO (Intel X58).
• Fedora 16 LiveCD for booting Linux on PCIe host machine.
• A Linux development PC with the ARM GNU tools installed. The ARM GNU tools are included with the Xilinx ISE Design Suite Embedded Edition or can be downloaded separately.
• A Linux development PC with the distributed version control system Git installed. For more information, refer to Using Git and to UG821: Xilinx Zynq-7000 EPP Software Developers Guide.
• Petalinux 2013.10 SDK
• Open JDK
• Other system utilities like make (3.82 or higher) and corkscrew if accessing git behind a firewall.
• A Linux development PC with QT and QWT libraries cross-compiled for Zynq platform. Set ZYNQ_QT_INSTALL environment variable by referring to Xilinx Zynq Qt/Qwt Libraries - Build Instructions

3 Building the FPGA Hardware Bitstream

This section explains how to generate the FPGA hardware bitsream using the Xilinx PlanAhead tool and how to export the hardware platform to Xilinx Software Development Kit (SDK) for software application development. Inside the PlanAhead project, a Xilinx Platform Studio (XPS) project is referenced that contains the embedded hardware design. The design top level file instantiates the embedded top level file along with the system with PCIe IP wrapper, PCIe DMA, PCIe performance monitor and hardware generator and checker blocks.

3.1 Building the Bitstream

Steps for building the FPGA hardware bitstream

Browse to $ZYNQ_TRD_HOME/hardware/plan_ahead/scripts directory:

• On Windows , open ISE Design Suite Command Prompt by navigating Start > All Programs > Xilinx Design Tools > ISE Design Suite 14.x > Accessories.
  • In the command prompt navigate to $ZYNQ_TRD_HOME/hardware/plan_ahead/scripts and run "launch_run.bat"
• On Linux, enter "./launch_run.sh" at the command prompt.

Plan Ahead GUI will be launched and Design Elaboration step will run. Wait till the Elaboration gets completed.

In the Flow Navigator pane on the left-hand side under Program and Debug, click Generate Bitstream.

• Note: Click on Save button when prompted for Save Project Before Generating Bit stream

• Note: A message window will pop up, saying there are critical warning messages. ignore these warnings and press OK to continue with the bit stream generation.

The bit stream will be generated at $ZYNQ_TRD_HOME/hardware/plan_ahead/runs/z7_pcie_trd_14.7.runs/impl_1/z7_pcie_trd.bit.

3.2 Exporting the Hardware Platform to SDK

A pre-generated hardware platform project can be found at $ZYNQ_TRD_HOME/software/zynq_ps/workspace/hw_platform.

Steps for exporting the hardware platform to SDK

From the PlanAhead project flow navigator click on Open Implemented Design. Click on "OK" when prompted for no trace timing results were found.. message
From the PlanAhead menu bar, select File > Export > Export Hardware.



In the Export Hardware window press OK. The SDK hardware platform will be exported to $ZYNQ_TRD_HOME/hardware/plan_ahead/runs/z7_pcie_trd_14.7.sdk/SDK/SDK_Export

Note: If the Launch SDK option is checked in the Export Hardware window, SDK will be launched immediately after SDK export has completed. This is not recommended at this point.

4 Building the First Stage Boot Loader (FSBL)

This section explains how to import and build the First Stage Boot Loader (FSBL) and the standalone OS based Board Support Package (BSP) from the provided SDK projects.

Launch Xilinx SDK:

• On Windows 7, select Start > All Programs > Xilinx Design Tools 14.x > ISE Design Suite 14.x > EDK > Xilinx Software Development Kit.
• On Linux, enter xsdk at the command prompt.In the Workspace Launcher window, click Browse and navigate to $ZYNQ_TRD_HOME/software/zynq_ps/workspace,then click OK.

To import the hardware platform (hw_platform) , FSBL (zynq_fsbl) ,FSBL BSP (zynq_fsbl_bsp) SDK projects into the SDK workspace, select File > Import.



Note: The zynq_fsbl project requires a hardware platform SDK project generated by SDK export. Instead of the provided hw_platform project, the one generated in Section 3.2 can be used. This requires the user to update the project reference of the zynq_fsbl project.

In the Import wizard, expand the General folder, select Existing Projects into workspace, and click Next.



All projects are located at the top-level inside your SDK workspace. Click Browse and navigate to $ZYNQ_TRD_HOME/software/zynq_ps/workspace. Press OK



Make sure the hw_platform, zynq_fsbl and zynq_fsbl_bsp projects are checked . Press Finish.


The build process will start automatically and builds the BSP first and then the FSBL. This option can be changed by unchecking Project > Build Automatically from the menu bar.
To manually build the project, right click zynq_fsbl in the Project Explorer and select Build Project. To clean the project, select Clean Project.


The generated Zynq FSBL executable can be found at $ZYNQ_TRD_HOME/software/zynq_ps/workspace/zynq_fsbl/Debug/zynq_fsbl.elf.

5 Installation of Petalinux SDK

5.1 Prerequisites

• 2GB RAM (recommended minimum for Xilinx tools)
• Pentium 4 2GHz CPU clock or equivalent.
• 5 GB free HDD space.
• Supported OS:
  • RHEL 5 (32-bit or 64-bit)
  • RHEL 6 (32-bit or 64-bit)
  • SUSE Enterprise 11 (32-bit or 64-bit)
• PetaLinux release package downloaded.
• Valid PetaLinux license.
• Common system packages and libraries are installed on your workstation. The installation process will check for these. See the section Required Tools and Libraries for more details. For detailed information refer petalinux installation guide UG976 .

Download Petalinux 2013.10 SDK software from Xilinx website download section.

5.2 Extract the PetaLinux Package

Assuming all the prerequisites described in the last subsection are satisfied, PetaLinux installation is very straight forward.Without any options, the installer will installl as a subdirectory of the current directory. Alternatively, an installation path may be specific . Run the downloaded petalinux installer.
PetaLinux will be installed in the petalinux-v2013.10-final directory, directly underneath the working directory of this command.
So, if you install the installer into your home directory /home/user, PetaLinux will be installed in /home/user/petalinux-v2013.10-final.
You may move the resulting petalinux-v2013.10-final directory to a preferred location before continuing.

5.3 Install License

PetaLinux licenses are managed using the same system as all other Xilinx Design Tools. For more details on licensing and setup of license please refer to the
"Xilinx Design Tools: Installation and Licensing Guide (UG798)" section "Obtaining and Managing a License".

5.4 Setup PetaLinux Working Environment

After extracting the package, the remainder of the setup is completed automatically.
1. Go to the PetaLinux root directory by running this command on the command console:
cd <path-to-installed-PetaLinux>
e.g.:
2. Source the appropriate PetaLinux setup script by running this command on the command console:
For Bash:
The first time the setup script is sourced, it will perform some post installation tasks to check system dependencies and initialise the Linux kernel source tree.
Below is an example of the output from sourcing the setup script for the first time:

PetaLinux environment set to ’/home/user/petalinux-v2013.10-final
INFO: Finalising PetaLinux installation
INFO: Checking free disk space
INFO: Checking installed tools
INFO: Checking installed development libraries
INFO: Checking network and other services
The post-install step only occurs once. Subsequent runs of the settings script should be much quicker, and simply output a confirmation message such as that shown below:
PetaLinux environment set to ’/home/user/petalinux-v2013.10-final'

5.4 Verify Petalinux Installation

Verify that the PetaLinux working environment has been set:
/home/user/petalinux-v2013.10-final
Environment variable "$PETALINUX" should point to the path to the installed PetaLinux. Your echo output may be different from this example, depending upon where you installed PetaLinux.

6 Zynq PCIe TRD BSP Installation

PetaLinux includes reference designs for you to start working with and customise for your own projects. These are provided in the form of installable BSP (Board Support Package) files, and include
all necessary design and configuration files, including pre-built and tested hardware and software images, ready for download to your board or for booting in the QEMU system simulation environment.
Run petalinux-create command on the command console:
petalinux-create -t project -s <path-to-bsp>

Example:
INFO: Create project:
INFO: Projects:
INFO: * zynq_pcie_trd_14.7
<snip>

7 Add Linux kernel 3.10 support

Zynq PCIe TRD uses Xilinx Linux 3.10 kernel version.Petalinux provides option to add individual project
specific kernel/u-boot version. Below steps demonstrates how to add/config/build the Linux kernel.
Create a new branch named zynq_pcie_trd_14.7 based on the xilinx-v14.7 tag.
Petalinux configuration help us to select custom kernel required for Zynq PCIe TRD project.


Select kernel menu in Linux configuration. It opens Kernel selection sub-menu.



Select linux-xlnx kernel , exit and select yes for "Do you wish to save your new configuration ?" .
It updates kernel selection for petalinux build process.

8 Build Petalinux images

Finally, it’s time to build your petalinux image. TRD installable BSP auto-configures required software settings.
Below steps are categorized into two sections:
a) Petalinux images for SD BOOT
b) Petalinux images for QSPI boot

Images to be build from sources.
a) image.ub
b) BOOT.bin
c) qspi_image.ub
d) zc706_pcie_trd.bin

For the purpose of keeping the images generated in step 8 and 9 , it is recommended to create $ZYNQ_TRD_HOME/build/ready_to_test/prog_qspi directory
and also copy the init.sh script to build directory.

8.1 Petalinux SD BOOT images

Run ’petalinux-build’ in the petalinux SDK project directory to build the PetaLinux system image:
Note: For more verbose build message use petalinux-build -v.
The console shows the compilation progress. e.g.:
INFO: Checking component...
INFO: Generating make files and build Linux
INFO: Generating make files for the subcomponents of linux
INFO: Building Linux
<snip>

NOTE:
Compilation log are stored in build.log in the $(PETALINUX)/build directory.
Build image.ub is generated in $(PETALINUX)/images/linux directory.
Copy built image.ub to $ZYNQ_TRD_HOME build directory

8.2 Petalinux QSPI BOOT images

Since petalinux BSP default configures to SD mode , we need to update petalinux configuration
for QSPI boot mode and TRD specific device tree.

a) Update default device tree source (system.dts) with TRD device tree.
b) Change boot mode to QSPI.
Run ’petalinux-config’ in the petalinux SDK project directory to configure the PetaLinux system for QSPI boot
In the linux configuration menu , select "System boot device --->" menu.


Press <Enter> for selecting System boot device and in the sub-menu of boot modes select <Enter> QSPI boot mode.


After configuring petalinux setting finally its time to build images.
Note: For more verbose build message use petalinux-build -v.
The console shows the compilation progress. e.g.:
INFO: Checking component...
INFO: Generating make files and build Linux
INFO: Generating make files for the subcomponents of linux
INFO: Building Linux
<snip>
NOTE:
Compilation log are stored in build.log in the $(PETALINUX)/build directory.
Build image.ub is generated in images/linux directory.
Rename image.ub to qspi_image.ub and copy it to $ZYNQ_TRD_HOME build directory

9 Generate Zynq BOOT image

Petalinux SDK provides petalinux-package utility to create Zynq boot images.
Creation of Zynq boot images is categorized into two sections.

NOTE: For using prebuilt Zynq FSBL ,bistream and u-boot images.
Pre Petalinux BSP installation : $ZYNQ_TRD_HOME/software/zynq_ps/petalinux/boot_image directory
Post Petalinux BSP installation : $PETALINUX/zynq_pcie_trd_14.7/pre-built/linux/images directory
For creating Xilinx BOOT.BIN image any of the mentioned directories can be looked in case prebuilt components are required

petalinux-package --boot --fsbl <Path to FSBL image> --fpga <Path to FPGA bitstream> --uboot=<Path to uboot image> -o <output file>
Required option for boot image package:
--fsbl <FSBL_ELF> Path to FSBL ELF image location
Options for boot image package:
--force Force overwrite the boot binary image
--fpga <BITSTREAM> Path to FPGA bitstream image location
--uboot[=<UBOOT_IMG>] Path to the u-boot elf image location
(default <PROJECT>/images/linux/u-boot.elf)

Prerequisite
petalinux-package command requires bootgen utility to be present in $PATH. Refer Xilinx ISE tools installation section for further information.

9.1 Generate Zynq SD BOOT image

Follow the steps below to generate the SD boot image (BOOT.BIN). PCIe TRD uses SD boot mode to flash QSPI partition with TRD images. So bistream is not needed in programming QSPI.

9.2 Generate Zynq QSPI BOOT image

Follow the steps below to generate the SD boot image (zc706_pcie_trd.bin).
In QSPI mode ,bitsream is required as a boot image partition.

10 Building the PCIe host SW application

The software application compilation procedure is provided here.
IMPORTANT: The traffic generator needs the C++ compiler which is not shipped with the Fedora 16 live
CD. Likewise, Java compilation tools are not shipped as part of the Fedora 16 live CD. Hence, GUI compilation will need additional installations.
The source code is provided for an end user to build upon this design; for TRD testing recompilation of application or GUI is not recommended.

Steps for building the PCIe host software application

The source code (threads.cpp) for the Zynq-7000 PCIe TRD is available under the directory:
$ZYNQ_TRD_HOME/software/host/gui/jnilib/src

User can add debug messages or enable log verbose to aid in debug.
IMPORTANT: Changes in data structure will also lead to changes in the GUI compilation, which is not recommended.
To compile the traffic generator application:
1. Open a terminal window.
2. Navigate to the $ZYNQ_TRD_HOME/software/host/gui/jnilib/src directory and execute ./genlib.sh
Shared object (.so) files are generated and copied into 32 or 64 directories present in the jnilib directory, according to the type of Operating System being used.
TIP: Enable verbose messages by adding the -DDEBUG_VERBOSE flag to genlib.sh.

11 Running Demo Applications

This section explains through step by step instructions how to bring up the ZC706 board for video demonstration part of the TRD and running different video demonstrations out of the box.

The ZC706 Evaluation Kit comes with an SD-MMC card pre-loaded with binaries that enable the user to run the video demonstration and software applications. It also includes the binaries necessary to configure and boot the Zynq-7000 AP SoC based ZC706 board.

Note:
If the evaluation kit design files were downloaded online, copy the entire folder ZYNQ_TRD_HOME/ready_to_test/prog_qspi from the package onto the primary partition of the SD-MMC card (which is formatted as FAT32) using a SD-MMC card reader for loading the QSPI device with boot image and Linux kernel image. Once QSPI programming is over, load ZYNQ_TRD_HOME/ready_to_test/sd_image content onto the primary partition of the SD-MMC card (which is formatted as FAT32) using a SD-MMC card reader.
Petalinux console Login Details:-
User : root
Password : root

11.1 Hardware Setup Requirements

The ZC706 board setup to run & test the video demonstration applications require the following items:
Requirements for TRD Linux application demo setup

• The ZC706 evaluation board with the XC7Z045 FFG900-2 part
• Reference Design zip file containing BOOT.BIN and Petalinux FIT image. Host system driver and GUI files
• A control PC with ISE design Suite Logic Edition Tools v14.7
• A control PC with teraTerm pro (or similar) terminal program
• A control PC with USB-UART driver from silicon labs installed
• Mini USB cable
• USB mouse (not included with the kit)
• Class 4 equivalent SD card
• Monitor supporting 1080p
• HDMI cable
• 4-pin to 6-pin PCIe adapter cable
• Fedora 16 LiveCD
• PC with PCIe v2.0 slot. Recommended PCI Express Gen2 PC system motherboards are ASUS P5E (Intel X38), ASUS Rampage II Gene (Intel X58) and Intel DX58SO (Intel X58). Note the Intel X58 chipsets tend to show higher performance. This PC could also have Fedora Core 16 Linux OS installed on it.

Please note that two PCs are required for running this TRD. A control PC which is connected to the zc706 board via mini-USB running teraTerm and a host PC running Fedora Core 16 Linux connected to zc706 via PCIe slot on the motherboard.
Note: The example mentioned in this package has been tested with Ivy Bridge and Sandy Bridge PCIe Host machine and a Dell model #P2412H display monitor. However, the example should work well with any HDMI-compatible output device.

11.2 Board Setup

For running the Host GUI and QT-based application on Zynq PS, please refer to Zynq-7000 SoC ZC706 Evaluation Kit UG961.

11.3 Running the Host GUI and Qt-based GUI Application on Zynq PS

For running the Host GUI and QT-based application on Zynq PS, please refer to Zynq-7000 SoC ZC706 Evaluation Kit UG961.

12 Know Issues

a) Disconnecting/reconnecting monitor hdmi cable while QT GUI is running, causes GUI Layer to appear on the top.
Workaround is to click on min followed by max button to resume normal operation.
Explanation- On reconnecting the monitor ,parameters like fb_var_screeninfo are set to default values as queried during EDID exchange.
In current design QT application is not querying monitor disconnect status , which will be added in coming release.

b) Petalinux binary built on certain development environment <=2.6.18-238.el5 ,throws below error during u-boot boot process.
‘Can't get hash value property for 'hash@1' hash node in 'kernel@1' image node
Bad hash in FIT image!’
So current workaround is to run dtc conversion from source to binary and vice-versa.

13 References

• Documentation for Zynq-7000 AP SoC
• Documentation for ZC706 Evaluation Kit
• Main Xilinx wiki