Functional Unit Simulation Environment Software
User Guide

About this Document

This document describes how to simulate a sample Accelerator Functional Unit (AFU) using the Intel
Accelerator Functional Unit (AFU) Simulation Environment (ASE) environment. Refer to the Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) User  Guide for comprehensive details on ASE capabilities and internal architecture.
The Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) is a hardware and software co-simulation environment for any Intel FPGA Programmable® Acceleration Card (Intel FPGA PAC). This software co-simulation environment currently supports the following Intel FPGA PACs: 10 GX FPGA

• Intel FPGA Programmable Acceleration Card D5005

• Intel Programmable Acceleration Card with Intel Arria®
  The ASE provides a transactional model for the Core Cache Interface (CCI-P) protocol and a memory model for the FPGA-attached local memory.
  The ASE also validates Accelerator Functional Unit (AFU) compliance to the following protocols and APIs:

• The CCI-P protocol specification

• The Avalon
  Memory Mapped (Avalon-MM) Interface Specification

• The Open Programmable Acceleration Engine (OPAE)®

Table 1. Acceleration Stack for Intel Xeon® CPU with FPGAs Glossary

collection of software, firmware and tools that provides performance-optimized connectivity between an Intel FPGA and an Intel Xeon processor.
Intel FPGA Programmable Acceleration Card (Intel FPGA PAC)| Intel FPGA PAC| PCIe* FPGA accelerator card.
Contains an FPGA Interface Manager (FIM) that pairs with an Intel Xeon processor over a PCIe bus.
Intel Xeon Scalable Platform with Integrated FPGA| Integrated FPGA Platform| Intel Xeon plus FPGA platform with the Intel Xeon and an FPGA in a single package and sharing a coherent cache of memory via Ultra Path Interconnect (UPI).

Related Information
Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) User Guide

Intel Corporation. All rights reserved. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Intel warrants the performance of its FPGA  and semiconductor products to current specifications in accordance with Intel’s standard warranty but reserves the right to make changes to any products and services at any time without notice. Intel assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Intel. Intel customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services.
*Other names and brands may be claimed as the property of others.
ISO 9001:2015 Registered

System Requirements

Here are the system requirements for Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE)::

• A 64-bit Linux operating system. This release validated the following operating systems:
  — For Intel FPGA PAC D5005:

• RHEL 7.6 with Kernel 3.10.0-957
  — For Intel PAC with Intel Arria 10 GX FPGA:

• RHEL 7.6 with Kernel 3.10.0-957

• Ubuntu 18.04 with Kernel 4.15

• One of the following simulators:
  — 64-bit Synopsys VCS-MX-2016.06-SP2-1 RTL Simulator
  — 64-bit Mentor Graphics Modelsim SE Simulator (Version 10.5c)
  — 64-bit Mentor Graphics QuestaSim Simulator (Version 10.5c)

• C compiler: GCC 4.7.0 or above

• CMake: version 2.8.12 or above

• GNU C Library: version 2.17 or above

• Python: version 2.7

• Intel Quartus® Prime Pro Edition software version 19.2 (1)

Setting Up the Environment

You must set up your simulation environment and install the OPAE software before running the ASE.

1. Set the following environment variables for your simulation software:
   • For VCS:
   $ export VCS_HOME=
   $ export PATH=$VCS_HOME/bin:$PATH
   The VCS installation directory structure is as follows:
   Make sure your system has a valid VCS license.
   • For Modelsim SE/QuestaSim:
   $ export MTI_HOME=
   $ export PATH=$MTI_HOME/linux_x86_64/:$MTI_HOME/bin/:$PATH
   The Modelsim/Questa installation directory structure is as follows:
   Make sure your system has a valid Modelsim SE/QuestaSim license.
   • For Intel Quartus Prime Pro Edition:
   $ export QUARTUS_HOME=<path to Intel Quartus Prime Pro Edition installation directory
   The Intel Quartus Prime installation directory structure is as follows:
   Add the environment variable to check the Modelsim license:
   $ export MGLS_LICENSE_FILE=

2. Export:
   $ export LM_LICENSE_FILE=

3.  Extract the runtime archive file, and install OPAE libraries, binaries, include files, and ASE libraries as described in the section: Installing the OPAE Software Package in the appropriate Intel Acceleration Stack Quick Start User Guide for your Intel FPGA PAC.

Your environment must be set up correctly to configure and build an AFU. In particular, you must install the OPAE Software Development Kit (SDK) properly. OPAE SDK  scripts must be on PATH and include files and libraries that must be available to the C compiler. In addition, you must ensure that the OPAE_PLATFORM_ROOT environment variable is set. Refer to Installing the OPAE Software Package for more information.
To ensure that the OPAE SDK and ASE are properly installed, in a shell, confirm that your PATH includes afu_sim_setup. The afu_sim_setup should be in the /usr/bin  directory or in if you built the OPAE from source files.

Related Information

• Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) User Guide

• Installing the OPAE Software Package
  For Intel PAC with Intel Arria 10 GX FPGA.

• Installing the OPAE Software Package For Intel FPGA PAC D5005.

Simulating hello_afu in Client-Server Mode

The hello_afu example is a simple AFU template that demonstrates the primary CCI-P interface. The RTL satisfies the minimum requirements of an AFU, responding to memory-mapped I/O reads to return the device feature header and the AFU’s UUID.
Figure 1. hello_afu Directory Tree

Note:
This document uses to refer to an example design directory, such as hello_afu in the figure above.
The software demonstrates the minimum requirements to attach to an FPGA using the OPAE. The RTL demonstrates the minimum requirements to satisfy the OPAE driver and the hello_afu example software.
filelist.txt specifies the files for RTL simulation and synthesis.
To successfully configure and build the AFU samples, your environment must be set up correctly, as described in Setting Up the Environment.

Related Information

• Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) User Guide
• Setting Up the Environment on page 5

Developing AFUs with the OPAE SDK
In the Accelerator Functional Unit (AFU) Developer’s Guide

4.1. Simulation in Client-Server Mode

The following example flow introduces the basic ASE scripts. You can simulate all examples with the ASE, except eth_e2e_e10 and eth_e2e_e40.
The simulation requires two software processes: one process for RTL simulation and a second process to run the connected software. To construct an RTL simulation environment,  run the following in $OPAE_PLATFORM_ROOT/hw/samples/hello_afu:
$ afu_sim_setup –source hw/rtl/filelist.txt build_sim
This command constructs an ASE environment in the build_sim subdirectory.
To build and run the simulator:
$ cd build_sim
$ make
$ make sim
The simulator prints a message that it is ready for simulation. It also prints a message prompting you to set the ASE_WORKDIR environment variable.
Open another shell for software simulation. You must ensure to set the OPAE_PLATFORM_ROOT environment variable.
To build and run the software in the new shell:
$ cd $OPAE_PLATFORM_ROOT
$ export ASE_WORKDIR=$OPAE_PLATFORM_ROOT/hw/samples/hello_afu/build_sim/work
$ cd $OPAE_PLATFORM_ROOT/hw/samples/hello_afu/sw
$ make clean
$ make USE_ASE=1
$ ./hello_afu

Note:
The specific pathname for ASE_WORKDIR may vary. Use the pathname provided by the simulator prompt.
The software and simulator run, log transactions, and exit.

4.1.1. Simulation Log Files
The simulation work directory stores the waveform, CCI-P transactions, and simulation log files.
Complete the following steps to view the waveform database:

1. Change to the directory in which you executed the make sim command.
2. Type:
   $ make wave
   The make wave command invokes the waveform viewer.

4.1.2. Design Declarations
The following file and directories define the AFU simulation:

• $OPAE_PLATFORM_ROOT/hw/samples//hw/rtl/filelist.txt specifies RTL sources.
• is the example directory as shown in the hello_afu Directory Tree figure.
• filelist.txt lists SystemVerilog, VHDL, and the AFU JavaScript Object Notation (.json) file.
• The AFU .json describes the interfaces the AFU requires. It also includes a UUID to identify the AFU once downloaded to an FPGA.
• hw/rtl/hello_afu.json defines ccip_std_afu as the top-level interface by setting afu-top-interface to ccip_std_afu. ccip_std_afu is the base CCI-P interface including clocks,  reset, and CCI-P TX and RX structures. More advanced examples define other interface options.
• The .json file declares the AFU UUID. An OPAE script generates the UUID. The RTL loads the UUID from afu_json_info.vh.
• sw/Makefile generates afu_json_info.h. The software loads the UUID from afu_json_info.h.

4.1.3. Troubleshooting Client-Server Simulation
If the afu_sim_setup command fails, confirm that:

• afu_sim_setup is on your PATH. afu_sim_setup should be in /usr/bin or in if you built OPAE from source files.
• You have Python version 2.7 or higher installed.

If you are unable to build and execute the simulator, it is likely that you did not install your RTL simulation tool properly.
When you try to build and run the software, if you see an “Error enumerating AFCs” message, you omitted setting USE_ASE=1 on the make command line. The software is  searching for a physical FPGA device. To recover, repeat the steps from the make clean command.

AFU Examples

Table 2.
AFU Examples
Each AFU example includes a detailed README file, providing an operational description and notes on how to simulate the design. For a full understanding of the simulation process, review the README file in each AFU example.

machine to access memory. The state machine is capable of several access patterns to local memory directly attached to FPGA pins, such as DDR4 DIMMs. This memory is distinct from the host memory accessed over CCI-P. The host manages the hello_mem_afu controller state machine using memory-mapped I/O (MMIO) requests to control and status registers (CSRs).
hello_intr_afu| hello_intrafu demonstrates the application interrupt feature in the ASE.
DMA an f1.1 (2) | dma_afu demonstrates a DMA Basic Building Block for host to FPGA, FPGA to host, and FPGA to FPGA memory transfers. When simulating this AFU, the buffer size used for DMA transfer is small to keep the simulation time reasonable. For more information, refer to the DMA Accelerator Functional Unit (AFU) User Guide.
nlb_mode_O| nlb_mode_O is a CCI-P system demonstrating the memory copy test. $0PAE_PLATFORM_ROOT/ sw/opae—cre/ease number>/sample/hello_fpga . c includes nlb_mode_0.|
$ sh regress.sh -a -r rtl_sim
-s < vcslmodelsimlquesta > [-i )
-b
streaming_dma| streaming_dma demonstrates how to transfer data between host memory and an FPGA streaming port. For more information, refer to the Streaming DMA Accelerator Functional Unit (AFU) User Guide.
hello_afu| hel lo_a fu is a simple AFU that demonstrates the primary CCI-P interface. The RTL satisfies the bare minimum requirements of an AFU, responding to MMIO reads to return the device feature header and the AFU’s UUID.

Related Information

• DMA Accelerator Functional Unit (AFU) User Guide
  For information on how to compile and execute the dma_afu on your Intel PAC with Intel Arria 10 GX FPGA.

• Streaming DMA Accelerator Functional Unit (AFU) User Guide
  For information on how to compile and execute the streaming_dma_afu on your Intel PAC with Intel Arria 10 GX FPGA.

• DMA Accelerator Functional Unit User Guide: Intel FPGA Programmable Acceleration Card D5005
  For information on how to compile and execute the dma_afu on your Intel FPGA PAC D5005.

• Streaming DMA Accelerator Functional Unit User Guide: Intel FPGA Programmable Acceleration Card D5005
  For information on how to compile and execute the dma_afu on your Intel FPGA PAC D5005.

Troubleshooting

If the following error appears during the simulation, correct it by following the steps below.
Error Message

[SIM] An ASE instance is probably still running in the current directory!

[SIM] Check for PID 28816

[SIM] Simulation will exit… you may use a SIGKILL to kill the simulation

process.

[SIM] Also check if .ase_ready.pid file is removed before proceeding.

Solution

1. Type kill ase_simv to kill zombie simulation processes and remove any temporary files left behind by failed simulation processes or lock ups.
2. Delete the .ase_ready.pid file, found in the $ASE_WORKDIR directory.

ASE Quick Start User Guide Archives

Quick Start User Guide
1.| Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) Quick Start User Guide
1.| Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) Quick Start User Guide
1.0| Intel Accelerator Functional Unit (AFU) Simulation Environment (ASE) Quick Start User Guide

Document Revision History for ASE Quick Start User Guide

• System Requirements
2019.08.05| 2.0| • Updated the Intel Quartus Prime Pro Edition version in System Requirements.
• Added the hello_afu in AFU Examples.
• Removed information about simulating in regression mode.
• Added a new section: ASE Quick Start User Guide Archives.
2018.12.04| 1.| Added Ubuntu support.
2018.08.06| 1.| Updated the system requirements, directory structure, and corresponding filenames.
2018.04.10| 1.0| Initial release.

683200 | 2020.03.06
Send Feedback

Documents / Resources

| intel Accelerator Functional Unit Simulation Environment Software [pdf] User Guide
Accelerator Functional Unit, Simulation Environment Software, Accelerator Functional Unit Simulation Environment, Software, Accelerator Functional Unit Simulation Environment Software
---|---

References

• Intel® Accelerator Functional Unit (AFU) Simulation Environment (ASE) User Guide — OPAE
• 1. About this Document
• 1. About this Document
• 1. About this Document
• 1. About this Document
• 1. About This Document
• 1. About this Document

Read User Manual Online (PDF format)

Read User Manual Online (PDF format)  >>

Download This Manual (PDF format)

Download this manual  >>