ALPHA DATA ADM-PCIE-9H3 High Performance FPGA Processing Card User Manual

June 10, 2024
ALPHA DATA

ALPHA DATA ADM-PCIE-9H3 High Performance FPGA Processing Card
ALPHA DATA ADM-PCIE-9H3 High Performance FPGA Processing
Card

Introduction

The ADM-PCIE-9H3 is a high-performance reconfigurable computing card intended for Data Center applications, featuring a Xilinx Virtex UltraScale+ Plus FPGA with High Bandwidth Memory (HBM).

Key Features

  • PCIe Gen1/2/3 x1/2/4/8/16 capable
  • Passive and active thermal management configuration
  • 1/2 length, low profile, x16 edge PCIe form factor
  • 8GB HBM on-die memory capable of 460GB/s
  • One QSFP-DD cage capable of data rates up to 28 Gbps per 8 channels (224 Gbps)
  • One 8 lane Ultraport SlimSAS connectors compliant with OpenCAPI and suitable for IO expansion
  • Supports either VU33P or VU35P Virtex UltraScale+ FPGAs
  • Front panel and rear edge JTAG access via USB port
  • FPGA configurable over USB/JTAG and SPI configuration flash
  • Voltage, current, and temperature monitoring
  • 8 GPIO signals and 1 isolated timing input

Order Code
ADM-PCIE-9H3
ADM-PCIE-9H3/NF (without optional fan)
See http://www.alpha-data.com/pdfs/adm-pcie-9h3.pdf for complete ordering options.

Board Information

Physical Specifications
The ADM-PCIE-9H3 complies with PCI Express CEM revision 3.0.
Table 1 : Mechanical Dimensions (Inc. Front Panel)

Description Measure
Total Dy 80.1 mm
Total Dx 181.5 mm
Total Dz 19.7 mm
Weight 350 grams

Physical Specifications

Chassis Requirements

PCI Express
The ADM-PCIE-9H3 is capable of PCIe Gen 1/2/3 with 1/2/4/8/16 lanes, using the Xilinx Integrated Block for PCI Express.

Mechanical Requirements
A 16-lane physical PCIe slot is required for mechanical compatibility.

Power Requirements
The ADM-PCIE-9H3 draws all power from the PCIe Edge. As per PCIe specification, this limits the power consumption of the card to a maximum 75W.
Power consumption estimation requires the use of the Xilinx XPE spreadsheet and a power estimator tool available from Alpha Data. Please contact support @alpha-data.com to obtain this tool.
The power available to the rails calculated using XPE are as follows:

Table 2 : Available Power By Rail

Voltage Source Name Current Capability
0.72-0.90 VCC_INT + VCCINT_IO + VCC_BRAM 42A
0.9 MGTAVCC 5A
1.2 MGTAVTT 9A
1.2 VCC_HBM * VCC_IO_HBM 14A
1.8 VCCAUX + VCCAUX_IO + VCCO_1.8V 1.5A
1.8 MGTVCCAUX 0.5A
2.5 VCCAUX_HBM 2.2A
3.3 3.3V for Optics 3.6A

Thermal Performance
If the FPGA core temperature exceeds 105 degrees Celsius, the FPGA design will be cleared to prevent the card from over-heating.
The ADM-PCIE-9H3 comes with a heat sink to reduce the temperature of the FPGA, which is typically the hottest point on the card. The FPGA die temperature must remain under 100 degrees Celsius. To calculate the FPGA die temperature, take your application power, multiply by Theta JA from the table below, and add to your system internal ambient temperature. The graph below shows two lines, one was tested in a duct with the shrouds installed, and the other was tested without the shrouds. The performance is generally better without the shrouds, but they do provide improved handling and reduce air re-circulation in compact servers. The shroud can be removed using a 1/16″ hex driver. If you are using the fan provided with the board, you will find theta JA is approximately 1.43 degC/W for the board in still air with or without the shroud installed.
The power dissipation can be estimated by using the Alpha Data power estimator in conjunction with the Xilinx Power Estimator (XPE) downloadable at http://www.xilinx.com/products/technology/power/xpe.html. Download
the UltraScale tool and set the device to Virtex UltraScale+, VU33P, FSVH2104, -2, -2L, or -3, extended. Set the ambient temperature to your system ambient and select ‘user override’ for the effective theta JA and enter the figure associated with your system LFM in the blank field. Proceed to enter all applicable design elements and utilization in the following spreadsheet tabs. Next aquire the 9H3 power estimator from Alpha Data by contacting
support@alpha-data.com. You will then plug in the FPGA power figures along with Optical module figures to get a board level estimate.
Thermal Performance

Active VS Passive Thermal Management
The ADM-PCIE-9H3 ships with a small optional blower for active cooling in systems with poor airflow. If the ADM-PCIE-9H3 will be installed in a server with controlled airflow, the order option /NF can be used to receive cards without this extra piece. The fans have a much shorter mean time between failure (MTBF) than the rest of the assembly, so passive cards have much longer life expectance before requiring maintenance. The ADM-PCIE-9H3 also includes a fan speed controller, allowing variable fan speed based on die temperature, and
detection of a failed fan (see section Fan Controllers).
Active VS Passive Thermal Management

Customizations
Alpha Data provides extensive customization options to existing commercial off-the-shelf (COTS) products.
Some options include, but are not limited to: additional networking cages in adjacent slots or full profile, enhanced heat sinks, baffles, and circuit additions.
Please contact sales@alpha-data.com to get a quote and start your project today.
Customizations

Functional Description

Overview
The ADM-PCIE-9H3 is a versatile reconfigurable computing platform with a Virtex UltraScale+ VU33P/VU35P FPGA, a Gen3x16 PCIe interface, 8GB of HBM memory, one QSFP-DD cage, an OpenCAPI compatible Ultraport SlimSAS connector also capable of 28G/channel, an isolated input for a timing synchronization pulse, a 12 pin header for general purpose use (clocking, control pins, debug, etc.), front panel LEDs, and a robust system monitor.
Overview

Switches
The ADM-PCIE-9H3 has an octal DIP switch SW1, located on the rear side of the board. The function of each switch in SW1 is detailed below:
Switches
Table 3 : Switch Functions

Switch Factory Default Function OFF State ON State
SW1-1 OFF User Switch 0 Pin AW33 = ‘1’ Pin BF52 = ‘0’
SW1-2 OFF User Switch 1 Pin AY36 = ‘1’ Pin BF47 = ‘0’
SW1-3 OFF Reserved Reserved Reserved
SW1-4 OFF Power Off Board will power up Immediately power down
SW1-5 OFF Service Mode Regular Operation Firmware update service mode
SW1-6 ON HOST_I2­ C_EN Sysmon over PCIe I2C Sysmon isolated
SW1-7 ON CAPI_VP­ D_EN OpenCAPI VPD available OpenCAPI VPD isolated
SW1-8 ON CAPI_VP­ D_WP CAPI VPD is write protected CAPI VPD is writable

Use IO Standard “LVCMOS18” when constraining the user switch pins.

LEDs
There are 7 LEDs on the ADM-PCIE-9H3, 4 of which are general purpose and whose meaning can be defined by the user. The other 3 have fixed functions described below:
LEDs

Table 4 : LED Details

Comp. Ref. Function ON State OFF State
D1 LED_G1 User defined ‘0’ User defined ‘1’
D3 LED_A1 User defined ‘0’ User defined ‘1’
D4 DONE FPGA is configured FPGA is not configured
D5 Status 1 See Status LED Definitions
D6 Status 0 See Status LED Definitions
D7 LED_A0 User defined ‘0’ User defined ‘1’
D9 LED_G0 User defined ‘0’ User defined ‘1’

See Section Complete Pinout Table for full list of user controlled LED nets and pins

Clocking
The ADM-PCIE-9H3 provides flexible reference clock solutions for the many multi-gigabit transceiver quads and FPGA fabric. Any clock out of the Si5338 Clock Synthesizer is re-configurable from either the front panel USB USB Interface or the Alpha Data sysmon FPGA serial port. This allows the user to configure almost any arbitrary clock frequency during application run time. Maximum clock frequency is 312.5MHz.
There is also an available Si5328 jitter attenuator. This can provide clean and synchronous clocks to the QSFP-DD and OpenCAPI (SlimSAS) quad locations at many clock frequencies. These devices only use volatile memory, so the FPGA design will need to re-configure the register map after any power cycle event.
All clock names in the section below can be found in Complete Pinout Table.
Clocking

Si5328
If jitter attenuation is required please see the reference documentation for the Si5328.
https://www.silabs.com/Support%20Documents/TechnicalDocs/Si5328.pdf
The circuit connections mirror Xilinx VCU110 and VCU108, please see Xilinx Dev Boards for references
Si5328

PCIe Reference Clocks
The 16 MGT lanes connected to the PCIe card edge use MGT tiles 224 through 227 and use the system 100 MHz clock (net name PCIE_REFCLK).
Alternatively, a clean, onboard 100MHz clock is available as well (net name PCIE_LCL_REFCLK).

Fabric Clock
The design offers a fabric clock (net name FABRIC_SRC_CLK) which defaults to 300 MHz. This clock is intended to be used for IDELAY elements in FPGA designs. The fabric clock is connected to a Global Clock (GC) pin.
DIFF_TERM_ADV = TERM_100 is required for LVDS termination

Auxiliary Clock
The design offers a auxiliary clock (net name AUX_CLK) which defaults to 300 MHz. This clock can be used for any purpose and is connected to a Global Clock (GC) pin.
DIFF_TERM_ADV = TERM_100 is required for LVDS termination

Programming Clock (EMCCLK)
A 100MHz clock (net name EMCCLK_B) is fed into the EMCCLK pin to drive the SPI flash device during configuration of the FPGA. Note that this is not a global clock capable IO pin.

QSFP-DD
The QSFP-DD cage is located in MGT tiles 126 and 127 and use a 161.1328125MHz default reference clock.
Note that this clock frequency can be changed to any arbitrary clock frequency up to 312MHz by re-programing the Si5338 reprogrammable clock oscillator via system monitor. This can be done using the Alpha Data API or over USB with the appropriate Alpha Data Software tools.
See net names QSFP_CLK for pin locations.
The QSFP-DD cage is also located such that it can be clocked from the Si5328 jitter attenuator clock multiplier.
See net names SI5328_OUT_1
for pin locations.

Ultraport SlimSAS (OpenCAPI)
The Ultraport SlimSAS connector is located in MGT tile 124 and 125.
For OpenCAPI an external 156.25MHz clock is provided over the cable. See net names CAPI_CLK_0 for cable clock pin locations.
Another alternative clock source for this interface is the Si5338 clock synthesizer which is defaulted to 161.1328125MHz. See net names CAPI_CLK_1
for pin locations. Note that this clock frequency can be changed to any arbitrary clock frequency up to 312MHz by re-programing the Si5338 reprogrammable clock oscillator via system monitor. This can be done using the Alpha Data API or over USB with the appropriate Alpha Data Software tools.
For jitter sensitive applications, this interface can be clocked from the Si5328 jitter attenuator. See net names SI5328_OUT_0* for pin locations.

PCI Express

The ADM-PCIE-9H3 is capable of PCIe Gen 1/2/3 with 1/2/4/8/16 lanes. The FPGA drives these lanes directly using the Integrated PCI Express block from Xilinx. Negotiation of PCIe link speed and number of lanes used is generally automatic and does not require user intervention.
PCI Express reset (PERST#) connected to the FPGA at two location. See Complete Pinout Table signals PERST0_1V8_L and PERST1_1V8_L.
The other pin assignments for the high speed lanes are provided in the pinout attached to the Complete Pinout Table
The PCI Express specification requires that all add-in cards be ready for enumeration within 120ms after power is valid (100ms after power is valid + 20ms after PERST is released). The ADM-PCIE-9H3 does meet this requirement when configured from a tandem bitstream with the proper SPI constraints detailed in the section:
Configuration From Flash Memory. For more details on tandem configuration, see Xilinx xapp 1179.

Note:
Different motherboards/backplanes will benefit from different RX equalization schemes within the PCIe IP core provided by Xilinx. Alpha Data recommends using the following setting if a user experiences link errors or training issues with their system: within the IP core generator, change the mode to “Advanced” and open the “GT Settings” tab, change the “form factor driven insertion loss adjustment” from “Add-in Card” to “Chip-to-Chip” (See Xilinx PG239 for more details).

QSFP-DD
One QSFP-DD cage is available at the front panel. This cage is capable of housing either QSFP28 or QSFP-DD cables (backwards compatible). Both active optical and passive copper QSFP-DD/QSFP28 compatible models are fully compliant. The communication interface can run at up to 28Gbps per channel. There are 8 channels across the QSFP-DD cage (total maximum bandwidth of 224Gbps). This cage is ideally suited for 8x 10G/25G, 2x 100G Ethernet, or any other protocol supported by the Xilinx GTY Transceivers. Please see Xilinx User Guide UG578 for more details on the capabilities of the transceivers.
The QSFP-DD cage has control signals connected to the FPGA. The connectivity is detailed in the Complete Pinout Table at the end of this document. The notation used in the pin assignments is QSFP* with locations clarified in the diagram below.
Use the QSFP_SCL_1V8 and QSFP_SDA_1V8 pins as detailed in Complete Pinout Table to communicate with QSFP28 register space.

Note:
The LP_MODE (Low Power Mode) to the cage is tied to ground, use the management interface to set power rules.
QSFP-DD

It is possible for Alpha Data to pre-fit the ADM-PCIE-9H3 with QSFP-DD and QSFP28 components. The table below shows the part number for the transceivers fitted when ordered with this board.
Table 5 : QSFP28 Part Numbers

Order Code Description Part Number Manufacturer
Q10 40G (4×10) QSFP Optical Transceiver FTL410QE2C Finisar
Q14 56G (4×14) QSFP Optical Transceiver FTL414QB2C Finisar
Q25 100G (4×25) QSFP28 Optical Transceiver FTLC9558REPM Finisar

OpenCAPI Ultraport SlimSAS

An Ultraport SlimSAS receptacles along the back of the board allow for OpenCAPI compliant interfaces running at 200G (8 channels at 25G). Please contact support@alpha-data.com or your IBM representative for more details on OpenCAPI and its benefits.
The SlimSAS connector can also be used to connect an additional 2x QSFP28 breakout board, contact sales@alpha-data.com for more details. Alternatively, cabling cab be used to connect multiple ADM- PCIE-9H3 cards within a chassis.
OpenCAPI Ultraport SlimSAS

System Monitor
The ADM-PCIE-9H3 has the ability to monitor temperature, voltage, and current of the system to check on the operation of the board. The monitoring is implemented using an Atmel AVR microcontroller.
If the core FPGA temperature exceeds 105 degrees Celsius, the FPGA will be cleared to prevent damage to the card.
Control algorithms within the microcontroller automatically check line voltages and on board temperatures and shares makes the information available to the FPGA over a dedicated serial interface built into the Alpha Data reference design package (sold separately). The information can also be accessed directly from the microcontroller over the USB interface on the front panel or via the IPMI interface available at the PCIe card edge.

Table 6 : Voltage, Current, and Temperature Monitors

Monitors Index Purpose/Description
ETC ETC Elapsed time counter (seconds)
EC EC Event counter (power cycles)
12V ADC00 Board input supply
12V_I ADC01 12V input current in amps
3.3V ADC02 Board input supply
3.3V_I ADC03 3.3V input current in amps
3.3V ADC05 Board input auxiliary power
3.3V ADC05 3.3V for QSFP optics
2.5V ADC06 Clock and DRAM voltage supply
1.8V ADC07 FPGA IO voltage (VCCO)
1.8V ADC08 Transceiver power (AVCC_AUX)
1.2V ADC09 HBM Power
1.2V ADC10 Transceiver Power (AVTT)
0.9V ADC11 Transceiver Power (AVCC)
0.85-0.90V ADC12 BRAM + INT_IO (VccINT_IO)
0.72-0.90V ADC13 FPGA Core Supply (VccINT)
uC_Temp TMP00 FPGA on-die temperature
Board0_Temp TMP01 Board temperature near front panel
Board1_Temp TMP02 Board temperature near back top corner
FPGA_Temp TMP03 FPGA on-die temperature

System Monitor Status LEDs
LEDs D5 (Red) and D6 (Green) indicate the card health status.

Table 7 : Status LED Definitions

LEDs Status
Green Running and no alarms
Green + Red Standby (Powered off)
Flashing Green + Flashing Red (together) Attention – critical alarm active
Flashing Green + Flashing Red (alternating) Service Mode
Flashing Green + Red Attention – alarm active
Red Missing application firmware or invalid firmware
Flashing Red FPGA configuration cleared to protect board

Fan Controllers
The onboard USB bus controlled by the system monitor has access to a MAX6620 fan controller. This device can be controlled through the multiple onboard system monitor communication interfaces, including USB, PCIe Edge SMBUS, and FPGA sysmon seral communication port. The fan controller is on I2C bus 1 at address 0x2a. For additional questions. Contact support@alpha- data.com with additional questions on utilizing these controllers.

USB Interface
The FPGA can be configured directly from the USB connection on either the front panel or the rear card edge.
The ADM-PCIE-9H3 utilizes the Digilent USB-JTAG converter box which is supported by the Xilinx software tool suite. Simply connect a micro-USB AB type cable between the ADM-PCIE-9H3 USB port and a host computer with Vivado installed. Vivado Hardware Manager will automatically recognize the FPGA and allow you to configure the FPGA and the SBPI configuration PROM.
The same USB connector is used to directly access the system monitor system. All voltages, currents, temperatures, and non-volatile clock configuration settings can be accessed using Alpha Data’s avr2util software at this interface.
Avr2util for Windows and the associated USB driver is downloadable here:
https://support.alpha-data.com/pub/firmware/utilities/windows/
Avr2util for Linux is downloadable here:
https://support.alpha-data.com/pub/firmware/utilities/linux/
Use “avr2util.exe /?” to see all options.
For example “avr2util.exe /usbcom com4 display-sensors” will display all sensor values.
For example “avr2util.exe /usbcom com4 setclknv 1 156250000” will set the QSFP clock to 156.25MHz. setclk index 0 = CAPI_CLK_1, index 1 = QSFP_CLK, index 2 = AUX_CLK, index 3 = FABRIC_CLK.
Change ‘com4’ to match the com port number assigned under windows device manager

Configuration
There are two main ways of configuring the FPGA on the ADM-PCIE-9H3:

  • From Flash memory, at power-on, as described in Section 3.8.1
  • Using USB cable connected at either USB port Section 3.8.2

Configuration From Flash Memory
The FPGA can be automatically configured at power-on from two 256 Mbit QSPI flash memory device configured as an x8 SPI device (Micron part numbers MT25QU256ABA8E12-0). These flash devices are typically divided into two regions of 32 MiByte each, where each region is sufficiently large to hold an uncompressed bitstream for a VU33P FPGA.
The ADM-PCIE-9H3 is shipped with a simple PCIe endpoint bitstream containing a basic Alpha Data ADXDMA bitstream. Alpha Data can load in other custom bitstreams during production test, please contact sales@alpha- data.com for more details.
It is possible to use Multiboot with a fallback image on this hardware. The master SPI configuration interface and the Fallback MultiBoot are discussed in detail in Xilinx UG570. At power-on, the FPGA attempts to configure itself automatically in serial master mode based on the contents of the header in the programing file. Multibook and ICAP can be used to selected between the two configuration regions to be loaded into the FPGA. See Xilinx UG570 MultiBoot for details.
The image loaded can also support tandem PROM or tandem PCIE with field update configuration methods.
These options reduce power-on load times to help meet the PCIe reset timing requirements. Tandem with field also enables a host system to reconfigure the user FPGA logic without losing the PCIe link, a useful feature when system resets and power cycles are not an option.
The Alpha Data System Monitor is also capable of reconfiguring the flash memory and reprograming the FPGA.
This provides a useful failsafe mechanism to re-program the FPGA even if it drops off the PCIe bus. The system monitor can be accessed over USB at the front panel and rear edge, or over the SMBUS connections on the PCIe edge.

Building and Programming Configuration Images

Generate a bitfile with these constraints (see xapp1233):

  • set_property BITSTREAM.GENERAL.COMPRESS TRUE [ current_design ]
  • set_property BITSTREAM.CONFIG.EXTMASTERCCLK_EN {DIV-1} [current_design]
  • set_property BITSTREAM.CONFIG.SPI_32BIT_ADDR YES [current_design]
  • set_property BITSTREAM.CONFIG.SPI_BUSWIDTH 8 [current_design]
  • set_property BITSTREAM.CONFIG.SPI_FALL_EDGE YES [current_design]
  • set_property BITSTREAM.CONFIG.UNUSEDPIN {Pullnone} [current_design]
  • set_property CFGBVS GND [ current_design ]
  • set_property CONFIG_VOLTAGE 1.8 [ current_design ]
  • set_property BITSTREAM.CONFIG.OVERTEMPSHUTDOWN Enable [current_design]

Generate an MCS file with these properties (write_cfgmem):

  • -format MCS
  • -size 64
  • -interface SPIx8
  • -loadbit “up 0x0000000 <directory/to/file/filename.bit>” (0th location)
  • -loadbit “up 0x2000000 <directory/to/file/filename.bit>” (1st location, optional)

Program with vivado hardware manager with these settings (see xapp1233):

  • SPI part: mt25qu256-spi-x1_x2_x4_x8
  • State of non-config mem I/O pins: Pull-none
  • Target the four files generated from the write_cfgmem tcl command.

Configuration via JTAG
A micro-USB AB Cable may be attached to the front panel or rear edge USB port. This permits the FPGA to be reconfigured using the Xilinx Vivado Hardware Manager via the integrated Digilent JTAG converter box. The device will be automatically recognized in Vivado Hardware Manager.
For more detailed instructions, please see “Using a Vivado Hardware Manager to Program an FPGA Device” section of Xilinx UG908: https://www.xilinx.com/support/documentation/sw_manuals/xilinx2014_1/ug908 -vivado-programming-debugging.pdf

GPIO Connector
The GPIO option consists of a versatile shrouded connector from Molex with part number 87832-1222 that give users with custom IO requirements four direct connect to FPGA signals.
Recommended mating plug: Molex 0875681273 or 0511101260

Direct Connect FPGA Signals
8 nets are broken out to the GPIO header, as four sets of differential pairs. These signal are suitable for any 1.8V supported signaling standards supported by the Xilinx UltraScale architecture. See Xilinx UG571 for IO options.
LVDS and 1.8 CMOS are popular options. The 0th GPIO signal index is suitable for a global clock connection.
The direct connect GPIO signals are limited to 1.8V by a quickswitch (74CBTLVD3245PW) in order to protect the FPGA from overvoltage on IO pins. This quickswitch allows the signals to travel in either direction with only 4 ohms of series impedance and less than 1ns of propagation delay. The nets are directly connected to the FPGA after the quickswitch.
Direct connect signal names are labeled GPIO_0_1V8_P/N and GPIO_1_1V8_P/N, etc. to show polarity and grouping. The signal pin allocations can be found in Complete Pinout Table

Timing Input
J1.1 and J1.2 can be used as an isolated timing input signal (up to 25MHz). Applications can either directly connect to the GPIO connector, or Alpha Data can provide a cabled solution with an SMA or similar connector on the front panel. Contact sales@alpha-data.com for front panel connector options.
For pin locations, see signal name ISO_CLK in Complete Pinout Table.
The signal is isolated through a optical isolator part number TLP2367 with a 220 ohm of series resistance.

User EEPROM
A 2Kb I2C user EEPROM is provided for storing MAC addresses or other user information. The EEPROM is part number CAT34C02HU4IGT4A
The address pins A2, A1, and A0 are all strapped to a logical ‘0’.
Write protect (WP), Serial Clock (SCL), and Serial Data (SDA) pin assignments can be found in Complete Pinout Table with the names SPARE_WP, SPARE_SCL, and SPARE_SDA respectively.
WP, SDA, and SCL signals all have external pull-up resistors on the card.

Appendix A: Complete Pinout Table

Table 8 : Complete Pinout Table (continued on next page)

Pin Number Signal Name Pin Name Bank Voltage
BC18 AUX_CLK_PIN_N IO_L11N_T1U_N9_GC_64 1.8 (LVCMOS18)
BB18 AUX_CLK_PIN_P IO_L11P_T1U_N8_GC_64 1.8 (LVCMOS18)
BF33 AVR_B2U_1V8 IO_L2P_T0L_N2_66 1.8 (LVCMOS18)
BF31 AVR_HS_B2U_1V8 IO_L1P_T0L_N0_DBC_66 1.8 (LVCMOS18)
BB33 AVR_HS_CLK_1V8 IO_L12N_T1U_N11_GC_66 1.8 (LVCMOS18)
BF32 AVR_HS_U2B_1V8 IO_L1N_T0L_N1_DBC_66 1.8 (LVCMOS18)
BA33 AVR_MON_CLK_1V8 IO_L12P_T1U_N10_GC_66 1.8 (LVCMOS18)
BF34 AVR_U2B_1V8 IO_L2N_T0L_N3_66 1.8 (LVCMOS18)
AK39 CAPI_CLK_0_PIN_N MGTREFCLK0N_124 MGT REFCLK
AK38 CAPI_CLK_0_PIN_P MGTREFCLK0P_124 MGT REFCLK
AF39 CAPI_CLK_1_PIN_N MGTREFCLK0N_125 MGT REFCLK
AF38 CAPI_CLK_1_PIN_P MGTREFCLK0P_125 MGT REFCLK
BF17 CAPI_I2C_SCL_1V8 IO_L1P_T0L_N0_DBC_64 1.8 (LVCMOS18)
BF16 CAPI_I2C_SDA_1V8 IO_L1N_T0L_N1_DBC_64 1.8 (LVCMOS18)
BF19 CAPI_INT/RESET_1V8 IO_L2P_T0L_N2_64 1.8 (LVCMOS18)
BF43 CAPI_RX0_N MGTYRXN0_124 MGT
BF42 CAPI_RX0_P MGTYRXP0_124 MGT
BD44 CAPI_RX1_N MGTYRXN1_124 MGT
BD43 CAPI_RX1_P MGTYRXP1_124 MGT
BB44 CAPI_RX2_N MGTYRXN2_124 MGT
BB43 CAPI_RX2_P MGTYRXP2_124 MGT
AY44 CAPI_RX3_N MGTYRXN3_124 MGT
AY43 CAPI_RX3_P MGTYRXP3_124 MGT
BC46 CAPI_RX4_N MGTYRXN0_125 MGT
BC45 CAPI_RX4_P MGTYRXP0_125 MGT
BA46 CAPI_RX5_N MGTYRXN1_125 MGT
BA45 CAPI_RX5_P MGTYRXP1_125 MGT
AW46 CAPI_RX6_N MGTYRXN2_125 MGT
AW45 CAPI_RX6_P MGTYRXP2_125 MGT
AV44 CAPI_RX7_N MGTYRXN3_125 MGT
AV43 CAPI_RX7_P MGTYRXP3_125 MGT
AT39 CAPI_TX0_N MGTYTXN0_124 MGT
AT38 CAPI_TX0_P MGTYTXP0_124 MGT
Pin Number Signal Name Pin Name Bank Voltage
--- --- --- ---
AR41 CAPI_TX1_N MGTYTXN1_124 MGT
AR40 CAPI_TX1_P MGTYTXP1_124 MGT
AP39 CAPI_TX2_N MGTYTXN2_124 MGT
AP38 CAPI_TX2_P MGTYTXP2_124 MGT
AN41 CAPI_TX3_N MGTYTXN3_124 MGT
AN40 CAPI_TX3_P MGTYTXP3_124 MGT
AM39 CAPI_TX4_N MGTYTXN0_125 MGT
AM38 CAPI_TX4_P MGTYTXP0_125 MGT
AL41 CAPI_TX5_N MGTYTXN1_125 MGT
AL40 CAPI_TX5_P MGTYTXP1_125 MGT
AJ41 CAPI_TX6_N MGTYTXN2_125 MGT
AJ40 CAPI_TX6_P MGTYTXP2_125 MGT
AG41 CAPI_TX7_N MGTYTXN3_125 MGT
AG40 CAPI_TX7_P MGTYTXP3_125 MGT
AV26 EMCCLK_B IO_L24P_T3U_N10_EMCCLK_65 1.8 (LVCMOS18)
BA31 FABRIC_CLK_PIN_N IO_L13N_T2L_N1_GC_QBC_66 1.8 (LVDS with

DIFF_TERM_ADV)
AY31| FABRIC_CLK_PIN_P| IO_L13P_T2L_N0_GC_QBC_66| 1.8 (LVDS with DIFF_TERM_ADV)
BA8| FPGA_FLASH_CE0_L| RDWR_FCS_B_0| 1.8 (LVCMOS18)
AW24| FPGA_FLASH_CE1_L| IO_L2N_T0L_N3_FWE_FCS2_B_65| 1.8 (LVCMOS18)
AW7| FPGA_FLASH_DQ0| D00_MOSI_0| 1.8 (LVCMOS18)
AV7| FPGA_FLASH_DQ1| D01_DIN_0| 1.8 (LVCMOS18)
AW8| FPGA_FLASH_DQ2| D02_0| 1.8 (LVCMOS18)
AV8| FPGA_FLASH_DQ3| D03_0| 1.8 (LVCMOS18)
AV28| FPGA_FLASH_DQ4| IO_L22P_T3U_N6_DBC_AD0P­

_D04_65

| 1.8 (LVCMOS18)
AW28| FPGA_FLASH_DQ5| IO_L22N_T3U_N7_DBC_AD0N­

_D05_65

| 1.8 (LVCMOS18)
BB28| FPGA_FLASH_DQ6| IO_L21P_T3L_N4_AD8P_D06_65| 1.8 (LVCMOS18)
BC28| FPGA_FLASH_DQ7| IO_L21N_T3L_N5_AD8N_D07_65| 1.8 (LVCMOS18)
BA19| GPIO_0_1V8_N| IO_L13N_T2L_N1_GC_QBC_64| 1.8 (LVCMOS18or LVDS)
AY19| GPIO_0_1V8_P| IO_L13P_T2L_N0_GC_QBC_64| 1.8 (LVCMOS18or LVDS)
AY20| GPIO_1_1V8_N| IO_L15N_T2L_N5_AD11N_64| 1.8 (LVCMOS18or LVDS)
AY21| GPIO_1_1V8_P| IO_L15P_T2L_N4_AD11P_64| 1.8 (LVCMOS18or LVDS)
AW20| GPIO_2_1V8_N| IO_L16N_T2U_N7_QBC_AD3N_64| 1.8 (LVCMOS18or LVDS)
Pin Number| Signal Name| Pin Name| Bank Voltage
---|---|---|---
AV20| GPIO_2_1V8_P| IO_L16P_T2U_N6_QBC_AD3P_64| 1.8 (LVCMOS18or LVDS)
AW18| GPIO_3_1V8_N| IO_L17N_T2U_N9_AD10N_64| 1.8 (LVCMOS18or LVDS)
AW19| GPIO_3_1V8_P| IO_L17P_T2U_N8_AD10P_64| 1.8 (LVCMOS18or LVDS)
BA27| IBM_PERST_1V8_L| IO_L20P_T3L_N2_AD1P_D08_65| 1.8 (LVCMOS18)
BA18| ISO_CLK_1V8| IO_L14P_T2L_N2_GC_64| 1.8 (LVCMOS18)
AD8| PCIE_LCL_REFCLK_PIN_N| MGTREFCLK0N_226| MGT REFCLK
AD9| PCIE_LCL_REFCLK_PIN_P| MGTREFCLK0P_226| MGT REFCLK
AF8| PCIE_REFCLK_1_PIN_N| MGTREFCLK0N_225| MGT REFCLK
AF9| PCIE_REFCLK_1_PIN_P| MGTREFCLK0P_225| MGT REFCLK
AB8| PCIE_REFCLK_2_PIN_N| MGTREFCLK0N_227| MGT REFCLK
AB9| PCIE_REFCLK_2_PIN_P| MGTREFCLK0P_227| MGT REFCLK
AL1| PCIE_RX0_N| MGTYRXN3_227| MGT
AL2| PCIE_RX0_P| MGTYRXP3_227| MGT
AM3| PCIE_RX1_N| MGTYRXN2_227| MGT
AM4| PCIE_RX1_P| MGTYRXP2_227| MGT
BA1| PCIE_RX10_N| MGTYRXN1_225| MGT
BA2| PCIE_RX10_P| MGTYRXP1_225| MGT
BC1| PCIE_RX11_N| MGTYRXN0_225| MGT
BC2| PCIE_RX11_P| MGTYRXP0_225| MGT
AY3| PCIE_RX12_N| MGTYRXN3_224| MGT
AY4| PCIE_RX12_P| MGTYRXP3_224| MGT
BB3| PCIE_RX13_N| MGTYRXN2_224| MGT
BB4| PCIE_RX13_P| MGTYRXP2_224| MGT
BD3| PCIE_RX14_N| MGTYRXN1_224| MGT
BD4| PCIE_RX14_P| MGTYRXP1_224| MGT
BE5| PCIE_RX15_N| MGTYRXN0_224| MGT
BE6| PCIE_RX15_P| MGTYRXP0_224| MGT
AK3| PCIE_RX2_N| MGTYRXN1_227| MGT
AK4| PCIE_RX2_P| MGTYRXP1_227| MGT
AN1| PCIE_RX3_N| MGTYRXN0_227| MGT
AN2| PCIE_RX3_P| MGTYRXP0_227| MGT
AP3| PCIE_RX4_N| MGTYRXN3_226| MGT
AP4| PCIE_RX4_P| MGTYRXP3_226| MGT
AR1| PCIE_RX5_N| MGTYRXN2_226| MGT
AR2| PCIE_RX5_P| MGTYRXP2_226| MGT
Pin Number| Signal Name| Pin Name| Bank Voltage
---|---|---|---
AT3| PCIE_RX6_N| MGTYRXN1_226| MGT
AT4| PCIE_RX6_P| MGTYRXP1_226| MGT
AU1| PCIE_RX7_N| MGTYRXN0_226| MGT
AU2| PCIE_RX7_P| MGTYRXP0_226| MGT
AV3| PCIE_RX8_N| MGTYRXN3_225| MGT
AV4| PCIE_RX8_P| MGTYRXP3_225| MGT
AW1| PCIE_RX9_N| MGTYRXN2_225| MGT
AW2| PCIE_RX9_P| MGTYRXP2_225| MGT
Y4| PCIE_TX0_PIN_N| MGTYTXN3_227| MGT
Y5| PCIE_TX0_PIN_P| MGTYTXP3_227| MGT
AA6| PCIE_TX1_PIN_N| MGTYTXN2_227| MGT
AA7| PCIE_TX1_PIN_P| MGTYTXP2_227| MGT
AL6| PCIE_TX10_PIN_N| MGTYTXN1_225| MGT
AL7| PCIE_TX10_PIN_P| MGTYTXP1_225| MGT
AM8| PCIE_TX11_PIN_N| MGTYTXN0_225| MGT
AM9| PCIE_TX11_PIN_P| MGTYTXP0_225| MGT
AN6| PCIE_TX12_PIN_N| MGTYTXN3_224| MGT
AN7| PCIE_TX12_PIN_P| MGTYTXP3_224| MGT
AP8| PCIE_TX13_PIN_N| MGTYTXN2_224| MGT
AP9| PCIE_TX13_PIN_P| MGTYTXP2_224| MGT
AR6| PCIE_TX14_PIN_N| MGTYTXN1_224| MGT
AR7| PCIE_TX14_PIN_P| MGTYTXP1_224| MGT
AT8| PCIE_TX15_PIN_N| MGTYTXN0_224| MGT
AT9| PCIE_TX15_PIN_P| MGTYTXP0_224| MGT
AB4| PCIE_TX2_PIN_N| MGTYTXN1_227| MGT
AB5| PCIE_TX2_PIN_P| MGTYTXP1_227| MGT
AC6| PCIE_TX3_PIN_N| MGTYTXN0_227| MGT
AC7| PCIE_TX3_PIN_P| MGTYTXP0_227| MGT
AD4| PCIE_TX4_PIN_N| MGTYTXN3_226| MGT
AD5| PCIE_TX4_PIN_P| MGTYTXP3_226| MGT
AF4| PCIE_TX5_PIN_N| MGTYTXN2_226| MGT
AF5| PCIE_TX5_PIN_P| MGTYTXP2_226| MGT
AE6| PCIE_TX6_PIN_N| MGTYTXN1_226| MGT
AE7| PCIE_TX6_PIN_P| MGTYTXP1_226| MGT
AH4| PCIE_TX7_PIN_N| MGTYTXN0_226| MGT
Pin Number| Signal Name| Pin Name| Bank Voltage
---|---|---|---
AH5| PCIE_TX7_PIN_P| MGTYTXP0_226| MGT
AG6| PCIE_TX8_PIN_N| MGTYTXN3_225| MGT
AG7| PCIE_TX8_PIN_P| MGTYTXP3_225| MGT
AJ6| PCIE_TX9_PIN_N| MGTYTXN2_225| MGT
AJ7| PCIE_TX9_PIN_P| MGTYTXP2_225| MGT
AW27| PERST0_1V8_L| IO_T3U_N12_PERSTN0_65| 1.8 (LVCMOS18)
AY27| PERST1_1V8_L| IO_L23N_T3U_N9_PERSTN1_I­ 2C_SDA_65| 1.8 (LVCMOS18)
AD39| QSFP_CLK_PIN_N| MGTREFCLK0N_126| MGT REFCLK
AD38| QSFP_CLK_PIN_P| MGTREFCLK0P_126| MGT REFCLK
AV16| QSFP_INT_1V8_L| IO_L24P_T3U_N10_64| 1.8 (LVCMOS18)
BA14| QSFP_MODPRS_L| IO_L22N_T3U_N7_DBC_AD0N_64| 1.8 (LVCMOS18)
AV15| QSFP_RST_1V8_L| IO_L24N_T3U_N11_64| 1.8 (LVCMOS18)
AU46| QSFP_RX0_N| MGTYRXN0_126| MGT
AU45| QSFP_RX0_P| MGTYRXP0_126| MGT
AT44| QSFP_RX1_N| MGTYRXN1_126| MGT
AT43| QSFP_RX1_P| MGTYRXP1_126| MGT
AR46| QSFP_RX2_N| MGTYRXN2_126| MGT
AR45| QSFP_RX2_P| MGTYRXP2_126| MGT
AP44| QSFP_RX3_N| MGTYRXN3_126| MGT
AP43| QSFP_RX3_P| MGTYRXP3_126| MGT
AN46| QSFP_RX4_N| MGTYRXN0_127| MGT
AN45| QSFP_RX4_P| MGTYRXP0_127| MGT
AK44| QSFP_RX5_N| MGTYRXN1_127| MGT
AK43| QSFP_RX5_P| MGTYRXP1_127| MGT
AM44| QSFP_RX6_N| MGTYRXN2_127| MGT
AM43| QSFP_RX6_P| MGTYRXP2_127| MGT
AL46| QSFP_RX7_N| MGTYRXN3_127| MGT
AL45| QSFP_RX7_P| MGTYRXP3_127| MGT
AW15| QSFP_SCL_1V8| IO_L23P_T3U_N8_64| 1.8 (LVCMOS18)
AW14| QSFP_SDA_1V8| IO_L23N_T3U_N9_64| 1.8 (LVCMOS18)
AH43| QSFP_TX0_N| MGTYTXN0_126| MGT
AH42| QSFP_TX0_P| MGTYTXP0_126| MGT
AE41| QSFP_TX1_N| MGTYTXN1_126| MGT
AE40| QSFP_TX1_P| MGTYTXP1_126| MGT
AF43| QSFP_TX2_N| MGTYTXN2_126| MGT
Pin Number| Signal Name| Pin Name| Bank Voltage
---|---|---|---
AF42| QSFP_TX2_P| MGTYTXP2_126| MGT
AD43| QSFP_TX3_N| MGTYTXN3_126| MGT
AD42| QSFP_TX3_P| MGTYTXP3_126| MGT
AC41| QSFP_TX4_N| MGTYTXN0_127| MGT
AC40| QSFP_TX4_P| MGTYTXP0_127| MGT
AB43| QSFP_TX5_N| MGTYTXN1_127| MGT
AB42| QSFP_TX5_P| MGTYTXP1_127| MGT
AA41| QSFP_TX6_N| MGTYTXN2_127| MGT
AA40| QSFP_TX6_P| MGTYTXP2_127| MGT
Y43| QSFP_TX7_N| MGTYTXN3_127| MGT
Y42| QSFP_TX7_P| MGTYTXP3_127| MGT
AV36| SI5328_1V8_SCL| IO_L24N_T3U_N11_66| 1.8 (LVCMOS18)
AV35| SI5328_1V8_SDA| IO_L24P_T3U_N10_66| 1.8 (LVCMOS18)
AE37| SI5328_OUT_0_PIN_N| MGTREFCLK1N_125| MGT REFCLK
AE36| SI5328_OUT_0_PIN_P| MGTREFCLK1P_125| MGT REFCLK
AB39| SI5328_OUT_1_PIN_N| MGTREFCLK0N_127| MGT REFCLK
AB38| SI5328_OUT_1_PIN_P| MGTREFCLK0P_127| MGT REFCLK
BB19| SI5328_REFCLK_IN_N| IO_L12N_T1U_N11_GC_64| 1.8 (LVDS)
BB20| SI5328_REFCLK_IN_P| IO_L12P_T1U_N10_GC_64| 1.8 (LVDS)
AV33| SI5328_RST_1V8_L| IO_L22P_T3U_N6_DBC_AD0P_66| 1.8 (LVCMOS18)
BE30| SPARE_SCL| IO_L5N_T0U_N9_AD14N_66| 1.8 (LVCMOS18)
BC30| SPARE_SDA| IO_L6P_T0U_N10_AD6P_66| 1.8 (LVCMOS18)
BD30| SPARE_WP| IO_L6N_T0U_N11_AD6N_66| 1.8 (LVCMOS18)
BE31| SRVC_MD_L_1V8| IO_L3P_T0L_N4_AD15P_66| 1.8 (LVCMOS18)
AV32| USER_LED_A0_1V8| IO_L18N_T2U_N11_AD2N_66| 1.8 (LVCMOS18)
AW32| USER_LED_A1_1V8| IO_T2U_N12_66| 1.8 (LVCMOS18)
AY30| USER_LED_G0_1V8| IO_L17N_T2U_N9_AD10N_66| 1.8 (LVCMOS18)
AV31| USER_LED_G1_1V8| IO_L18P_T2U_N10_AD2P_66| 1.8 (LVCMOS18)
AW33| USR_SW_0| IO_L22N_T3U_N7_DBC_AD0N_66| 1.8 (LVCMOS18)
AY36| USR_SW_1| IO_L23P_T3U_N8_66| 1.8 (LVCMOS18)

Revision History

Date Revision Changed By Nature of Change
24 Sep 2018 1.0 K. Roth Initial Release

31 Oct 2018

| ****

1.1

| ****

K. Roth

| Updated product images, changed default programable clock frequency for CAPI_CLK_1 to 161MHz


14 Dec 2018

| ****

1.2

| ****

K. Roth

| Updated configuration flash part number, changed wording of gpio description for accuracy, added weight.


24 Oct 2019

| ****

1.3

| ****

K. Roth

| Updated Configuration to remove address map and correct description of memory part capacity.



25 Jan 2022

| ****


1.4

| ****


K. Roth

| Updated Thermal Performance to include thermal efficiency figures and comments about the impact of the shroud, removed references to QSFP0 and QSFP1 from section QSFP-DD and updated 25Gb transceiver part number.

Customer Service

© 2022 Copyright Alpha Data Parallel Systems Ltd.
All rights reserved.
This publication is protected by Copyright Law, with all rights reserved. No part of this publication may be reproduced, in any shape or form, without prior written consent from Alpha Data Parallel Systems Ltd.
Head Office
Address: Suite L4A, 160 Dundee Street,
Edinburgh, EH11 1DQ, UK
Telephone: +44 131 558 2600
Fax: +44 131 558 2700
email: sales@alpha-data.com
website: http://www.alpha-data.com
US Office
Address: 10822 West Toller Drive, Suite 250
Littleton, CO 80127
Telephone: (303) 954 8768
Fax: (866) 820 9956 – toll free
email: sales@alpha-data.com
website: http://www.alpha-data.com

All trademarks are the property of their respective owners.
Address: Suite L4A, 160 Dundee Street,
Edinburgh, EH11 1DQ, UK
Telephone: +44 131 558 2600
Fax: +44 131 558 2700
email: sales@alpha-data.com
website: http://www.alpha-data.com
Address: 10822 West Toller Drive, Suite 250
Littleton, CO 80127
Telephone: (303) 954 8768
Fax: (866) 820 9956 – toll free
email: sales@alpha-data.com
website: http://www.alpha-data.com

ALPHA DATA logo

References

Read User Manual Online (PDF format)

Read User Manual Online (PDF format)  >>

Download This Manual (PDF format)

Download this manual  >>

Related Manuals