1、Introduction to Field Programmable Gate Arrays,Lecture 3/3CERN Accelerator School on Digital Signal Processing Sigtuna, Sweden, 31 May 9 June 2007 Javier Serrano, CERN AB-CO-HT,Outline,Using FPGAs in the real world Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing
2、to the outside world. Clock domains and metastability. Safe design and radiation hardness.,Outline,Using FPGAs in the real world Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardn
3、ess.,Reminder: basic digital design,High clock rate: 144.9 MHz on a Xilinx Spartan IIE.,Higher clock rate: 151.5 MHz on the same chip.,6.90 ns,6.60 ns,Buffering,Delay in modern designs can be as much as 90% routing, 10% logic. Routing delay is due to long nets + capacitive input loading. Buffering i
4、s done automatically by most synthesis tools and reduces the fan out on affected nets:,net1,net2,net1,net2,net3,Before buffering,After buffering,Replicating registers (and associated logic if necessary),Producer,Consumer 1,Consumer 2,Consumer 4,Consumer 3,Consumer 1,Consumer 2,Consumer 4,Consumer 3,
5、Producer,Before,After,Retiming (a.k.a. register balancing),Large combinational logic delay,Small Delay,Balanced delay,Balanced delay,Before,After,Pipelining,Large combinational logic delay,Small delay,Before,After,Small delay,Small delay,Time multiplexing,Data In,100 MHz,50 MHz logic,50 MHz logic,50
6、 MHz logic,50 MHz logic,50 MHz logic,50 MHz logic,Data Out,50 MHz,De-multiplexer,Multiplexer,An example: boosting the performance of an IIR filter (1/2),Simple first order IIR: yn+1 = ayn + b xn,Z-1,X,+,b,x,X,a,y,Performance bottleneck in the feedback path,Problem found in the phase filter of a PLL
7、used to track bunch frequency in CERNs PS,An example: boosting the performance of an IIR filter (2/2),Look ahead scheme: From yn+1 = ayn + b xn we get yn+2 = ayn+1 + bxn+1 = a2yn + abxn + bxn+1,Z-1,X,+,x,X,a2,y,ab,X,b,Z-1,+,Z-2,FIR filter (can be pipelined to increase throughput),Now we have two clo
8、ck ticks for the feedback!,Another example: being smart about what you need exactly.,u x v = ux vy uy vx |u x v| = |u| x |v| sin = IcFwd u = Vacc, v = IcFwd,Cross product used as phase discriminator by John Molendijk in the LHC LLRF.,Outline,Using FPGAs in the real world Performance boosting techniq
9、ues. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardness.,Floating point designs,To work in floating point you (potentially) need blocks to: Convert from fixed point to floating point and back. Convert between
10、different floating point types. Multiply. Add/subtract (involves an intermediate representation with same exponent for both operands). Divide. Square root. Compare 2 numbers. The main FPGA companies provide these in the form of IP cores. You can also roll your own.,Format,s: sign. e: exponent. f: fr
11、actional part (b0.b1b2b3b4.bwf-1) Convention: normalized numbers have b0=1,Exponent value:,IEEE-754 standard single format: 24-bit fraction and 8-bit exponent (w=32 and wf=24 in the figure). IEEE-754 standard double format: 53-bit fraction and 11-bit exponent.,Total value:,Some performance figures (
12、single precision),Some performance figures (double precision),Rolling your own. Example:,Ray Andraka, “Hybrid Floating Point Technique Yields 1.2 Gigasample Per Second 32 to 2048 point Floating Point FFT in a single FPGA.” http:/ three of these together and triplicate throughput!,Limited by DSP48 ma
13、x. clock rate in Virtex 4 XCV4SX55-10: 400 MHz. Total throughput: 1.2 Gs/s,Outline,Using FPGAs in the real world Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardness.,FPGA power
14、requirements (1/2),Voltage: different voltage rails: core, I/Os, AUX, SERDES, PLL.Tolerance: typically +/- 5%.Monotonicity: Vcc must rise steadily from GND to desired value (could work otherwise but FPGAs are not tested that way).,FPGA power requirements (2/2),Power-on current. Watch out for PCB cap
15、acitor in-rush current: Ic=C*DV/DT. Slow down voltage ramp if needed. Sequencing: required for old technologies and recommended for new ones. Read datasheet. Example for Virtex-4/5: VCCINT VCCAUX VCCO. Use Supply Voltage Supervisor (SVS) to control sequencing. Power-on ramp time. Devices specify a m
16、inimum and a maximum ramp time. Again, this is how they are tested.,Power solutions,Low Drop-Out (LDO). Linear. Unbeatable for quietness. Inefficient. Switching solutions (some have external clk pins that you can drive at a frequency you can easily filter afterwards) Controller (external FET) Conver
17、ter (built-in FET) Module Multi-rail solutions,Amps,LDO: Be aware - Under-voltage lockout,Problem: LDO with non-monotonic voltage output. Cause: 5V primary supply was powering on at the same time. Caps and 3 LDOs caused the 5V to droop. Result: Primary 5V current-limiter shut it down. LDOs under-vol
18、tage lockout tripped, shutting down the LDO. How can we fix this?,LDO: under-voltage lockout solution,Use SVS to sequence regulators after caps are charged.,LDO: be aware in-rush and current limit,A fast-starting LDO induces a huge in-rush current from charging capacitors (remember Ic=C*DV/DT) LDO e
19、nters current-limit mode due to capacitor in-rush. The transition to current-limit mode causes a glitch. What to do?,LDO: in-rush and current-limit solution,Slow down the ramp time using a soft-start circuit. Reduces V/t which reduces capacitor in-rush current. Regulator never hits current-limit and
20、 stays in voltage mode. Good for meeting FPGA minimum ramp time specs. External or built-in.,Note: in-rush FPGA current during configuration is a thing of the past thanks to the introduction of proper housekeeping circuitry.,How much current is our design consuming?,Insert a small high-precision res
21、istor in series with primary voltage source before the regulator, and measure the voltage drop with a differential amplifier. Below an example from a LLRF board designed by Larry Doolittle (LBNL).,Then compare with the predicted power consumption from your vendors software tool ;),Decoupling capacit
22、ors,Capacitors are not ideal! They have parasitic resistance and inductance:,Decoupling capacitors,Knee frequency in the spectrum of a digital data stream is related by the rise and fall times (Tr) by: Fknee=0.5/Tr (1). We want our Power Distribution System to have low impedance at all frequencies o
23、f interest low voltage variations for arbitrary current demands. Solution: parallel combination of different capacitor values. For more info: Xilinx XAPP623.,(1) Howard W. Johnson, Martin Graham. High Speed Digital Design, A Handbook of Black Magic. Prentice Hall, 1993.,Outline,Using FPGAs in the re
24、al world Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardness.,FPGAs have very versatile connectivity. Example: Xilinx Spartan 3 family.,HSTL,Chip-to-Chip Interfacing: Backplane
25、Interfacing: High-speed Memory Interfacing:,LVDS,GTL,GTL+,PCI,BLVDS,Single ended and differential. 784 single-ended, 344 differential pairs. 622 Mb/sec LVDS. 24 I/O standards, 8 flexible I/O banks. PCI 32/33 and 64/33 support. Eliminate costly bus transceivers. 3.3V, 2.5V, 1.8V, 1.5V, 1.2V,SSTL,Inte
26、rfacing with ADCs and DACs,Large parallel busses working at high clock rates potential for timing and noise problems.Possible solutions: ADCs nowadays have analog bandwidths well above twice their maximum sampling rate sample band pass signals at slower rates (in other Nyquist zones). Use high speed
27、 differential serial links for ADCs and DACs (so far, no embedded clock: clk + data on two separate LVDS links). Run digital supply in parallel ADCs as low as possible: 2.0-2.5V feasible.,Interfacing with busses using 5V signaling (e.g. VME),Dual supply level translators are the most flexible soluti
28、on. Alternatives: 5V compliant 3.3V buffers exist, such as the LVTH family. They also provide more current than standard FPGA I/Os. Open-drain devices (uni-directional, can do wired-or). FET switches (very fast, no active drive).,Open-drain 3.3V 5V,FET-based 5V 3.3V,Outline,Using FPGAs in the real w
29、orld Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardness.,Characterizing metastability,Use measurements with this setup to find K1 and K2, assuming an MTBF of the form:,Virtex I
30、I Pro Metastability results,From Xilinx XAPP094,Synchronizer circuit,Place the two flip-flops close together to minimize net delay.When a signal comes on-chip, synchronize it first then fan-out (dont fan-out then synchronize at multiple places).Make sure clk period is OK for desired MTBF. E.g. for V
31、irtex II Pro, giving the flip-flop 3 ns to resolve will give you an MTBF higher than 1 Million years!,Crossing clock domains,For single-bit signals, use the double flip-flop synchronizer. For multi-bit signals, using a synchronizer for each bit is wrong. Different synchronizers can resolve at differ
32、ent times. No way to know when data is valid, other than waiting a long time. For slow transfers, you can use 4-phase or 2-phase handshake (a single point of synchronization). Otherwise, give up acknowledgement and make sure system works “by design”. FIFOs are also useful.,Four phase handshake,VALID
33、,ACK,DATA,VALID,ACK,DATA,n,Adapted from VLSI Architectures Spring 2004 www.ee.technion.ac.il/courses/048878 by Ran Ginosar,The VALID signal is synchronous to the source clock and gets synchronized at the receiving end by a double flip-flop synchronizer. The same happens in the opposite sense with th
34、e ACK signal.,Two phase handshake,VALID,ACK,DATA,n,Adapted from VLSI Architectures Spring 2004 www.ee.technion.ac.il/courses/048878 by Ran Ginosar,VALID,ACK,DATA,A complete circuit,Michael Crews and Yong Yuenyongsgool, Practical design for transferring signals between clock domains. EDN magazine, Fe
35、bruary 20, 2003.,Outline,Using FPGAs in the real world Performance boosting techniques. Floating point designs. Powering FPGAs. Interfacing to the outside world. Clock domains and metastability. Safe design and radiation hardness.,Reset strategies,Different flip-flops see reset de-asserted in differ
36、ent clock cycles!It matters in a circuit like this.You can fix this problem with a proper reset generator.,Even better if you can use this as a synchronous reset,Safe state machines,One-hot encoding: s0 = 0001 s1 = 0010 s2 = 0100 s3 = 1000 12 “illegal states” not covered, or covered with a “when oth
37、ers” in VHDL or equivalent. Use option in synthesis tool to prevent optimization of illegal states.,Single Event Effects (SEE) created by neutrons,Classification of SEEs,Single Event Effect (SEE),Single Event Functional Interrupt (SEFI) Bit-Flip specifically in a control register POWER ON RESET/JTAG
38、 etc.,Single Event Upset (SEU) Bit-Flip Somewhere,Single Event Latch-Up (SEL) Parasitic transistors activated in a device, causing internal short,Single Event Transient (SET) A signal briefly fluctuates somewhere in design,SEL description,Has virtually disappeared in new technologies (low Vccint not
39、 enough to forward bias transistors). Only cure used to be epitaxial substrate (very expensive).,Activation of either of these transistors causes a short from V+ to V-,SEU Failures in Time (FIT),Defined as the number of failures expected in 109 hours. In practice, configuration RAM dominates. Exampl
40、e:Average of only 10% of FPGA configuration bits are used in typical designs Even in a 99% full design, only up to 30% are used Most bits control interconnect muxes Most mux control values are “dont-care” Must include this ratio for accurate SEU FIT rate calculations.,FPGA Interconnect,ON,OFF,DONT-C
41、ARE,Active Wire,Virtex XCV1000 memory Utilization,Not all parts of the design are critical,Average of only 40% of circuits in FPGA designs are critical Substantial circuit overhead for startup logic, diagnostics, debug, monitoring, fault-handling, control path, etc. Must also include this ratio in S
42、EU FIT rate calculations,FPGA Design,Critical,Non-critical,Actual FIT,Note 1: From analysis of real FPGA designs,Actual FIT = Base FIT * SEUPI Ratio * CC Ratio,Half-latches (weak keepers) in Virtex devices,Provide constants Save logic resources Used throughout device Subject to SEU upset Can reset o
43、ver time Not observable Not defined by configuration bits Reinitialized as part of device initialization Full reconfiguration required,0,0,1,0,0,Configuration Bits,Half-latch,T1,T2,T3,A,Mitigation techniques: scrubbing,Readback and verification of configuration. Most internal logic can be verified d
44、uring normal operation. Sets limits on duration of upsets. Partial configuration Not supported by all FPGA vendors/families. Allows fine grained reconfiguration. Does not reset entire device. Allows user logic to continue to function. Complete reconfiguration Required after SEFI. No user functionali
45、ty for the duration of reconfiguration. Verification by dedicated device Usually radiation tolerant antifuse FPGA Secure storage of checksums and configuration an issue FLASH is radiation sensitive Self verification Often the only option for existing designs Not possible in all device families Utili
46、zes logic intended for dynamic reconfiguration Verification logic has small footprint Usually a few dozen CLBs and 1 block RAM (for checksums).,Triple Module Redundancy (TMR),Feedback TMR Three copies of user logic State feedback from voter Counter example Handles faults Resynchronizes Operational t
47、hrough repair Speed penalty due to feedback Desirable for state based logic,Counter,Counter,Counter,Voter,Voter,Voter,Alternatives,Antifuse Configuration based on physical shorts Invulnerable to upset Cannot be altered Over 90% smaller upset cross section for comparable geometry Signal routing more
48、efficient Much lower power dissipation for similar device geometry Lags SRAM in fabrication technology Usually one generation behind Latch up more of a problem than in SRAM devices Rad-hard Antifuse All flip-flops TMRed in silicon Unmatched reliability High (extreme) cost Unimpressive performance Feedback TMR built in Usually larger geometry Not available in highest densities offered by antifuse FLASH FPGAs Middle ground in base susceptibility Readback/Verification problematic Usually only JTAG (slow) supported Maximum number of write cycles an issue,
