FPGAs Have the Wrong Abstraction for Computing

I wonder if the author is aware that the bistream formats for both Lattice iCE40 series and Xilinx Virtex 7 series FPGAs have now been reverse engineered, and there is a complete open source toolchain that can be used for these. So Verilog is no longer as low as you can go.

Efforts of this type are also underway for other parts and there is a growing movement in this direction - see talks from Clifford Wolf at recent CCC events.

* Most of the algorithms that we want to work with in this domain are doing arithmetic operations on ints and floats. This isn't super difficult to do in an RTL, but it's like implementing C++ objects in assembler. You can do it, but you need to think harder than you should.

* FPGAs make you worry about timing. This is a massive shift in thinking for software people. It's also not a value-add; I don't want to care about timing. And it enforces chip-wide dependencies (you can have separate clock domains, but not many of them).

If you simplify the model to "pipelines of arithmetic ops" and then provide an abstraction that eliminates timing (e.g. all ops run in a fixed number of clock cycles and the compiler automatically pipelines them where necessary) then I think you'd have something usable. But this is basically a GPU with a lot of SRAM. Such a constrained problem would run extremely well on any modern GPU or SIMD machine, without the power and cost and obscurity constraints of FPGAs.

"FPGAs aren't evolved like modern processors with standardized programming models, so we must throw out the current model but I have no idea what is better."

Having worked with FPGAs, I can understand his complaints about the toolchain, they absolutely suck and are mostly closed source. This technology is just like microprocessors in its infantry, its evolving.

FPGAs have made crazy progress in the last decade and are getting to the point where it's now affordable for hobbyists and consumers to work with them instead of merely just aerospace & defense contractors with massive hardware budgets.

The problem with Verilog as an ISA is that it is too far removed from the hardware. The abstraction gap between RTL and FPGA hardware is enormous: it traditionally contains at least synthesis, technology mapping, and place & route—each of which is a complex, slow process. As a result, the compile/edit/run cycle for RTL programming on FPGAs takes hours or days and, worse still, it’s unpredictable: the deep stack of toolchain stages can obscure the way that changes in RTL will affect the design’s performance and energy characteristics.

Yes, RTL varies significantly from the actual hardware of the device (LUTs, Memory, Peripherals, etc..) but from a design standpoint, I don't see anything else that would make more sense to work with. FPGAs are significant BECAUSE of the fact that you get build and design at that level. Let's not forget that ISAs are a higher level abstraction of RTL...

Transistors are abstracted into logic gates. Logic gates are abstracted into higher-order digital functions like flip-flops, muxes, etc. It is the mapping of algorithms/functions onto gates that is the essence of digital design. This is difficult work that would be impossible at today's scales (5-billion+ transistors) without synthesis tools and HDLs. And, given that an ASIC mask set costs 1MM+ for a modern geometry, it needs to be done right the first time (or at least the 2nd). Furthermore, the mapping to gates needs to be efficient, throwing more gates at a problem increases area, heat, and power, all of which need to be minimized in most contexts.

My first job out of college was designing 386 motherboards. Back then we were still using discrete 74xx ICs for most digital functions. The boards were huge. PLDs allowed better intergration and were cost effective since a single device could implement many different functions, and reduced board area and power consumption. CPLDs moved this further along.

FPGAs grew out of PLD/CPLDs and allowed a significantly higher level of integration and board area reduction. They offered a way to reduce the cost of a system without requiring the investment and expertise required for an ASIC. But, FPGAs themselves are an ASIC, implemented with the same technology as any other ASIC. So, FPGAs are a compromise; the LUTs, routing, etc are all a mechanism to make a programmable ASIC. Compared to an ASIC, however, FPGAs require more power and can implement less capability for a given die size. But, they allow a faster and lower cost development cycle. To bring this back around, the LUTs and routing mechanisms are functions that have been mapped to gates. To use an FPGA, algorithms still need to be mapped onto the LUTs and this is largely the same process as mapping to gates.

This article was pointless, even the author acknowledges: "I don’t know what abstraction should replace RTL for computational FPGAs." And, "Practically, replacing Verilog may be impossible as long as the FPGA vendors keep their lower-level abstractions secret and their sub-RTL toolchains proprietary." As I have argued above, knowing the FPGA vendors lower-level abstractions won't make the problem any better. The hard work is mapping onto gates/LUTs. And that analogy is wrong: "GPU : GPGPU :: FPGA : " An FPGA is the most general purpose hardware available.

The best FPGA/ASIC abstraction we have today is a CPU/GPU.

Verilog is an event-driven modeling language. It easily describes large collections of processes where a given process is triggered for reevaluation any time one of its inputs changes. That is what it was designed for back in the 80s. Using it for automatic synthesis of logic came later.

If your mental model is that Verilog is an ISA then things will be very confusing. Programming language semantics and ISAs are two different things.

Yes, the Verilog language is pretty ugly. But it allows low level modeling at the primitive level, even if that primitive is a single logic gate. And like any programming language, hierarchy is used to build useful abstractions for the particular problem domain.

That is simply not true.

You can manually instantiate FPGA primitives (LUTs/BRAMs/DSPs) with all major FPGAs, and if you’re truly desperate you can add placement attributes to place these primitives exactly where you want them.

That’s as close to the metal as I can imagine (other than specifying the actual routing network), and from there one could build up any abstraction level one desires.

"To let GPUs blossom into the data-parallel accelerators they are today, people had to reframe the concept of what a GPU takes as input. We used to think of a GPU taking in an exotic, intensely domain specific description of a visual effect. We unlocked their true potential by realizing that GPUs execute programs."

Up until the late 2000s, there was a lot of wandering-in-the-wilderness going on with respect to multicore processing, especially for data-intensive applications like signal processing. What really made the GPU solution accelerate (no pun intended!) was the recognition and then real-world application (CUDA & OpenCL) of a programming paradigm that would best utilize the inherent capabilities of the architecture.

I have no idea if those languages have gotten any better in the last few years, but anything past a matrix-multiply unroll was some real "here be dragons" stuff. But: you could take these kernels and then add sufficient abstraction on top of them until they were actually usable by mere humans (in a BLAS flavor or even higher). And even better if you can add in the memory management abstraction as well.

Point being: still not there for FPGA computation, though there was some hope at one time that OpenCL would lead us down a path to decent heterogeneous computing. Until there's some real breakthroughs in this area though, the best computation patterns that are going to map out using these techniques are the things we're already targeting to either CPUs or GPUs.

However, I don't think an ISA for FPGA can exist, not one that that allows for quick synthesis.

Sure, you could drop down to a layer where everything is described as LUTs, FFs and routing; But you still need to run "place and route" before you can execute it, and that the expensive part of synthesis.

See my rants for the last decade about "Bitgrid" if you want to know more.

[0] https://en.m.wikipedia.org/wiki/Field-programmable_gate_arra...

The biggest issue is that standard processors (i.e. x86_64) continue to get faster and better, and in a straight line, they're already much faster than an FPGA. Secondarily, software written for an FPGA ends up being specific to the FPGA, mostly because the number of LUTs per chip and the routing.

So there's two things going on, one is that you need something very parallel in order to have a compelling argument for implementing it on an FPGA, and secondarily, in a couple of years, the C implementation of the same thing ends up being faster because processors have improved and number of cores has increased, caches have gotten better on your standard processor. So for your code to improve, you have to rework it entirely, possibly to the extent of rethinking the algorithm you've used. (FPGAs advance in the same way, but a new compiler doesn't speed up your code, because you wrote it very close to the metal).

So in reality, the issue isn't that FPGAs have the wrong abstraction, it's that they have practically no abstraction, at least when it comes to having a real compiler that will optimize your code in a meaningful way for the chip that you are using, the way something like GCC would. Even if you are writing verilog or vhdl, you still need to consider the number of LUTs you have, how the placement will work out based on the size of your different modules (and of course, clock timings & pipeline stalls). You get some help with that stuff from the compiler, but when you then upgrade to a bigger chip, there are diminishing returns in the help that it provides. It is really like you are building your own arbitrary CPU. In that regard, it is difficult to find good people.

None of the commercial attempts at high level languages have been successful, largely because they suck. You're stuck writing C code when you'd rather write something that actually considers the advantages that an FPGA has (the extremely parallel processing of bytes). Implementors need to think more in terms of parallel graphs and tree structures that coalesce than they do loops. From that perspective, you need actual language primitives that actually match with the advantages of the platform (which verilog and vhdl do, but in a low-level, ham-fisted way). So it's an incredibly tough problem to tackle.

[0] https://symbiflow.github.io/

This is basically what pushes me away from FPGAs, even though there is now icestorm so i can avoid Windows. As an outsider I totally agree with the author, I love the idea of using FPGAs as accelerators, but it's way easier to do GPGPU right now.

[1] https://github.com/SymbiFlow/ideas/issues/19

The author seems to view "mainstream" as meaning being as easy as CPU or GPGPU programming. I don't think it makes much sense trying to accomplish this on FPGAs; you're better off using CPUs, GPUs, or making something domain-specific like TPUs. The benefit of FPGAs is that they allow you to build your own architecture, and define data movement in a way that is specific to your application, at the cost of increased development effort. The complexity encountered in doing RTL arises from the inherent complexity in using FPGAs effectively.

There is a case to be made for something in between a CPU and an FPGA that allows easier development, but gives you some ability to control your data movement to get higher performance. Processor meshes, like what Xilinx is including with their upcoming Versal chips, might be a good solution to this (though in typical FPGA vendor fashion this too is locked behind proprietary tooling).

  The problem with Verilog as an ISA is that it is too far removed from the hardware. The abstraction gap between RTL and FPGA hardware is enormous: it traditionally contains at least synthesis, technology mapping, and place & route—each of which is a complex, slow process. 

A GPU ISA changes very little between generations of the GPU, however a rather small change in the structure of the FPGA usually implies a completely new bitstream. So even if there was bitstream documentation it would be very FPGA-specific, unlike an ISA.

P&R does not look that much different from compiling, in the sense that you can spend as much effort as you want on it in order to produce a better or worse result for your machine.

I've met a number of software people who are now developing FPGA designs, and hardware people who are the same place (even looking to hire US Persons with those skills for SDR work :-) and invariably the software folks get a brain cramp because it "looks" like code, but it doesn't "work" like code.

I have often wondered how close this experience is to English speaking people listening to a conversation about a software design and implementation becoming frustrated because they think they understand each word but they aren't understanding any semantics of the conversation.

I'm one of those people who essentially double majored in EE and CS[1] which is sort of like growing up in a house where one parent speaks one language and the other speaks another. In those situations you tend to naturally accept when one discipline (or language) doesn't overlap cleanly with the other.

As a result I disagree with the author that an FPGA is just a computation engine with a "wonky" ISA. Thinking about it that way is some what limiting. That said, some of the things that give FPGAs their programmability might be useful additions to GPUS, mostly redefining the internal data paths of the GPU to allocate more "bits" to change dynamic range for different operators might allow some interesting things to be done.

HDLs themselves are interesting problems, because you have a target, switching matrix and logic elements of an FPGA, and text. Designing a way to express the capabilities in one that translates to the other is very hard. So hard in fact that there are circuits you can construct in the FPGA that you cannot express directly in a HDL, and there are things you can write in an HDL that cannot by synthesized into a set of linked logic elements and a clock. On the surface this might seem like saying "Well yeah you can write things in assembly that the compiler won't generate." but it goes deeper than that. Both from programming what the pins on the package do to changing clock networks to get timing closures on complex designs, an FPGA is not a stored program computation device unless you configure it as one.

[1] Computer Engineering as a major wasn't a "thing" yet so I ended up taking all the EE major classes and nearly all the CS major classes (I missed out on some of the logic oriented math classes of CS because of the physics and materials science requirements on the EE side).

Is it possible to create a language around FSMs? Most hardware seems to have two parts: the actual logic that implements some functionality and then some FSM that implements the control logic. The FSM may also have a lot of implicit/assumed states (like a counter for some timeout). Maybe a higher level language can expose these design pattern in a nicer way and hide all the messy low level details (like sequential/combinational logic, connecting ports and wires, matching signal widths, etc).

You do not PROGRAM an FPGA in the software engineering sense of the word, you CONFIGURE it.

You do not use a programming language to create your configuration bitstream; you use a HARDWARE DESCRIPTION LANGUAGE.

You describe the hardware you want and how it is to be interconnected. You do this either explicitly —by literally using code that wires resources as you specify—or by inference— using idioms, if you will, that you know result in specific hardware within the device.

Thinking of Verilog or VHDL as software programming languages is wrong and can only lead to frustration. FPGA’s are still very much the domain of hardware engineering. Well, at least if what we are after are efficient high-performance results.

If, as a software engineer, you want to take advantage of an FPGA to accelerate processing, you should work with a capable FPGA hardware engineer to create a device with an “API” (using the term loosely) that exposes the desired functionality. I’ve done just that more times than I can remember; hanging a large FPGA off something like a small 8 bit 8051-derivative processor that allows the micro to access powerful computing resources exposed through means easily accesible with simple C functions in real time.

If you use the right tool for the job and do it correctly it can be blissful; try to force a paradigm that does not match reality and all you get is frustration.

Bulk computation is what we need. The maximum use of all of the transistors in a chip, at the minimum necessary clock speed and voltage to get the job done. Delays don't matter at all if you get a result each clock cycle.

That's way at InAccel we developed the FPGA resource manager that allows to instantiate and deploy FPGAs in the same way as you invoke typical software functions. The FPGA manager takes care the scheduling, the resource management and the configuration of the FPGAs from a pool /marketplace of hardware accelerators.

That way is easier than ever to use FPGAs in the same way you use optimized libraries for CPUs/GPUs.

And we have a free community edition: More info at: https://www.inaccel.com/

Any inputs/thoughts on how appropriate it might be for a Software Engineer to learn and program FPGAs?

I think part of the problem here for example is

>That is, Verilog is to an FPGA as an ISA is to a CPU.

No! Not at all! For a start I can literally write verilog that won't work on any FPGA ever

always @ (rising edge multiplier_result[8]) begin $print("This is nuts!") end

Verilog is not an FPGA language, Verilog is a hardware language that Vendors implement a subset of on any given FPGA.

Here would be my advice to someone who wants a better abstraction for FPGAs: stop relating them to other things that behave very differently. Timing is important, placement is important, mapping is important. If your abstraction doesn't include these elements it is fundamentally flawed.