From zero to RISC-V in hardware, in 6 minutes

Program your FPGA with a one-liner command. It’s a kind of magic.

This is a project in sustainable FPGA development that I have been working on. It ties together several smaller projects that I developed in a perpetual quest for hermetic, ephemeral, and reproducible builds. As far as I know, you can not find any setup like this anywhere else. It is a long and non-eventual video, but hardware people should be able to recognize the appeal.

Words are cheap: check out the video first, and I will explain the details later. If you are sufficiently familiar with the material, you will recognize the value right away and you may not need to read any further.

What you may note in the video is as follows:

• An EDA workflow that starts from a clean machine in GCP,
• The build system downloads and installs all required EDA software without user intervention. The GCP virtual machine only has bazel installed.
• It then builds a RISC-V 32-bit core with custom made firmware (with an open source toolchain), then runs synthesis, PnR and programming on a FPGA device which is physically remote, hundreds of miles away, from the machine running the EDA software.
• It lets the RISC-V 32 core run and catches its “hello world” on a UART (remotely!).

This RISC-V 32 setup is not the final setup.

I needed a controller to execute programmable hardware tests for an as-of-yet unpublished project. (Which I hope to be able to publicize, but we’ll see about that.)

But I only have an FPGA device, not a Zynq, so I couldn’t rely on any hard IP on the device. So I created a testbench that enables me to continue my work. That testbench is what you see in the video.

Why?

From times past, when “do you use source control” was a reasonable question to ask, comes the venerable blog Joel on Software. Once upon a time, Joel formulated 12 questions for judging the quality of your development team quickly. These questions have since become known as “The Joel Test”.

Its “question 2” is relevant here, and it tells the story way more eloquently than I ever could:

2. Can you make a build in one step?
By this I mean: how many steps does it take to make a shipping build from the latest source snapshot? On good teams, there’s a single script you can run that does a full checkout from scratch, rebuilds every line of code, makes the EXEs, in all their various versions, languages, and #ifdef combinations, creates the installation package, and creates the final media — CDROM layout, download website, whatever.

If the process takes any more than one step, it is prone to errors. And when you get closer to shipping, you want to have a very fast cycle of fixing the “last” bug, making the final EXEs, etc. If it takes 20 steps to compile the code, run the installation builder, etc., you’re going to go crazy and you’re going to make silly mistakes.

For this very reason, the last company I worked at switched from WISE to InstallShield: we required that the installation process be able to run, from a script, automatically, overnight, using the NT scheduler, and WISE couldn’t run from the scheduler overnight, so we threw it out. (The kind folks at WISE assure me that their latest version does support nightly builds.)

What?

Programmable hardware buffs among you will remember how hard it is to create a programmable hardware workbench for FPGAs from scratch.

The typical workflow goes something like this:

• Get the programmable hardware and accessories.
• Get the appropriate machine.
• Get the appropriate software.
• Try to install drivers, hardware and software.
• Get frustrated because none of them work.
• Try to install the software, and see it fail.
• Try to find a manual to no avail (nobody makes or reads manuals these days)
• Try to find advice from the manufacturer. The document is 20 years old and gives outdated information.
• Try to find advice in forums. Sift through thousands of semi-literate and semi-completed messages and discussions.
• Find that one message about exactly the issue you are facing, posted 21 years ago, without any answers.
• Accidentally find a post alluding to the version of Python being the problem.
• You try to install that version of Python.
• Suddenly, nothing works anymore.
• Learn about venv.
• Randomly install some stuff.
• The versions do not match up, try again.
• OK Python works, what now?
• Try to install vendor software.
• You can not get an installer, unless you go through the onerous licensing form. So you do.
• The licensing form crashes at the very last of the 50 questions.
• Try again. This time it works.
• Spend hours retrying a 500G download from a rickety website riddled with buggy javascript.
• Realize you don’t have enough disk space.
• Go buy another external drive.
• Try to install the vendor software. It installs but does not work.
• Ponder why you started doing this in the first place. Perhaps pause for a few days until you recoup the will to brave this again.
• Try to install the vendor software. It installs, but crashes.
• Delete everything, try to install again.
• This time the install seems to succeed, but you didn’t install all the packages you need. Try again.
• Install seems to succeed. You start the vendor tooling.
• Your computer has too little RAM, startup lasts a small forever.
• Perhaps you buy more RAM? Perhaps you buy a new computer altogether?
• If you have a spouse, this is when you cue in an argument about spending too much on your toys.
• You buy a new computer. You also buy a 1T SSD just in case.
• The computer is dead-on-arrival. Return it, and wait.
• The computer arrives, and works.
• You reinstall all the software you need. Go back to the beginning of this workflow once again.
• Now the software GUI starts and boots.
• You realize the cat chewed through your cables.
• You cry. (Or, at least, I do. You may be made out of tougher cloth.)

What if things could just magically happen? Is such a thing even possible?

Yes

Why yes, yes it is. The video linked above shows you a FPGA programming workflow I developed, using nothing but either open source or freely available tooling.

The video shows me programming the Alinx A200T FPGA device, based on AMD (fka Xilinx) Artix-7 XC7A200T device, starting from a setup that contains no mission-specific software preinstalled.

The entire process is started and completed with a single command only.

And the board being programmed was hundreds of miles away from the build machine.

Obvious parts

That single command does a number of important things that happen automatically for you. There is no fiddling with any finicky piece of software. We started from scratch, on a machine that only has bazel installed.

No workflow that I am aware of today can compare.

• All the needed software is downloaded and hermetically set up for you.
  • This is taken from my work on hermetic, ephemeral and reproducible builds. I developed a special build process that ensures all needed software is pulled in, just once, before the actual build starts.
  • All the board definitions are prepared for you.
• All the required software toolchains are hermetically downloaded and configured for you by bazel (the build software).
• I understand as far as build software, bazel is fairly not popular. But it has unique qualities that I am using in this setup, which are hard to replicate with any other build system.
• All the required hardware toolchains are hermetically configured for you.
  • GHDL is downloaded and set up for you automatically for use in simulation and cross-language development.
  • Even vendor-specific software such as AMD Xilinx Vivado is prepared in the same way.
  • You do not need to do anything special but wait for the process to complete once.
• All the hardware definitions are built for simulation.
  • This includes testbenches with automated signal viewing setup, as well as unit tests that can be run at moment’s notice.
• All of the above code and setup are automatically repeated if you make changes that invalidate parts of the workflow.
  • Only the modified parts of the workflow are rebuilt.
• Once all the build environment is set up, however…
• All the soft IP definitions are built. You will notice that I have a fully functional RISC-V CPU connected via serial port.
  • All of the firmware is built using the hermetically downloaded and configured compiler.
  • The current firmware is configured just to send a title banner to the UART peripheral.
• The entire design is elaborated, then synthesized, then placed-and-routed using a dedicated set of constraints.
  • This includes merging the firmware with the IP definition so that once the board is programmed, it automatically starts doing the right thing.
• A bitstream is generated.
• Once the bitstream is ready, a remote programmer is started (see below for details).
• Once the device is programmed, you will see a greeting banner printed on the serial console, which up to now sat empty just below the compilation window. The pattern of LED lights on the FPGA has now changed to testify that the device has indeed been programmed. (Before we started with the workflow, the light pattern was “1 LED running off”, after that it is a “count to 7, repeat”.)
• Notice that the entire process from the start of software installation, to a programmed board, is just over 6 minutes long. Compare that to the ordeal described above.
• And if you modify any of the hardware, software or other IP related things, the entire process is refreshed, but only the parts that have actually been changed.

Less obvious parts

Remote build

You may not have noticed this.

The entire build process, including synthesis, happens on a virtual machine in Google Cloud Platform (GCP). I do this because I can size the virtual machine based on the requirements of the vendor software I must use. If I need more power, I can configure my virtual machine with more power, and continue. If I need more than a single machine, I can repeat the same setup on each one trivially.

For example: a few days ago, I used a 4-core machine with 16G RAM. Today, I used a 32-core machine with 64G RAM to speed place-and-route up. This is all completely transparent for you as the user.

Remote programming

You may not have noticed this either. The hardware FPGA board that was being programmed lives on my desk at home. While the build process happens on GCP, I use the remote programming approach I developed to apply a bitstream built in the Cloud to my device.

This happens automatically, without any special action on your part. The workflow simply does the right thing.

This remote programming approach is far superior to that described on AMD’s website itself. The approach they describe uses an unsecured connection to a co-located other machine.

My setup uses the secure and fully encrypted SSH tunneling across the US continent.

Vendored software installation

While vendored software installation is also made hermetic, I could not have it be prepared by bazel itself. This is for two reasons:

• The unclear status of sharing the installation does not allow me to publicize the actual hermetic setup.
• I can only publicize the recipe for making one, and expect that you should be able to prepare and load it yourself.
• Bulky installation.
• The process of loading a hermetic installation of vendored tooling requires a preposterous amount of disk space (500G+) and a preposterous amount of time (2-3 hours).
• Luckily this only needs to be done once.
• And in contrast to the frustrating process outlined above, it is completely automated.

Since the entire installation is in GCP’s Cloud, if you need more storage, you can add more storage in seconds.

However, barring the above issues, the approach is unchanged, for any piece of software you may need to use.

Remote everything

I have my FPGA powered by a smart plug. This means that I can control its power on and off remotely. With the GCP-based remote setup, I am able to do the following:

• Develop hardware no matter where in the world I am (so long as there is network connectivity)
• Create bitstreams ready for deployment anywhere.
• Program my device at home, regardless of where I am physically at that time, and regardless of where the GCP virtual machine is located.

Conclusion

I hope you can appreciate the streamlined process outlined above. Its interesting features are:

• Super-easy setup.
• Configurable hardware.
• Write anywhere, synthesize anywhere, program anything.

If you want to share your comments with me, consult the contact section at the bottom of my home page