Hi everyone. This is my first post on Lemmy. I'm a senior Computer Engineering student. And I'm working on my senior design project with my friends.

  • BACKGROUND INFO

My friends have basically near 0 hardware knowledge but are willing to work. I have more experience with computer architecture and digital electronics but still I never made something as complex as this before.

The project is a comparison between hardware solutions to image processing. We will implement 3 types of hardware to process live (or pre recorded) video and output it to vga. Then compare their efficieny.

First one is a CPU core (will implement in system verilog which I have some experience with thank god). It will support RV32IM instructions and my friend will write an assembly program to control it.

Second one is an extension to the first CPU core.

Third one will be a specialized hardware for this purpose.

We have 2.5 months to complete half of the project and the other parts will be completed in next semester.

  • ACTUAL QUESTION

So my plan is to create the CPU and measure its performance in these months and leave the other parts for next semester.

I'm planning on creating a 5 stage pipelined in order CPU. I also want to use wishbone since it's easier. But I have a problem.

First is the fact that videos take up a lot of space and we don't have enough RAM (Basys 3). We can buy an Arty S7 50 for it's DDR3 but the controller for it is AXI. I don't have time to learn AXI and I could not find any open source bridges that would work between that and WB B3.

One other option might be to send and process the video from an external camera or my laptop. But how can I make this work? USB is not nearly fast enough for 640x480 24fps.

Note: We won't use any AI in this project. So the time is more crucial.

Any tips? Should we buy a new board or will streaming work? Maybe we can use ethernet port? Idk.

Thanks.

you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 8 hours ago* (last edited 7 hours ago)

Outputting to VGA with an FPGA is fairly easy, but depending on what you mean by "image processing", this may be too big of a project given you're the only hardware engineer or maybe all you mean is to take the video input and output it to VGA without any further processing. You did mention wishbone, so I guess if you're planning to reuse a bunch of Open Cores, that could simplify some things.

How much RAM do you have on your dev board? How long of a video do you want to do?

USB2 is fast enough for uncompressed 640x480 @ 24 fps @ 24bit color: 640 * 480 * 24 * 24 = ~177Mbit/s (plus misc overhead) USB isn't so bad to get working on an FPGA dev board (I didn't see you mention which you're using) as it probably has a transceiver chip already along with drivers available. Even if not, you could add a cheap USB interface chip (FTDI makes some popular ones, but there's many others like CP2102, etc). I'd not try to do more modern USB (3+) as it's not needed, but also higher speeds bring more problems. I'd actually try lower quality video (lower res, framerate, and not 24bit color). As you increase the amount of data you need to transfer, you have more potential for issues related to buffering the video (and the extra memory required) or timing requirements (ensuring the video is arriving by the time it needs to, but not too much before).

If I'm understanding your main concern, it's how to get the video data in to your hardware? Generally:

  • camera module (that outputs uncompressed image/pixel data, idk how difficult these are to get running, drivers often abstract a lot of complexity)
  • video stored on memory chip like an eeprom (may be impractical given size)
  • storage device like SD, compact flash, or even an IDE hard drive (idk how difficult SATA is to work with). if you go this route, I recommend avoiding file systems.
  • stream over USB2 from a laptop (whether it's pre recorded file, built in camera, some USB camera or capture device kind of doesn't matter from your hardware perspective)

Other communication interfaces (like ethernet) are probably going to be a lot more effort although I've seen a serial to ethernet interface that worked similarly to those UART <-> USB chips. idk cost or how easy they are to use though.

EDIT: I see dev board is Basys 3 (I didn't realize that's what that was, I thought that was a camera). With only 1.8Mbits of memory, you're going to need to add memory (could be complicated), drastically reduce video quality and/or have some pretty strict timing requirements for the image data entering your hardware and maybe a camera module is the way to go at that point (vs USB from laptop, but I'm thinking flash memory like SD card is going to be more practical). You had mentioned designing a CPU, so I assume you intend to run software on this, which is also going to need to use memory. Depending on what that software wants to do, adding more memory may be a hard requirement. If you're just taking the video and outputting to VGA, you wouldn't even need the CPU and could probably get it to work without extra memory, but without being able to buffer much video data, you still have to ensure your video input meets timing requirements and I'm leaning away from streaming from a laptop due to that I thought you'd be able to buffer more data.

Also this dev board only does 12bit VGA output, so you can reduce your video color bit depth, which will help some.

  • source