While it's running, have you checked docker stats to see how much memory/cou the container is using? What's the host, what're it's total resources and what are you using to run the vm?

You may get more traction asking in the communities that exist for those tools: IntelliJ and Docker

8GB for two separate IntelliJ projects sounds low. You could try importing both into one instance as separate "modules" so that there is only one IntelliJ instance/window.

Depending on how you are running the VM, the host may be choking it through the host OS and leading to OOM. Especially with a tool like docker.

Edit: I see you commented usage of windows, you may need to look into wslconfig

Are you running the project on an OS in the VM? Because the 8GB assigned to the VM will be used by the OS and anything else running in it?