Does something like a 1 to 2 Ethernet cable sort of device exists ? Searched earlier on Amazon but it's never clear what their product is used for
No, the only things you'll find is a toggle to switch between the two. It's not possible to split the connection.
You can buy a little 4 port switch like this. Plug ethernet from the router into this and then both computers can take a port from this.
Note: The speed will be split between both computers based on usage. If they're both downloading something, it'll split it evenly. If one computer is idling, the other could have full speed.