I'm going to increase JAVA Heap Size for VirtualServiceEnvironmentService.
This is only DevTest Server component which is run on this host.
Are there any guidelines what this heap size should be when virtual machine has 8GB RAM?
As per JDK/JRE recommendation, you can allocate maximum of 80% of RAM to the resources running on system.
It also depends on the operating system process, number of services which you will deploy on this VSE, etc.
If your system is 64-bit process and VSE alone is running on this system, you can initially try by allocate memory between 1GB to 2GB by adding below settings in VirtualServiceEnvironmentService.vmoptions file and restart the component.