I faced this issue today when starting RubyMine. And going thorugh Ruby Mine manuals https://www.jetbrains.com/ruby/help/tuning-rubymine.html I figured it out how to fix this.
Right click on you Ruby Mine shortcut icon and click on Open File location just to reach to the folder which contains the Ruby mine .exe
Now open rubymine.exe.vmoptions edit -Xmx to 512m
Now start ruby mine and the error should be gone and it should open fine.
Hope this helps.