XAMPP is a package that installs Apache, MariaDB, PHP and Perl onto Windows, Linux or Mac. It is a very simple way to get a web development environment up and running with little knowledge.
Virtual Hosts are used in Apache to allow a single web server to serve numerous websites. Basically www.FailedNormal.com point to /var/www/FailedNormal, and www.EliTheComputerGuy.com points to /var/www/EliTheComputerGuy.
VPS , Virtual Private Servers, are simply virtual servers that are hosted by vendors such as Linode and Digital Ocean.