Vmware ESXi NAT Networking

The case: You have multiple virtual machines that you don’t want to expose directly to the internet and you prefer routing them through a single virtual machine. This is also handy if you are short on IPs. Our case is OVH specific and it will give you a guide on how to set things up on OVH weird network topology. Reason why we use ipcop instead of pfsense is because pfsense doesn’t support a 32bit net mask. No matter what i’ve tried i could get it to work with a 32 bit netmask and gateway on separate network. So IpCop was the next best thing and it does the job quite well.

ESXi Configuration:

Login to your Vsphere client and go to Configuration -> Networking.

On Network Access select “Create a vSphere standard switch” without select any vmnic .

Proceed to Connection Settings and name your new nic under “Network Label” . “Vlan ID” select None.

Proceed on creating a new VM including both NICs the one that you just created and the one that connects you to the outside world.

Then move on with standard installation of IPcop. Keep in mind that your RED network will be the one connecting outside and the GREEN one will be your internal nat network. If you are not sure of the network settings while installing proceed and will edit network config later on anyways.

After installation login to the machine and now let’s edit the files needed for proper networking under OVH routers.

vi /var/ipcop/ethernet/settings

DEFAULT_GATEWAY=Dedicated Server IP but end in .254
DNS2=8.8.4.4
DNS1=8.8.8.8
RED_1_NETADDRESS=IP Failover
RED_1_NETMASK=255.255.255.255
RED_1_ADDRESS=IP Failover
RED_1_BROADCAST=IP Failover
GREEN_1_NETADDRESS=192.168.1.0
GREEN_1_NETMASK=255.255.255.0
GREEN_1_ADDRESS=192.168.1.1
GREEN_1_MAC=Your Virtual Mac Address
GREEN_1_DRIVER=pcnet32
GREEN_1_OPTIONS=
GREEN_1_DEV=lan-1
RED_1_MAC=Your Virtual Mac Address
RED_1_DRIVER=pcnet32
RED_1_OPTIONS=
RED_1_DEV=wan-1
RED_1_TYPE=STATIC
ORANGE_COUNT=0
BLUE_COUNT=0
RED_COUNT=1
GREEN_COUNT=1

vi /etc/rc.d/rc.event.local

#!/bin/sh
route add Dedicated Server IP but end in .254 dev wan-1
route add default gw Dedicated Server IP but end in .254

Reboot and you are done! You can now login to the web interface under https://192.168.1.1:8443

Advertisements

#esxi, #ipcop, #nat, #ovh, #pfsense, #virtual-machine, #vmware-2

Migrating VMware ESXi

Hello.
Recently one of our VMware ESXi servers started acting up weird.
Situation was that we had no means of accessing ESXi through SSH ( user/pass got refused although it was the right one) and the console was also unaccesible for each VM.
After invastigating a bit through the Vsphere Client it appearead that the system couldn’t find the necessary pam modules and thus no authentication for us.
Hopefully at least we had access through the VSphere Client and also to the Web-Based datastore Browser.
Anyway to make long story short we had to find a way of getting our VMs from there and moving them to another machine.
Typically and afaik we would simply login on our ESXi console and scp our files through the servers. In our case that was no option since we had no ssh access at all. So we came up with two solutions.

General Prerequisites 

  1. Power off your VM while transferring.
  2. Make sure there are no snapshots.
  3. If you got snapshots  take care of them and delete. You can’t move snapshots and you have to have only “the current working branch” .

Transferring (Having a shell access)

  1. scp [[user@]from-host:]source-file [[user@]to-host:][destination-file]  
  2. Example: scp root@myno1.esx.com:/vmfs/volumes/datastore1/linux/linux.vmdk root@myno2.esx.com:/vmfs/volumes/datastore1/linux

Transferring (Alternative way, no ssh required)

    1. I’ve found this very handy little script http://blogs.vmware.com/vsphere/2010/01/scripting-datastore-access.html . What it does is accessing your datastore through the web interface and downloads to local .
    2. So to make things faster we can have this script download the files on a remote box and then transfer from there to our new ESXi server.
#!/bin/bash

CURL_ARGS="--insecure"
# Change if you want to use an alternate user
# (you'll be prompted for the password each time)
USER=root
if ! $(which curl > /dev/null); then
echo "ERROR: curl not found in your path" >&2
echo "" >&2
echo "You'll need to install curl on your system for this script to work." >&2
exit 1
fi
usage() {
echo "USAGE: $0 <get|put> <hostname> <ds> <ds path> <local_path> [thread_count]" >&2
echo "" >&2
echo "one of source or target must be a datastore path" >&2
echo "Example: $0 put hostname datastore /file.iso ./file.iso'" >&2
echo "" >&2
echo "If you set the thread_count then this script will" >&2
echo "use that many parallel threads when downloading the file." >&2
echo "Warning: you'll need ~2x the file size in available space locally for this approach" >&2
}
urlescape() {
TMP=$(echo $1 | sed \
-e ' {
s/%/%25/g
s/ /%20/g
s/</%3C/g
s/>/%3E/g
s/#/%23/g
s/{/%7B/g
s/}/%7D/g
s/|/%7C/g
s/\\/%5C/g
s/\^/%5E/g
s/~/%7E/g
s/\[/%5B/g
s/\]/%5D/g
s/`/%60/g
s/;/%3B/g
s|/|%2F|g
s/?/%3F/g
s/:/%3A/g
s/@/%40/g
s/=/%3D/g
s/&/%26/g
s/\$/%24/g
}' )
echo ${TMP}
}
if [ $# -lt 5 ] ; then
usage
exit 1
fi
OPERATION=$1
HOSTNAME=$2
DATASTORE=$(urlescape "$3")
REMOTE_PATH=$(urlescape "$4")
LOCAL_PATH=$5
if [ $# == 6 ] ; then
THREADS=$6
else
# Default to 4 threads for better performance
THREADS=4
fi
URL="https://${HOSTNAME}/folder/${REMOTE_PATH}?dcPath=ha-datacenter&dsName=${DATASTORE}"
if [ "${OPERATION}" == "get" ]; then
if [ ${THREADS} -gt 1 ] ; then
echo -n "Enter password for ${USER}@${HOSTNAME}: "
stty -echo
read PASSWORD
stty echo
echo ""
# Note: This is somewhat insecure as the password will show up on the
#    command line. Consider switching to use netrc or SSPI
#	   see the curl man page for more details.
#
#    This first curl invocation grabs just the header to get the size
LENGTH=$(curl ${CURL_ARGS} -s -u "${USER}:${PASSWORD}" "$URL" -I | awk '/Content-Length:/ { print $2 }'|sed -e "s/\r//g")
CHUNK=$((LENGTH / THREADS + 1))
START=0
echo "Starting download (${THREADS} parallel threads)..."
COUNT=1
while [ ${COUNT} -le ${THREADS} ] ; do
END=$((START + CHUNK - 1))
# Progress reporting with multiple threads gets jumbled up, so be silent
curl -s ${CURL_ARGS} -u "${USER}:${PASSWORD}" --create-dirs --range ${START}-${END} "$URL" -o "${LOCAL_PATH}.${COUNT}" &
COUNT=$((COUNT + 1))
START=$((END + 1))
done
wait
# Merge the files back together
rm -f "${LOCAL_PATH}"
touch "${LOCAL_PATH}"
COUNT=1
while [ ${COUNT} -le ${THREADS} ] ; do
cat "${LOCAL_PATH}.${COUNT}" >> "${LOCAL_PATH}"
rm -f "${LOCAL_PATH}.${COUNT}"
COUNT=$((COUNT + 1))
done
echo "Done"
else
curl ${CURL_ARGS} -u ${USER} --create-dirs "$URL" -o "${LOCAL_PATH}"
fi
elif [ "${OPERATION}" == "put" ]; then
if [ ! -f ${LOCAL_PATH} ] ; then
echo "ERROR: ${LOCAL_PATH} does not exist" >&2
echo "" >&2
usage
exit 1
fi
curl ${CURL_ARGS} -u ${USER} -T "${LOCAL_PATH}" "$URL"
else
usage
exit 1
fi

Configuration of the VM to the new ESXi

  1. After having transfer all our files ( *-flat.vmdk *.vmdk *.vmx) we now have to create a new VM on the new server.
  2. Create a new VM using the exact same options as on the old ESXi.
  3. Choose Advanced options and select an existing hard disk.
  4. Finsh and boot.

Final Configuration

  1. Since you are now propably on a new network you need to reconfigure your network and your VMAC address.
  2. Create a VMAC address from your host panel.
  3. On CentOS goto /etc/udev/rules.d/70-persistent-net.rules and edit making sure eth0 uses the newly created VMAC.
  4. Goto /etc/sysconfig/network-scripts and edit ifcfg-eth0 accordingly with your new ip, gateway and hwaddr.
  5. Edit /etc/sysconfig/network-scripts/route-eth0 and configure your gateway.

 

Et voila! You have now succesfully migrated ! 🙂

Suggestions or alternative methods are really appreciated 🙂

#centos, #datastore, #esxi, #esxi4, #esxi5, #migration, #moving, #vmware-2