How to install vnc server on memory limited VPS?

There have been enough articles on how to install vnc server on CentOS6, CentOS7, or ubuntu, debian. But all of those guides tell you to install a desktop such as gnome, kde, before installing vncserver. The first step of their instructions would be:

yum groupinstall Desktop

However, my linux box is a cheap OpenVZ vps with very limited resources. It has only 128M memory and even does not allow me to use swap(swapon /var/swapfile says “swapon failed: Operation not permitted”). In such memory-limited VPS, even the installation of Desktop can not be completed due to the out of memory problem. Even the command:

will fail due to OOM. If you check the log file /etc/log/message, you will see the yum process is killed because the rss memory exceeds the available memory in the system. The minimal desktop setup in this post or the installation of the basic-desktop group also fail with the lack of memory problem. If you can not get more physical memory or swap space, you must not install those xwindows and desktop packages. Fortunately, those heavy desktop or GUI packages are not  prerequisites of installing vnc server. You can use a simple x window manager called dwm,

How to install dwm? I could not find a repo to install dwm with yum, neither can I find a rpm to install. The only way to install dwm is by downloading its source code, building, and installing.

  1. yum install make automake gcc gcc-c++ kernel-devel
  2. yum -y install git libX11-devel libXft-devel libXinerama-devel
  3. wget
  4. tar xzf dwm-6.1.tar.gz
  5. cd dwm-6.1
  6. cp config.def.h config.h
  7. vi config.h to change,static const char *termcmd[]  = { “st”, NULL };
    to static const char *termcmd[]  = { “xterm”, NULL };
  8. vi, comment the line FREETYPEINC = ${X11INC}/freetype2
  9. make
  10. make install

Note that step 8  is necessary, otherwise you will get the error:

fatal error: freetype/config/ftheader.h: No such file or directory

Now you can install and configure the tiger vnc server using :

  1. yum install tigervnc-server
  2. yum install xorg-x11-fonts-Type1
  3. service vncserver start
  4. vncpasswd
  5. vi  /etc/sysconfig/vncservers to add the following lines: VNCSERVERS=”1:root”
    VNCSERVERARGS[1]=”-geometry 800×600″
  6. service vncserver restart
  7. iptables -A INPUT -p tcp –dport 5901 -j ACCEPT
  8. iptables -A INPUT -p tcp –dport 6001 -j ACCEPT
  9. service iptables save
  10. vi ~/.vnc/xstartup to change the last line from “twm&” to “dwm &”
  11. vi ~/.xinitrc to add the line: “exec dwm”
  12. service vncserver restart

Now the setup of vnc server is completed. As to the vnc client, there are plenty of free vnc programs such as tightvnc and commercial vnc programs such as realvnc.  The first consideration in choosing a vnc client is the security issue. Is tightvnc encrypted? I mean is the communication between the vnc client and the vnc server encrypted. Realvnc advocates the security in its white paper, while tightvnc clearly states it does not support the encryption(preciously, tightvnc only encrypts the password not the data afterwards) . (There is an enhanced tightvnc that supports ssl. )But after carefully reading the reviews and tutorials for tightvnc and realvnc, it seems both do not support the SSL encryption by default. But the good news is, we can resort to ssh tunnel instead of the vnc program itself for security/encryption support.   So my choice goes to tightvnc as I have not much money. The SSH tunnel is an interesting stuff. An ssh client such as putty connects to sshd server to make an ssl encrypted connection. The communication between vnc client and vnc server are thru this connection, thus encrypted.  I used putty as a terminal to login my vps, but never realized it has a build-in ssh tunnel function. The configuration steps of putty for an ssh tunnel are:

  1. Run putty, in the Session category, create a new session, specify the destination host ip and port as you do for normal login terminal.
  2. In Connection/SSH/X11 category, tick on the “Enable X11 forwarding” option.
  3. In Connection/SSH/Tunnels category, fill a Source port such as 5900, fill a destination like, click the “Add” button.
  4. Back to the Session Category, click “Save” to save the configuration for the session.

Now you can click the “Open” button at the bottom of putty window to log in the sshd server like you do for login terminal. At this time, putty becomes a server running on your local machine and listening on port 5900. You can run your vnc client now. For tightvnc, just fill “localhost:5900″ and click the “connect” button. tighvnc client now connects and talks to the putty server thru port 5900 on your local computer. Putty will transfer what it gets from the vnc client to the sshd daemon running on the remote server. When the sshd server on the remote vps gets the data, it will act as a client and connect to the vnc server also running on the remote machine and listening on port 5901(why 5901?  Because we set VNCSERVERS=”1:root” in /etc/sysconfig/vncservers. If we set  VNCSERVERS=”2:root”, the vnc server will listen on port 5902. To let sshd know it should connect to the port 5901, you must specify the destination port as we have done in the step 3 of the putty configuration.) If you use the command “netstat -antp” now, you will find sshd now becomes both a server connected by a remote client, and a client connecting to the vnc server on the same box.

Tightvnc client may report the error “connection has been gracefully closed” when connecting to vnc server through ssh tunnel. This error indicates the ssh tunnel is working normally. You can use “netstat -antp” to see the connections from vnc client and putty, and from sshd to vnc server are established both on your local computer and on the vps. The error is caused by the vnc server, for example, the window manager does not work properly. You should check  /etc/sysconfig/vncservers, ~/.vnc/xstartup,and ~/.xinitrc to verify a correct window manager has been specified and is working.

If everything is ok, soon after connection, tightvnc client will ask you for a password, this is the password you set with the vpnpasswd command, not the Linux system password for the user. If the password is correct, the exiting GUI window will pop up in front of you.

You can see installing vncserver on CentOS with small memory is not easy because main-stream x-window system and GUI desktops consume much memory. If possible, add more physical memory, or at least add some swap space, you will be more comfortable enjoying the modern GUI environment.  And you can follow this post  to set up your GUI environment.









Posted in tips of hosting