Solved: Netscaler VPX (on VMWare) Problem.

During my latest project I ran into a problem where Netscaler VPX stops responding to any sort of request.

  • Logon to Netscalers management interface: Netscaler immediately stops responding.
  • Connect to Netscalers Access Gateway VPN server: Netscaler immediately stops responding.
  • Connect to Netscalers Access Gateway Logon Interface: Netscaler immediately stops responding.

The state of the netscaler is as follows:

  • Netscaler is no longer reachable via management Interfaces (SSL/SSH, etc)
  • Netscaler is not replying to ping requests (to NSIP)
  • Netscaler IS reachable and response via VMWare console

Via WMWare console the netscaler is responsive and when issueing command “Show Interface” netscaler responds by listing al it’s network interfaces. I thing I noticed that the interface of the NSIP was shutdown because of administrative reasons (cannot recall the exact message). When enabling this Interface with command: enable interface 0/1 everything seemed to be working again until you try one of earlier mentioned actions.

If you experience this, there is a good change that the VMWare server on which the Netscaler VPX is running was upgraded with patches from VMware ESXi 5.5.0 U2 both VMWware and Citrix Have released KB documents about this issue:

from VMware document we learn more about the issue:

  • This issue occurs when the NetScaler virtual machine driver resets TDT to 0 after 511 while the TX ring size is shown as 1024.
  • This is not a VMware issue. To resolve this issue, upgrade the NetScaler appliance.

According to Citrix there are 3 workaround

  1. Revert to a non updated VMWware host (not recommended)
  2. Upgrade to NetScaler 10.5 build 55.8 or above (recommended if possible)
  3. change the TX ringsize (last option if no other option works out)

from the document we can extract the procedure:

  1. SSH and log on to Citrix NetScaler VPX appliance as nsroot.
  2. Type shell.
  3. Change directory (cd) to /flash/boot.
  4. Create file /flash/boot/loader.conf.local (if not present) with same permissions as /flash/boot/loader.conf. Add the following line and reboot:
    Note: To create the file, use command touch loader.conf.local.

vi Commands

The following are the vi commands to edit the document:

  1. From NetScaler shell type:
    vi <filename>
  2. Move the cursor to the last character of text in the file, type “a” and click Enter.
  3. Type the line:
  4. Press the ESC key and then “:” key. The cursor will move to the bottom of the page, then type wq!.


After this procedure reboot the netscaler and all should be working fine again.


Solved: XenApp 5 HRP 7 results in 0x000000CF BSOD

While installing a new Citrix XenApp  farm for a customer I upgraded the servers to Hotfix Rollup Pack 7 (HRP7, PSE450W2K3R07). The XenApp 5 farm is build as virtual servers on a  VMWare vSphere 4.1 cluster. Installing Hotfix Rollup Pack 7 via RDP console session (mstsc /console) succeeded  with no error and the farm booted nicely after installing the HRP.

The troubles started when I tried to connect to the farm via an ICA connection, only few seconds after initiating the connection the selected server crashed with an 0x000000CF (TERMINAL_SERVER_DRIVER_MADE_INCORRECT_MEMORY_REFERENCE)BSOD. After the BSOD the server rebooted nicely but an ICA connection was never setup.

Investigating this problem led me to some specific solutions to the problem like:

  • Copying an older version of twexport.sys to “C:\Program Files\Citrix\Drivers” and “C:\Windows\System32\Drivers” [Source]
  • Installing hotfix PSE450R07W2K3011

but none of them solved my solved the problem. Eventually the problem was solved by installing HRP7 NOT via RDP BUT via the VMWare console

Citrix accuses VMWare of lying. (updated)

What an interesting start of the week, today Citrix’s desktop CTO Harry Labana wrote a blog post accusing VMware of lying while refering to a Gartner Report about the TCO of SBC comparing to (un)managed desktops. In this article about VMware’s View, the company refers tho the report and insinuates that VMWare View incorporates SBC technology (which it doesn’t).

As you might know, one of the big differences between VMware’s and Citrix’s desktop virtualization solutions is that  Citrix XenDesktop incorporates VDI/Desktop virtualization capabilities but also includes XenApp which delivers shared Terminal Server-based desktops and applications. While VMware View only incorporates  VDI/desktop virtualization.

Therefore their press release is inaccurate and subject to rectification, in my opinion.

Stay tuned for more….

UPDATE: Brian Madden also picked up the story and has some background information, he also responds to the reaction from VMWare.

The recent joint announcement between Wyse and VMware, on February 9, featured a quote by Gartner looking at the TCO benefits associated with server-based computing (SBC). The Wyse portfolio of thin, zero and cloud PC client solutions supports both SBC and VDI. It is appropriate for Wyse to choose the feature this when talking about their products. VMware’s portion of the announcement featured customer momentum and results related to our portfolio of desktop and application virtualization technologies.

Who knows if it was on purpose or an oversight? They’ll claim it was a mistake. The conspiracy theorists will believe otherwise. If you asked me over a beer I’d tell you that I don’t believe they did it on purpose, but that it was not wise to respond with the statement they used. Instead they should have put a new quote with VDI-specific data in it and reissued the press release. Then they’d be done. But now we’re left with a release where TS is doing the heavy lifting to power the “success” of the TCO savings of VDI. And that’s exactly what I accused them of doing three years ago, which I wish was a thing of the past.

Also Harry Labana responded on his blog to VMWare’s reaction:

Fundamentally VMware is trying to defend an inaccurate press release. After a history of getting away with elastic facts, getting caught twice, the appropriate thing to do would be to retract the statement and claims of SBC having anything to do with VMware.