Latency, the delay before the transfer of data, is one of the critical user experience measurements for many technology solutions, and one of the top when judging the quality of end user experience on Citrix Virtual Apps and Desktops.
- Citrix Workspace Slow
- Citrix Workspace Online
- Citrix Workspace Slowest
- Citrix Workspace Va
- Citrix Workspace Troubleshooting
Edit: I was referring to Citrix Workspace sorry. I am using Citrix receiver in Windows 10. I have a pretty decent internet service, but my connection once im logged in to Citrix is painfully slow. I am using Mcafee as a Firewall, and access seems to be allowed. Net Guard is active doh. There’s Citrix Workspace, Storefront, Application Delivery Controller (ADC), Delivery Controller, License Server, Virtual Delivery Agent (VDA), Secure Ticket Authority (STA) server and so on. Imagine this: Let’s say one of your users reports a slow login.
Why is Citrix Slow?
Have you ever received a “Citrix is slow” user complaint? I will expect the answer to be yes. It is very common, even more so for remote workers. The reason Citrix is slow might not be related to high Citrix ICA latency at all, but there is a good chance it is. You see, there are multiple reasons why latency could be high for a user remotely connecting to their Citrix app or desktop using something like Citrix Gateway or full VPN, as such:
- User is connected to WiFi but the signal/coverage is poor.
- User is connected to public WiFi which has a bandwidth cap, or contention from other people sharing the same connection.
- Local device network cards or wireless adapters running old, outdated drivers, or there could be hardware faults impacting network performance.
- Packet loss experienced due to old or faulty broadband equipment that needs to be refreshed by the broadband provider.
- Local device running low on RAM and available CPU, causing lag to ICA session.
- Local broadband overloaded by other household members streaming videos or playing online games.
- Overall public internet overloaded due to increase in usage caused by COVID stay-at-home requirements.
To help gauge user experience, Citrix latency is defined by two metrics:
- ICA RTT – This is ICA latency and Thinwire latency, aka the time between the user clicking or typing into something on a virtual app or desktop, and the moment that action is graphically displayed to the end-user. For example, clicking on a menu drop-down within an application, or typing into an Outlook mail window. This can be more easily known as screen lag. Application performance could increase ICA RTT.
- Latency – Otherwise known as ICA Latency, this metric captures the bottom-line network latency and ICA stack latency between the client and VDA server. For example, latency between client-side Winstation Driver and VDA Winstation Driver. Think similar to a ping test. System performance could increase ICA RTT and ICA latency.
How does ICA round-trip latency impact the user experience?
When ICA round-trip latency is high, the end-user may notice a delay when they click or type within their ICA session. Citrix technologies are quite good at controlling latency and minimizing it where possible with features like Adaptive Display and Adaptive Transport, but there will be times when the latency is too high and the end user experience starts to become impacted. It is at these times when you may get those “Citrix is slow” complaints and have to start some investigations.
Remember that many of your remote workers are accustomed to working from the office, where they receive a low latent connection to their virtual apps and desktops. This results in the end-user having a certain expectation for the performance of their session. They know the experience they normally get when working within the office, and they expect the same experience when connected remotely.
Citrix Workspace Slow
Now that we have some metrics from Citrix, we can easily look up the affected user session and immediately understand ICA RTT for that session. However, what do the numbers mean?
Everyone will have their own perception of good and bad latency, and personal opinion mixed with experience will determine what the numbers mean to you. Also, depending on the application and desktop in use, latency may not impact users the same way.
I typically like to categorize the numbers into Great, Good, and Poor. The numbers I use to categorize are estimated, so the categorization is more general. For example, you could adjust the numbers by 30ms either way, but from my own testing and experience I would categorize them as follows:
- Great: Typically, anything between 20-180ms.
- Good: Typically, anything between 180-240ms
- Poor: 240ms+
The challenge is finding what causes the latency. Is it the user’s home network, Citrix Gateway, internal network, VDAs, high CPU, etc? If you want to find the root cause, you always need more information rather than having numbers thrown at you. Unfortunately, most of the time, Citrix’ internal tools, like Citrix Director, will not provide enough information to understand what is causing the latency. This is when you need to turn to other products, specifically purpose-built to monitor Citrix. Solutions such as Goliath Performance Monitor can now only show the numbers, but also explain the reasons for them.
Make Citrix Troubleshooting Easy
With Goliath, IT administrators, or even the support desk, can find any existing or historical user session. Now when the admin begins to investigate a ‘Citrix is Slow’ ticket, they can navigate to that user session and quickly identify if that user is experiencing high latency (Image 1).
Then by clicking into that user session, you as an IT admin can see every IT element that could impact that end user’s session. From the user’s endpoint, to logon details, to connection and user behavior, to all Citrix and supporting infrastructure components. Knowing the user has high latency, you can click into the ICA/HDX tab. Here you can now see additional correlated metrics you don’t get with Citrix alone, that prove root cause of the slowness. In the example in Image 2, this user has both high latency and network latency above 240ms and in that same period slow connection speed well below 10mb/s. As an admin, you can draw the conclusion and prove to the user the slowness is an issue with the connection point before reaching Citrix Gateway. If all else looks good in the data center, root cause is most likely an issue with the user’s home ISP.
Goliath offers additional insights into a variety of ICA channels that can help IT Pros identify and prove root cause from audio or graphical files consuming too much bandwidth, a large print job stuck in queue, or even if a large clipboard file exists, all potentially negatively impacting user experience. When it comes to troubleshooting remote worker slowness issues, latency is to confirm the issue is present, but as an admin you need additional correlated, metrics to prove root cause and resolve.
See how Penn National Insurance used key metrics to quickly troubleshoot remote worker experience issues.
Source: Giphy
Citrix Workspace Online
When users complain that they are experiencing slowness in their VDIs, there are some points to check within in citrix.
If you have Citrix ADM (Application Delivery Management) installed and configured, you should see users sessions and bandwidth details in ADM. You may see high ICA RTT even though users say that they have very good internet plan. They always try to download a file or stream a YouTube video in HD and say, my internet is good. VDI has issue. It is slow.
Lets see what factors contribute to ICA RTT. Citrix Article, CTX204274 says:
- Client OS introduced delay – Client OS meaning – Operating System in user’s physical laptop/ personal laptops/ipad/Mobile etc ie., Device OS where user is connecting from.
- Client to NS introduced network delay(Wan Latency) – User’s physical laptop/ personal laptops to Netscaler.
- NS introduced delay in processing client to NS traffic(Client Side Device Latency) – It is Netscaler processing traffic, it may show as high if Netscaler is running on high cpu and memory. Because Netscaler is the one that processes it. If Netscaler is running at normal resource utilization, it shows correct value.
- NS introduced delay in processing NS to Server (XA/XD) traffic (Server Side Device Latency) – Self explanatory
- NS to Server network delay(DC Latency) – Self explanatory
- Server (XA/XD) OS introduced delay (Host Delay) – Self explanatory
DC stands for DataCenter. DC latency consists of all the switches/routers/firewalls etc that traffic goes through to get from the VDI to the internet.
From Citrix blog, How-network-latency-impacts-user-experience average network latency values are:
- Up to 150ms: great user experience
- 150ms – 300ms: good/acceptable user experience
- Over 300ms: degraded user experience
Some more details from Docs.citrix.com site:
Citrix Workspace Slowest
| Metrics | Description |
|---|---|
| # Active Sessions | This number indicates the count of active Citrix Virtual Apps and Desktops sessions. |
| # Active Apps | This number indicates the count of active Citrix Virtual App sessions. |
| ICA RTT | ICA RTT is the screen lag that the user experiences while interacting with an application or desktop hosted on Citrix Virtual App or Desktop respectively. |
| WAN latency | Latency caused by the client side of the network. That is, from Citrix ADC to end user. |
| DC latency | Latency caused by the server side of the network. That is, from Citrix ADC to backend servers. |
| Bandwidth | Total bytes per second taken for end to end communication during the selected time interval. |
| Server Side Retransmits | The number of packets retransmitted on the connection between Citrix ADC and backend server. |
| Client Side Retransmits | The number of packets retransmitted on the connection between Citrix ADC and the end user. A high value of this metric does not mean that the user experience will not be seamless but indicates high bandwidth utilization due to retransmits. |
| Client side fast RTO | Number of times the retransmission timeout occurred the connection between Citrix ADC and the end user. |
| Server side fast RTO | Number of times the retransmission timeout occurred on the connection between Citrix ADC and backend server. |
| Client side Zero Window size event | This counter indicates the number of times the client advertised a zero TCP window. |
| Server side Zero Window size event | This counter indicates the number of times the server advertised a zero TCP window. |
Citrix Workspace Va
Common troubleshooting:
- Tracert from user’s laptop and tracert from working laptop. Compare results.
- Check that VDI or app server’s resource utilization cpu and memory.
- Change user’s wifi to mobile hotspot and test.
- If one location is facing slowness, and other location is fine, ask user from working location to have screenshare with user from affected location. Over webex/teams etc.. Affected user should connect to their vdi or app from laptop in working location and check the results.
- If it is an application slowness, check how many users are accessing the app ie., load on the app server. If cpu/memory are high, it may lead to application slowness.
- Check hypervisor storage/CPU/memory where the app servers are hosted on.
Citrix Workspace Troubleshooting
Check ddc and storefront vips, gateway vips. Check storefront and ddc memory and cpu usage. Check rsa vip.