TrueDEM: A Paradigm Shift in Digital Experience Monitoring

In today’s fast-paced digital world, where hybrid work is becoming the norm and skilled professionals are in high demand, organizations recognize the need for exceptional user experience to boost engagement, satisfaction, and productivity. OfficeExpert TrueDEM is leading the way in Digital Experience Monitoring (DEM) for Microsoft 365, setting a new standard for seamless user experience monitoring with a new and more effective way to monitor and troubleshoot your user’s Microsoft 365 user experience.

Giving you the tools to adequately and quickly address problems, prevent unnecessary helpdesk calls, pre-emptively alert users of (impending) disturbances, and provide your experts and third-party contractors with the information they need to take action quickly.

Traditionally the DEM spectrum is dominated by products based on or using one or more of the following monitoring philosophies:

This large number of vastly different technologies has led Digital Experience Monitoring (DEM) down an interesting path. Research organizations have changed the definition and description of what DEM is multiple times. Even the naming isn’t consistent with Digital Experience Monitoring (abbreviated to DEM or sometimes DXM), Digital Employee Experience (DEX) and End-User Experience Monitoring (EUEM) being coined by various analysts and providers.

But changing the name itself would never solve the underlying problem that none of the above philosophies in themselves cover what DEM / DEX / EUEM in a modern environment should be.

As we examined the philosophies, we discovered that they were often siloed and inadequate in providing a comprehensive DEM view in a modern digital world that is dominated by two groundbreaking developments: SaaS and an unprecedented increase in hybrid work. These developments require a unified and comprehensive view of the entire user journey, which the traditional methods often fail to deliver.

Even Gartner® recognizes this in their 2022 Market Guide for Digital Experience Monitoring by Mrudula Bangera, Padraig Byrne, and Gregg Siegfried. Stating: “There is little effort to triangulate observation and understanding across multiple data ingestion technologies and no cross-channel measurement of user experience”.

Regular discussions with partners, customers and their teams of experts showed that traditional methods consistently fell short in delivering true digital experience monitoring for modern organizations. To avoid this pitfall, panagenda designed OfficeExpert TrueDEM from scratch, constantly challenging the status quo of existing methods.

Launched worldwide in July 2020, with enhanced capabilities for organizations using Microsoft 365, OfficeExpert TrueDEM is centered around a simple yet critical question: “Is everything all right and if not where is the problem?” To answer this question, we rely on three pillars that help us get to the root of the problem:

By leveraging these three pillars, OfficeExpert TrueDEM delivers a comprehensive and unified view of the user journey, allowing organizations to proactively identify and address issues before they impact productivity and user satisfaction.

The TrueDEM technology puts the user at its center, and it does so for a reason. In the past, it was relatively easy to generalize the digital experience as users were issued standard devices, worked in offices on dedicated, IT managed networks, and worked with applications fully controlled by their own IT. A single ‘canary test‘, where only a subset of users was monitored, was often enough to provide insight into the experience of an entire office or department because all used the same infrastructure, device types, networks and servers. However, those days are long gone.

Today, with hybrid work and global talent shortages, users are no longer restricted to a specific device, network, or location. For example, a user may check their email in Outlook and catch up on chats in Microsoft Teams while using a coffee shop’s free Wi-Fi. Later, they may join a call using their home network and a headset. Stop, and consider the number of ‘hidden’ variables this one example already injects into a support discussion about network and devices.

For example, Microsoft directs users to the best data center based on current load, maintenance cycle, even the application being used. And each ISP will take a different journey to reach this dynamically-selected Microsoft data center. This means that not only the user’s network and devices change, but that the Microsoft 365 data centers where their requests are routed through may also differ depending on the location of their data and the load on those data centers at the time.

So, the only true way to say anything about a user’s digital experience is to monitor every individual user. Because you can’t monitor user experience without monitoring the user – it’s in the name after all.

For every user, every situation, and every location the expected normal can differ and must be determined based on historical data. When you properly evaluate each contributing factor for each real user, you can generate ‘normal’ ranges for sub-components in a journey that no one thought to evaluate. And once you know what the expected normal is, you can start seeing when numbers are starting to trend downward, often even before the users or service providers themselves notice.

Secondly, it also allows you to verify if the user is truly experiencing a problem or simply having a cranky day because the local coffee-shop just ran out of his favorite coffee blend.

Imagine if you could identify delays in a given Microsoft 365 App, regardless of the user’s home ISP, coffee shop, headset vendor, and so on. No need to imagine, OfficeExpert TrueDEM can. And it grants you the ability to pro-actively alert users, take action or inform the responsible parties.

In modern digital monitoring, the risk of analysis paralysis is significant. Simply collecting more data or metrics with different tooling or technologies from different parts of the user journey doesn’t necessarily result in better insights. In almost all cases, it has the exact opposite impact. The problem is compounded by the fact that data from different tooling is often misaligned. This makes it difficult to pinpoint the actual cause among multiple red flags that can show up in the different data sets, leading to finger-pointing between departments and costly research by expert staff to weed out false positives.

Instead of causing this type of data confusion, OfficeExpert TrueDEM is designed to collect specific metrics from each part of the journey that are designed to be aligned and then used to deduct the Area of Responsibility. Allowing you to immediately address the problem with the right experts or parties and provide them with the right insights and data.

Lastly, to truly derive benefits from a DEM system, it is crucial that it provides accurate insights into the exact impact of issues.

For instance, if an ISP network is faltering, the system should be able to identify the specific offices and users affected. Similarly, if a SaaS data center is experiencing issues, it should be clear which users are routed through that center. And if a particular version of a Microsoft Teams Client is straining device CPU, the system should indicate which device types are most affected. Additionally, the system should provide information on alternatives if certain parts of the SaaS service are faltering. For example, if the OneNote desktop client is experiencing problems, it is essential to be able to know the status of the browser-based OneNote version so that you can advise users to switch to it temporarily.

Having this type of information and being able to pro-actively act and alert on them can eliminate a lot of frustration, unnecessary helpdesk requests and most importantly eliminate productivity-loss.

Despite the developments towards cloud and hybrid work environments, many DEM solutions continue to rely on the traditional methods that were never designed for this use case. While these methods resonate with IT teams that began in the on-premises era, and may seem to remain functional, they are less effective in today’s IT landscape, delivering actual value in just a small subset of scenarios. In the following sections, we will explore how these methods relate to DEM in the context of cloud and hybrid work environments.

Real User Monitoring (RUM) is often misunderstood, but it is a critical aspect of Digital Experience Monitoring. It is meant to give insights into the experience of users as they interact with services and applications.

At the core, RUM is a passive method of collecting monitoring insights. Meaning that it only collects metrics as the user is using it. Traditionally this was done by injecting code snippets in the application (server-side). In itself not necessarily bad, there are a few definite drawbacks:

Instead of using traditional RUM based techniques, OfficeExpert TrueDEM collects its metrics continuously from within the user’s context. This is done through a lightweight, low impact agent installed on the user’s device and operating in the user’s context (device, location, authentication) to ensure that the metrics obtained represent the exact same as the user is or would experience them if they use the services.

The above-mentioned RUM method has significant shortcomings, but they pale in comparison to the widespread misguided assumptions around synthetic transaction monitoring.

Synthetic transaction monitoring (STM) is often touted as the DEM method of choice. And in fairness, it was the method of choice – more than a decade ago. STM performs constant artificial pre-scripted actions (often named probes, agents or robots) against the application. Using connections from a physical device or software-based application, it simulates actions taken by users without actually having a user involved.

Whereas this method does have the advantage of potentially being able to see outages somewhat more in advance than RUM does, the likelihood it does is very small as there are several problems with this method.

The concept of synthetic transactions as such therefore delivers limited value. It was valuable when Office 365 was new, early in the history of Microsoft’s cloud as synthetic transactions were all that was needed to provide exemplary support.

Nowadays synthetic transactions may be mostly useful if you want your monitoring product to co-fail alongside your user population or monitor when the Microsoft platform is 100% down. However, with Microsoft easily meeting their SLA nearly all the time, these scenarios are not very common. Moreover, most synthetic transaction solutions put more importance on checking off the box that says a test exists than accuracy, precision, or focus. As a result, the ‘best’ synthetic transaction solutions co-fail and break when your users do, while the worst STM products miss your outages altogether.

Instead, OfficeExpert TrueDEM uses a modernized concept for synthetic transaction monitoring. We built a new method to non-intrusively monitor all API’s relevant for the user’s experience. It does so directly from within the user’s context of device, location and authentication to ensure the results are specific to each user and situation. This modern variant is based on answering the question of ‘can you perform’ instead of needing to perform the action which causes much less of an impact on the service and allows for a far more accurate monitoring. This even allows us to monitor API’s that would not allow traditional synthetic monitoring because of its intrusive character.  We prefer to say that we do Real User Simulations.  It’s the modern, and accurate way to proactively monitor how SaaS platforms are working for your users.

Endpoint monitoring is a concept that can mean several things. Originally it was used to mean monitoring the entry point to a service. To monitor the Outlook.office365.com host for example. Ping and round-trip times for IP traffic do not do much good though, if the target network blocks them or, as is the case with advanced portions of Microsoft 365, the network does not use the IP protocol at all.

The second use of the term Endpoint monitoring refers to monitoring the user’s endpoint: their device. This is a valuable component of a true user experience monitoring solution. But it is just one of many components. What happens when the problem is not on the user’s device? By focusing too much on the endpoint alone, you can miss what else is going wrong. Especially while monitoring SaaS platforms like Microsoft 365.

In conclusion, Endpoint Monitoring data is crucial but cannot stand on its own in a modern DEM landscape and stands or falls with the quality of collected data and the correlation to other metrics. 

OfficeExpert TrueDEM therefore collects its endpoint monitoring data in direct conjunction with the collection of the overall user journey data. Setting it in context to all other data points and aligning the level of depth of data collected to the level of depth needed.

Network Monitoring products have gained considerable traction among network teams in enterprise customers. In the past, network issues were more prevalent than any other problem. Even today, network issues can arise early in the deployment process. However, once a stable state is achieved, the network is just one of many potential areas for problems to occur. And it’s then that the importance of full overview DEM becomes important. Hybrid work developments have only exacerbated this need as users now work in a multitude of locations and network environments, often not even controlled or managed by the networking team (see the example of the user working in the coffee shop).

Networking is no more likely to be the cause of an outage, than any of the other areas of responsibility.

Despite this, many network monitoring solutions fail to adapt and instead focus on even deeper analysis. While these tools may have dozens or even hundreds of tests that they perform. This is no guarantee that they will find the problem when users report impacted user experiences. As they focus singularly on their area of networking and will often keep digging until they find something wrong instead of correlating to other data available and realizing it is a combination of factors or even not related to networking at all.

Potential problems of network monitoring therefore are:

Again, OfficeExpert TrueDEM recognizes the need for network monitoring but sees it as an integral part of the complete journey and will refrain from ‘digging itself into a hole’ by selectively collecting additional data where needed based on network performance indicators.

Application Performance Monitoring (APM) is a specialized area of monitoring and is great for proprietary and custom software.  But its success requires vendors to code to both the client and the server sides of the system.  Unfortunately, server-side modifications can’t be made for SaaS solutions like Microsoft 365.  Light support for plugins exist in Microsoft 365 desktop applications, but they are not deeply coded into the core of the server side. 

APM products therefore provide very little actual value for SaaS solutions like Microsoft 365. 

Technology has allowed us to collaborate remotely, and it is technology that will enable the success of the hybrid work model. Sustaining a healthy, performant, and optimized IT landscape is paramount for organizations to recruit better talent, achieve innovation, and build a flexible, productive future.

Not surprisingly therefore, the key factors for innovation in this are:

The new hybrid, predominantly digital workplace is complex to say the least. Network issues and problems with devices, video conferencing systems, digital tools and communication platforms are bound to happen and impact both your users and subsequently the interaction they have with your customers. This underscores the need for IT leaders to monitor, as well as gather data and insights so that their organizations can prepare for – and quickly respond to – any crises.

Modern organizations are expected to be able to:

panagenda’s TrueDEM technology is a paradigm-shifting solution that addresses these very issues in a holistic way, helping organizations act with agility and flexibility, accelerate their digital experience ambitions, and double down on transformation success.

You can’t fix what you can’t see, and performance issues like glitchy video, poor quality audio, dropped calls, slow upload, and download speed can derail an entire workday – and that translates into lost revenue and poor user satisfaction. To achieve a thriving and successful hybrid workplace, company leaders need to be aware hybrid working is far more complex than just having some employees in the office and some working from home.

Whether you call it EUEM, DXM, DEX or DEM without a holistic, end-to-end view across all digital touch points, it will be next to impossible to identify where problems are occurring.

TrueDEM technology removes the need to ‘triangulate’ from multiple data methods by instead providing you with unique and holistic insights into the digital experience for any user in your organization as well as organization wide. Removing the need for multiple point solutions that cover only part of the journey like network or endpoint by giving you a single tool to support and maintain your Microsoft 365 environment’s user experience effectively and successfully. Ensuring the technology works seamlessly with minimal disruption to work – to drive productivity, attract talent and foster an overall positive workplace culture.