In today’s fast-paced digital world, where hybrid work is becoming the norm and skilled professionals are in high demand, organizations recognize the need for exceptional user experience to boost engagement, satisfaction, and productivity. OfficeExpert TrueDEM is leading the way in Digital Experience Monitoring (DEM) for Microsoft 365, setting a new standard for seamless user experience monitoring with a new and more effective way to monitor and troubleshoot your user’s Microsoft 365 user experience.
Giving you the tools to adequately and quickly address problems, prevent unnecessary helpdesk calls, pre-emptively alert users of (impending) disturbances, and provide your experts and third-party contractors with the information they need to take action quickly.
Digital Experience Monitoring
Traditionally the DEM spectrum is dominated by products based on or using one or more of the following monitoring philosophies:
- Real User Monitoring (RUM)
- Synthetic Transaction Monitoring (STM)
- Endpoint Monitoring (EPM)
- Network Monitoring (NM)
- Application Performance Monitoring (APM)
This large number of vastly different technologies has led Digital Experience Monitoring (DEM) down an interesting path. Research organizations have changed the definition and description of what DEM is multiple times. Even the naming isn’t consistent with Digital Experience Monitoring (abbreviated to DEM or sometimes DXM), Digital Employee Experience (DEX) and End-User Experience Monitoring (EUEM) being coined by various analysts and providers.
But changing the name itself would never solve the underlying problem that none of the above philosophies in themselves cover what DEM / DEX / EUEM in a modern environment should be.
As we examined the philosophies, we discovered that they were often siloed and inadequate in providing a comprehensive DEM view in a modern digital world that is dominated by two groundbreaking developments: SaaS and an unprecedented increase in hybrid work. These developments require a unified and comprehensive view of the entire user journey, which the traditional methods often fail to deliver.
Even Gartner® recognizes this in their 2022 Market Guide for Digital Experience Monitoring by Mrudula Bangera, Padraig Byrne, and Gregg Siegfried. Stating: “There is little effort to triangulate observation and understanding across multiple data ingestion technologies and no cross-channel measurement of user experience”.
Shifting the Paradigm
Regular discussions with partners, customers and their teams of experts showed that traditional methods consistently fell short in delivering true digital experience monitoring for modern organizations. To avoid this pitfall, panagenda designed OfficeExpert TrueDEM from scratch, constantly challenging the status quo of existing methods.
Launched worldwide in July 2020, with enhanced capabilities for organizations using Microsoft 365, OfficeExpert TrueDEM is centered around a simple yet critical question: “Is everything all right and if not where is the problem?” To answer this question, we rely on three pillars that help us get to the root of the problem:
- What is the expected normal? (for the user, the organization, the situation and the location)
- Who or what is responsible? (what or where is the area of responsibility for the problem)
- What is the impact? (who is likely going to be impacted by the problem and how big will the impact be? – delays, outages, affected services, etc.)
By leveraging these three pillars, OfficeExpert TrueDEM delivers a comprehensive and unified view of the user journey, allowing organizations to proactively identify and address issues before they impact productivity and user satisfaction.
Having the User at the Center is More Critical than Ever Before
The TrueDEM technology puts the user at its center, and it does so for a reason. In the past, it was relatively easy to generalize the digital experience as users were issued standard devices, worked in offices on dedicated, IT managed networks, and worked with applications fully controlled by their own IT. A single ‘canary test‘, where only a subset of users was monitored, was often enough to provide insight into the experience of an entire office or department because all used the same infrastructure, device types, networks and servers. However, those days are long gone.
Today, with hybrid work and global talent shortages, users are no longer restricted to a specific device, network, or location. For example, a user may check their email in Outlook and catch up on chats in Microsoft Teams while using a coffee shop’s free Wi-Fi. Later, they may join a call using their home network and a headset. Stop, and consider the number of ‘hidden’ variables this one example already injects into a support discussion about network and devices.
For example, Microsoft directs users to the best data center based on current load, maintenance cycle, even the application being used. And each ISP will take a different journey to reach this dynamically-selected Microsoft data center. This means that not only the user’s network and devices change, but that the Microsoft 365 data centers where their requests are routed through may also differ depending on the location of their data and the load on those data centers at the time.
So, the only true way to say anything about a user’s digital experience is to monitor every individual user. Because you can’t monitor user experience without monitoring the user – it’s in the name after all.
Because Each User’s Situation is Unique it Should be Evaluated as Such
For every user, every situation, and every location the expected normal can differ and must be determined based on historical data. When you properly evaluate each contributing factor for each real user, you can generate ‘normal’ ranges for sub-components in a journey that no one thought to evaluate. And once you know what the expected normal is, you can start seeing when numbers are starting to trend downward, often even before the users or service providers themselves notice.
Secondly, it also allows you to verify if the user is truly experiencing a problem or simply having a cranky day because the local coffee-shop just ran out of his favorite coffee blend.
Imagine if you could identify delays in a given Microsoft 365 App, regardless of the user’s home ISP, coffee shop, headset vendor, and so on. No need to imagine, OfficeExpert TrueDEM can. And it grants you the ability to pro-actively alert users, take action or inform the responsible parties.
Data is Key, But Insights is What Leads to Solutions
In modern digital monitoring, the risk of analysis paralysis is significant. Simply collecting more data or metrics with different tooling or technologies from different parts of the user journey doesn’t necessarily result in better insights. In almost all cases, it has the exact opposite impact. The problem is compounded by the fact that data from different tooling is often misaligned. This makes it difficult to pinpoint the actual cause among multiple red flags that can show up in the different data sets, leading to finger-pointing between departments and costly research by expert staff to weed out false positives.
Instead of causing this type of data confusion, OfficeExpert TrueDEM is designed to collect specific metrics from each part of the journey that are designed to be aligned and then used to deduct the Area of Responsibility. Allowing you to immediately address the problem with the right experts or parties and provide them with the right insights and data.
Who Truly Suffers?
Lastly, to truly derive benefits from a DEM system, it is crucial that it provides accurate insights into the exact impact of issues.
For instance, if an ISP network is faltering, the system should be able to identify the specific offices and users affected. Similarly, if a SaaS data center is experiencing issues, it should be clear which users are routed through that center. And if a particular version of a Microsoft Teams Client is straining device CPU, the system should indicate which device types are most affected. Additionally, the system should provide information on alternatives if certain parts of the SaaS service are faltering. For example, if the OneNote desktop client is experiencing problems, it is essential to be able to know the status of the browser-based OneNote version so that you can advise users to switch to it temporarily.
Having this type of information and being able to pro-actively act and alert on them can eliminate a lot of frustration, unnecessary helpdesk requests and most importantly eliminate productivity-loss.
Pitfalls of Traditional Monitoring Methods in a Microsoft 365 Dominated IT Landscape
Despite the developments towards cloud and hybrid work environments, many DEM solutions continue to rely on the traditional methods that were never designed for this use case. While these methods resonate with IT teams that began in the on-premises era, and may seem to remain functional, they are less effective in today’s IT landscape, delivering actual value in just a small subset of scenarios. In the following sections, we will explore how these methods relate to DEM in the context of cloud and hybrid work environments.
Real User Monitoring
Real User Monitoring (RUM) is often misunderstood, but it is a critical aspect of Digital Experience Monitoring. It is meant to give insights into the experience of users as they interact with services and applications.
At the core, RUM is a passive method of collecting monitoring insights. Meaning that it only collects metrics as the user is using it. Traditionally this was done by injecting code snippets in the application (server-side). In itself not necessarily bad, there are a few definite drawbacks:
- SaaS solutions offer little to no options to inject code. To combat this, some solutions resorted to ingesting applications logs instead. This is not true monitoring; it is post-mortem reporting and lacks the precision required to identify and resolve problems pro-actively or even as they are happening. They also heavily rely on the level of detail provided in the logs, which can vary greatly.
- SaaS solutions are often associated with web applications. And as browsers do support browser plugins, these are often touted as RUM. With platforms like Microsoft 365 though, most users prefer using the desktop client applications over the browser applications. Resulting in only partial or conditional monitoring.
- By collecting data as the user is using it, there is no way to predict potential problems. At best, you’re “failing with the user”.
Instead of using traditional RUM based techniques, OfficeExpert TrueDEM collects its metrics continuously from within the user’s context. This is done through a lightweight, low impact agent installed on the user’s device and operating in the user’s context (device, location, authentication) to ensure that the metrics obtained represent the exact same as the user is or would experience them if they use the services.
Synthetic Transaction Monitoring
The above-mentioned RUM method has significant shortcomings, but they pale in comparison to the widespread misguided assumptions around synthetic transaction monitoring.
Synthetic transaction monitoring (STM) is often touted as the DEM method of choice. And in fairness, it was the method of choice – more than a decade ago. STM performs constant artificial pre-scripted actions (often named probes, agents or robots) against the application. Using connections from a physical device or software-based application, it simulates actions taken by users without actually having a user involved.
Whereas this method does have the advantage of potentially being able to see outages somewhat more in advance than RUM does, the likelihood it does is very small as there are several problems with this method.
- As the transactions have to be pre-scripted using specific API’s they often assume a uniformity that is seldom there. For example, Microsoft 365’s backend has dozens of API’s for creating and sending mail which can vary depending on how, where and with what client the user is doing it. And this is just a single use case! As an effect, more and more transaction probes are included and touted as the solution, but unless you can accurately attach the results of a specific probe to actual impact on specific users, they still generate little value.
- Another drawback the STM method generates, is that it does not account for the user-centric data storage used in Microsoft 365. A user’s data can be located on any of tens of thousands of servers in different data centers throughout a company’s assigned geographic region. And the route and speed with which a user is taken to its data while using a service has a significant impact on the experience. As the probes generally use dedicated user accounts, they can’t report on the experiences other than for those accounts.
- Worse, STM solutions often use the same system accounts in multiple places for its testing, which compounds the disconnect to an actual user account experience as the data routing will be severely impacted by the hopping of locations. Making it difficult to impossible to obtain accurate insights on individual user experience or even generalized results.
The concept of synthetic transactions as such therefore delivers limited value. It was valuable when Office 365 was new, early in the history of Microsoft’s cloud as synthetic transactions were all that was needed to provide exemplary support.
Nowadays synthetic transactions may be mostly useful if you want your monitoring product to co-fail alongside your user population or monitor when the Microsoft platform is 100% down. However, with Microsoft easily meeting their SLA nearly all the time, these scenarios are not very common. Moreover, most synthetic transaction solutions put more importance on checking off the box that says a test exists than accuracy, precision, or focus. As a result, the ‘best’ synthetic transaction solutions co-fail and break when your users do, while the worst STM products miss your outages altogether.
Instead, OfficeExpert TrueDEM uses a modernized concept for synthetic transaction monitoring. We built a new method to non-intrusively monitor all API’s relevant for the user’s experience. It does so directly from within the user’s context of device, location and authentication to ensure the results are specific to each user and situation. This modern variant is based on answering the question of ‘can you perform’ instead of needing to perform the action which causes much less of an impact on the service and allows for a far more accurate monitoring. This even allows us to monitor API’s that would not allow traditional synthetic monitoring because of its intrusive character. We prefer to say that we do Real User Simulations. It’s the modern, and accurate way to proactively monitor how SaaS platforms are working for your users.
Endpoint monitoring is a concept that can mean several things. Originally it was used to mean monitoring the entry point to a service. To monitor the Outlook.office365.com host for example. Ping and round-trip times for IP traffic do not do much good though, if the target network blocks them or, as is the case with advanced portions of Microsoft 365, the network does not use the IP protocol at all.
The second use of the term Endpoint monitoring refers to monitoring the user’s endpoint: their device. This is a valuable component of a true user experience monitoring solution. But it is just one of many components. What happens when the problem is not on the user’s device? By focusing too much on the endpoint alone, you can miss what else is going wrong. Especially while monitoring SaaS platforms like Microsoft 365.
- Collected data is singularly aimed at only part of the user’s journey and can cause tunnel vision or lack of insight if used on its own.
- To combat lack of correlated insights, EPM solutions often resort to log correlation from other parts of the user journey as the go-to answer. However, as mentioned before, it’s often impossible to effectively identify anything but the most catastrophic outage from ingesting logs.
- And lastly, more than other categories, endpoint monitoring and management solutions are prone to gather obscene amounts of irrelevant data. Contributing heavily to analysis paralysis.
In conclusion, Endpoint Monitoring data is crucial but cannot stand on its own in a modern DEM landscape and stands or falls with the quality of collected data and the correlation to other metrics.
OfficeExpert TrueDEM therefore collects its endpoint monitoring data in direct conjunction with the collection of the overall user journey data. Setting it in context to all other data points and aligning the level of depth of data collected to the level of depth needed.
Network Monitoring products have gained considerable traction among network teams in enterprise customers. In the past, network issues were more prevalent than any other problem. Even today, network issues can arise early in the deployment process. However, once a stable state is achieved, the network is just one of many potential areas for problems to occur. And it’s then that the importance of full overview DEM becomes important. Hybrid work developments have only exacerbated this need as users now work in a multitude of locations and network environments, often not even controlled or managed by the networking team (see the example of the user working in the coffee shop).
Networking is no more likely to be the cause of an outage, than any of the other areas of responsibility.
Despite this, many network monitoring solutions fail to adapt and instead focus on even deeper analysis. While these tools may have dozens or even hundreds of tests that they perform. This is no guarantee that they will find the problem when users report impacted user experiences. As they focus singularly on their area of networking and will often keep digging until they find something wrong instead of correlating to other data available and realizing it is a combination of factors or even not related to networking at all.
Potential problems of network monitoring therefore are:
- Unnecessary depth can be detrimental to the end result, as it causes a lack of visibility and can cloud the issue. To illustrate this point, consider the following question: if one were to dig a 50-foot or 15-meter hole and climb into it, how well would they be able to see across the street, down the block, or even miles/kilometers away?
Again, OfficeExpert TrueDEM recognizes the need for network monitoring but sees it as an integral part of the complete journey and will refrain from ‘digging itself into a hole’ by selectively collecting additional data where needed based on network performance indicators.
Application Performance Monitoring
Application Performance Monitoring (APM) is a specialized area of monitoring and is great for proprietary and custom software. But its success requires vendors to code to both the client and the server sides of the system. Unfortunately, server-side modifications can’t be made for SaaS solutions like Microsoft 365. Light support for plugins exist in Microsoft 365 desktop applications, but they are not deeply coded into the core of the server side.
APM products therefore provide very little actual value for SaaS solutions like Microsoft 365.
Innovation :: Making the Hybrid Workplace Better
Technology has allowed us to collaborate remotely, and it is technology that will enable the success of the hybrid work model. Sustaining a healthy, performant, and optimized IT landscape is paramount for organizations to recruit better talent, achieve innovation, and build a flexible, productive future.
Not surprisingly therefore, the key factors for innovation in this are:
- Flexibility and agility
- Supporting rapid digital transformation
The new hybrid, predominantly digital workplace is complex to say the least. Network issues and problems with devices, video conferencing systems, digital tools and communication platforms are bound to happen and impact both your users and subsequently the interaction they have with your customers. This underscores the need for IT leaders to monitor, as well as gather data and insights so that their organizations can prepare for – and quickly respond to – any crises.
Modern organizations are expected to be able to:
- Understand which devices are not performing at optimum levels.
- Know and alert the right responsible people if there are problems developing or likely to arise.
- Ensure that user experience is up to standard wherever people work.
- Debug and analyze each individual user’s experience as each call or use of a cloud application can be impacted by many variables in a cloud/hybrid environment. For example, if you make a Microsoft Teams video call, not just the location, network and used devices may differ from user to user or even call to call, so does the route the call takes within the Microsoft 365 environment as Microsoft’s systems decide dynamically which servers and regions to route the call over. Finding the root cause of problems is like looking for the proverbial needle in a haystack if organizations don’t have insights into the digital touchpoints for the whole journey and for every user at any moment.
- Differentiate assumption from fact. Experience very often is a subjective perception. What is bad for one person may be acceptable or fine someone else. So, knowing a user’s ‘expected normal’ plays a key role in understanding what the user truly experiences before pressing the alarm button. This concept is where Digital Experience Monitoring (DEM) originated.
TrueDEM Technology for True Digital Experience Monitoring
panagenda’s TrueDEM technology is a paradigm-shifting solution that addresses these very issues in a holistic way, helping organizations act with agility and flexibility, accelerate their digital experience ambitions, and double down on transformation success.
- Cost effective and pragmatic. Return on investment (ROI) is not calculated in ‘soft’ savings such as employee productivity. panagenda understands that a true digital experience monitoring solution must support users, while fitting in the real-world confines of the business. TrueDEM eliminates purchases for multiple existing products and delivers tangible savings.
- See the truth behind digital experiences. TrueDEM technology provides visibility into the user experience and performance of Microsoft 365 services from the organizational level down to the individual user.
- Push the boundaries of understanding user journeys. TrueDEM technology covers the whole user journey and provides your IT department with unique metrics to truly cover what is needed to the depth needed to understand what is going on to support and optimize your user’s digital experience.
- Stop guessing where performance issues start. TrueDEM technology monitors all important APIs at various levels to ensure visibility into the user experience for all monitored services.
- All data collected by TrueDEM is aligned and correlated, allowing you to have full overarching insights in a ‘single glass of pane’.
- Instantly analyze digital experience data with full context. TrueDEM collects metrics from within the user’s context (device, location, authentication state) using low-impact locally installed tooling, to ensure organizations can accurately report on each user’s experience.
- TrueDEM technology uses weighted metrics to compare each individual user’s experience against their own historical situation, while ensuring that they are compared only to contextually relevant information.
- Catch issues before they impact your business. TrueDEM technology combines proprietary insights and methodologies with the platform principles of the Microsoft 365 environment to accurately report on performance, identify issues, and often predict when they will occur and who will be impacted so you can take immediate action and proactively engage with affected individuals.
- Do the right things. TrueDEM technology eliminates unnecessary actions and data collections that could negatively impact the cost of operations.
You can’t fix what you can’t see, and performance issues like glitchy video, poor quality audio, dropped calls, slow upload, and download speed can derail an entire workday – and that translates into lost revenue and poor user satisfaction. To achieve a thriving and successful hybrid workplace, company leaders need to be aware hybrid working is far more complex than just having some employees in the office and some working from home.
Whether you call it EUEM, DXM, DEX or DEM without a holistic, end-to-end view across all digital touch points, it will be next to impossible to identify where problems are occurring.
TrueDEM technology removes the need to ‘triangulate’ from multiple data methods by instead providing you with unique and holistic insights into the digital experience for any user in your organization as well as organization wide. Removing the need for multiple point solutions that cover only part of the journey like network or endpoint by giving you a single tool to support and maintain your Microsoft 365 environment’s user experience effectively and successfully. Ensuring the technology works seamlessly with minimal disruption to work – to drive productivity, attract talent and foster an overall positive workplace culture.
Find out how users really experience Microsoft Teams
Get a free trial of Office Expert TrueDEM, gain real-time insight into your organization’s Microsoft Teams usage, performance and experience, and understand what to do to improve.