Quality metrics for end-to-end analytics in a complex funnel

This case describes the work on the iDataFusion end-to-end analytics project with a corporation that is a leader in proptech and development. The following business areas are being developed in the group's perimeter: an online platform of real estate services, a management company, commercial and rental real estate funds, project real estate development.

In 2021, the devloper teamed up with iDataFusion to build an end-to-end analytics system.

End-to-end analytics for this developer, according to the head of analytics, is a decision support system. Marketing and the business, working with data and customized reports, are able to understand how well all the business processes work, as well as which of these processes can be improved. This is a system that is built into the workflow as much as possible and is used on a daily basis.

All stages of the funnel to the sale of the developer in question are currently being digitized.

The real estate funnel is long, contains many stages, and importantly - includes a large number of online and offline events: website visits through advertising, organic search, visits to ads (CIAN, Avito and others), calls, applications, office visits, reservations and transactions.

Basic identification is possible only at the stage of the transaction, when the client leaves his phone number. Before this stage, the third funnel, the path of a particular customer is unknown, because it is technically impossible. However, after the client's phone number is transferred to the developer, the client's path is reconstructed retrospectively. At the same time, the client is a rather fuzzy concept in real estate. It can be a husband or wife, or any other relative. They are all lumped together into a single entity, "household."  In addition to ordinary clients, agents, real estate agents also visit the site.

As a result, it is often unclear on what basis the disparate chains can be connected to get an accurate picture. There are many hypotheses about how this data can be combined and linked, that is, data cleansing. A tool was needed to understand which hypotheses worked and which did not, which parameter of end-to-end analytics needed improvement.

The share of clients with a digital footprint in the Meeting phase is one such metric. In the Meetings phase, the percentage of such digitized customers is ⅔. Tracking this metric will allow you to see your progress in digitizing customers.

An even more interesting and important metric is the depth of customer digitization on transactions. Since there are a few more touches between the meeting and the deal, up to 70% of clients are already digitized at this stage. The analytics team was able to achieve a significant increase in this metric, from 50% to 70%.

Digitization depth is in direct correlation with the accuracy of the marketing budget, that is, it is about the same 70%, which means that at least 70% of the marketing budget is attributed correctly and, therefore, at least 70% of the marketing budget is spent effectively. In terms of business impact - the result was a 2-fold reduction in CRR.

In addition to this metric, the developer also decided to calculate the impact of digital channels by multichannel attribution. Specifically, the share of deals that were concluded on this attribution from digital channels. The increase in this metric compared to all other metrics confirms that the team is moving in the right direction.

This metric correlates with the share of customers with a digital footprint, but it also gives a different perspective on digitizing a business by showing how well online advertising is working. By increasing the share of influence of digital channels, the team is improving the quality of the end-to-end analytics system. This upward trend should continue, or at least not drop from month to month.

Average number of sessions before the transaction

The third metric introduced by the analytics team is the number of sessions to an event. It allows you to measure end-to-end analytics in breadth. Let's say a transaction is 100% digitized, and all ad touches are identified. But this digitization does not account for the number of ad touches.

Consider an example in which a transaction is fully digitized in depth, that is, all of the advertising touches are taken into account. Suppose there were 20 touches, but the actual clicks were 26. The "digital analytics depth" metric will not show the difference whether there were 20 or 26 clicks. This is the so-called ROC AUC problem, where the model predicts everything, but how well we don't know. There is no understanding of how well the data is separated and clustered from others.

In this regard, the team of analysts calculated the number of sessions per customer before a certain event occurred. As the quality of end-to-end analytics improves, the number of sessions should also increase. This means that more and more advertising channels are adequately associated with meetings, calls and deals, and the CPL calculation is getting better.

This metric says nothing about how reliable the data are, but it says something about the quality of the system as a whole. An increase in the number of sessions confirms that digitization quality is increasing, while digitization depth is not changing much, which is why this metric is so important.

One of the client gluing tools for desktop and mobile browser communication is sending a text message. The client clicks the link in the text message, goes to the mobile browser, and the phone number is used to glue sessions in the mobile browser and on the desktop. The effect of such gluing is measured simply by the "number of sessions" metric. If the quality of such gluing improves, the number of sessions will increase.

In sales, there are bound to be customers who are well digitized in CRM, but they cannot be digitized online because they did not interact with it initially. The share of such customers should not be large, and with systematic work on data cleansing and gluing, the share of such unidentified customers should decrease.

In this regard, the term "unidentified" meetings and deals was introduced, and it includes everything that cannot be glued together. For meetings, this figure is 27% and continues to decline; for deals, it is 7%. Thus, the maximum accuracy of the marketing team's advertising spending decision is 93%.