Datinal

Category: Data Science

  • The Sniff Test – More important than it ever has been.

    The Sniff Test – More important than it ever has been.

    Applying a “Sniff Test” or carrying out a “Sanity Check”, has always been an important part of any analysis or calculation. It should be your first check before diving deeper, and especially before sharing results with others.

    I am not saying it was easy or foolproof, or even always possible; however, the onus was on you, since you came up with the answer and had often spent a lot of time arriving at it.

    With the rapid rise of generative AI, you no longer need to do the heavy lifting; complex questions can receive intelligent-sounding answers in seconds, and it is tempting to accept them after a cursory glance. After all, we are busy, and the answers provided are intended to be coherent and phrased with a confident tone.

    How To Apply a Sniff Test

    I don’t think anyone would disagree with the need to ask ourselves, “does this answer make sense?“. Here are some suggestions:

    • Does this align with what I already know?
    • Is there a coherent thread to follow in the argument provided by the AI?
    • If I am asked to explain the answer, can I do it in my own words?
    • Can I find a trustworthy source that provides an alternative point of view or answer?
    • If the answer was written by a brilliant, but recently graduated intern, would I have spent a similar amount of time reviewing it?
    • Did the prompt I wrote steer the AI toward the answer I wanted?
    • Did I refine the prompt through several iterations, or go with the first version because I liked the answer?

    If the answer to any of these questions gives you pause, that’s your signal to dig deeper.

    Risks of the Sniff Test

    Like all techniques, the sniff has its own limitations:

    • Bias, assumptions, or lack of domain knowledge can distort the Sniff Test.
    • Correct but counterintuitive answers are often thrown out.
    • It is easy to develop Sniff Fatigue after a period of questioning everything. Trusting AI out of convenience is a very easy slope to slip down.
    • A Sniff Test is a first step, not a last line of defense, it doesn’t replace the need for a thorough review, that is appropriate for how you are going to use the answered provided.

    Final Thoughts

    Ask yourself Does this feel right? Does it make sense? Don’t take the easy way out.

  • What are we searching for?

    What are we searching for?

    This sounds like an existential question, but it’s not quite that deep. I had a wave of nostalgia at the weekend pondering “What happened to MDM and is it still a thing?”. So I decided to look at search patterns over the 15 years for terms that were, or have become, common.

    The source of the data is Google Trends and is based on worldwide searches, from 2010 to 2025, The Y axis is the relative Interest, where 100, in percent, reflects the maximum interest in a particular topic.

    The categories, and their meaning are as follows:

    CategoryExplanation
    Consistent GrowthMature yet still expanding, integral to modern IT and data infrastructure
    Maturing/Slight MovementMatured, some (e.g., Big Data, ETL) were once hot but are now mainstream, or no longer gaining new attention.
    DecliningOften older technologies or methodologies that are being replaced or have reached saturation. (e.g. SQL which is far from obsolete)
    Strong GrowthDominant growth trend

    Interesting anomolies exists around GDPR which peaked around 2018 (its enforcement year), then declined. I am also curious about the continued appetite for Quality, Governance and the Vault, ideas that are sound but, in my experience, still haven’t delivered on the promise or been integrated into the operational world we work in.

    My main take away, apart from MDM still being a “thing” is not the presence of AI/ML, Generative AI, LLM etc. it is their “take off”, which is meteoric

    Suffice to say, there aren’t any topics over the last 15 years that come close to these tragectories, saddle up and lets see where they go, and what’s coming next!!

    I guess I, need to point out that, a value of 100 interest could represent 200 hits or 200 million hits, supplementing this analyse by using Key Word Planner to look at search volume would be valuable

    Please let me know if you feel I have missed a topic that should have been included, or more importantly if your crystal ball is clearer than mine.

  • Membership, how to engage, grow and retain

    Membership, how to engage, grow and retain

    How Analytics Supports Membership-Based Businesses

    Businesses built on a membership model have different analytical needs compared to those using distribution or retail models. This post explores how analytics enables organisations to grow and sustain membership businesses effectively.

    What is a Membership Model?

    A membership model is a business structure where customers pay a recurring fee to access services. Common examples include:

    • Netflix
    • Dollar Shave Club
    • Fitness Passport
    • Royal College of Physicians Australia
    • Costco
    • LinkedIn

    While sub-types exist (e.g. fixed-term, usage-based, or tiered memberships), this article focuses on the shared analytics needs across these models.

    Core Objectives

    Membership businesses typically aim to:

    • Retain existing members
    • Acquire new members to grow and offset churn
    • Transition members to higher-value packages or plans

    Advantages of the Membership Model

    • Recurring revenue provides predictability
    • Direct customer relationships (no resellers or agents) enable better data and feedback
    • Distributed risk — no single client loss is business-ending
    • Scalability — costs to deliver digital services don’t rise proportionally with usage
    • Lower marketing costs — once a stable member base is achieved

    Challenges

    • Requires a large volume of members to cover operational overhead
    • Staffing costs can be high, especially in the early growth phase

    Analytics Initiatives for Membership Organisations

    Member Segmentation

    Group members by:

    • Geography
    • Psychographics
    • Behaviour

    This enables targeted messaging and personalised experiences.

    Churn Modelling

    Identify members at risk of leaving by analysing usage, interaction, and payment data. Use this to trigger timely interventions.

    Renewal Modelling

    Detect friction points in the renewal process that reduce satisfaction or delay payments.

    Data Acquisition

    Integrate data from:

    • CRM
    • Website interactions
    • Event attendance
    • Payment and renewal records
    • Social media signals

    Data Modelling

    Support strategic decisions and reporting with KPIs such as:

    • Monthly/Annual Recurring Revenue (MRR/ARR)
    • Churn/Attrition rates
    • Net Promoter Score
    • Memberships in arrears

    Operational Reporting and Visualisation

    Present trends and insights visually for clarity and stakeholder buy-in.

    How Analytics Drives Outcomes

    Engage

    • Enhance member experience to reduce churn
    • Increase event attendance and involvement
    • Identify and support at-risk members

    Transition

    • Upsell members into more valuable packages
    • Use insights to offer personalised upgrade paths
    • Predict which members are most likely to convert

    Follow along at datinal.com.au to explore more.

    Further Reading: https://chrislema.com/memberships-and-subscriptions/