We use qualitative observation protocols to record performance.
Our aim is to work with decision-makers to describe and understand behaviours, patterns, and trends in performance in order to initiate conversations about the ‘why’ and ‘how’ of performance … and how applied qualitative analytics might be of service in identifying drivers and connectors of performance.
In sport contexts, we provide decision support for coaches in real-time to identify lead indicators that capture the flow of games.
Digging Deeper
This page provides some background information about our epistemic culture, our sense of creating and warranting knowledge (Cestina, 1999), dealing with the messiness of performance (Williamson, 2018) that seeks to make sense of time, space, and causality (Marcus and Davis, 2019). We take to heart the notion that continuous questioning and rethinking helps us move forward in our interdisciplinary practice, responsibilities and enablement (Giardina, 2017; Karwowski, 2019; Burkhardt, Hohn and Wigley, 2019; Ley, 2019; Stathoulopoulus, Moteos-Garcia and Owen, 2019; Tuffley, 2019: Zador, 2019; National Research Council of Canada, 2020; Zimmermann, Di Rosa and Kim, 2020).
You can find a detailed account of our approach in this document. (Download document.) See also our discussions about Analytics 5.0. (Download document.)
We would be delighted to provide more detail should you require it. (Contact)
Our qualitative analytics approach resonates with Adam Cooper’s (2015) definition of analytics as “a personal and organisational perspective on using data for decision-making and action-planning and less about how it is processed in a computer; evaluating, planning and doing are human activities”. (Our emphasis. See also Feng and Wu, 2019; Colson, 2019; Krotov and Hopfield, 2019; Bergstein, 2019, Dehaene 2020).
Our conversations with partners explore situational awareness (Baysal, Holmes and Godfrey, 2013) and address their understanding of their immediate environment (Schwartz and colleagues, 2019). Our aim is to support our partners as active agents in anticipating and planning for changes in their environment (Beech, 2019; Grace and colleagues, 2018).
The focus of our qualitative observations has been informed by the information and insight matrix shared by Thomas Davenport, Jeanne Harris and Robert Morison (2010):
We have observational protocols that measure performance in real-time and in lapsed-time. These protocols are monitored continuously to maintain optimal levels of intra- and inter-observer reliability. We are mindful also of the threats to the internal and external validity to our work in single case studies (Kratochwill and Levin, 2015) and have a continuous, independent audit of our methods, findings and advice.
We are sensitive to the intelligence augmentation (Engelbart, 1962) potential of our work in an age increasingly challenged and changed by computational intelligence (Wing, 2006), artificial intelligence (Ito, 2018; Wallace, 2019; Winslow, 2019, Loukides, 2019, Claudinio et al., 2019) and algocracy (Danaher, 2020). Our qualitative approach is designed to be a symbiotic connection (Licklider, 1960) between our partners as sentient humans and the potential of machine learning to provide insights to support behavioural change (Cook, 2017, Karwowski, 2019;Tuffley, 2019). Joi Ito’s (2019) discussion of extended intelligence resonates strongly with us, as does support for human intelligence and interactions (Golembiewski, 2019) as well as conceptual understanding (Marcus and Davis, 2019).
Our work has an affinity with the sentiments expressed by Ross Goodwin (2016):
When we teach computers to write, the computers don’t replace us any more than pianos replace pianists—in a certain way, they become our pens, and we become more than writers. We become writers of writers
Michael Jordan (2018):
The idea that our era is somehow seeing the emergence of an intelligence in silicon that rivals our own entertains all of us — enthralling us and frightening us in equal measure. And, unfortunately, it distracts us
The Economist (2018)”the first step towards ensuring the fairness of the new information age is to understand that it is not data that are valuable. It is you”. This involves us in thinking critically about issues raised by deep learning approaches to observation, classification and validation (Xiaoxuan Liu et al, 2019).
And Gary Smith (2019):
Computers can do many very specific tasks much better than humans, but they do not have anything remotely resembling the wisdom, common sense, and critical thinking that humans use to deal with ill-defined situations, vague rules, and ambiguous, even contradictory, goals.
Our methods are constantly on the move. Like Monika Buscher and John Urry (2009), we are mindful of allowing ourselves to be moved by, and to move with, our partners, and aspire to be tuned into how people, objects, information and ideas move and are mobilised in interaction with others. In doing so, we aspire to examine performance up-close and employ a range of methods and resources to “generate insights from complex social spaces and practices” (Williamson, 2018), and the insights to be gained from digital learning networks (Agresta, 2019).
Risk, Exposure and Opportunity
The interaction of risk, exposure and opportunity is central to our work. Our qualitative approach to this interaction has been refined over two decades and, we think, provides a point of difference in our approach to the observation and analysis of performance.
A qualitative approach to risk has an important contribution to make to our intelligence augmentation conversations with our partners (Gal and Rucker, 2018). We seek to find ways to identify and manage risks that embrace an awareness of exposure and inform decision-making in a range of performance contexts. Our experience is that this awareness contributes to proactive leadership, organisational resilience and the agility to respond to opportunity.
We work with our partners to explore how to address exposure, threat and threat agents. We aim to support organisations and individuals to address a variety of performance issues to explore how we can help to protect them from exposure.
You might find this presentation of interest to discover how we use a decision tree to engage with our partners about exposure.
We have a very close relationship with the Australian Risk Policy Institute. Tony Charge, our CEO, is President of the Institute. The Institute was formed to promote and encourage greater focus on risk policy in leadership, decision-making and management across all sectors in Australia and more recently has extended its reach internationally as Convenor of the Global Risk Policy Network.
Photo Credits
Photo by Nick Croft on Unsplash
Keith Lyons (CC BY 4.0)
