Weeknotes S03E06
This week saw quite a few things become clearer in my mind. I’m getting closer to explaining data in the context of digital product development. In doing so, I’m closer to understanding why the delivery triangle model needs to be iterated. Exciting and perhaps controversial.
Things I’ve been doing and thinking about this week.
Planning, interconnections and ambitions
This week I’ve been talking in a lot of detail about the interconnections between different aspects of our work and how to plan for that. We’re starting to get towards defining more clearly how we need to work with each other and with other teams. Stuart submitted our quarterly plans and priorities. They are well thought out and ambitious but with all risks highlighted. Ed worked with Jason and Aleks to refine some of our processes.
I’ve also had a few conversations about personal ambitions, core values and wellbeing with people in the team. I enjoy balancing professional development, delivering business value with professional satisfaction and pastoral care.
Professional pride and image projection
My son had some minor surgery on Tuesday so he was off school and I was off work. He joined me on my morning walk and we got chatting about jobs. In particular work-life balance, job satisfaction and personal motivation. We wandered into a conversation about the difference between public and private sector work.
It is easy to get disillusioned as a public servant, especially now. Professional pride is an important dimension of both the work and engagement at work.
Will ran a session on responsibilities early in the week. I have strong views and values when it comes to professionalism and accountability. This sometimes comes across as confrontational, contentious, or defiant. It often is. I’m conscious of the image I can project in certain circumstances but I make absolutely no excuses for having high expectations.
Building smart services
I’ve been searching for the right way to explain a few things this week. Our team often uses food analogies when talking to each other. They usually work pretty well for conversations between ourselves and some of our friends but not always universally. Ed and I spoke recently about how showing our working isn’t always productive.
I had a good conversation with Rich this week which also touched upon showing your working. That session seeded a few things in my brain. I think it indirectly resulted in me considering the house building analogies often used in software development (and in our team too).
Analogies are powerful for getting shared understanding. So I’ve been stretching and flexing this one while thinking about how to explain data in the context of digital product development. Still a work in progress but here goes.
Service development in government is like iterating a house that you don’t live in.
In many real-world circumstances, those living in a house make their own decisions. A new lamp, new wallpaper, adapt the layout, even moving house. In our iterative government house, they don’t. It’s a house that changes around its residents.
A development team builds every part of the house from infrastructure to features. They supply electricity, water, lamps, wallpaper. Every decision is made by them. In particular the product manager. The incentive is not profit but welfare. There are constraints but the focus should be on giving residents the best experience.
I want to help design and build smart services
Smart homes send information about the resident’s actions and behaviors. This information can be used to learn and make decisions to provide better experience and efficiency. In our iterative government house, it is down to the product manager to decide how smart the house needs to be.
From a practical point of view this will depend on many other things. The stage of development. The plan for iteration. But being aware of what you need to know and what needs to happen to get that information is vital.
Consider three aspects of smartness that touch upon the role of data professionals in each.
1. Using standard exhaust data from all houses. Think of the electricity supplier. Monitoring the substation for things like downtime, faults and defects.
This is like system events: developers can control and configure which data gets stored in server logs — the raw, unfiltered information about activity on the application and infrastructure. Telemetry tools like Graphana, Kibana etc can be used with this data to keep watch over the behaviour of servers and live services activity over time.
The data is in the language of the system. It can be use to understand some level of user behaviour but it is not designed for that and often requires disproportionate effort to do it. In this scenario data professionals aren’t really involved with the development team. They are given post-hoc data and asked to get the best out of it.
2. Using data we collect from inside a house. Think of asking residents if they will install smart plugs or motion sensors in some rooms of the house.
This is like web analytics: Snippets of code added to web pages — code which needs to be run on a user’s computer or smartphone — to gather information about user activity and behaviour on the application/service and return it to a data collection service — such as Google Analytics.
This needs the user’s permission and other technical elements. This means you can’t be confident of capturing data from a large and consistent proportion of users. In this scenario data professionals aren’t really involved with the development team. They are brought in to conduct what amounts to a survey. This affects the value of using it. Especially for changes and trends. It does give some aspect of journey information but only in some rooms of the house. See also user research.
3. Improving exhaust data through smarter infrastructure. This involves thinking up-front about what needs to be measured and writing code to give that information.
This is like business events: Developers can write code at the application layer to produce business events and programmatically generate logs for these. Applying business logic and data modelling to aggregate or define system event data to higher-order business event data adds meaning.
In this scenario, data professionals need to be more far more involved with the development team. Understanding success and goals upfront. Advising the best way to get the information. Supporting developers and others to get it. The result however is a far more comprehensive — and always on — understanding of user behaviour and journeys.
Being smart changes the way you work with data professionals
Much like with smart home technology, the work to develop smart services can be stifled by perceived complexity and inconvenience. However ill-informed decisions can result in irrecoverable data debt and a complete lack of knowledge of what is going on in the house/service.
Older houses can sometimes be retrofitted with some smarter technologies. Replace standard meters which require visits, or residents to submit information or estimates. With smart meters automatically send readings every few minutes. This needs time and effort.
New houses can be constructed with smart home infrastructure in place. plumbing, electrical, lighting, heating systems with the right standards in place could all communicate and be used to together. You can do this with cities too like Jenny and Phil. This needs time and effort.
How much effort depends how much you want to learn about journeys and activity. It depends how smart you want to be.
