Tools or Culture — what came first?
Learning from the past
Reading “Sapiens: A Brief History of Humankind” by Yuval Noah Harari, was a joyous experience, full of eye-openers and moments of revelation. One of the parts that stayed with me is where Yuval is discussing the agricultural revolution, and the question about how the first cities came to be.
The conventional viewpoint is that humans, for reasons summarised as the “Luxury Trap” i.e. the quest for convenience, ended up cultivating wheat and gradually building more complex settlements, societies and cultures. There are some archeological discoveries though, that tell a different story.
There is a site in present south-east Turkey, called Gobleki Tepe. An excavation there, unearthed monumentally sized pillared structures with sophisticated decorative engravings. The discovery was in a way similar to Stonehenge in Britain. However, although Stonehenge dates to 2500BC, Gobleki Tepe dates to ~9500BC, meaning that it was constructed by hunter gatherers, before the agricultural revolution. In addition to that, one of the first domesticated variants of wheat was also traced to the same area. So, it would seem that humans built temples, well before they had villages and agricultural societies and cultures. The temple came first and the village grew later around it.
The first seeds of agriculture and settled lifestyle in contrast to the hunter-gatherer way of life, were sown during the process of building temples, that were for some reason very important to a large number of different hunter gatherer groups. Those groups, even though independent and autonomous, collaborated closely to build structures that were only possible through the development of technology, that in turn would bring with it a series of changes leading to a cultural revolution.
Cultural diversity
As we’ve discussed in previous posts, DevOps occupies a space in the fertile intersection of contradictions and dualities, such as “Development” and “Operations”, or “Tools” and “Culture”.
It’s been argued that culture trumps everything else e.g. strategy. But how does culture actually come about? It can emerge from and be shaped by a variety of factors. Tools and the behaviours they create, can act as catalysts for cultural changes and revolutions. Culture can be created and transformed in a bottom up fashion, by emerging behavioural patterns.
On the tooling front, DevOps has historically been focusing on the automation of provisioning and delivery. Configuration Management, Cloud Orchestration and CI/CD pipeline tooling, has been the focus of work described as “DevOps”. Products and projects from that space occupy the lists of CVs and job ads that have the word DevOps in them. The crowning of Kubernetes and its ecosystem, as the de-facto tool for managing all things Cloud, has made its use a necessary condition for any technical organisation that claims to be “doing DevOps”.
The issue with that, is the inevitable navel gazing that it brings with it. DevOps is a culture is that calls for breaking out from the technology silo and breaking free from the narrow strip of the Dev-vs-Ops friction. It wants to reach out to the rest of the organisation, to all of its functions. To encompass and engage with Product Management, Marketing. To collaborate with Sales and Finance, and work along with Legal. DevOps helps IT and “the Business” to develop a common language, and work in harmony towards their Vision and Mission.
Arguably, this is a tall order and can only be achieved step by step and inch by inch. I argue that tools can play an important and pivotal role in this process. But tools that begin and end with only the Ops or Dev concerns in mind, cannot play that role.
Sit back and watch
This is where Observability and its tooling are different. Observability’s objective is to help us understand what our increasingly complex and hard to grasp systems and applications are doing at any one time. It tries to help us answer questions about our systems’ state.
- Are they healthy?
- Are they performing optimally and efficiently?
- Which parts are the busiest?
- Where are the bottlenecks?
- Which parts are the least secure or under the most attack?
- Which areas are the hottest and most expensive to run?
Observability can help us detect and fix issues in a timely fashion. It allows for application related metrics and concerns to be combined with business related ones and bring about Intelligence that encompasses as many organisational aspects and functions as possible.
Bionic vision
In the heart of Observability are Application Performance Monitoring (APM) solutions. I first came across one of those towards the end of the Noughties. It felt like magic. My first reaction I shared with my colleagues was that the tool was giving us “X-Ray” vision as systems engineers. As it happens, an APM product with that name was later launched by a well known public cloud provider. In another instance, I exclaimed that at last this was able to help us find the “needle the haystack”. Naturally, a project with that name exists in the APM space.
When I joined Elastic as a Support Engineer, I did not know that soon after that, the Elastic APM solution would be launched, following the joining forces with Opbeat.
Naturally, I was delighted to be able to get to work so much closer with an APM solution. This time not as a user, but as part of the team that works with the developers that build it, whilst helping its users to make the most of it.
Staring at the same dashboard
But it was later, that I fully understood the power that the APM solution as part of an Elastic deployment can offer. The key aspect is that Elastic APM data are just another Elasticsearch index. It’s well known that Elasticsearch can help ingest, search and aggregate on a huge variety of data.
The APM DevOps catalyst behaviour of materialises, when we have the ability to combine data from different sources in custom visualisations and dashboards. Building dashboards that link application errors with sales figures, or application utilisation metrics that are correlated with product revenue and third SaaS costs. Creating compelling observability views that provide insights, unlock information and help different people, roles and functions, to connect the dots and see the bigger picture in high definition.
It’s the kind of tooling that can “effortlessly” lead to the emergence of a culture of collaboration, empathy and shared objectives, from the ground up, built on solid and tangible foundations.
Disclaimer
I work for www.elastic.co
