Sonar is SME Water’s acoustic noise logging analysis suite. By combining water network data from GIS with deployment and fleet status data from logger suppliers, we’re able to offer our clients insights including:
- Maximum achievable acoustic coverage
- Optimised logger deployment plans
- Fixed network logging performance analysis
- “Lift and shift” logger deployment performance analysis
In this blog post, I’ll share my highlights of developing Sonar throughout 2026, including how it was made over 100 times faster!
My previous blog post described how Sonar’s optimiser had been made over 10 times faster by using multi-core processing. A timeline plot (Figure 1) was used to illustrate how DMAs (the coloured bars) were analysed in parallel. It was remarked that as some DMAs took much longer to analyse than others, it might be possible to reduce overall run time by scheduling these large DMAs to run first. This would hopefully avoid such situations where most cores are idle for several hours at the end of the run while a few large areas finish processing.
Figure 1 – Sonar multi-core timeline (before scheduling improvements)
It was confirmed that, as suspected, optimiser run time for a DMA increases with the number of available acoustic logging sites. An option was added to Sonar to sort DMA input list by highest number of sites first. Using this across the analysis of all DMAs in a water company, the beneficial effect is illustrated in Figure 2.
Besides making for a much prettier plot than Figure 1, it shows how the scheduling improvements can save time overall (note that the workloads here are different, so the actual time durations on the x-axis can’t be directly compared).
Figure 2 – Sonar timeline (after scheduling improvement)
It had become clear to me that any significant reduction in optimiser run time would now have to come from a change in approach, rather than incremental improvement.
Sonar uses an open-source database extension named PostGIS. This provides geometry data structures, and functions which perform useful operations on them. For example, it has a function which finds the difference between two geometries, and another which finds their union (combines them together). Sonar’s optimiser core logic called some of these functions many thousands of times while generating a deployment plan, and most of the run was spent waiting for these calculations to complete.
When Sonar was first conceived these functions had seemed like a wonderful gift, but over time it became apparent that they were computationally expensive for our use case. It seemed like there might be a more efficient way to process Sonar’s geometry calculations. I turned this over in the back of my mind for some months until an idea began to take form. Many sketched diagrams and revisions later, I had enough confidence in the idea to add it to Sonar’s development plan.
The key was to rephrase the same question to the computer using much simpler data structures and operations than in the PostGIS library functions. I won’t describe the new approach in any more detail here, but I applied many lessons I’ve learned over the past 5 years of software development. During final testing, the old and new optimiser implementations were run across a real-world load (the same as shown in Figure 2), analysing every DMA in one water company. The time to complete analysis of the whole distribution network was reduced from 27 hours to 16 minutes! This is factor of over 100 and needless to say, I was ecstatic with the result.
Figure 3 – Optimiser run time reduction
With such radical reduction in analysis run times, the Sonar server’s schedule is freed up. This effectively gives us time to run more analysis. Multiple clients had expressed interest in being able to generate optimised deployment plans using different logging technologies across certain areas of the network. This had a clear use case to, for example, generate hydrophone plans for areas with mostly plastic mains where accelerometers might perform poorly.
Some development was required to support this. Sonar was initially implemented as a single pipeline. The user would configure the logger capabilities, the analysis would run, and results would pop out of the other end. The system could hold only one configuration and one result set at a time (Figure 4).
Figure 4 – Sonar multi-scenario support
To support use of mixed technologies across a network, the concept of multiple “scenario” configurations has been introduced. These are stored in a register, where each scenario represents a chosen logger technology and set of distances covered across different mains materials, and rules to specify which parts of the network it’s appropriate to analyse. The pipeline has been reworked to support running analysis and storing result sets for multiple scenarios. It now also tracks the last time a scenario was analysed, and knows if the network data has changed, to facilitate keeping the analysis of all scenarios up to date.
All this work behind the scenes now allows Sonar to deliver more insights to our clients, and faster.
The optimiser core re-implementation was particularly rewarding to work on. On reflection, I’ve found it very interesting how most of the work for this was done before touching the computer: first in my head, and then on paper. A lot of human head-scratching up front eventually allowed for a sleek and simple implementation in the computer.
We have been talking recently at SME Water about the concepts of Kaizen (continuous, gradual improvement) and Kaikaku (revolutionary, step improvement). I think the development journey of Sonar is a great example of why both are necessary. I’m grateful to have had the opportunity to try something innovative and revolutionary and look forward to continuing to grow my skill set in the coming year.
