Creating a Symphony of Data Analytics with Kevin Hamilton

10 - GAT Podcast: Force Multiplier - Creating a Symphony of Data Analytics

MJ 00:01 Hi, everyone. This is Michael Jelen from the Global Applied Technology podcast, The GAT Team, as we call ourselves, are a globally distributed team of software engineers, data scientists, graphic designers, and industry experts who serve clients through our products built atop the BRG DRIVETM analytics platform. We're helping some of the world's largest and most innovative clients and governments transform raw data into actionable insights, drive efficiency through automation, and empower collaboration to improve business decisions. You can learn more about us, our products, and our team on our website, brggat.com. And if you have any questions or comments, please email us at gat@thinkbrg.com.

Today, I'll be speaking with Kevin Hamilton, the managing director of our Global Applied Technology team. I love this episode, since we discuss how to solve one of the pain points that brought us together to form this team several years ago. Many companies have several business intelligence tools used by different departments, creating a difficult environment to effectively share insights and collaborate across the organization. For that reason, we created our Symphony product, which provides a technology agnostic platform to identify and curate the best analytics to drive businesses forward. You can learn more about Symphony and our other products in our website, brggat.com. And please enjoy this conversation about Symphony with Kevin Hamilton. Hi, Kevin.

KH 01:19 Hi there. How are you doing?

MJ 01:20 Doing great. Thanks for making the time. Very excited to chat today about the ever-changing landscape of business intelligence tools.

KH 01:27 Yeah, for sure. It's good to be back. Thank you for having me.

MJ 01:30 Great. Well, I know that you've been on one [podcast] before, so people probably know who you are. But I would love a quick little intro, especially in this space and in some of the things that have led you into this area.

KH 01:40 Yeah, absolutely. So, my name is Kevin Hamilton, one of the founders, with Mike, of the Global Applied Technology team. Prior to that, I was a healthcare consultant for about twenty years. So, my background is in utilizing data and analytics to try to solve problems in the healthcare space, specifically with hospitals and how to improve the care they're providing, as well as help them run better operationally. So [I’ve] been deeply entrenched in data and that space for, like I said, twenty years, and it was a messy space and it's getting better and there's more tools. And so, looking forward to talking a little bit about that ever-changing landscape of business intelligence and what I've seen over the last twenty years.

MJ 02:23 Great, thanks. And I know that this topic in particular is very near and dear to both of our hearts. I think, honestly, this is really what sparked our collaboration right from the get-go.

For those who don't know, my background was largely in data analytics, but not specifically in a certain industry. And over the course of the last fifteen years or so—that's transformed wildly from fifteen years ago—here's a data set, go lock yourself in a room and come back with an answer to a situation now where no individual probably knows enough information to be able to get to the bottom of, or understand and interpret, data properly in an organization. You need that collaboration between people that maybe have industry expertise, maybe you have to have a multifunctional team with some lawyers, accountants, and tech people. And then, together, they're all using some sort of BI platform or data tools to get to and collaborate on what ultimately ends up being something that changes the future of their business together.

And so that, really, I think, is one of the reasons that we started the practice that we're in and built some of the tools that we did. So very excited to dig into that with you a little bit more. And before we get started in the meat of it, I was just wondering, there are a lot of buzzwords in this space, whether it's business intelligence or data analytics platform, self-service analytics, data, or BI catalog. What are these things, and how do you differentiate between them? Or are they really all saying the same thing? What can people look for in the marketplace to understand what is business intelligence?

KH 03:52 Yeah, that's a great question. I think that there are some similarities among those words, and then a few differences. So, I would classify business intelligence as kind of the overarching idea of taking raw data and turning it into information that end users can act on. So that would be kind of business intelligence, and there's a lot of tools that have allowed that to really come to fruition. It used to be that it took a bunch of analysts in a room—kind of back to your background: you kind of locked yourself into a room and you built the analytics for people.

In the more recent years, as these tools have gotten easier to use, you've got self-service analytics. So self-service analytics kind of goes along with business intelligence, but it is what it sounds like: providing self-service to people. So, more that an end user can create their own analyses and dashboards and tools versus having somebody that is super specialized do it.

And then all those ride on a data analytics platform. So, a data analytics platform could be one technology, or it could be multiple technologies. I would classify the data analytics platform probably as including the underlying data, which can be in various different data stores.

And then on top of that, you would have the kind of user experience visualization layer, which is provided by some of these tools. And then what you have at the end of the day is you have a bunch of dashboards and investigations and things like that that somehow have to be brought together and catalogued. And that's part of what we'll talk about today is some of our ideas around how you go about doing that.

MJ 05:26 Okay. So is it safe to say that it's kind of a—it's a piece of software that ingests data coming from one or more locations and allows the user to either, if they're a builder or potentially developer, to be able to create visualizations from that data. Or if they're just a consumer, they're able to take a look at visualizations and often interact with those visualizations to look at the underlying data in a graphical representation?

KH 05:55 Yeah, I think that's a good characterization. I mean, back to the earliest days—and there's early, early days of Quattro Pro spreadsheet and things like that—but probably the one that most everybody is used to now is Excel, being able to take an Excel data feed or a report and creating a bar chart or a pie chart or a pivot table. Where Excel starts to fall down is that those visuals don't tend to interact with each other. So it's harder to do what you need to to drill down. And there's kind of still a lot of exercise in doing that.

But Excel, I would be remiss to dismiss it. I mean, it's something that's used every day, and it will continue to be used every day. There's a lot more advanced tools now that have come out, and I think in some ways they do a way better job. But if you just need a pie chart, Excel's fine. And I think that's what people are used to.

MJ 06:46 Yeah, and I suppose in reaction to Excel's lack of linking together different interactive visualizations, a ton of companies popped up, especially over the last ten years, to try to solve this problem. And also, I think to be able to deal with larger quantities of data that are more common in firms now than they were previously, and somewhere that Excel often hits limitations. So as all of these new, different software tools popped up in the marketplace, and some of the big players like Microsoft obviously has their own, a ton of independent companies started popping up in offering their own. What are some of the different pros and cons? Or just, what does a person do to start deciding which of these tools they want to implement in their business? And does it really matter?

KH 07:30 Yeah. No, those are all good questions. And you're right, the rise of the visualization engines—for lack of a better term—has been pretty profound in the last, I'd say, ten years or so. There are plenty that are out there that are fairly general tools that work good for quite a few different things. And then there are some that are specialized.

I think it's appropriate to look at the different tools and pick the right one for the job. And that's really the stance that we come at it from: that Power BI, Microsoft's product, is better at some things than some of their competitors, and vice versa. And so, our ideal is that an end user or a designer can pick the tool that's right for the job, and the one they're most comfortable in, and you don't have to be hemmed in by a specific license to one tool. You choose the best tool for the job. And what we see is that that is the case.

In the past, there was kind of a shadow IT idea where people kind of got to go out and do their own and build their own IT function in some ways. Well, that's kind of been curtailed. But what we see now, and what we have been seeing in the last probably five years or so, is there's a shadow BI function. So, an organization might have its own BI group, which is great, but end users are going out and procuring their own business intelligence software, whether that's commercial software or an open-source software, which there's now a lot of as well. And so, you have the rise of these decentralized business intelligence functions. And so, what it's turning into is, you've got a little bit of a Wild West going on where organizations find that they have much more business intelligence software than they thought that they had. The problem you run into is that all of that is siloed, nobody knows where any of that is, and there is unfortunately not always the right quality controls on top of those pieces of software and what people are doing with them. So, like I said, it's gotten a little messy, and I think that's what we're looking to solve.

I read a survey the other day that said that 25% of organizations use ten or more BI platforms. That number astounded me. So, I would guess that that survey is fairly encompassing of what a BI platform is. And 61% use four or more, and 86% use two or more. It's an informal survey, but from my background in working with our customers, that's probably spot on. I don't know that we have any that have ten BI platforms, but we have many that have four or more.

We have one client that has two different versions of one company's software, plus they have Power BI, and they have Tableau's. They have four that we know about. And they are running into this problem where the end users say they don't know which dashboard to go find. Do they want to look at a specific dashboard? They don't know what tool to go get it from, because they all have different logins, they’re in different disparate places. And so actually, that was a perfect opportunity for our platform we call Symphony, because it harmonizes these things together to get installed. So, we put our Symphony platform in place and we're able to draw the different analytics into one central location and then take that technology and make it so that no matter which underlying visualization engine is being used, the end user interacts with it the same way. So trying to, in some ways, abstract away having to know a specific piece of software for the end users, but still empowering the designers and the people that are building this business intelligence to use the tools to their fullest capability.

MJ 11:21 Yeah, that makes sense. I think it seems like a problem of scale for any of these large organizations, because within an individual team or silo, as you refer to them, when you have the accounts payable department using one tool and everyone in department knows that tool and knows the underlying data, that seems okay. But then someone in finance has a completely different tool that they're using. Marketing has a different tool. And ultimately, as that trickles up to leadership, they, number one, are looking at possibly different underlying data sources that may not be from the same time period. And that's often very confusing. And they're also looking at very different tools that display data in a different way. And so, it's often a difficult challenge for them to be able to understand what fits into to where and which team is actually using the correct data in certain circumstances. So definitely see how this can spiral out of control in a large organization or an organization that grew through acquisition where you have different skill sets in the underlying teams.

So, I guess when we talk about Symphony and the way that that solves that problem, ultimately, how does that work, or what does it do exactly?

KH 12:27 Yeah. No, happy to go through that. And I think you said something really quick that is spot on: people that are in one organization might say this is not a problem for us because we have tight controls over business intelligence, which could be the case. But what we're also finding now is that organizations within the same industry are interacting with each other. So, whether they're competitors or collaborators, and it might be two different organizations that are trying to do something in the market together, they need this as well, because inevitably, they've also chosen different visualization engines. So even if you feel like you have a tight business intelligence function, when you're trying to share information or visuals across multiple organizations, you still end up with the same problem.

So, with that said, what we've done with Symphony is we've layered Symphony as kind of an umbrella over the top of all these visualization engines. So we are not, by our nature, dashboard builders anymore. That's kind of in our past; we did that. But actually, at this point, we're spending all of our time on the Symphony product and utilizing people that are people that build dashboards—what I would term a data artist—whether that's inside of an organization or a separate firm, to build the dashboards.

But the Symphony tool sits on top of all of these platforms, and they connect into it. So what we're doing is, we are abstracting the functionality that exist in these platforms to a point where anybody that goes into Symphony sees the business intelligence that's available in their organization can curate it, can rate it, can do what's with it, and search it so that you can really start to find the best of the dashboards and tools that are available in the organization, independent of whatever the software that it was built in. And that's what we keep coming back to is that we want to stay as technology agnostic as we can because the technology is changing a lot. There's rising and falling. There's a lot of acquisitions that are going on in the market. Salesforce bought Tableau. Microsoft is putting a lot of resources into Power BI. Qlik is out there with two different products. And there's just a lot going on. And so if we stay technology agnostic, then we can plug those tools into Symphony, and the end users don't really care what the underlying tool is. And that was what our goal was.

MJ 14:59 Yeah. And I think we're also very lucky that in the evolution of software development, almost everything these days now is based on a web browser. So, all of the interaction and clicks and things like that can be handled through JavaScript. So, it's pretty easy to bring in any different webpage and wrap something around it to enhance that functionality.

So yeah, I think it's been very nice that all of the different tools, as wildly different as the underlying engine is to process and visualize that data, they all sort of present things in the same sort of manner, and I think that's made it much easier to try to integrate these things together as well.

When it comes to the functionality that Symphony imparts on the user when we wrap that visualization, what are some of the other things that we wanted to focus on that we think are actually the most important things in interacting with the visualization?

KH 15:50 Yeah, absolutely. I think that, par for the course, we have to be centralized, right? So, we centralize all the analytics. That's the key. From there, then we need to do things with them. And so, one of the first things we've seen as a requirement is that curation that I mentioned, because once you start to go through an organization and centralize all the dashboards and tools, you find that actually there's a lot of duplication. And so actually, straight out the gate, we tend to find there's quite a lot of return on investment for bringing Symphony in, because you'll have various people in an organization that have been creating very similar dashboards for their own siloed department, and they're refreshing those dashboards on a regular basis, and it's just a total waste of time.

Once you've curated and seen them all together, if you can say, okay, if I'm looking from a hospital perspective, I want to look at the operating theatres, for example, the operating rooms, I go in and I vote them all in. Now I search for operating room, and I find that I've got several from different resources, and so I can actually collapse those down into one core dashboard or tool that takes the best from everything. And now we don't have to be refreshing five different tools. We're only refreshing the one, and everybody can be comfortable that the data is right. It's coming from the right sources, and it's providing the metrics the way it's supposed to be.

So that's kind of our curation function. We’ve got a few different ways to do that. One is by providing search. We have tagging capability so it can be tagged with keywords. And then we have a rating capability. So, the idea that we want to see people rate these dashboards, and there's a lot that can come from that. If there's a designer or a business intelligence analyst inside of an organization that's consistently getting their dashboards rated very highly by end users, well, maybe you need to look at that person and look at how they could take on more of a leadership role if they weren't already. Versus if you have actually a dashboard that's being rated very lowly, so it's just not getting good ratings, but you're finding that your end users are using it a lot, that that's where you'd want to focus your resources to build that up and make it better and understand why is it being rated poorly if it's being used a lot.

Flip side of it, if you have ones that are being rated very highly, but they have low utilization: why is that? And so that's kind of that curation function of how do we make sure that the best dashboards and tools are rising to the top, and we're either getting rid of or improving the ones that are either unnecessary or aren't as good. So that's the curation function, kind of that rating idea. And then having a search we find is quite good.

On top of all that, then, is our idea around you need to be able to use the data or drive change within an organization. And so, what we've done is, we've layered a collaboration solution on top of it. It's part and parcel with Symphony. Now many of these tools individually have collaboration. So, whether it's Power BI or Tableau or others, you can make comments and you can start threads. But you're doing that inside of that particular silo of a visualization tool, and you can't do it across tools.

And so, what we've done is, we've connected these tools into the Symphony hub; you can collaborate on the tools no matter what the tool is. So, as you're looking at a dashboard, again, doesn't matter what the underlying visualization engine is, you can collaborate on it. You can have discussions about it. Then you can drive to a change event or an idea of how to improve the operations or whatever you're trying to do in the same way, again, no matter what the tool is. So, we've extracted that collaboration function, and we do it natively in Symphony for that reason.

MJ 19:38 So much good stuff to unpack there. I can't help but think that it also seems like the right time for a solution like that, as people are spending a lot more time working remotely, and the team structure has changed, where if you're not sitting side by side with someone in an office, perhaps certain work products are getting lost in the ether because there are so many other different products and things that people are building in terms of visualizations that, you're right, without a rating function or the ability to bring that to the top of mind or even make a recommendation. If Netflix and Amazon can recommend what movies you should watch and what products you can buy, why can't software recommend which visualizations are going to be the most useful to you based on other things?

So, I think that that's incredibly powerful as people work more remotely and then, of course, collaboration. We're not really sitting down in meetings anymore and going through a couple printed-out slides or images or even interactive visualizations together. We're trying to do that work asynchronously. I think that's incredibly powerful to be able to freeze frame certain things, start that conversation, and upload other supporting information to be able to facilitate the business decision that is based on that underlying analytical tool. So, I think that's all super important.

One of the things that also popped into my head that I wanted to spend a little bit time on, I did hear you mentioned centralization of the visualizations. I know a lot of organizations are probably thinking, "My data's not centralized. It's all over the place. This system does that, that other system manages this other thing." Does the data have to be centralized in order for us to centralize the visualizations? Or how does that work?

KH 21:14 Yeah, good question. No, it doesn't. I think that there are tools out there to help centralize data. That's not the space that we've chosen to play in. We've really left the data where it lies. So, for companies that have gone down the road of getting a large enterprise data warehouse, that's wonderful. It is not required for Symphony. That we are back to being technology agnostic, we're also data-location agnostic. So, if companies have data in multiple silos coming from different systems, some in the cloud, some on premise, it doesn't really matter to us, because the underlying visualization engines are connecting into those pieces, and we're sitting on top of the visualization engine. And that's actually become quite important, because there are—even in the most, I guess, most advanced organizations that have enterprise data warehouse—there are inevitably still data that sits outside the data warehouse that needs to be connected to, or maybe even open-source data or public data that's been coming out of other organizations. And so, we don't mind where the data is, we just really want to sit on top of the analytics engine.

MJ 22:27 Yeah, I think it dovetails really well with a previous conversation I had with Michael Hatfield on this podcast about building a data-driven culture and all of the things that come along with that. So, the ability to go to one single source of truth to find all of the different visualizations, regardless of what platform they were built in, that seems like a great step forward. The ability to tap into the same connection to a number of different data sources and look at them side by side and compare them to tell the whole story of what's going on with the data, I mean, that seems to also be part of Symphony.

Are there other things that you're thinking about right now or trying to build in the development roadmap that you think would enhance that capability?

KH 23:07 Yep, absolutely. In fact, the data question is one of our central focuses right now. And the reason is that we don't require centralization of data. We're not going to move the data. But from an end-user perspective, it's really important that that data has been validated and certified, so the dashboard is using the right data. And so one of the concepts we're working on right now in our roadmap is the idea of peer reviewed. So, like a scientific journal, we want peer review of tests and studies that are being done. We are looking at ways to make a peer-review dashboard or investigational tool. So not only has a designer or a business intelligence analyst built a dashboard, somebody has looked over their shoulder, looked at the data sources they're using, made sure the calculations are being done correctly, and has actually peer reviewed that. So, we're looking at how we add either a blue checkmark if we wanted to go the Twitter route or some other functionality around that peer-review idea so that when an end user, be it a manager or an executive, is using that tool to create a PowerPoint or to present, they can be sure that the data is right, because that's a trap we see people fall into.

And again, it goes all the way back to this self-service analytics. Self-service analytics is great, but people that are not dyed in the wool, data analysts perhaps, could be making a mistake here and there or grabbing the data wrong, picking the wrong time frame, whatever. So, it's pretty important for us that we move into a peer-reviewed model. So that's one thing we're thinking about.

MJ 24:48 Yeah, and another thing we're thinking about is the continued rise of remote work as we live through the pandemic. How do we work together with people on an individual analysis or dashboard? We have our collaboration functionality. We're looking at how we potentially layer in some real-time analysis or work with each other so that people can actually be viewing the same analytic and working together. We're starting to get used to that, whether it's Google Sheets or things like that where you can work on the same tool. We want to actually start to look at how that comes to play with these different visualization engines as well. So that's certainly something that we're looking at.

KH 25:30 We have lots of ideas around more connection points. So, one thing that's come up recently with one of our clients is that these visualization engines are all well and good. Some people still like PowerPoints and PDFs, and we need to meet people where they are. If that's what they're comfortable in, that's okay. So, let's go and look at how we do that, but let's do it right.

So, as we have connection points with some of the big file shares online, whether it's OneDrive, SharePoint, Box, whoever else, how do we integrate those pieces into the platform so that business intelligence in the form of more of a static report or a PowerPoint or something like that is also integrated into what we're doing so that you have the thing at your fingertips that you need. Not trying to compete with them. We don't want to become a big online file storage system. That's really not our point, but with the idea that we want to bring the right business intelligence to people's fingertips so that they make it easy for them to use. Really that mantra. PowerPoint and PDF still exist, so we need to look at how we want to handle those.

So, there are a few of the things we're thinking about. Lots of activity in the space, but really are just our core tenet is how do we make the UX, so user experience, for end users as good as it can be and [for] end users that are nontechnical? So again, my background coming back from healthcare, there's quite a few nontechnical people in healthcare as there are in a lot of organizations, but it has to be easy to use, because if it's not easy to use or they have to go searching for the analytics that they want, they're inevitably just going to go with their gut a lot of times, or they're gonna get frustrated, or they're going to spend a lot of time, and we don't want that. We don't want people just going with their gut. We want the analyses and the dashboards to be right there at their fingertips, easy to use so that they can use data and information to make their decision.

MJ 27:25 Love that. Wow. Tons of stuff coming down the pipeline. Very, very cool. And I do love the fact that this tool really evangelizes the idea of bringing a data-driven culture to everybody in the organization, regardless of technical ability and regardless of the toolset that's being used. So, it sounds like such a great, great tool, and [I’m] very excited to hear more about it as it continues to evolve and develop. For people that are currently living in this situation day to day, maybe they have a bunch of BI tools that are inside the organization and they have different data sets there, they're just dealing with the headache of not being able to get to the right information and collaborate with their teammates and colleagues. What advice do you have for them, and how are ways that they can try to get ahead of this problem and help take back the decision-making capability through data for their business?

KH 28:15 Yeah, I think my takeaway, obviously, is we believe our solution is powerful to help with this issue of multiple business intelligence tools. What I would also encourage people [to do] is don't reactively decide you have to centralize to one BI tool. There are definitely goods and bads of all of the major business intelligence tools, visualization engines. And the process of transitioning from one to another can be painful and expensive. And when you're making a bet like that, you're betting that that business intelligence tool that you're moving to is going to be the one that's on top for the next few years. The way that the business intelligence space is moving, that's potentially a bad bet. There are new entrants all the time, so I would suggest [you] don't spend time moving to one tool or trying to centralize to one particular visualization engine, let people build in what they feel is the best and then layer a solution such as Symphony on top of it so that the end users can utilize the dashboards easily. That's, I think, where we see the future going, is stay agnostic to the engines but allow the end users to do what they need to do.

MJ 29:39 Perfect. Well, there you have it, stay agnostic, let people build in the best tools for the job, and try to decentralize the building, but centralize the availability to see those visualizations and interact with them. So, thank you so much, Kevin. Really appreciate it. It's been a pleasure to learn about Symphony and the very, very rapid pace of change in the business intelligence area. Looking forward to our next chat. Thanks so much.

KH 30:04 Yeah, thank you. Have a good day.

MJ 30:05 You too. Bye-bye.

The views and opinions expressed in this podcast are those of the participants and do not necessarily reflect the opinions, position, or policy of Berkeley Research Group or its other employees and affiliates.