Thought Leaders in Big Data
via sramanamitra.com
Peter provides some excellent insights into open problems around which new entrepreneurs can build companies.
Sramana Mitra: Let’s start by having you introduce yourself as well as Instaclustr to our audience.
Peter Nichol: I’m the CEO of Instaclustr. I’ve been CEO and various other positions in the technology industry for nearly 30 years. I’m based in Silicon Valley. Instaclustr is a company that was founded in Australia in 2013 about four years ago. We focus on providing Database-as-a-Service to our customers with a specific focus on not just open source technology starting with Cassandra but also expanding to other technology offerings such as Spark, Elasticsearch, and Kafka. We take these open source technologies and manage them on behalf of our customers in the public cloud through AWS, Azure, and other cloud platforms.
Sramana Mitra: Can you talk a little bit about what’s happening in that ecosystem and what are the trends that are driving your business?
Peter Nichol: The major trends that we align with are the following. First of all as everybody knows, we’ve seen a massive movement to the cloud in the past few years as companies decide not to invest in their own data centers and move to Amazon or Microsoft. It certainly is a more flexible way of providing IT infrastructure, especially for smaller companies. They don’t have to make a massive investment in building their own infrastructure, so it’s a lot easier for smaller companies to get started. That’s one major trend.
Even industries that, five years ago, were hesitant to move to the cloud for security reasons are starting to move to the cloud as well. It’s pretty amazing given the reluctance they’ve had in previous years to do so. The other major trend is a large focus on companies wanting to use open source software so they don’t end up spending a lot of money on proprietary solutions and they don’t get locked in. They have a very strong preference for open source software.
The final trend is the trend towards companies outsourcing what is not their core competency. They don’t want to focus on every aspect of providing IT. They want to outsource as much as possible and focus on what they’re really good at, which means focusing on developing their applications and outsourcing the infrastructure aspects to companies like Instaclustr.
Sramana Mitra: What do you think is driving the aggressive adoption of open source?
Peter Nichol: A couple of things. First of all, open source technology has become very mature. Five or ten years ago, a lot of companies would not have trusted open source technology from an overall robustness perspective. Companies are seeing vibrant, well-supported open source communities that they think will survive and be healthy for a long time. It’s also the ability to use software which doesn’t lock them into a particular supplier.
I would say it’s those three things. It’s the fact that open source software is a lot more mature, the communities are vibrant and healthy, and companies can rely on this type of software for production systems without locking into a vendor which can be very expensive in the long term.
Sramana Mitra: At the same time, these open source technologies are fairly complex and require external expertise for them to be able to manage them. You are one of the specialized vendors whom they’re reaching out to to get that expertise in?
Peter Nichol: I think that’s right. The companies that base their business on open source are providing a lot of expertise around open source technology. For example in our case, the technology we deal with is a database technology. One example is Cassandra which is a NoSQL database technology. It’s used for storing terabytes, or even petabytes of data, by companies who are writing applications that may have thousands or even millions of users like Facebook.
They distribute that across multiple regions running in Amazon, for example. Managing that is very complex. We take away from the customer the burden of managing the technology, but we don’t charge the customer license fees. It’s a different model. They’re not buying software from us. They’re still using open source software for which they don’t have to pay license fees, but we take away the operational burden of managing that software. That’s the big difference.
Sramana Mitra: Talk a little bit about the use cases in which they’re using these kind of technologies in your orbit.
Peter Nichol: There are a few use cases that we focus on. Obviously, social media is a big one. In fact, a lot of these NoSQL technologies were invented by companies like Google, Facebook, and Yahoo! Anybody who’s writing an application that involves hundreds of thousands of users and massive amounts of data is going to use a NoSQL database like Cassandra and is going to approach a company like us for help if they need expertise in managing that. Another use case is IoT where we see a lot of traditional companies.
There are companies that are involved in providing transportation. They use a lot of sensor data to identify the exact location of a particular vehicle minute by minute. They will feed that information back into a central system and then do analytics on that to help run their business in a more efficient way. This is happening with pretty much every device that you can think of – automobiles, refrigerators, home devices. They are all being connected to the Internet for collecting data which provides intelligence to the business or to the customer.
That obviously involves a massive amount of data. That data has to be managed in the cloud. What companies are looking for when they talk about managing their data is they want their application and their system to be up and running 99.99% of the time. They cannot tolerate any outages because an outage means lack of revenue or customer dissatisfaction. It’s all about providing a robust database system which will never go down. That’s basically the value proposition.
Sramana Mitra: In this general universe that you’re operating in, where do you see open problems? If you were starting a company today, where would you look into?
Peter Nichol: That’s a great question. I would say there’re probably two areas that are big trends emerging in the future that haven’t been addressed in a meaningful way. We’ve seen the movement of a lot of companies to the cloud. I think the next trend would be companies looking for cloud independence.
In other words, a lot of companies have moved to AWS, for example. That’s the biggest cloud provider out there. Companies are now going to be starting to look for independence. In other words, they don’t want to be locked in to a specific cloud provider. They want to have some degree of independence and they want to be able to run their applications across multiple cloud providers and not just Amazon.
Sramana Mitra: How does that play out? Does that mean that they will have some sort of middleware that’s their own and that interfaces with multiple public cloud vendors that risk-distributes across difference vendors? Or do you expect that they will be putting in investment in infrastructure themselves?
Peter Nichol: The bigger companies will be doing this themselves. There are opportunities and this is where I would focus my efforts. This is where I see gaps. There are opportunities for companies to start providing that infrastructure software – that middle layer – which will give companies the ability to operate over multiple cloud providers. That’s one of the gaps that I see in the market at the moment.
Sramana Mitra: Very interesting. Excellent. Is there anything else you want to include in this conversation?
Peter Nichol: This is another trend which relates very much to the one I just mentioned. There’re probably two other trends that I would mention. One trend is the move towards shared infrastructure in general. Typically, companies run a lot of applications in a very siloed way. They have a separate infrastructure for different applications. That’s a very inefficient way of using infrastructure.
We’re starting to see a lot of companies utilizing tools that allow them to run multiple applications on a single-shared infrastructure. That’s going to be a big trend going forward. The last trend, which is interesting, is new technologies which make application development much easier.
One example of this is Kubernetes which is a technology that came out of Google. It allows the application developer to develop at a higher level of abstraction and takes away a lot of his burden of knowing exactly what database it’s running on, knowing exactly what cloud provider it’s running on so that application developers can develop an app that can run anywhere, on any cloud, on any database.
Sramana Mitra: This is a trend that we have seen in many different formats. The AI engines are trying to accomplish that. Even Microsoft is trying to accomplish a level of abstraction so that people with good domain knowledge can build AI apps without having to be AI geniuses. The layers have become very complicated. The abstraction is becoming a real requirement now to move forward in this very complex technology stack.
Peter Nichol: Absolutely. I think it’s going to be something that further pushes companies towards using applications and infrastructure across multiple cloud providers, including potentially their own internal infrastructure.
Sramana Mitra: Terrific. Thank you for your time.