Reliable data models are absolutely essential for a company to align itself on a common framework and a set of metrics that enable everybody to build and share an understanding of the business. The more you have a shared understanding of business, then you can think about what are the key inputs to my business decisions, what are the key inputs to my performance evaluation, how do I think about the data with a common set of nomenclature to be able to do all the right things and get all the value that I can get out of a data set that I can trust.
The faster your queries run, you actually lose revenue. In hindsight, when you’re not in the thick of things, it’s easy to see that it’s a no-brainer decision. But the company did debate this. They wondered what they should do – if we do this in the next quarter, revenue is going to be down. However, over a period of time, it will come back.
I bucket the world of start-up investments into three categories:
– Invested, and happy
– Invested, but regretted
– Didn’t invest, and regretted
Every venture capitalist goes through this. Having said that, there are two things that happen here: sometimes we get a chance to invest in a company, or we don’t invest and the company just continues to run and we are never able to get into it.
Data analysts and data scientists are focused on building a model because they are trying to get to a particular outcome, and they want to get there as fast as possible.
The data engineer is all about building data pipelines, data ingestion, and data cleansing. They want to build an infrastructure that is going to enable them to operate on their data at scale in the most optimal way.
The point of convergence between data science and data engineering is where organizations have the opportunity to really have a force-multiplied effect, or have a significant step function in terms of what they are able to do.
When Snowflake started, it was all about building a data warehousing solution, built for the cloud. They started with a fresh design from scratch that’s a cloud-based service that takes advantage of all the attributes of the cloud. It was primarily a data warehouse installation.
Over a period of time, they started thinking about a huge problem in the world of data, which is the ability to share data across teams, companies, and organizations.
The first thing to release is now every company is thinking about digital transformation. Every company realizes that they need to be on the cutting edge of technology adoption to be able to better engage with their customers, to be able to do better at whatever they want to do.
The other realization that goes hand-in-hand with digital transformation is, people realize what a phenomenal asset they have in data and how they use data to advance their business in meaningful ways.
If you talk to an enterprise CIO or CEO, one of the problems they tell you about data is – they know they have a lot of valuable data, but their data is siloed. Every system has its own data and its own storage.