Unpacking the next data platform is a crucial process in the constantly changing world of data and artificial intelligence. It involves understanding metadata knowledge graphs and how different layers of the modern data stack come together.<\/p>\n
If one wants to do anything with data, they need a stack of tools to get it done. The stack has not changed even with all of the innovation that\u2019s been happening in the data industry, according to Gaurav Pathak<\/a> (pictured, right), vice president of product management AI and metadata at Informatica Inc.<\/p>\n George Fraser of Fivetran and Gaurav Pathak of Informatica talk with theCUBE about metadata knowledge graphs.<\/p>\n<\/div>\n \u201cPlayers have changed. But that stack, moving the data from raw data to really processed insight, has remained quite similar with [metadata knowledge graphs],\u201d Pathak said. \u201cWe are looking at triples, we are looking at relationships between individual metadata objects.\u201d<\/p>\n Pathak and George Fraser<\/a> (left), chief executive officer of Fivetran Inc., spoke with theCUBE Research\u2019s Rob Strechay<\/a> and George Gilbert<\/a> at the Supercloud 7: Get Ready for the Next Data Platform<\/a> event, during an exclusive broadcast on Media\u2019s livestreaming studio. They discussed the evolving data stack and the role of metadata knowledge graphs.<\/p>\n Informatica collects technical, business, operational and usage metadata about data assets, according to Pathak. That also involves collecting information such as schema and structures about what data looks like in Snowflake and Databricks.<\/p>\n \u201cWe look at how is that pipeline created, what are the transformations, how [are] all of these things related to each other?\u201d he said. \u201cHaving that triple, having that metadata knowledge graph, then allows you to now start doing, both human-wise and AI-wise, intelligence queries to the data ecosystem itself.\u201d<\/p>\n Companies can ask questions as a result of that, according to Pathak. Those questions could include how many Iceberg tables a company has.<\/p>\n \u201cHow many of them are used by people in [the] marketing department? And how many of them are compliant with GDPR<\/a>?\u201d Pathak said. \u201cTheir data is not moving from one jurisdiction to another. These kind of questions are really, really hard to get early on. But with metadata knowledge graphs, with catalogs like these, these are now possible.\u201d<\/p>\n It\u2019s understood now that metadata has been centralized, things such as usage, consumption, financial management and governance are much easier to manage. There are a lot of new workloads happening right now, according to Fraser.<\/p>\n \u201cThat\u2019s one of the big phenomenon that we\u2019re seeing, is customers are doing more new workloads with their data. From a Fivetran perspective, that means new data types,\u201d he said. \u201cIt means there are things that previously didn\u2019t belong in the central data estate, now belong there. Mostly freeform text stuff. Fivetran has had connectors to systems like Zendesk and Slack for many years that have freeform text, but there\u2019s a whole new emphasis on those systems.\u201d<\/p>\n Beyond AI\u2019s evolution tied to freeform text, there\u2019s also an evolution tied to a demand for more diverse sources of data, according to Fraser. The other point of evolution has to do with latency.<\/p>\n \u201cSome of these more operational type of workflows that people want to do with AI agents and things like that, they require fresher data,\u201d Fraser said. \u201cThe first Fivetran pipeline 10 years ago ran once a day. And now, the milestone we\u2019re trying to get to is where we can reliably do one-minute latency for all data sources.\u201d<\/p>\n Fivetran considers all these to involve workloads that run on the data it delivers. That evolution means more sources and new entities within existing sources, according to Fraser.<\/p>\n \u201cIt maybe means more adoption of data lakes as the compute engine people want to use to power some of these new workloads is maybe one that doesn\u2019t even exist yet,\u201d he said. \u201cThose are the main, I think, evolutionary pressures that we are feeling from the data pipeline perspective.\u201d<\/p>\n These technologies are extremely powerful, according to Pathak. There are more changes on the horizon to come, too.<\/p>\n \u201cWhat will change is that people have thought about code as something that needs to be maintained pristine. It has to be taken in for a long time. There was a whole ecosystem around it,\u201d he said. \u201cBut if you have gen AI systems that can convert English into natural language statements and then take decisions on what\u2019s the right formats, what are the right models to store that data in, I think that will be a very different world that we will live in.\u201d<\/p>\n Stay tuned for the complete video interview, part of SiliconANGLE\u2019s and theCUBE Research\u2019s coverage of the Supercloud 7: Get Ready for the Next Data Platform event<\/a>.<\/p>\n![\"George](\"https:\/\/d15shllkswkct0.cloudfront.net\/wp-content\/blogs.dir\/1\/files\/2024\/08\/George-Gilbert-theCUBE-George-Fraser-chief-executive-officer-of-Fivetran-Gaurav-Pathak-vice-president-of-product-management-AI-and-metadata-at-Informatica-Supercloud-7-300x200.jpg\")
<\/p>\nMetadata knowledge graphs enable more action<\/h3>\n
AI demands diverse, fresh data sources<\/h3>\n
<\/h5>\n