Every time we look at data analytics and data systems, the idea of having a way to manage control and explain the data is actually as important as the data itself. This episode is all about metadata, specifically, metadata related to data analytics analysis, Big Data computation, sort of the data lake metadata problem.
Today we discuss the challenges of data management, but also the potential of understanding so much more about how data is used. If you are an IT professional or a data professional, you will find this conversation about how we’re going to draw inferences, manage and control all of that data that were collected fascinating.
Data comes from many different places, sources, and ways. Some data we call dark data, which is data not accessible to you, and all of it relevant. Today we talk about metadata as part of the governance control management exposure of data.
An important layer beyond the data itself is the governance intent, how people access it, and how you combine data. We discuss exactly what that is, but still only touch the surface.
In the Cloud 2030 Podcast episode on metadata and building a data control plane, Rob Hirschfeld emphasizes the challenge of controlling data egress due to its diverse sources and destinations. He contends that rather than attempting to create a locked box for all data, the focus should be on embedding information, particularly metadata, to control data consumption. Hirschfeld envisions a distributed system for a data control plane, involving multiple parties managing and providing consistent rules for data use, acknowledging the complexity of data storage across various locations. He encourages listeners to explore the insightful February 2nd episode and engage in ongoing conversations at the2030.cloud.
Metadata is information that travels with the raw data that provides context, provenance, security, authorship, controls, and indexing.
The number of ways that you can expand the use of data is controlled by adding metadata. It creates a change in how we look at and manage data. Instead of creating control systems that contain the data, it’s actually packaged the control infrastructure, or the data control plane, as part of the data so that all of the systems can participate in it.
We also talk about data mesh a lot in that context.
In the Cloud 2030 Podcast episode from January 19th, Rob Hirschfeld discusses the challenges and opportunities associated with metadata, particularly in the context of managing and sharing data effectively. The conversation explores the concept of packaging raw data in a way that makes it available on request without being included, accompanied by metadata detailing its provenance, access permissions, context, and even expiration dates. Hirschfeld emphasizes the importance of building a data control plane to navigate the complexities of data consumption and production, envisioning a future where metadata plays a crucial role in weaving together diverse data sources. Listeners are encouraged to explore the full 40-minute discussion for comprehensive insights, and to join ongoing conversations at the2030.cloud.
If you love data and data context formats for exchanging data, you will love this conversation.
Today’s episode is a deep conversation about the potential ability to define ways in which we produce, store and share data, providing context using markup languages, and then being able to extend that. It’s a fascinating conversation about how much we could improve our use of data if we were able to provide more context about who wanted to see it and what relevance it had.
We also have some interesting conversations about data migration and how we share information.
We discussed the implications of chat GPT for it and the industry.
In today’s episode, we spend a lot of time figuring out how data provenance governance, bias, and ownership will impact chat GPT in IT and technology and cloud contexts. This discussion really looks into how chat GPT can be used in disruptive ways, but also in protective ways as what we describe as guardrails for how these systems are going to get built.
In the Cloud 2030 podcast’s January fifth episode, CEO Rob Hirschfeld explores the complexities of data provenance in ChatGPT, questioning ownership and control of the generated content. He emphasizes the need to understand the sources of data, pondering whether the output belongs to users, the algorithm, or no one, highlighting the challenges of systems that belong to nobody. Hirschfeld also connects this issue with Software Bill of Materials, emphasizing the importance of knowing the components of systems for accuracy and confidence. He encourages listeners to delve into the full episode for valuable insights and invites them to engage further in discussions at 2030.Cloud.
Backups are really, really tricky! We talk through a lot of different things that you have to consider in making successful backups like security, resilience, how you store the data, how you recover the data and rebuild the systems. Basically, we ran the gamut on backup challenges.
You really need to think through a lot of the considerations! Our discussion will help make you better at backups.
Rob Hirschfeld, CEO and co-founder of RackN and host of the Cloud 2030 Podcast, reflects on the October 26th discussion about backups, emphasizing their critical role in successful recovery efforts. He highlights the complexities of securing data at rest and the potential vulnerabilities backups may pose in scenarios like ransomware attacks or disaster recovery. Hirschfeld urges listeners to consider the entire system architecture and storage mechanisms to avoid potential losses, inviting them to explore the comprehensive discussion at the2030.cloud for deeper insights.
Cloud versus Edge? This panel dove into what makes edge different than cloud.
There are a lot of different technical and commercial drivers. And fundamentally, it matters who owns the sources of data and how data sources are different. This underscores how it is critical to understand data sources, infrastructure ownership, and how everything fits together.
This discussion will change to you rethink what makes Edge different than Cloud.
Building an edge control plane is challenging! It’s not clear even what is currently available. As always, data, data pipelines, data orchestration, and data choreography are all influential for edge infrastructure.
Transcript and Photo by Taryn Elliott from Pexels [ID 3889936]
Right to Repair is the idea that when you buy a product, you’re able to fix it. We’ve been building products lately that don’t have that inherent part of the contract.
In this episode, we really took Right to Repair to another level talking about Intellectual Property (IP) and ownership of that IP in the software components.
This topic impacts every single business and every single consumer!
We talk about Digital Twins and the Edge with Simon Crosby from Swim.AI. They are literally building digital twins in edge locations so he has a lot to share.
We work to expand and understand how Simon’s experience translates into general cases and what we’re seeing in the edge. The systems that we’re trying to build are at the intersection of models and “connectedness” of all the components for the edge.
These designs don’t fit traditional models and it is what makes edge unique. Edge is not a single application, but a connected system that going to have to emerge to make all this work together.
