This episode is a one person retrospective on Cloud 2030. As well as what we want to be talking about and doing over the next year. I hope you’ll take a second and listen to my reflection on what we’ve been talking about and where things are going.
Think about what you want the show to be. This podcast is structured around the discussions we come up with together, as well as current events. We try to think deeply and in an unusual way here, and I hope you’re getting a lot of that.
Hazel walks us through the Hackyderm.io leaving the basement migration. We also talk much more generally about Mastodon fediverse and scaling distributed systems.
This podcast is like a super class in what it takes to scale infrastructure and systems, especially live and under duress. Every minute of this conversation is worth listening to twice.
In the Cloud 2030 podcast episode discussing Hacky Derm’s scaling challenges, Rob Hirschfeld commends Hazel weekly for exploring the intricacies of exponential growth and federated platform integration. He highlights the significance of core architectural design decisions, such as Twitter’s use of immutable IDs for tweets and the necessity for sharing media files in federated systems. Hirschfeld emphasizes the impact of early design choices on an application’s lifecycle, resilience, and scalability, encouraging listeners to delve into the insightful January 31st episode and join the Cloud 2030 community for ongoing discussions at the2030.cloud.
Data comes from many different places, sources, and ways. Some data we call dark data, which is data not accessible to you, and all of it relevant. Today we talk about metadata as part of the governance control management exposure of data.
An important layer beyond the data itself is the governance intent, how people access it, and how you combine data. We discuss exactly what that is, but still only touch the surface.
In the Cloud 2030 Podcast episode on metadata and building a data control plane, Rob Hirschfeld emphasizes the challenge of controlling data egress due to its diverse sources and destinations. He contends that rather than attempting to create a locked box for all data, the focus should be on embedding information, particularly metadata, to control data consumption. Hirschfeld envisions a distributed system for a data control plane, involving multiple parties managing and providing consistent rules for data use, acknowledging the complexity of data storage across various locations. He encourages listeners to explore the insightful February 2nd episode and engage in ongoing conversations at the2030.cloud.
We get an update for the first time in four years about Chick-fil-A edge Kubernetes clusters that gets to the heart of how building distributed infrastructure works and what the challenges are.
We had a fantastic conversation about laying the foundations for this. We came away with two really important thoughts about what edge infrastructure looks like, how you pick it, can Kubernetes be used, what is IoT and integration, and the design considerations that go into building this environment.
Listen to this podcast as a preview for a longer article.
Our mini episode today is a short discussion of API delineation and abstractions for platform engineering.
This was a short intro discussion, and it is especially interesting because platform is a major topic we will be exploring in the coming year. We highlight the challenges of finding the right abstraction points as well as building front end and back end automation.
Today is a one on one discussion between me and Rocky Grover about infrastructure, infrastructure patterns, AI, and how all of these systems connect.
We think deeply about what it takes to design great systems and cover a ton of ground to connect it all back together into systems design. Rocky has a lot of depth of experience here.
In the Cloud 2030 Podcast episode on process infrastructure automation systems, Rob Hirschfeld introduces the concept of a “stochastic parrot,” emphasizing the challenge posed by AI systems that provide answers without a true understanding of the underlying process. Hirschfeld highlights the importance of comprehending the process through which decisions are made, especially as AI, like ChatGPT, becomes more integrated into various domains. The episode delves into the significance of systems thinking, stressing that understanding the process is often as crucial as knowing the output. Listeners are encouraged to explore the insightful conversation from January 17th at the2030.cloud and join the ongoing discussions.
Metadata is information that travels with the raw data that provides context, provenance, security, authorship, controls, and indexing.
The number of ways that you can expand the use of data is controlled by adding metadata. It creates a change in how we look at and manage data. Instead of creating control systems that contain the data, it’s actually packaged the control infrastructure, or the data control plane, as part of the data so that all of the systems can participate in it.
We also talk about data mesh a lot in that context.
In the Cloud 2030 Podcast episode from January 19th, Rob Hirschfeld discusses the challenges and opportunities associated with metadata, particularly in the context of managing and sharing data effectively. The conversation explores the concept of packaging raw data in a way that makes it available on request without being included, accompanied by metadata detailing its provenance, access permissions, context, and even expiration dates. Hirschfeld emphasizes the importance of building a data control plane to navigate the complexities of data consumption and production, envisioning a future where metadata plays a crucial role in weaving together diverse data sources. Listeners are encouraged to explore the full 40-minute discussion for comprehensive insights, and to join ongoing conversations at the2030.cloud.
If you love data and data context formats for exchanging data, you will love this conversation.
Today’s episode is a deep conversation about the potential ability to define ways in which we produce, store and share data, providing context using markup languages, and then being able to extend that. It’s a fascinating conversation about how much we could improve our use of data if we were able to provide more context about who wanted to see it and what relevance it had.
We also have some interesting conversations about data migration and how we share information.
Today we talk about backstage.io, and we have that conversation centered around a demo done by one of the RackN and interns, Zander Franks. Check out the demo video here: youtu.be/cAQQOmKz4OI
Zander has been exploring with the backstage to Digital Rebar integration, and the conversation that results explains backstage in some fundamental ways and also what it takes to build good developer portals.
You will find in this episode both the broader information about how to do integrations where you have a developer portal as a front end and the key insights about how backstage works.
To get the most out of the backstage pieces, you will definitely want to see the video on Youtube. Take time to enjoy this whole podcast, both in video and audio format.
In the Cloud 2030 Podcast episode from January 10th, Rob Hirschfeld discusses the backstage integration, emphasizing the importance of understanding the dynamics involved in building an integration between a developer self-service portal and the systems responsible for a robust deployment experience. Hirschfeld underscores the critical point that while developer interfaces are crucial in platform engineering, they should not be expected to replace production tools like orchestrators, observability platforms, and infrastructure automation components. The episode features a demonstration and code exploration, providing valuable insights into the complexities and considerations of such integrations. For those interested in similar discussions, the Cloud 2030 community welcomes participation at the2030.cloud.
We discussed the implications of chat GPT for it and the industry.
In today’s episode, we spend a lot of time figuring out how data provenance governance, bias, and ownership will impact chat GPT in IT and technology and cloud contexts. This discussion really looks into how chat GPT can be used in disruptive ways, but also in protective ways as what we describe as guardrails for how these systems are going to get built.
In the Cloud 2030 podcast’s January fifth episode, CEO Rob Hirschfeld explores the complexities of data provenance in ChatGPT, questioning ownership and control of the generated content. He emphasizes the need to understand the sources of data, pondering whether the output belongs to users, the algorithm, or no one, highlighting the challenges of systems that belong to nobody. Hirschfeld also connects this issue with Software Bill of Materials, emphasizing the importance of knowing the components of systems for accuracy and confidence. He encourages listeners to delve into the full episode for valuable insights and invites them to engage further in discussions at 2030.Cloud.
