Engineering for Data Privacy: Navigating Infrastructure, Security, and Compliance with Skyflow’s Roshmik Saha
Partially Redacted: Data, AI, Security, and PrivacyShow Notes
In this episode Roshmik Saha, Head of Engineering at Skyflow, dives into the fascinating realm of data privacy and security solutions. Whether you're considering building your own privacy solution or seeking insights into the infrastructure requirements for handling credit card data securely, this episode has you covered. One important aspect that often goes underestimated is the maintenance costs associated with data privacy solutions. Roshmik emphasizes the significance of factoring in long-term maintenance expenses, as these solutions require ongoing updates, monitoring, and enhancements to adapt to evolving threats and regulations. It's crucial to recognize that compliance is merely a baseline and that solely building for compliance may not offer state-of-the-art security. Roshmik shares his expertise on how to go beyond compliance and implement robust security measures to protect sensitive data effectively. During the conversation, Roshmik highlights key considerations and features when building a data privacy solution to securely store and govern access to data. From encryption techniques and access control mechanisms to comprehensive auditing capabilities, he offers insights into the foundational elements required for a robust privacy solution. Additionally, he emphasizes the importance of incorporating state-of-the-art security technologies and features to reduce the risk of data breaches and potential reputational damage. Scalability is another critical aspect to address when developing a data privacy solution. Roshmik sheds light on the challenges faced by engineering teams in ensuring that the solution can meet the needs of a growing organization. He discusses strategies for building a scalable architecture that can handle increasing data volumes, user demands, and operational complexities. Throughout the episode, Roshmik provides practical advice and shares his thoughts on various topics, including the future of data privacy and security technologies. By drawing from his vast experience and expertise, you'll gain valuable insights into building a data privacy solution that not only meets regulatory requirements but also ensures resilience against cyber attacks. Topics: If I told you I was starting a B2C company and I was going to build my own privacy and security solution, what would your advice be Considering just credit card data, what would I need from an infrastructure standpoint to securely store, handle, and process credit card data? Beyond infrastructure costs, what other types of costs would I need to think through? What are the types of features or technologies I’d need to build to meet existing privacy requirements but also reduce the risk that I end up in the news for a data breach? What are the key considerations or features when building a data privacy solution to securely store data and govern access? What’s the engineering cost to build and maintain these? What kind of expertise does an engineering team require to build something that you think not only meets regulatory requirements, but also is resilient to cyber attacks? What are the most important security measures that need to be put in place to protect data privacy? How do you test and evaluate the effectiveness of the data privacy solution? How do you ensure that the data privacy solution remains up-to-date with evolving data privacy regulations and best practices? What are the biggest challenges that engineering teams face when building a data privacy solution? How do you ensure that the data privacy solution is scalable to meet the needs of a growing organization? Why do you think companies try to do this themselves? How do you ensure that the Skyflow is resilient to cyber attacks and other security threats? What advice would you give to other engineering teams building a data privacy solution for their organization? Are there any future data privacy or security technologies you’re excited about?
More from Partially Redacted: Data, AI, Security, and Privacy
In this episode, Sean sat down with Jack Godau to dive deep into the world of pseudoanonymization. They started by discussing Jack's career trajectory working with highly sensitive data and how that experience shapes his engineering mindset. Jack shared how pseudoanonymization differs from anonymization, explaining its value for maintaining data utility while complying with stringent regulations like GDPR. Jack also walked us through the challenges and key components of building a pseudoanonymization engine, including the complexities of handling re-identification risks, ensuring scalability, and optimizing performance for large datasets. He shared insights on the trade-offs between data protection and usability, and whether building these systems in-house is worth the investment for startups. Finally, they explored where the field is heading, especially as data privacy concerns continue to grow.
In this episode, Sean sits down with Ben Burkert, Co-founder and CTO of Anchor, to dive into the world of certificate management and internal TLS. We explore how certificates and TLS function, the inherent difficulties in managing internal TLS certificates, and why nearly every engineer has a horror story related to it. Ben also shares insights into how Anchor is addressing these challenges and making internal TLS certificate management simpler and more reliable. Key Topics:Understanding Certificates and TLS:Basics of how certificates and TLS work.The role of TLS in securing internal communications. The Challenges of Internal TLS Certificate Management:Why managing internal TLS certificates is so difficult.Common pitfalls and challenges engineers face. Engineer Horror Stories:Real-world examples of certificate management gone wrong.The impact of these failures on teams and organizations. How Anchor is Fixing the Problem:Anchor’s approach to simplifying internal TLS certificate management.Key features and benefits of Anchor’s solution. If you've ever struggled with internal TLS certificates or are looking for a way to avoid the pain altogether, Ben’s expertise provides a clear path to overcoming the challenges of certificate management with a modern, reliable approach. Resources:https://anchor.dev/https://lcl.host/
In this episode, we sit down with Ori Rafael, CEO and Co-founder of Upsolver, to explore the rise of the lakehouse architecture and its significance in modern data management. Ori breaks down the origins of the lakehouse and how it leverages S3 to provide scalable and cost-effective storage. We discuss the critical role of open table formats like Apache Iceberg in unifying data lakes and warehouses, and how ETL processes differ between these environments. Ori also shares his vision for the future, highlighting how Upsolver is positioned to empower organizations as they navigate the rapidly evolving data landscape.
In this episode, Sean Falconer is joined by Aubrey King, solutions architect and community evangelist at F5, to discuss the top 10 security issues for LLM applications. They explore critical threats such as prompt injections, insecure output handling, and training data poisoning, among others. Aubrey provides insights into why these issues arise, the attacks being observed, and the methods used to mitigate these risks. This episode is essential listening for anyone interested in the security of large language models and their applications.
In this episode, host Sean Falconer sits down with Eric Flaningam, a researcher at Felicis Ventures, to explore the fascinating world of data warehouses. They dive into the history, evolution, and future trends of data warehousing, shedding light on its importance. Key topics discussed include an overview of the article "A Primer on Data Warehouses," and the definition and key characteristics of data warehouses. They also cover the historical evolution and major milestones in data warehousing, the shift from batch processing to real-time data, and the convergence of data warehouses and SQL. Eric and Sean discuss the impact of unstructured and complex data, advancements in technology and their effect on data warehouses, and the technical architecture and components of a typical data warehouse. They share real-world benefits and use cases of data warehouses, common challenges in implementing and maintaining data warehouses, and future trends and the influence of AI and machine learning on data warehouses. For further reading, check out Eric Flaningam’s article, A Primer on Data Warehouses: https://www.generativevalue.com/p/a-primer-on-data-warehouses