By Mordechai Worch on October 07, 2022
2022,

IRONSCALES Engineering: The Challenges of Using AWS Glue as a Schema

Avro serialization with Kafka and AWS Glue

Avro is a popular serialization format for high throughput streaming applications. However, to get the benefits of improved performance, it requires an external schema registry.

In this video we will explore the challenges of using AWS glue as a schema, including:

  • What is Apache Avro?
  • Why a schema registry is important?
  • What are some challenges with integrating Kafka and AWS schema registry?
  • How can`aws-glue-schema-registry` enable us to use AWS schema registry with Python Kafka clients?
  • Overview of library and code examples.

Avro Serialization with AWS Glue Video

Note - here is the main Python library discussed in this talk.

I'm not a maintainer of this project just a fan.

 

Learn more about engineering at IRONSCALES, the team, and some of their work here.

 

Published by Mordechai Worch October 7, 2022
Shapes-Left

Join thousands of your peers! Subscribe to our blog.

Ironscales needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at any time. For information on how to unsubscribe, as well as our privacy practices and commitment to protecting your privacy, please review our Privacy Policy.

Shapes-Right