Kafka Avro — Topic Naming Strategies in Schema Registry

Kafka by default applies naming strategy to its subjects in schema registry. As such I didn’t find any way to not use any naming strategy, hence I had to use one of them. So lets dive deep into the naming strategies, but before that consider below given schema definition example as a reference to understand better.

{"schema":"{\"type\":\"record\",\"name\":\"StudentDTO\",\"namespace\":\"com.example.demo.models\",\"fields\":[{\"name\":\"firstname\",\"type\":[\"null\",\"string\"]},{\"name\":\"lastname\",\"type\":[\"null\",\"string\"]},{\"name\":\"age\",\"type\":[\"null\",\"int\"]}]}"}

Subject Naming Strategy for Schema Registry in Lafka Clusters explained

Naming Strategy Types

1. TopicNameStrategy

This is the default strategy and is implemented if no strategy has been specified. In this strategy, we are expected to create the subjects with appended suffixes such as -key and -value.

So, if you are using schema registry and if you have a topic named student_information on your kafka broker, then you should have subjects named student_information-key and student_information-value with their respective schema definitions in schema registry.

Notice the mentioned subject names. Schema registry creates the below mentioned schema definitions with the subject names appended with suffixes if using TopicNameStrategy.

{"subject":"student_information-value","version":1,"id":1,"schema":"{\"type\":\"record\",\"name\":\"StudentDTO\",\"namespace\":\"com.example.demo.models\",\"fields\":[{\"name\":\"firstname\",\"type\":[\"null\",\"string\"]},{\"name\":\"lastname\",\"type\":[\"null\",\"string\"]},{\"name\":\"age\",\"type\":[\"null\",\"int\"]}]}"}

{"subject":"student_information-key","version":1,"id":1,"schema":"\"string\""}

So the advice is to create the subject names as per the naming strategy to avoid auto creation of the subjects. Creating subjects beforehand is completely optional and depends on the business needs.

2. RecordNameStrategy

In this naming strategy the subject name is created by combining the Namespace(com.example.demo.model) + Name(StudentDTO) as specified in the schema definition. That is also called as fully qualified record name.

As per the reference example mentioned at the start of this blog, the subject should look like

com.example.demo.models.StudentDTO

Again, the subject name in above format is created by kafka schema registry if RecordNameStrategy is specified by overriding the default naming strategy.

3. TopicRecordNameStrategy

In this strategy, the topic name and the fully qualified record name are combined together and are suffixed with -key and -value.

As per our example. the subject name would look like given below.

student_information-com.example.demo.models.StudentDTO-key

student_information-com.example.demo.models.StudentDTO-value

How to use

When creating ConsumerFactory with schema registry integration, we need to pass few parameters as given below

Map<String, Object> props = new HashMap<>();
props.put(KafkaAvroSerializerConfig.SCHEMA_REGISTRY_URL_CONFIG, "SCHEMA_REGISTRY_URL");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class); //Your prefferred deserializer
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaAvroDeserializer.class); //Your prefferred deserializer
try {
    props.put("key.subject.name.strategy", Class.forName("YOUR_NAMING_STRATEGY_FULLY_QUALIFIED_NAME"));
} catch (ClassNotFoundException e) {
    log.info("Key subject naming strategy not found {} ", e);
}
try {
    props.put("value.subject.name.strategy", Class.forName("YOUR_NAMING_STRATEGY_FULLY_QUALIFIED_NAME"));
} catch (ClassNotFoundException e) {
    log.info("Value subject naming strategy not found {} ", e);
}