[转帖]Welcome to the di-kafkameter wiki!

welcome,to,the,di,kafkameter,wiki · 浏览次数 : 0

小编点评

**Producer Elements:** * Kafka Producer Config: Holds the connection information, including security and other properties. * Kafka Producer Sampler: Sends messages to the topic using the connection established by the Producer Config. **Consumer Elements:** * Kafka Consumer Config: Holds the connection information and other properties for consumers. * Kafka Consumer Sampler: Reads messages from the topic using the connection established by the Consumer Config. **Additional Properties for JAAS Security:** * `sasl.jaas.configConfig`: Contains the configuration for SASL authentication. **Configuration Example:** ```properties # Producer Config kafka.producer.config.bootstrap.servers=localhost:9092 kafka.producer.config.client.id=my-producer-id # Kafka Producer Sampler kafka.producer.sampler.send.buffer.bytesPositive=16384 # Kafka Consumer Config kafka.consumer.config.bootstrap.servers=localhost:9092 kafka.consumer.config.group.id=my-consumer-group-id # Kafka Consumer Sampler kafka.consumer.sampler.poll.interval.msPositive=10000 ```

正文

https://github.com/rollno748/di-kafkameter/wiki#producer-elements

 

Introduction

DI-Kafkameter is a JMeter plugin that allows you to test and measure the performance of Apache Kafka.

Components

The DI-Kafkameter comprises of 2 components, which are

  • Producer Component
    1. Kafka Producer Config
    2. Kafka Producer Sampler
  • Consumer Component
    1. Kafka Consumer Config
    2. Kafka Consumer Sampler

Producer Component (Publish a Message to a Topic)

To publish/send a message to a Kafka topic you need to add producer components to the testplan.

  • The Kafka Producer config is responsible to hold the connection information, which includes security and other properties required to talk to the broker.
  • The Kakfa Producer Sampler helps to send messages to the topic with the connection established using Config element.

image

Right click on Test Plan -> Add -> Config Element -> Kafka Producer Config

Provide a Variable name to export the connection object (Which will be used in Sampler element)

Provide the Kafka connection configs (list of Brokers with comma separated)

Provide a Client ID (Make it unique, to define where you sending the message from)

Select the right security to connect to brokers (This will be completely based on how Kafka security is defined)

For JAAS Security, You need to add the below key and value to the Additional Properties
Config key: sasl.jaas.config
Config value: org.apache.kafka.common.security.scram.ScramLoginModule required username="<USERNAME>" password="<PASSWORD>";

image

Right click on Test Plan -> Add -> Sampler -> Kafka Producer Sampler

Use the same Variable name which was defined in the config element

Define the topic name where you want to send the message (Case sensitive)

Kafka Message - The Original message which needs to be pushed to the topic

Partition String (Optional) - This option helps you to post messages to particular partition by providing the partition number

Message Headers (Optional) - This helps in adding headers to the messages which are being pushed (Supports more than one header)

Consumer Component (Read Message from a topic)

To Consume/Read a message from a Kafka topic you need to add Consumer components to the testplan.

  • The Kafka Consumer config is responsible to hold the connection information, which includes security and other properties required to talk to the broker.
  • The Kafka Consumer Sampler helps to read messages from the topic with the connection established using Config element.

image

Right click on Test Plan -> Add -> Config Element -> Kafka Consumer Config

Provide a Variable name to export the connection object (Which will be used in Sampler element)

Provide the Kafka connection configs (list of Brokers with comma separated)

Provide a Group ID (Make it unique, to define the group your consumer belongs to)

Define the topic name where you want to send the message (Case sensitive)

No Of Messages to Poll - This allows you to define the number of messages to read within a request (Defaults to 1)

Select the right security to connect to brokers (This will be completely based on how Kafka security is defined)

Auto Commit - This will set the offset as read, once the message is consumed

Select the right security to connect to brokers (This will be completely based on how Kafka security is defined)

For JAAS Security, You need to add the below key and value to the Additional Properties
Config key: sasl.jaas.config
Config value: org.apache.kafka.common.security.scram.ScramLoginModule required username="<USERNAME>" password="<PASSWORD>";

image

Right click on Test Plan -> Add -> Sampler -> Kafka Consumer Sampler

Use the same Variable name which was defined in the config element

Poll timeout - This helps to set the polling timeout for consumer to read from topic (Defaults to 100 ms)

Commit Type - Defines the Commit type (Sync/Async)

Producer Properties

Supported Producer properties which can be added to Additional Properties field.

PropertyAvailable OptionsDefault
acks [0, 1, -1] 1
batch.size positive integer 16384
bootstrap.servers comma-separated host:port pairs localhost:9092
buffer.memory positive long 33554432
client.id string ""
compression.type [none, gzip, snappy, lz4, zstd] none
connections.max.idle.ms positive long 540000
delivery.timeout.ms positive long 120000
enable.idempotence [true, false] false
interceptor.classes fully-qualified class names []
key.serializer fully-qualified class name org.apache.kafka.common.serialization.StringSerializer
linger.ms non-negative integer 0
max.block.ms non-negative long 60000
max.in.flight.requests.per.connection positive integer 5
max.request.size positive integer 1048576
metadata.fetch.timeout.ms positive long 60000
metadata.max.age.ms positive long 300000
partitioner.class fully-qualified class name org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes positive integer 32768
reconnect.backoff.ms non-negative long 50
request.timeout.ms positive integer 30000
retries non-negative integer 0
sasl.jaas.config string null
sasl.kerberos.kinit.cmd string /usr/bin/kinit
sasl.kerberos.min.time.before.relogin positive long 60000
sasl.kerberos.service.name string null
sasl.mechanism [GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512] GSSAPI
security.protocol [PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL] PLAINTEXT
sender.flush.timeout.ms non-negative long 0
send.buffer.bytes positive integer 131072
value.serializer fully-qualified class name org.apache.kafka.common.serialization.StringSerializer

Consumer Properties

Supported Consumer properties which can be added to Additional Properties field.

PropertyAvailable OptionsDefault
auto.commit.interval.ms positive integer 5000
auto.offset.reset [earliest, latest, none] latest
bootstrap.servers comma-separated host:port pairs localhost:9092
check.crcs [true, false] true
client.id string ""
connections.max.idle.ms positive long 540000
enable.auto.commit [true, false] true
exclude.internal.topics [true, false] true
fetch.max.bytes positive long 52428800
fetch.max.wait.ms non-negative integer 500
fetch.min.bytes non-negative integer 1
group.id string ""
heartbeat.interval.ms positive integer 3000
interceptor.classes fully-qualified class names []
isolation.level [read_uncommitted, read_committed] read_uncommitted
key.deserializer fully-qualified class name org.apache.kafka.common.serialization.StringDeserializer
max.partition.fetch.bytes positive integer 1048576
max.poll.interval.ms positive long 300000
max.poll.records positive integer 500
metadata.max.age.ms positive long 300000
metadata.fetch.timeout.ms positive long 60000
receive.buffer.bytes positive integer 32768
reconnect.backoff.ms non-negative long 50
request.timeout.ms positive integer 30000
retry.backoff.ms non-negative long 100
sasl.jaas.config string null
sasl.kerberos.kinit.cmd string /usr/bin/kinit
sasl.kerberos.min.time.before.relogin positive long 60000
sasl.kerberos.service.name string null
sasl.mechanism [GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512] GSSAPI
security.protocol [PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL] PLAINTEXT
send.buffer.bytes positive integer 131072
session.timeout.ms positive integer 10000
value.deserializer fully-qualified class name org.apache.kafka.common.serialization.StringDeserializer

与[转帖]Welcome to the di-kafkameter wiki!相似的内容:

[转帖]Welcome to the di-kafkameter wiki!

https://github.com/rollno748/di-kafkameter/wiki#producer-elements Introduction DI-Kafkameter is a JMeter plugin that allows you to test and measure th

[转帖]

Linux ubuntu20.04 网络配置(图文教程) 因为我是刚装好的最小系统,所以很多东西都没有,在开始配置之前需要做下准备 环境准备 系统:ubuntu20.04网卡:双网卡 网卡一:供连接互联网使用网卡二:供连接内网使用(看情况,如果一张网卡足够,没必要做第二张网卡) 工具: net-to

[转帖]

https://cloud.tencent.com/developer/article/2168105?areaSource=104001.13&traceId=zcVNsKTUApF9rNJSkcCbB 前言 Redis作为高性能的内存数据库,在大数据量的情况下也会遇到性能瓶颈,日常开发中只有时刻

[转帖]ISV 、OSV、 SIG 概念

ISV 、OSV、 SIG 概念 2022-10-14 12:29530原创大杂烩 本文链接:https://www.cndba.cn/dave/article/108699 1. ISV: Independent Software Vendors “独立软件开发商”,特指专门从事软件的开发、生产、

[转帖]Redis 7 参数 修改 说明

2022-06-16 14:491800原创Redis 本文链接:https://www.cndba.cn/dave/article/108066 在之前的博客我们介绍了Redis 7 的安装和配置,如下: Linux 7.8 平台 Redis 7 安装并配置开机自启动 操作手册https://ww

[转帖]HTTPS中间人攻击原理

https://www.zhihu.com/people/bei-ji-85/posts 背景 前一段时间,公司北京地区上线了一个HTTPS防火墙,用来监听HTTPS流量。防火墙上线之前,邮件通知给管理层,我从我老大那里听说这个事情的时候,说这个有风险,然后意外地发现,很多人原来都不知道HTTPS防

[转帖]关于字节序(大小端)的一点想法

https://www.zhihu.com/people/bei-ji-85/posts 今天在一个技术群里有人问起来了,当时有一些讨论(不完全都是我个人的观点),整理一下: 为什么网络字节序(多数情况下)是大端? 早年设备的缓存很小,先接收高字节能快速的判断报文信息:包长度(需要准备多大缓存)、地

[转帖]awk提取某一行某一列的数据

https://www.jianshu.com/p/dbcb7fe2da56 1、提取文件中第1列数据 awk '{print $1}' filename > out.txt 2、提取前2列的文件 awk `{print $1,$2}' filename > out.txt 3、打印完第一列,然后打

[转帖]awk 中 FS的用法

https://www.cnblogs.com/rohens-hbg/p/5510890.html 在openwrt文件 ar71xx.sh中 查询设备类型时,有这么一句, machine=$(awk 'BEGIN{FS="[ \t]+:[ \t]"} /machine/ {print $2}' /

[转帖]Windows Server 2022 简体中文版、英文版下载 (updated Oct 2022)

https://sysin.org/blog/windows-server-2022/ Windows Server 2022 正式版,2022 年 10 月更新,VLSC Posted by sysin on 2022-10-27 Estimated Reading Time 8 Minutes