[转帖]JMeter InfluxDB v2.0 listener plugin

jmeter,influxdb,v2,listener,plugin · 浏览次数 : 0

小编点评

## Generating Content with Simple Formatting Here's how to generate content with simple formatting and optimal performance: **1. Define Variable Formats:** * Use `recordSubSamples` to save individual JMeter sub samples. * Save response length in `responseBodyLength` for accurate measurements. * Define thresholds for error analysis and report them separately. **2. Use Regular Expressions for Efficient Filtering:** * Generate regular expressions for sampler names and filter them efficiently. * Use the same regex for both sampler list and request details. **3. Formatting for Clarity and Readability:** * Use string formatting for clear and concise output. * Format timestamps and metrics for optimal readability. * Use conditional formatting for better control of information. **4. Optimize Data Presentation and Display:** * Use aggregation and filtering to display the relevant data. * Consider adding network statistics and processing time for better insights. * Format results for optimal display on the dashboard. **5. Keep the Code Concise and Efficient:** * Use meaningful variable names and consistent formatting. * Use comments to explain complex logic and improve readability. * Consider using a linter to ensure code quality. **6. Optimize for Performance and Memory:** * Use `recordSubSamples` to save only relevant JMeter sub samples. * Use `saveResponseBodyOfFailures` to avoid saving unnecessary data. * Consider using thread-safe methods for efficient data processing. **7. Formatting for Specific Dashboards:** * Use conditional formatting and string formatting for specific dashboards. * Use different formats for different data types and presentation requirements. **8. Keep the Documentation Clear and Concise:** * Write clear comments and variable descriptions for easy understanding. * Use consistent naming conventions throughout the code. * Consider using a linter to ensure code quality and documentation. **9. Testing and Error Handling:** * Use unit tests to ensure code functionality and coverage. * Implement robust error handling and reporting mechanisms. * Test the aggregate report with different scenarios and data types. **10. Code Optimization for Performance:** * Use `StringBuilder` for efficient string manipulation. * Use `StringJoiner` for cleaner and more efficient string formatting. * Consider using alternative data structures for better performance. **Remember:** These are just general guidelines. Adjust them based on your specific requirements and dashboard functionalities.

正文

https://github.com/mderevyankoaqa/jmeter-influxdb2-listener-plugin

 

Support my Ukrainian Family ❤️

Like what you see? 😍

Description

The goal of the project is to make a quite similar online dashboard in the same way as JMeter generates. Supported the latest InfluxDB v2.0 and created appropriate dashboard ("Flux" language has been used to create the queries - now there are a lot fo capacities to build amazing charts, tables with a lot of math function). So that it would be possible to build the monitor hardware solution on the latest InfluxDB v2.0 and telegraf (agent to send the hardware metrics to InfluxDB) as well.

The plugin sends metrics to InfluxDB and provides the possibility to visualize the charts in Grafana, have the Aggregate report as JMeter creates. Added the possibly to save the following extra fields in the database:

  • Response code;
  • Error message;
  • Response body of the failed requests (can be configured);
  • Connect time;
  • Latency;
  • The response time (uses from the SampleResult.class, needs to make aggregate report).

Notes: if you need to save the errors you got wile the test to csv like file and then share to the dev team you can use the jmeter-csv-listener-plugin. This plugin can be used while the functional testing and load tests as well.

Important notes

🚨 The plugin allows 5 errors happened one by one, then plugin will stop importing the results after that! See details in the logs. Counter will be refreshed at least you have 4 fails. This is protection to avoid OOM error. The option can be configured in the settings (the key name is influxDBThresholdError see the Plugin configuration for the details). You need to be careful with that option and know the hardware resources to store data in the memory.

Pleas monitor the elapsed time of the data importing (see logs) to avoid issues with requests sending from JMeter. Pay attention on "Sending metrics performance tuning" chapter, see below.

Compatibility

The supported versions:

Maven Support

Please see the latest release here https://search.maven.org/artifact/io.github.mderevyankoaqa/jmeter-plugins-influxdb2-listener.

CI/CD Support

The CI/CD can be done using jmeter-maven-plugin The example of the adding plugin to the project with the jmeter-maven:

      <configuration>
            <jmeterExtensions>
              <artifact>io.github.mderevyankoaqa:jmeter-plugins-influxdb2-listener:2.1</artifact>
            </jmeterExtensions>
        </configuration>

Some notes cab be found in the article Adding jar's to the /lib/ext directory.

Development and branching strategy

Hello all contributors! Welcome to the project, I'm happy to see you here. Just to avoid the mess and have a nice history, let's keep the simple rules:

  1. Code should pass standard static code analysis in IntelliJ IDEA.
  2. Comments should be for the new code to have clear java docs.
  3. All new development lets - in the "development" branch. So the steps: a. make a branch for the feature you want to develop from "development" (source branch -> "main"). There will be a mirror of the latest release + can be a merge of the new features. b. all pull requests from your feature branch -> "development" branch only. c. once we decide to make the new release after testing, we will merge "development" -> "main" with push to maven central.

While the first pull request please add your self to the build.gradle the section 'developers'

developers {
    developer {
    id = 'your github if'
    name = 'your name'
    email = 'email'
    }

Deployment

  • Put 'jmeter-plugin-influxdb2-listener-<version>.jar' file from Releases to ~<JmeterPath<\lib\ext;

Note: "fatJar" gradle task should be used to create the package for the JMeter.

  • Run JMeter and select the test plan, Add-> Listener -> Backend Listener.

  • Go to Backend Listener

  • Select from the dropdown item with the name 'io.github.mderevyankoaqa.influxdb2.visualizer.JMeterInfluxDBBackendListenerClient'.

Upgrade

  • Close JMeter if its started.

  • Remove old jar.

  • Put 'jmeter-plugin-influxdb2-listener-<version>.jar' file from Releases to ~<JmeterPath<\lib\ext;

  • Run JMeter again and got Listener.

  • Select from the dropdown item with the name 'io.github.mderevyankoaqa.influxdb2.visualizer.JMeterInfluxDBBackendListenerClient'.

  • Click 'Clear all' button

  • Set right settings (update all properties).

InfluxDB configuration

  • Create Bucket to store the further testing results.

  • Create the token with read and write rights. 

Plugin configuration

Let’s explain the plugin fields:

  • testName - the name of the test.

  • nodeName - the name of the server.

  • runId - the identification number of hte test run, can be dynamic.

  • influxDBURL - InfluxDB server URL [protocol://][host][:port], protocol (can be http or https) and the default port is 8086.

  • influxDBToken - the influxdb bucket token, the default value should be updated, copy it from InfluxDB site.

  • influxDBOrganization - the influxdb bucket organization, the default value should be updated, copy it from InfluxDB site.

  • influxDBFlushInterval - its interval to send data to InfluxDB, the default value is 4000 (4 seconds).

  • influxDBMaxBatchSize - the max size of the batch with metrics, the default 2000 (2000 items of JMeter results).

  • influxDBThresholdError - the error threshold before stopping the import, the default value is 5. (see Important notes for more detail.)

  • influxDBBucket - the InfluxDB bucket name to store the test results.

  • samplersList - the regex value to sort out the JMeter samplers results; the default is .*. For example if you have the pattern of JMeter test plan development like this - create the 'Transaction controller', add inside of the 'Transaction controller' the Sampler with request, and the name pattern 'GET->Something', like on the see screen below. The regex ^(Home Page|Login|Search)(-success|-failure)?$ can be used to save only samplers names. The regex can be generated from JMeter menu.

You can modify the generated string in terms of your purposes.

  • useRegexForSamplerList - allows to use the regexps if set to 'true'.
  • recordSubSamples - allows to save the JMeter sub samples if set to 'true'.
  • saveResponseBodyOfFailures - allows to save the response body of the failures.
  • responseBodyLength - allows saving the response body, not more than the set specified length.

Sending metrics performance tuning

The plugin imports batch with JMeter results each 4 seconds (by default settings). In the logs you will see records like this: INFO o.m.j.i.v.InfluxDatabaseBackendListenerClient: Data has been imported successfully, batch size --> 68, elapsed time is --> 14 ms (the elapsed time is the response time of the batch importing.) So you can control the flush interval depends on the load you have and adjust influxDBFlushInterval setting. Is not recommended having less 1 second.
Max batch protection -> send data when batch max size is occurred. For example, when batch size is 2000 items (it's the default setting of influxDBMaxBatchSize) plugin imports that batch, even when flush interval was not occurred. Using both options you can tune data importing and have optimal performance.

Make sure you have enough ram to aggregate huge batch and optimal flush period.

Notes: when test has been interrupted from UI; the processes may not be finished properly, restart JMeter.

Grafana dashboard capabilities

See deployment instructions here https://grafana.com/grafana/dashboards/13644

Dashboard helps:

  • Filter the results by Run Id or Type (can be requests or transactions).


  • Monitor throughput with active users. 

  • Overview and analise the response time, distribution as well. Added the filters to see requests with status "pass", "fail".  

  • See aggregate report. The table rendering may take an extra time. The table has hardware resources consuming queries from Influxdb side. If you have low hardware on the Influxdb server - recommended make the clone of the original dashboard and remove aggregate report. So the idea - it's to have one 'fast' dashboard for the online monitoring (has no aggregate report) to see the results while the test, the second (original with aggregate report) to see the final results. 

    Now added the possibility to see the aggregate report for requests with status "pass" only. 

  • Investigate errors. The table is interactive, it's possible to filter data in the columns and see details for the specific error. Added the paging. 

  • See network statistics, latency, processing time.   

  • Check individual request details. 

与[转帖]JMeter InfluxDB v2.0 listener plugin相似的内容:

[转帖]JMeter InfluxDB v2.0 listener plugin

https://github.com/mderevyankoaqa/jmeter-influxdb2-listener-plugin Support my Ukrainian Family ❤️ Like what you see? 😍 Description The goal of the pr

[转帖]Jmeter连接InfluxDB2.0.4

Jmeter连接InfluxDB2.0.4 问题描述:在用Jmeter+InfluxDB构建监控时,因为docker构建的InfluxDB的版本是2.0.4,按照网上的教程进行后端监听器的填写,但是一直出现错误提示401等问题。网上的教程大多是1.X版本的,怀疑是数据库版本不一致导致的数据无法写入,

[转帖]手摸手搭建简单的jmeter+influxdb+grafana性能监控平台

我安装的机器是阿里云的centos8机器,其他的系统暂未验证 1.安装influxdb influxdb 下载地址https://portal.influxdata.com/downloads/,也可以直接在服务器上执行以下命令下载 我的软件一般下载在/usr/local/soft文件夹下,soft

[转帖]jmeter及serveragent的安装、常见报错及解决方法

一.说明: 1.jmeter必须在java环境下运行 2.jmeter抓包可以用自带服务器录制抓取,也可以手动网页抓取(含fiddler),也可以用badboy(更推荐手动抓取和badboy) 3.后面单独开一篇来说明各个功能的用处。 二.安装: 1.jmeter.properties配置:在jme

[转帖]JMETER性能监控之serverAgent

对linux服务器的服务进行压测时,服务器的运行情况可以通过添加插件serverAgent来观察,可以实时监控性能指标。 1 (一)环境准备 1、下载zip包ServerAgent-2.2.3.zip 2、在服务器中,创建一个文件夹serveragent,名字随便起 mkdir serveragen

[转帖]jmeter分布式执行原理

jmeter分布式执行原理https://www.jianshu.com/p/bcfc48054b1e Jmeter分布式执行原理: 1、Jmeter分布式测试时,选择其中一台作为控制机(Controller),其它机器做为代理机(Agent)。 2、执行时,Controller会把脚本发送到每台A

[转帖]JMeter分布式测试环境搭建(禁用SSL)

https://www.cnblogs.com/MasterMonkInTemple/p/11978058.html JMeter分布式环境,一台Master,一到多台Slave,Master和Slave可以是同一台机器。 前提条件: 所有机器,包括master和slave的机器: 1.运行相同版本

[转帖]jmeter压力测试

使用jmeter 进行并发压力测试。 首先需要安装好jmeter,下面以widows操作平台为例: 1、确保电脑安装并配置好java环境;具体怎么下载和配置请自行百度; 2、登录jmeter官网http://jmeter.apache.org/download_jmeter.cgi,点击如下图操作

[转帖]【Jmeter】Jmeter压力测试工具安装及使用教程(redis测试)

摘自:https://www.cnblogs.com/monjeo/p/9330464.html 一、Jmeter下载 进入官网:http://jmeter.apache.org/ 1.第一步进入官网如下图 2.选择进行下载,下载下来为一个压缩包,解压即可。 3.我下载的是jmeter4.0版本,对

[转帖]jmeter编写测试脚本大全

目录 一、背景 二、按照功能划分 2.1 加密处理、验签处理 2.2 jmeter 使用beanshell 编写脚本 2.3 jmeter脚本报错大全 2.4 jmeter打印log 2.5 jmeter压测java代码 2.6 jmeter读取数据保证每个线程拿到的数据唯一 2.7 jmeter设