Dejvi blog

Of intra-mart using KNIME log analysis -- 2 --

Posting Date: 2019 - 05 - 31 Modified:

Hello, I'm Sekine of the digital Business Promotion Office (dejvi).
In continuation from the last time, is introduced method of log analysis of intra-mart.

Prerequisites for this article

・KNIME 3.6.1
・Log of intra-mart Accel Platform 2018 Summer
・Settings of request log is equal to or less than.
[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%thread] %X{log.report.sequence} %-5level %logger{255} %X{tenant.id} %X{log.id} - %X{client.session.id} %X{request.remote.host} %X{request.method} %X{request.url} %X{request.query_string} %X{request.url.referer} %X{request.page.time} %X{request.accept.time} %X{request.id}%nopex%n

Goal of this article

Request log of intra-mart Accel Platform into KNIME
The list of number of simultaneous requests for each period and processing time, etc.
Thereby, which time zone number of user's access to rise, or concurrency able to see such as several.

Here: Periodic statistics reports KNIME import file



First…

Import/reshaped of log the last blog post as it is used so omitted!
At this time log file other than
Populating xml file which states the setting of thread-max.

Next…

Calculates the number of concurrent requests.

[point] Calculation Method for concurrency
Added calculation variable, split to send the request and to receive the request
Sender + 1, but the recipient set to the variable - 1
Acquires the maximum value to summary time periods (1 hours) for each.

At the end…

To create a list of time-phased aggregate from the reshaped log.

[point] Aggregate for the period
The number of threads in the schedule period for each by using the GroupBy (count)、
For unique session ID (Unique Count), the maximum of the page processing time, the minimum, center, 90% ile, Average, Sum, standard deviation.

[point] Visualization of thread-max value and concurrency
By using the Line Plot node
thread-max value and concurrency that was obtained from the configuration file
Creates a graph that you want to compare.

Together

This time, I required number of requests or processing time base for the period.
Place be calculated concurrency a little effort
Implementation could be not too complicated processing.
Has been can also also combine configuration file of intra-mart.
I would like to extend this flow continue!

BONUS

Since one component falling due to the increase in node
Let's do metanode into as required. (by Ikuta)

Advance notice under the next time

Will I would like to introduce function "dejvidetechwood" the internal verified by research and development!

-Dejvi blog

Author:


  1. [...] Article related to KNIME in the past of intra-mart with intra-mart log analysis -- 1 --/KNIME using here! intra-mart BIORA booth -- thermo sensor and AI voice chat KNIME log analysis -- 2 -- [...]

comment

Field that does not mail address is published. * is attached is required item

Related article

Souvenir delivery -- by intra-mart BIORA booth -- Roomba

Hello. Sekine of the digital business promotion office. I to other months, isn't it? time passes after EWS2018 earlier. This time, I will describe souvenir delivery by Roomba.

How to make ice button

Hello, I'm Takamatsu of the digital Business Development Office. For IoT button introduced last received that "would like you to tell me how to make" so, let me introduce you. described structure of ice button at the beginning for the configuration,…

"takamatsu shopping" Introducing IoT example of

Hello. I Takamatsu of the digital business Business Promotion Office. "digital business Business Promotion Office", verification of business into by utilizing the new elemental technologies such as IoT or AI, RPA is, this time of our company a little changes…

intra-mart BIORA booth -- thermo sensor and AI voice chat --

Hello. Sekine of the digital Business Promotion Office is. This time, I will describe thermo sensor and AI voice chat.

[EWS] Will do this year [DX display]

Hello. Sekine is the digital business promotion office. season EWS I'm here again this year!→ EWS2019 special site ← theme of this year change how to work in data and AI to the destination of RPA! intra-mart…

There is no data yet.

RSSRSSRSSRSS