Dejvi blog

Of intra-mart using KNIME log analysis -- 2 --

Posting Date: 2019 - 05 - 31 Modified:

Hello, I'm Sekine of the digital Business Promotion Office (dejvi).
In continuation from the last time, is introduced method of log analysis of intra-mart.

Prerequisites for this article

・KNIME 3.6.1
・Log of intra-mart Accel Platform 2018 Summer
・Settings of request log is equal to or less than.
[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%thread] %X{log.report.sequence} %-5level %logger{255} %X{tenant.id} %X{log.id} - %X{client.session.id} %X{request.remote.host} %X{request.method} %X{request.url} %X{request.query_string} %X{request.url.referer} %X{request.page.time} %X{request.accept.time} %X{request.id}%nopex%n

Goal of this article

Request log of intra-mart Accel Platform into KNIME
The list of number of simultaneous requests for each period and processing time, etc.
Thereby, which time zone number of user's access to rise, or concurrency able to see such as several.

Here: Periodic statistics reports KNIME import file



First…

Import/reshaped of log the last blog post as it is used so omitted!
At this time log file other than
Populating xml file which states the setting of thread-max.

Next…

Calculates the number of concurrent requests.

[point] Calculation Method for concurrency
Added calculation variable, split to send the request and to receive the request
Sender + 1, but the recipient set to the variable - 1
Acquires the maximum value to summary time periods (1 hours) for each.

At the end…

To create a list of time-phased aggregate from the reshaped log.

[point] Aggregate for the period
The number of threads in the schedule period for each by using the GroupBy (count)、
For unique session ID (Unique Count), the maximum of the page processing time, the minimum, center, 90% ile, Average, Sum, standard deviation.

[point] Visualization of thread-max value and concurrency
By using the Line Plot node
thread-max value and concurrency that was obtained from the configuration file
Creates a graph that you want to compare.

Together

This time, I required number of requests or processing time base for the period.
Place be calculated concurrency a little effort
Implementation could be not too complicated processing.
Has been can also also combine configuration file of intra-mart.
I would like to extend this flow continue!

BONUS

Since one component falling due to the increase in node
Let's do metanode into as required. (by Ikuta)

Advance notice under the next time

Will I would like to introduce function "dejvidetechwood" the internal verified by research and development!

-Dejvi blog

Author:


  1. [...] Article related to KNIME in the past of intra-mart with intra-mart log analysis -- 1 --/KNIME using here! intra-mart BIORA booth -- thermo sensor and AI voice chat KNIME log analysis -- 2 -- [...]

comment

Field that does not mail address is published. * is attached is required item

Related article

[EWS] Step-up for "business automation" [DX display]

Hello, I'm Sekine of the digital Business Promotion Office (dejvi), this time for the DX exhibition contents of this year, only to. of intra-mart flow business process automation research Step current digital business…

[EWS Tokyo] Is the last under adjustment! [DX display]

Hello. Sekine is the digital business promotion office. condition of the final adjustments toward DX display will be sent. http://dev.intra-mart.jp/wp-content/uploads/2019/…

"takamatsu shopping" Introducing IoT example of

Hello. I Takamatsu of the digital business Business Promotion Office. "digital business Business Promotion Office", verification of business into by utilizing the new elemental technologies such as IoT or AI, RPA is, this time of our company a little changes…

[EWS2019] Condition of the DX exhibition -- current day --

Also Hello. I Sekine of dejvi. EWS2019 visitors Thank you very much! this year the size of the site, to attract 2500 people super between April, could be held smoothly. For DX exhibition TH-THE

intra-mart BIORA booth -- thermo sensor and AI voice chat --

Hello. Sekine of the digital Business Promotion Office is. This time, I will describe thermo sensor and AI voice chat.

There is no data yet.

RSSRSSRSSRSS