주식회사 누리아이티

정보자산의 보안강화를 위한 3단계 인증 보안SW(BaroPAM) 전문기업인 누리아이티

▶ BaroSolution/가이드

정보자산의 이상접속 탐지/차단을 위한 Intrusion Detector 관리

누리아이티 2020. 2. 25. 11:18

1. Intrusion Detector 설치

 

Intrusion Detector 설치는 컴파일 후 생성된 flume-ng-jdbc-sink-2.0.jar 파일을 $FLUME_HOME/lib 디렉토리에 다음과 같이 복사하면 된다.

 

[root] /home/flume-ng-sink/target > cp flume-ng-jdbc-sink-2.0.jar $FLUME_HOME/lib/.

 

 

2. 환경 변수 설정

 

Intrusion Detector를 기동하려면 환경설정 파일인 flume-env.sh에 다음과 같이 환경 변수들을 정의해야 한다.

 

변수 설명 비고
FLUME_HOME Apache Flume이 설치된 디렉토리를 지정하는 변수  
FLUME_CLASSPATH Apache Flume Library 디렉토리를 지정하는 변수  
JAVA_HOME JDK가 설치된 디렉토리를 지정하는 변수  
CLASSPATH Java 프로그램을 컴파일(javac)이나 실행(java)할 때나 관련된 클래스를 지정하는 변수  
LANG 동일한 언어를 지원하는 데 필요한 로케일을 지정하는 변수  
PATH $FLUME_HOME/bin, $JAVA_HOME/bin PATH에 반드시 포함되어야 한다.  
     

 

[root] //usr/baropam/master > vi flume-env.sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
# If this file is placed at FLUME_COTB_DIR/flume-env.sh, it will be sourced
# during Flume startup.
 
# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
#JAVA_OPTS="-Xms100m -Xmx200m -Dcom.sun.management.jmxremote"
JAVA_OPTS="-XX:MaxDirectMemorySize=128m"
 
# Note that the Flume conf directory is always included in the classpath.
FLUME_HOME=/home/apache-flume-1.7.0-bin
FLUME_CLASSPATH=$FLUME_HOME/lib
 
# Java variables can be set here
JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64
CLASSPATH=$CLASSPATH:$FLUME_CLASSPATH:$JAVA_HOME/lib:
 
# Enviroment variables can be set here.
LANG=ko_KR.euckr
#LANG=ko_KR.utf8
PATH=$PATH:$FLUME_HOME/bin:$JAVA_HOME/bin:/etc/alternatives

 

 

3. Log4j 속성 설정

 

log4j는 프로그램을 작성하는 도중에 로그를 남기기 위해 사용되는 자바 기반 로깅 유틸리티이다. 디버그용 도구로 주로 사용되고 있다.

 

log4j의 최근 버전에 의하면 높은 등급에서 낮은 등급으로의 6개 로그 레벨(FATAL, ERROR, WARN, INFO, DEBUG, TRACE)을 가지고 있다. 설정 파일에 대상별(자바에서는 패키지)로 레벨을 지정이 가능하고 그 등급 이상의 로그만 저장하는 방식이다.

 

[root] //usr/baropam/master > vi log4j.properties
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#
 
# Define some default values that can be overridden by system properties.
#
# For testing, it may also be convenient to specify
# -Dflume.root.logger=DEBUG,console when launching flume.
 
#flume.root.logger=DEBUG,console
flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log
 
log4j.logger.org.apache.flume.lifecycle = INFO
log4j.logger.org.jboss = WARN
log4j.logger.org.mortbay = INFO
log4j.logger.org.apache.avro.ipc.NettyTransceiver = WARN
log4j.logger.org.apache.hadoop = INFO
 
# Define the root logger to the system property "flume.root.logger".
log4j.rootLogger=${flume.root.logger}
 
 
# Stock log4j rolling file appender
# Default log rotation configuration
log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.LOGFILE.MaxFileSize=100MB
log4j.appender.LOGFILE.MaxBackupIndex=10
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n
 
 
# Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job!
# This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy.
# See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html
# Add "DAILY" to flume.root.logger above if you want to use this
log4j.appender.DAILY=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.DAILY.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.DAILY.rollingPolicy.ActiveFileName=${flume.log.dir}/${flume.log.file}
log4j.appender.DAILY.rollingPolicy.FileNamePattern=${flume.log.dir}/${flume.log.file}.%d{yyyy-MM-dd}
log4j.appender.DAILY.layout=org.apache.log4j.PatternLayout
log4j.appender.DAILY.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n
 
 
# console
# Add "console" to flume.root.logger above if you want to use this
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

 

 

4. Intrusion Dtector 속성 설정

 

Intrusion Detector JDBCSink를 사용하려면 환경설정 파일인 flume.conf에 다음과 같이 Property들을 정의해야 한다.

 

[root] /home/apache-flume-1.7.0-bin/master > vi flume.conf
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
 
# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'
 
master1.sources = master1
master1.channels = mem-channel-100 mem-channel-200 mem-channel-921
master1.sinks = jdbc-sink-100 jdbc-sink-200 jdbc-sink-921
 
# For each one of the sources, the type is defined
master1.sources.master1.type = avro
master1.sources.master1.bind = 1.234.83.169
master1.sources.master1.port = 61616
master1.sources.master1.restartThrottle = 100
master1.sources.master1.restart = true
#master1.sources.master1.restart = false
master1.sources.master1.batchSize = 1
master1.sources.master1.charset.default = euc-kr
 
# The channel can be defined as follows.
master1.sources.master1.channels = mem-channel-100 mem-channel-200 mem-channel-921
 
# Static Interceptor
#master1.sources.master1.interceptors = i1
#master1.sources.master1.interceptors.i1.type = static
#master1.sources.master1.interceptors.i1.key = task_type
#master1.sources.master1.interceptors.i1.value = 100
 
# Multiplexing Channel Selector
master1.sources.master1.selector.type = multiplexing
master1.sources.master1.selector.header = task_type
master1.sources.master1.selector.mapping.100 = mem-channel-100
master1.sources.master1.selector.mapping.200 = mem-channel-200
master1.sources.master1.selector.mapping.921 = mem-channel-921
#master1.sources.master1.selector.default = mem-channel-100
 
# Each sink's type must be defined
master1.sinks.jdbc-sink-100.type = org.apache.flume.sink.JDBCSink
master1.sinks.jdbc-sink-200.type = org.apache.flume.sink.JDBCSink
master1.sinks.jdbc-sink-921.type = org.apache.flume.sink.JDBCSink
 
# URL to connect to database
#master1.sinks.jdbc-sink-100.sink.jdbc.driver = sunje.sundb.jdbc.SundbDriver
#master1.sinks.jdbc-sink-100.sink.connection.url = jdbc:sundb://160.61.194.54:22581/700
master1.sinks.jdbc-sink-100.sink.jdbc.driver = oracle.jdbc.OracleDriver
master1.sinks.jdbc-sink-100.sink.connection.url = jdbc:oracle:thin:@1.234.83.169:1521:ORCL
 
master1.sinks.jdbc-sink-200.sink.jdbc.driver = oracle.jdbc.OracleDriver
master1.sinks.jdbc-sink-200.sink.connection.url = jdbc:oracle:thin:@1.234.83.169:1521:ORCL
 
master1.sinks.jdbc-sink-921.sink.jdbc.driver = oracle.jdbc.OracleDriver
master1.sinks.jdbc-sink-921.sink.connection.url = jdbc:oracle:thin:@1.234.83.169:1521:ORCL
 
# Database connection properties
master1.sinks.jdbc-sink-100.sink.charset = euc-kr
#master1.sinks.jdbc-sink-100.sink.charset = utf-8
master1.sinks.jdbc-sink-100.sink.user = baropam
master1.sinks.jdbc-sink-100.sink.password = baropam
 
master1.sinks.jdbc-sink-200.sink.charset = euc-kr
master1.sinks.jdbc-sink-200.sink.user = baropam
master1.sinks.jdbc-sink-200.sink.password = baropam
 
master1.sinks.jdbc-sink-921.sink.charset = euc-kr
master1.sinks.jdbc-sink-921.sink.user = baropam
master1.sinks.jdbc-sink-921.sink.password = baropam
 
# Agent status properties
master1.sinks.jdbc-sink-100.sink.agent.status.stmt = UPDATE TB_AGENT_INFO SET AGENT_YN = 'Y', UPD_DTTM = TO_CHAR(SYSTIMESTAMP, 'YYYYMMDDHH24MISSFF6'), UPD_USER = '20170425094135653654' WHERE AGENT_ID = '20170426095141389910'
 
# UnitRule properties(Y or N)
master1.sinks.jdbc-sink-100.sink.unit.rule = Y
 
# Rule set properties(Y or N)
master1.sinks.jdbc-sink-100.sink.rule.set = N
 
# Accidents registered properties
master1.sinks.jdbc-sink-100.sink.intrusion.detect = N
master1.sinks.jdbc-sink-100.sink.intrusion.route = H
master1.sinks.jdbc-sink-100.sink.push.message =
 
# Specify the channel the sink should use
master1.sinks.jdbc-sink-100.channel = mem-channel-100
master1.sinks.jdbc-sink-200.channel = mem-channel-200
master1.sinks.jdbc-sink-921.channel = mem-channel-921
 
# Each channel's type is defined.
master1.channels.mem-channel-100.type = memory
#master1.channels.mem-channel-100.type = file
master1.channels.mem-channel-100.checkpointDir = ./checkpoint_100
master1.channels.mem-channel-100.dataDirs = ./checkdata_100
 
master1.channels.mem-channel-200.type = memory
master1.channels.mem-channel-200.checkpointDir = ./checkpoint_200
master1.channels.mem-channel-200.dataDirs = ./checkdata_200
 
master1.channels.mem-channel-921.type = memory
master1.channels.mem-channel-921.checkpointDir = ./checkpoint_921
master1.channels.mem-channel-921.dataDirs = ./checkdata_921
 
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
master1.channels.mem-channel-100.capacity = 1080000
master1.channels.mem-channel-100.transactionCapacity = 10000
master1.channels.mem-channel-100.keep-alive = 3
 
master1.channels.mem-channel-200.capacity = 1080000
master1.channels.mem-channel-200.transactionCapacity = 10000
master1.channels.mem-channel-200.keep-alive = 3
 
master1.channels.mem-channel-921.capacity = 1080000
master1.channels.mem-channel-921.transactionCapacity = 10000
master1.channels.mem-channel-921.keep-alive = 3
 

 

 

5. Intrusion Detector 기동

 

Intrusion Detector를 기동하는 startup.sh 쉘 스크립트는 다음과 같다.

 

[root] //usr/baropam/master > vi startup.sh
#!/bin/sh
 
#export FLUME_HOME=/home/apache-flume-1.7.0-bin;
#export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64;
 
#export CLASSPATH=$CLASSPATH:$FLUME_HOME/lib:$JAVA_HOME/lib
#export PATH=$PATH:$FLUME_HOME/bin:$JAVA_HOME/bin
 
export LANG=ko_KR.euckr
#export LANG=ko_KR.utf8
 
\rm /usr/baropam/master/logs/flume*
 
flume-ng agent -n master1 -c /usr/baropam/master/ -f flume.conf -Dflume.monitoring.type=http -Dflume.monitoring.port=41414 &
 

 

Intrusion Detector 기동은 startup.sh 쉘 스크립트를 백드라운드 프로세스로 다음과 같이 실행하면 된다.

 

[root] //usr/baropam/master > sh startup.sh &

 

Intrusion Detector가 실행되고 있는지 확인하기 위해서는 다음과 같은 명령어를 수행한다.

 

[root] //usr/baropam/master > ps -ef|grep flume | grep master1 | grep -v grep

 

그러면, 다음과 같이 Intrusion Detector 프로세스가 존재하는지 확인할 수 있다.

 

[root] //usr/baropam/master > ps -ef|grep flume | grep master1 | grep -v grep
root     19158     1  0 15:06 pts/1    00:00:05 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java -XX:MaxDirectMemorySize=128m -Dflume.monitoring.type=http -Dflume.monitoring.port=41414 -cp /usr/baropam/master:/home/apache-flume-1.7.0-bin/lib/*:/home/apache-flume-1.7.0-bin/lib:/lib/* -Djava.library.path= org.apache.flume.node.Application -n master1 -f flume.conf

 

 

6. Intrusion Detector 종료

 

Intrusion Detector를 종료하는 shutdown.sh 쉘 스크립트는 다음과 같다.

 

[root] //usr/baropam/master > vi shutdown.sh
#!/bin/sh
 
ps -ef|grep flume | grep master1 | grep -v grep |awk '{print "kill -9 "$2}'|sh -v
 

 

Intrusion Detector 종료는 shutdown.sh 쉘 스크립트를 다음과 같이 실행하면 된다.

 

[root] //usr/baropam/master > sh shutdown.sh

 

 

7. Intrusion Detector 로그

 

Intrusion Detector 로그는 Intrusion Detector가 실행되면서 발생한 로그(INFO, WARN, ERROR) 및 수집하면서 남긴 로그들이 남아 향후 Intrusion Detector 상태 및 장애 발생시 원인 구명 등에 활용하는 중요한 로그다.

 

[root] //usr/baropam/master/logs > ls -al
합계 20
drwxr-xr-x 2 root root  4096 12  4 11:05 .
drwxr-xr-x 6 root root  4096 12  4 11:04 ..
-rw-r--r-- 1 root root 12188 12  4 11:05 flume.log

 

30 5 2017 12:34:32,554 INFO  [lifecycleSupervisor-1-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start:61)  - Configuration provider starting
30 5 2017 12:34:32,565 INFO  [conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:133)  - Reloading configuration file:flume.conf
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,584 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,586 WARN  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.<init>:102)  - Configuration property ignored: master1.sinks.jdbc-sink-100.sink.push.message =
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,586 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:931)  - Added sinks: jdbc-sink-100 jdbc-sink-200 jdbc-sink-921 Agent: master1
30 5 2017 12:34:32,587 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,588 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,588 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,589 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-100
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-921
30 5 2017 12:34:32,590 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1017)  - Processing:jdbc-sink-200
30 5 2017 12:34:32,625 INFO  [conf-file-poller-0] (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:141)  - Post-validation flume configuration contains configuration for agents: [master1]
30 5 2017 12:34:32,625 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:145)  - Creating channels
30 5 2017 12:34:32,632 INFO  [conf-file-poller-0] (org.apache.flume.channel.DefaultChannelFactory.create:42)  - Creating instance of channel mem-channel-200 type memory
30 5 2017 12:34:32,636 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:200)  - Created channel mem-channel-200
30 5 2017 12:34:32,636 INFO  [conf-file-poller-0] (org.apache.flume.channel.DefaultChannelFactory.create:42)  - Creating instance of channel mem-channel-921 type memory
30 5 2017 12:34:32,637 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:200)  - Created channel mem-channel-921
30 5 2017 12:34:32,637 INFO  [conf-file-poller-0] (org.apache.flume.channel.DefaultChannelFactory.create:42)  - Creating instance of channel mem-channel-100 type memory
30 5 2017 12:34:32,637 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:200)  - Created channel mem-channel-100
30 5 2017 12:34:32,637 INFO  [conf-file-poller-0] (org.apache.flume.source.DefaultSourceFactory.create:41)  - Creating instance of source master1, type avro
30 5 2017 12:34:32,668 INFO  [conf-file-poller-0] (org.apache.flume.sink.DefaultSinkFactory.create:42)  - Creating instance of sink: jdbc-sink-100, type: org.apache.flume.sink.JDBCSink
30 5 2017 12:34:32,682 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSink.configure:45)  - Reading and processing configuration values for sink jdbc-sink-100
30 5 2017 12:34:32,683 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:46)  - JDBCSinkUtils.!!!
30 5 2017 12:34:32,683 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:47)  - jdbcDriver      = [oracle.jdbc.OracleDriver]
30 5 2017 12:34:32,683 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:48)  - connectionURL   = [jdbc:oracle:thin:@1.234.83.169:1521:ORCL]
30 5 2017 12:34:32,683 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:49)  - charset         = [euc-kr]
30 5 2017 12:34:32,683 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:50)  - user            = [baropam]
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:51)  - password        = [baropam]
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:53)  - agentStatusStmt = [UPDATE TB_AGENT_INFO SET AGENT_YN = 'Y', UPD_DTTM = TO_CHAR(SYSTIMESTAMP, 'YYYYMMDDHH24MISSFF6'), UPD_USER = '20170425094135653654' WHERE AGENT_ID = '20170426095141389910']
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:55)  - unitRule        = [Y]
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:56)  - ruleSet         = [N]
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:58)  - intrusionDetect = [N]
30 5 2017 12:34:32,684 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:59)  - intrusionRoute  = [H]
30 5 2017 12:34:32,685 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:60)  - pushMessage     = []
30 5 2017 12:34:32,685 INFO  [conf-file-poller-0] (org.apache.flume.sink.DefaultSinkFactory.create:42)  - Creating instance of sink: jdbc-sink-921, type: org.apache.flume.sink.JDBCSink
30 5 2017 12:34:32,685 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSink.configure:45)  - Reading and processing configuration values for sink jdbc-sink-921
30 5 2017 12:34:32,685 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:46)  - JDBCSinkUtils.!!!
30 5 2017 12:34:32,685 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:47)  - jdbcDriver      = [oracle.jdbc.OracleDriver]
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:48)  - connectionURL   = [jdbc:oracle:thin:@1.234.83.169:1521:ORCL]
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:49)  - charset         = [euc-kr]
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:50)  - user            = [baropam]
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:51)  - password        = [baropam]
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:53)  - agentStatusStmt = []
30 5 2017 12:34:32,686 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:55)  - unitRule        = [N]
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:56)  - ruleSet         = [N]
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:58)  - intrusionDetect = [N]
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:59)  - intrusionRoute  = [H]
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:60)  - pushMessage     = []
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.DefaultSinkFactory.create:42)  - Creating instance of sink: jdbc-sink-200, type: org.apache.flume.sink.JDBCSink
30 5 2017 12:34:32,687 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSink.configure:45)  - Reading and processing configuration values for sink jdbc-sink-200
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:46)  - JDBCSinkUtils.!!!
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:47)  - jdbcDriver      = [oracle.jdbc.OracleDriver]
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:48)  - connectionURL   = [jdbc:oracle:thin:@1.234.83.169:1521:ORCL]
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:49)  - charset         = [euc-kr]
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:50)  - user            = [baropam]
30 5 2017 12:34:32,688 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:51)  - password        = [baropam]
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:53)  - agentStatusStmt = []
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:55)  - unitRule        = [N]
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:56)  - ruleSet         = [N]
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:58)  - intrusionDetect = [N]
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:59)  - intrusionRoute  = [H]
30 5 2017 12:34:32,689 INFO  [conf-file-poller-0] (org.apache.flume.sink.JDBCSinkUtils.<init>:60)  - pushMessage     = []
30 5 2017 12:34:32,693 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:114)  - Channel mem-channel-200 connected to [master1, jdbc-sink-200]
30 5 2017 12:34:32,693 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:114)  - Channel mem-channel-921 connected to [master1, jdbc-sink-921]
30 5 2017 12:34:32,693 INFO  [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:114)  - Channel mem-channel-100 connected to [master1, jdbc-sink-100]
30 5 2017 12:34:32,707 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:138)  - Starting new configuration:{ sourceRunners:{master1=EventDrivenSourceRunner: { source:Avro source master1: { bindAddress: 1.234.83.169, port: 61616 } }} sinkRunners:{jdbc-sink-100=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@69211341 counterGroup:{ name:null counters:{} } }, jdbc-sink-921=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@7a774652 counterGroup:{ name:null counters:{} } }, jdbc-sink-200=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@2ca9f04e counterGroup:{ name:null counters:{} } }} channels:{mem-channel-200=org.apache.flume.channel.MemoryChannel{name: mem-channel-200}, mem-channel-921=org.apache.flume.channel.MemoryChannel{name: mem-channel-921}, mem-channel-100=org.apache.flume.channel.MemoryChannel{name: mem-channel-100}} }
30 5 2017 12:34:32,707 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:145)  - Starting Channel mem-channel-200
30 5 2017 12:34:32,708 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:145)  - Starting Channel mem-channel-921
30 5 2017 12:34:32,709 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:145)  - Starting Channel mem-channel-100
30 5 2017 12:34:32,884 INFO  [lifecycleSupervisor-1-0] (org.apache.flume.instrumentation.MonitoredCounterGroup.register:120)  - Monitored counter group for type: CHANNEL, name: mem-channel-200: Successfully registered new MBean.
30 5 2017 12:34:32,884 INFO  [lifecycleSupervisor-1-1] (org.apache.flume.instrumentation.MonitoredCounterGroup.register:120)  - Monitored counter group for type: CHANNEL, name: mem-channel-921: Successfully registered new MBean.
30 5 2017 12:34:32,884 INFO  [lifecycleSupervisor-1-2] (org.apache.flume.instrumentation.MonitoredCounterGroup.register:120)  - Monitored counter group for type: CHANNEL, name: mem-channel-100: Successfully registered new MBean.
30 5 2017 12:34:32,884 INFO  [lifecycleSupervisor-1-0] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:96)  - Component type: CHANNEL, name: mem-channel-200 started
30 5 2017 12:34:32,884 INFO  [lifecycleSupervisor-1-1] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:96)  - Component type: CHANNEL, name: mem-channel-921 started
30 5 2017 12:34:32,885 INFO  [lifecycleSupervisor-1-2] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:96)  - Component type: CHANNEL, name: mem-channel-100 started
30 5 2017 12:34:32,885 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:173)  - Starting Sink jdbc-sink-100
30 5 2017 12:34:32,885 INFO  [lifecycleSupervisor-1-1] (org.apache.flume.sink.JDBCSink.start:56)  - Starting JDBCSink jdbc-sink-100 ...
30 5 2017 12:34:32,885 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:173)  - Starting Sink jdbc-sink-921
30 5 2017 12:34:32,886 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:173)  - Starting Sink jdbc-sink-200
30 5 2017 12:34:32,886 INFO  [lifecycleSupervisor-1-9] (org.apache.flume.sink.JDBCSink.start:56)  - Starting JDBCSink jdbc-sink-921 ...
30 5 2017 12:34:32,886 INFO  [lifecycleSupervisor-1-1] (org.apache.flume.sink.JDBCSink.start:56)  - Starting JDBCSink jdbc-sink-200 ...
30 5 2017 12:34:32,887 INFO  [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:184)  - Starting Source master1
30 5 2017 12:34:32,887 INFO  [lifecycleSupervisor-1-7] (org.apache.flume.source.AvroSource.start:228)  - Starting Avro source master1: { bindAddress: 1.234.83.169, port: 61616 }...
30 5 2017 12:34:32,889 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.JDBCSink.process:82)  - jdbc-sink-200 start to process event
30 5 2017 12:34:32,889 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.JDBCSink.process:82)  - jdbc-sink-100 start to process event
30 5 2017 12:34:32,889 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.JDBCSink.process:82)  - jdbc-sink-921 start to process event
30 5 2017 12:34:32,889 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.JDBCSink.saveAgentStatus:681)  - Save the Agent status jdbc-sink-100
30 5 2017 12:34:33,318 INFO  [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67)  - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
30 5 2017 12:34:33,407 INFO  [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67)  - jetty-6.1.26
30 5 2017 12:34:33,530 INFO  [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67)  - Started SelectChannelConnector@0.0.0.0:41414
30 5 2017 12:34:33,723 INFO  [lifecycleSupervisor-1-7] (org.apache.flume.instrumentation.MonitoredCounterGroup.register:120)  - Monitored counter group for type: SOURCE, name: master1: Successfully registered new MBean.
30 5 2017 12:34:33,723 INFO  [lifecycleSupervisor-1-7] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:96)  - Component type: SOURCE, name: master1 started
30 5 2017 12:34:33,723 INFO  [lifecycleSupervisor-1-7] (org.apache.flume.source.AvroSource.start:253)  - Avro source master1 started.
30 5 2017 12:34:33,926 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.JDBCSink.saveAgentStatus:696)  - executeUpdate = [1]
30 5 2017 12:43:14,579 INFO  [New I/O server boss #1 ([id: 0xe9962f91, /1.234.83.169:61616])] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)  - [id: 0x1429cc92, /1.234.83.169:31968 => /1.234.83.169:61616] OPEN
30 5 2017 12:43:14,581 INFO  [New I/O  worker #1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)  - [id: 0x1429cc92, /1.234.83.169:31968 => /1.234.83.169:61616] BOUND: /1.234.83.169:61616
30 5 2017 12:43:14,581 INFO  [New I/O  worker #1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)  - [id: 0x1429cc92, /1.234.83.169:31968 => /1.234.83.169:61616] CONNECTED: /1.234.83.169:31968