Logstash是一个开源的服务器端数据处理管道，可以同时从多个源获取数据。面对海量的日志量，rsyslog和sed，awk等日志收集，处理工具已经显的力不从心。logstash是一个整合型的框架，可以用以日志的收集，存储，索引构建。

1. logstash安装

1.1 安装环境

  • centos 7.2
  • JDK：1.8.0_112

1.2 安装

直接下载压缩包，解压运行即可。

1
2
# wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.0.tar.gz
# tar -zxvf logstash-6.5.0.tar.gz

2. 验证

进入解压的logstash文件夹目录下的bin目录中，运行以命令行的形式运行logstash：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# cd logstash-6.5.0/bin/
# ./logstash -e 'input{ stdin{} } output{ stdout{} }'
Sending Logstash logs to /root/logstash-6.5.0/logs which is now configured via log4j2.properties
[2018-11-19T14:30:25,975][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/root/logstash-6.5.0/data/queue"}
[2018-11-19T14:30:26,010][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/root/logstash-6.5.0/data/dead_letter_queue"}
[2018-11-19T14:30:27,642][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-11-19T14:30:27,848][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.5.0"}
[2018-11-19T14:30:28,051][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"7e16bcaa-b50e-4751-9fac-7acd00dc1773", :path=>"/root/logstash-6.5.0/data/uuid"}
[2018-11-19T14:30:49,501][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-11-19T14:30:50,690][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2418c105 run>"}
The stdin plugin is now waiting for input:
[2018-11-19T14:30:51,118][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-11-19T14:30:52,429][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
hello world
{
      "@version" => "1",  # 版本号
       "message" => "hello world", # 输入内容
          "host" => "xxxxxx",  # 主机名，数据是从哪个节点发过来的
    "@timestamp" => 2018-11-19T06:31:03.697Z # 自动生成的时间戳
}