Homework: Difference between revisions

From John's wiki
Jump to navigation Jump to search
No edit summary
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
See messages from Jedd:
This [[homework]] project is for John to develop a network monitoring capability under the tutelage of his Sensei, Jedd.
 
== Messages from Jedd ==
 
* [https://www.progclub.org/pipermail/programming/2019-June/004272.html Fri Jun 28 09:01:40 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004183.html Wed Jun 12 19:11:53 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004180.html Wed Jun 12 00:40:49 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004178.html Wed Jun 12 00:22:50 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004175.html Tue Jun 11 23:58:41 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004173.html Tue Jun 11 23:21:46 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-June/004173.html Tue Jun 11 23:21:46 AEST 2019]
* [https://www.progclub.org/pipermail/programming/2019-March/003823.html Thu Mar 21 08:37:55 AEDT 2019]
* [https://www.progclub.org/pipermail/programming/2019-March/003823.html Thu Mar 21 08:37:55 AEDT 2019]
* [https://www.progclub.org/pipermail/programming/2019-March/003777.html Wed Mar 13 21:34:52 AEDT 2019]
* [https://www.progclub.org/pipermail/programming/2019-March/003777.html Wed Mar 13 21:34:52 AEDT 2019]
* [https://www.progclub.org/pipermail/programming/2018-August/003438.html Fri Aug 31 23:40:10 AEST 2018]
* [https://www.progclub.org/pipermail/programming/2018-August/003438.html Fri Aug 31 23:40:10 AEST 2018]
== Component summary ==


{|class="wikitable"
{|class="wikitable"
Line 59: Line 69:
|}
|}


Box: single box: 6GB, 2 core, separate LVM-backed file systems for postgresql, elasticsearch, and prometheus
== Further reading ==


* [https://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems Comparison of network monitoring systems]
* [https://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems Comparison of network monitoring systems]
* [https://en.wikipedia.org/wiki/List_of_systems_management_systems List of systems management systems]
* [https://en.wikipedia.org/wiki/List_of_systems_management_systems List of systems management systems]
== Buzzwords ==
Terms Jedd is liable to drop on you:
{|class="wikitable"
! Term        !! Meaning
|-
| APM          || Application Performance Monitoring
|-
| RCA          || Root Cause Analysis
|-
| SIEM        || Security Information and Event Management
|}
= Tasks =


== TODO ==
== TODO ==
Stuff to do. Priorities on top.


Do all this in your test environment so you don't need to worry about firewalls and user auth:
Do all this in your test environment so you don't need to worry about firewalls and user auth:


# gandalf: Spin up a VM for logging - install graylog as per instructions.  Point some rsyslog stuff towards it.  Review the tutorials.
* read the [https://www.zabbix.com/documentation/current/start Zabbix documentation]
# ripley: Spin up a VM with Prometheus and Grafana - install node-exporter on one of your other boxes, and point that towards your Prometheus server.   
* [https://linuxhandbook.com/grafana-setup/ How to Install and Configure Grafana] (also [https://grafana.com/docs/grafana/latest/installation/ Install Grafana])
# faith: Spin up a VM for Elasticsearch + Logstash + Kibana and set up Metricbeat on one of your other boxes, and point that towards your Elasticsearch server.
* gandalf: Spin up a VM for logging - install graylog as per instructions.  Point some rsyslog stuff towards it.  Review the tutorials.
* ripley: Spin up a VM with Prometheus and Grafana - install node-exporter on one of your other boxes, and point that towards your Prometheus server.   
* faith: Spin up a VM for Elasticsearch + Logstash + Kibana and set up Metricbeat on one of your other boxes, and point that towards your Elasticsearch server.
 
== Done ==
 
Stuff that's done. Latest on top.
 
* 2020-03-29 jj5 - I completed [https://www.zabbix.com/download?zabbix=4.4&os_distribution=ubuntu&os_version=18.04_bionic&db=mysql&ws=apache Install and configure Zabbix server for your platform]
* 2020-03-28 jj5 - I completed [https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-getting-started.html Get started with Metricbeat]
* 2020-03-28 jj5 - I found [https://www.elastic.co/guide/en/elastic-stack-get-started/7.6/get-started-elastic-stack.html Getting started with the Elastic Stack]
* 2020-03-28 jj5 - I completed [https://www.elastic.co/guide/en/kibana/current/deb.html Install Kibana with Debian Package]
* 2020-03-28 jj5 - I completed [https://linuxize.com/post/how-to-install-elasticsearch-on-ubuntu-18-04/ How to Install Elasticsearch on Ubuntu 18.04]
* 2020-03-28 jj5 - created '[[#knowledge-test|knowledge-test]]' (sm-log-1) and did a basic base config with Salt Stack: see [https://www.progclub.org/pipermail/programming/2019-June/004272.html instructions from Jedd]
 
= Virtual machines =
 
== gandalf-test ==
 
These notes are obsolete, see [[#knowledge-test|knowledge-test]] instead.
 
{|class="wikitable"
! Setting          !! Value
|-
| Sysid            || 141
|-
| System          || sm-log-1
|-
| Host            || gandalf-test
|-
| Deployment      || test
|-
| Provider        || vbox
|-
| Salt ID          || 141-sm-log-1-gandalf-test-vbox
|-
| Machine Folder  || /srv/vbox/fast
|-
| Type            || Linux
|-
| Version          || Ubuntu (64-bit)
|-
| Memory Size      || 3072 MB
|-
| Processor(s)    || 2 CUPs
|-
| Root Volume Size || 30 GB
|-
| Root Volume Path || /srv/vbox/fast/gandalf-test/gandalf-test.vdi
|-
| LVM PV 1 Size    || 500 GB
|-
| LVM PV 1 Path    || /srv/vbox/slow/gandalf-test/gandalf-test-pv-1-500.vdi
|-
| Attached to      || Bridged Adapter
|-
| Ethernet Adapter || enp3s0f0
|-
| IP Address      || 10.1.3.141/16
|-
| Gateway          || 10.1.1.1
|-
| DNS              || 10.1.1.113
|}
 
== knowledge-test ==
 
Box: single box: 2 core, 6GB RAM, 100GB disk
 
{|class="wikitable"
! Setting          !! Value
|-
| Sysid            || 141
|-
| System          || sm-log-1
|-
| Host            || knowledge-test
|-
| Deployment      || test
|-
| Provider        || vbox
|-
| Salt ID          || 141-sm-log-1-knowledge-test-vbox
|-
| Machine Folder  || /srv/vbox/fast
|-
| Type            || Linux
|-
| Version          || Ubuntu (64-bit)
|-
| Memory Size      || 6 GB
|-
| Processor(s)    || 2 CUPs
|-
| Root Volume Size || 100 GB
|-
| Root Volume Path || /srv/vbox/fast/knowledge-test/knowledge-test.vdi
|-
| Attached to      || Bridged Adapter
|-
| Ethernet Adapter || enp3s0f0
|-
| IP Address      || 10.1.2.141/16
|-
| Gateway          || 10.1.2.1
|-
| DNS              || 10.1.1.113
|}

Latest revision as of 20:22, 29 March 2020

This homework project is for John to develop a network monitoring capability under the tutelage of his Sensei, Jedd.

Messages from Jedd

Component summary

Software Note Function
Prometheus
Loki by the Grafana guys
Grafana
Elasticsearch
metricbeat
Graylog
Zabbix
PostgreSQL say with TimescaleDB
Logstash
Kibana
NetFlow (elastiflow)
NetData

Further reading

Buzzwords

Terms Jedd is liable to drop on you:

Term Meaning
APM Application Performance Monitoring
RCA Root Cause Analysis
SIEM Security Information and Event Management

Tasks

TODO

Stuff to do. Priorities on top.

Do all this in your test environment so you don't need to worry about firewalls and user auth:

  • read the Zabbix documentation
  • How to Install and Configure Grafana (also Install Grafana)
  • gandalf: Spin up a VM for logging - install graylog as per instructions. Point some rsyslog stuff towards it. Review the tutorials.
  • ripley: Spin up a VM with Prometheus and Grafana - install node-exporter on one of your other boxes, and point that towards your Prometheus server.
  • faith: Spin up a VM for Elasticsearch + Logstash + Kibana and set up Metricbeat on one of your other boxes, and point that towards your Elasticsearch server.

Done

Stuff that's done. Latest on top.

Virtual machines

gandalf-test

These notes are obsolete, see knowledge-test instead.

Setting Value
Sysid 141
System sm-log-1
Host gandalf-test
Deployment test
Provider vbox
Salt ID 141-sm-log-1-gandalf-test-vbox
Machine Folder /srv/vbox/fast
Type Linux
Version Ubuntu (64-bit)
Memory Size 3072 MB
Processor(s) 2 CUPs
Root Volume Size 30 GB
Root Volume Path /srv/vbox/fast/gandalf-test/gandalf-test.vdi
LVM PV 1 Size 500 GB
LVM PV 1 Path /srv/vbox/slow/gandalf-test/gandalf-test-pv-1-500.vdi
Attached to Bridged Adapter
Ethernet Adapter enp3s0f0
IP Address 10.1.3.141/16
Gateway 10.1.1.1
DNS 10.1.1.113

knowledge-test

Box: single box: 2 core, 6GB RAM, 100GB disk

Setting Value
Sysid 141
System sm-log-1
Host knowledge-test
Deployment test
Provider vbox
Salt ID 141-sm-log-1-knowledge-test-vbox
Machine Folder /srv/vbox/fast
Type Linux
Version Ubuntu (64-bit)
Memory Size 6 GB
Processor(s) 2 CUPs
Root Volume Size 100 GB
Root Volume Path /srv/vbox/fast/knowledge-test/knowledge-test.vdi
Attached to Bridged Adapter
Ethernet Adapter enp3s0f0
IP Address 10.1.2.141/16
Gateway 10.1.2.1
DNS 10.1.1.113
Retrieved from "https://www.jj5.net/wiki/mediawiki/index.php?title=Homework&oldid=4058"