FreedomBox for Communities/Monitoring Server

The monitoring client on the FreedomBox server of each of the communities will need to send the collected metrics to a central location. This central location, the monitoring server, needs to store the data sent, visualize the data for administrator and alert them when something is off. This section describes the setup on the server. It is assumed that the server is also running a FreedomBox. However, with minor changes to the instructions any server running Debian GNU/Linux Buster or above can be used as a server.

Setup Graphite Carbon
Graphite Carbon server is responsible collecting the data from the various FreedomBox servers and storing them in a time-series database called Whisper. Run the following commands to setup Graphite Carbon.



Install Graphite Carbon. 

Open up ports in the firewall for receiving data from FreedomBox Servers. 

After this, graphite-carbon server will be running and listening on port 2003. This can be checked using the following commands: 



Setup Graphite Web
Graphite Web service is responsible for serving the data stored by Graphite Carbon to consumers that can visualize the data such as Grafana. Run the following commands to setup Graphite Web.



Install Graphite Web 

Generate a secret key for Django framework used by Graphite Web. 

Note the key generated from previous command and set it as secret key in the file /etc/graphite/local_settings.py. 

Create a database for Graphite Web to store its own data. </li>

<li>Create Apache configuration for graphite-web and start serving. </li>

</ol>

Setup Grafana
Grafana consumes data from Graphite Web and produces beautiful visualizations. It can also be configured for alerting when certain event occurs in the data. Run the following commands to setup Grafana.

<ol>

<li>Install Grafana from it's project's Debian repositories. </li>

<li>Configure Grafana to be served from the a existing domain under a sub-path. Edit the file /etc/grafana/grafana.ini and ensure the following two directives are configured as shown. </li>

<li>Configure Grafana to start every time the machine is booted and start Grafana. </li>

<li>Create Apache configuration to proxy Grafana from a sub-path and start serving. </li>

<li>Setup credentials for Grafana access. Visit https://<domain_name>/monitor/ and login with the following credentials. You will be prompted to change the password. Set a strong password. </li>
 * Username: admin
 * Password: admin

<li>Create user accounts for each for users who will using the monitoring setup. In the left menu, go to Server Admin, Users. Click on the New User button and provide details such as name, email, username and password to create a new user account.</li>

<li>Connect Grafana to the Graphite Carbon database via Graphite Web we have configured in the previous sections. Go to Configuration, Data Sources and select Carbon data source. In the configuration dialog that shows up, provide the following details: The press Save & Test. You should see a message confirming the data source is accessible. </li>
 * Name: Carbon (or anything you choose)
 * URL: http://localhost:8080
 * Access: Server
 * Version: 1.1

</ol>

Creating a Dashboard for Each Community
In Grafana interface, you need to create a dashboard for monitoring each of the communities you are the administering. The following instructions provide an overview of how this can be done:


 * 1) Login to Grafana interface.
 * 2) In the menu, select Create, Dashboard. A dashboard is created and a prompt is shown to add a new panel. Select a Graph.
 * 3) A new dashboard is created with a simple graph.
 * 4) Save dashboard by clicking the save icon. In the save dialog, provide a name for the dashboard based on the community for which the dashboard is being created. Click Save.
 * 5) To start showing proper data in the panel, click on Panel Title and select Edit. In the Metric tab, click on select metric drop down. You should see the name of the domain you assigned to your community as configured in collectd. Then select a metric such as cpu-0 and then cpu-user. You should start seeing the plot of the metric in the graph. Save the dashboard.

Parameters to Monitor
It is recommended that you create a dashboard to your taste but monitor the following parameters at least for each community:


 * CPUs
 * Series: • * • cpu-user
 * Functions: averageSeries • alias(User)
 * Series: • * • cpu-nice
 * Functions: averageSeries • alias(Nice)
 * Series: • * • cpu-system
 * Functions: averageSeries • alias(System)
 * Series: • * • cpu-wait
 * Functions: averageSeries • alias(Wait)
 * Memory
 * Series: • memory • percent-used
 * Functions: alias(% Used)
 * Load
 * Series: • load • load • shorterm
 * Functions: alias(5min)
 * Series: • load • load • midterm
 * Functions: alias(10min)
 * Series: • load • load • longterm
 * Functions: alias(15min)
 * Disk
 * Series: • • disk_ops • read
 * Functions: alias(Read ops)
 * Series: • • disk_ops • write
 * Functions: alias(Write ops)
 * Series: • • disk_octets • read
 * Functions: alias(Read Bytes)
 * Series: • • disk_octets • write
 * Functions: alias(Write Bytes)
 * Disk I/O Time
 * Series: • • pending_operations
 * Functions: alias(Pending Operations)
 * Series: • • disk_io_time • io_time
 * Functions: alias(I/O Time)
 * Series: • • disk_io_time • weighted_io_time
 * Functions: alias(Weighted I/O Time)
 *  Partition (for each partition)
 * Series: • df- • percent_bytes-used
 * Functions: alias(Used)
 * Series: • df- • percent_bytes-reserved
 * Functions: alias(Reserved)
 * <LAN/WAN> Network Bandwidth (for each interface)
 * Series: • interface- • if_octets • rx
 * Functions: alias(Received)
 * Series: • interface- • if_octets • tx
 * Functions: alias(Transmitted)
 * Connections Tracked
 * Series: • conntrack • conntrack
 * Functions: alias(Connections Tracked)
 * Wi-Fi Access Points
 * alias(sumSeries(isNonNull( .ping.ping-10_42_0_[1-9][0-9])), 'Wi-Fi Access Points')
 * Internet Uplink
 * alias(isNonNull( .ping.ping-10_42_0_4), 'Internet Router')
 * alias(isNonNull( .ping.ping-10_42_0_5), 'PTP Link - Router A')
 * alias(isNonNull( .ping.ping-10_42_0_6), 'PTP Link - Router B')
 * Internet Ping
 * Series: • ping • ping-1_1_1_1
 * Functions: alias(Internet Ping)
 * Processes
 * Series: • processes • ps_state-running
 * Functions: alias(Running)
 * Series: • processes • ps_state-sleeping
 * Functions: alias(Sleeping)
 * Series: • processes • ps_state-paging
 * Functions: alias(Paging)
 * Series: • processes • ps_state-blocked
 * Functions: alias(Blocked)
 * Series: • processes • ps_state-zombies
 * Functions: alias(Zombies)
 * Series: • processes • ps_state-stopped
 * Functions: alias(Stopped)
 * Uptime
 * Series: • uptime • uptime
 * Functions: alias(Time Since Reboot)
 * Users
 * Series: • users • users
 * Functions: alias(Logged-In Users)
 * collectd Queue
 * Series: • collectd-cache • cache_size
 * Functions: alias(Cache Size)
 * Series: • collectd-write_queue • derive-dropped
 * Functions: alias(Dropped Metrics)
 * Series: • collectd-write_queue • queue_length
 * Functions: alias(Queue Length)