Events and metrics

To provide an effective feedback loop, HTTP/TCP logs should be collected, stored and analyzed. Collection and storing is done with a combination of HAProxy, VGA and Logstash setup. Logs are stored in Elasticsearch and can be later visualised by Kibana.

How traffic is logged

Vamp uses Logstash to format and store logs from running applications HAProxy (via Vamp Gateway Agent) in Elasticsearch indices. Logstash listens on UDP port and formats incoming log raw data in json format. Vamp API actions (including those generated by Vamp workflows and the Vamp UI) and running service events are stored by the Vamp API to specific Elasticsearch indices. Data and events are read by Vamp components either directly from Elasticsearch or via the Vamp API:

  • The Vamp UI reads data and events via the Vamp API. Health and Metrics events (generated by Vamp workflows) are required by the Vamp UI.
  • Vamp workflows read formatted log data directly from Elasticsearch and events are via the Vamp API. In theory, workflows could also read events directly from Elasticsearch.

Formatted raw data (logs)

HAProxy

HAProxy generates logs and makes them accessible via open socket - check the HAProxy configuration of log (github.com/magneticio - haproxy.cfg). VGA listens on log socket and any new messages are forwarded to the Logstash instance. The HAProxy log format is configurable in Vamp configuration vamp.gateway-driver.haproxy (github.com/magneticio - reference.conf). In general, for each HTTP/TCP request to HAProxy, several log messages are created (e.g. for gateway, service and instance level).

Vamp uses Filebeat to ship HAproxy logs from the Vamp Gateway Agents to Elasticsearch. Vamp components read data either directly from Elasticsearch or via the Vamp API.

Events

Vamp collects events on all running services. Vamp API actions (including those generated by Vamp workflows and the Vamp UI) and running service events are stored by the Vamp API to specific Elasticsearch indices. Third party applications can also create events and trigger Vamp actions. All events are stored and retrieved using the Event API, part of the Vamp API.

Kibana

Vamp can be configured to create Kibana searches, visualisations and dashboards automatically with the vamp.gateway-driver.kibana.enabled configuration parameter. Vamp will do this by inserting ES documents to the Kibana index, so only the URL to access ES is needed (by default reusing the same as for persistence). Read more about Vamp configuration

What next?