OpenTelemetry Collector

Description

You can export your log data directly to Mezmo with the OpenTelemetry Exporter. If you are already using OpenTelemetry Collector, you can start sending log data by adding the Mezmo Exporter to your existing Pipelines. If you are new to OTEL, you can use this quick start example to get you up and running.

Example OpenTelemetry Exporter Setup

1. Download the appropriate OTEL collector for your environment from the OpenTelemetry website.
2. Create a new pipeline in Mezmo.
3. Add an OpenTelemetry Logs Source node. Note the Pipeline URL and access key.
4. Add an OpenTelemetry Metrics Source node. Note the Pipeline URL and access key.
5. Add an OpenTelemetry Traces Source node. Note the Pipeline URL and access key.
6. Create a configuration file config.yaml with these contents:

receivers:
  filelog: #This will collect new lines from log files in /var/log
    include: [/var/log/*.log] 
  hostmetrics: #This will collect metrics about the host on which the collector is running.
    scrapers:
      cpu:
      disk:
      filesystem:
      load:
      memory:
  jaeger: #This will expose an endpoint where you can send traces
    protocols:
      grpc:
        endpoint: "0.0.0.0:14250"
exporters:
  otlphttp/logs:
    endpoint: "<endpoint from Logs node>"
    headers:
      Authorization: "<access key from Logs node>"
  otlphttp/metrics:
    endpoint: "<endpoint from Metrics node>"
    headers:
      Authorization: "<access key from Metrics node>"
  otlphttp/traces:
    endpoint: "<endpoint from Traces node>"
    headers:
      Authorization: "<access key from Traces node>"
        
processors:
  batch:
  
service:
  pipelines:
    logs:
      receivers: [ filelog ]
      exporters: [ otlphttp/logs ]
    traces:
      receivers: [jaeger]
      processors: [batch]
      exporters: [otlphttp/traces]
    metrics:
      receivers: [hostmetrics]
      processors: [batch]
      exporters: [otlphttp/metrics]

The endpoints in the exporters config must exactly match the URL provided by the OpenTelemetry Metrics / Logs / Traces sources, You don’t need to specify the /v1/metrics /v1/logs /v1/traces paths, which will otherwise be ignored by the Sources.

4. Start the Collector with this command: ./otelcol-contrib --config /path/to/config.yaml
5. The Collector will start sending logs, metrics and traces to your Pipeline.

For complete information on setting up and configuring the OpenTelemetry Collector, visit https://opentelemetry.io/docs/collector/configuration

Installing the Collector via a Helm Chart

You can install the OpenTelemetry collector directly in a Kubernetes cluster using a Helm chart.

If you don’t have the OpenTelemetry repo added to your Helm list, you must add it first.

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts

Update your repos once you’ve added the OpenTelemetry repo.

helm repo update

Then you can install the chart to deploy the Collector. The example shown here installs as a single global endpoint for all data with a Splunk HEC format as indicated in the provided .yamlconfiguration. You may also provide a configuration of your own like the one above.

To utilize this chart, create a Splunk HEC Source, and note the Endpoint and Token. As shown in this command, replace <YOUR PIPELINE INGEST ID HERE> with the ID portion of endpoint (the part that follows /v1/ ). Replace <YOUR TOKEN HERE> with your provisioned token.

helm upgrade -i --create-namespace -n mezmo \
  mezmo-otel-collector open-telemetry/opentelemetry-collector \
  --values https://repo.mezmo.com/otel-collector/values.yaml \
  --set global.mezmo.endpoint="<YOUR ROUTE ID>" \
  --set global.mezmo.token="<YOUR TOKEN HERE>"
  --set image.repository="otel/opentelemetry-collector-contrib"