JFrog supports integration with New Relic, a SaaS-based data analytics platform, for log analytics and real-time observability. Using this integration, you can collect logs, violations and metrics from the JFrog Platform to understand the usage of artifacts and repositories across your organization, monitor system performance of platform services, and analyze storage consumption, errors, and connection statistics associated with your deployment. In addition, the violations data from JFrog Xray provide actionable insights into the license and security vulnerabilities impacting your artifacts and components.
This integration is supported for Artifactory versions 7 only.
You must follow the order of the steps throughout the New Relic configuration.
New Relic Setup
New Relic setup can be done by going through the onboarding steps below or by using license key directly, if one exists. If a license key exists, use the New Relic Fluentd plugin to forward logs, violations and metrics directly to your New Relic account.
Create an account in New Relic
From the account dropdown, click API keys
Copy the license key which is also referenced in the UI as ingest - license
Page Contents
Environment Configuration
We rely heavily on environment variables so that the correct log files are streamed to your observability dashboards. Ensure that you set theJF_PRODUCT_DATA_INTERNALenvironment variable to the correct path for your product.
The environment variableJF_PRODUCT_DATA_INTERNALmust be defined to the correct location.
Helm-based installs will already have this defined based on the underlying Docker images.
For non-k8s-based installations, below is a reference to the Docker image locations per product. Note these locations may be different based on the chosen installation location.
Once RVM installation is complete, execute the commandrvm -vto verify the installation executing.
Now install ruby v2.7.0 or above executing the commandrvm install , ex:rvm install 2.7.5.
Verify the ruby installation. Executeruby -v, gem installationgem -vandbundler -vto ensure all the components are intact.
Post completion of Ruby, Gems installation, the environment is ready to further install new gems. Execute the following gem install commands one after the other to set up the needed ecosystem:
gem install fluentd
After FluentD is successfully installed, the below plugins are must be installed:
Configurefluent.conf.*according to the instructions mentioned in theFluentd Configuration for New Relicsection and then run the fluentd wrapper with one argument pointed to thefluent.conf.*file configured.
./fluentd $JF_PRODUCT_DATA_INTERNAL/fluent.conf.
Docker
To run fluentd as a Docker image to send the log, siem and metrics data to New Relic, the following commands must be executed on the host that runs the Docker.
Check the Docker installation is functional. Execute the commandsdocker versionanddocker ps.
Once the version and processes are listed successfully, build the intended Docker image for the observability platform using the Docker file.
Download Dockerfile fromhereto any directory that has write permissions.
Download theDockerenvfile_.txtfile needed to run Jfrog/FluentD Docker Images for the intended observability platform.
DownloadDockerenvfile_newrelic.txtfromhereto the directory where the Docker file was downloaded.
For New Relic as the observability platform, execute these commands to setup the docker container running the fluentd installation:
执行的docker build --build-arg SOURCE="JFRT" --build-arg TARGET="NEWRELIC" -t .' Command example: 'docker build --build-arg SOURCE="JFRT" --build-arg TARGET="NEWRELIC" -t jfrog/fluentd-newrelic-rt.' The above command will build the docker image.
Fill the necessary information in theDockerenvfile_newrelic.txtfile, if the value for any of the field requires to have a '/' use '\/' and if '\' is required use '\\'.
执行的docker run -it --name jfrog-fluentd-newrelic-rt -v :/var/opt/jfrog/artifactory --env-file Dockerenvfile_newrelic.txt ' The should be an absolute path where the Jfrog Artifactory Logs folder resides, i.e for an Docker-based Artifactory Installation. Example:/var/opt/jfrog/artifactory/var/logson the Docker host. Command example: 'docker run -it --name jfrog-fluentd-newrelic-rt -v /var/opt/jfrog/artifactory/var:/var/opt/jfrog/artifactory--env-file Dockerenvfile_newrelic.txt jfrog/fluentd-newrelic-rt'
Kubernetes Deployment with Helm
Recommended installation for Kubernetes is to utilize the helm chart with the associated values.yaml in this repo.
Product
Example Values File
Artifactory
helm/artifactory-values.yaml
Artifactory HA
helm/artifactory-ha-values.yaml
Xray
helm/xray-values.yaml
Update the values.yaml associated to the product you want to deploy with your New Relic settings.
Replace placeholders with yourmasterKeyandjoinKey. To generate each of them, use the commandopenssl rand -hex 32
Artifactory:
Replace thenewrelic_licensekeyinnewrelic.licensekeyat the end of the yaml file with License key copied from New Relic inNew Relic Setup
Replacejpd_urlinjfrog.observability.metrics.jpd_urlwith Artifactory JPD URL (note - if deployed on K8s use the localhost and port number combination per sidecar)
Replacejfrog_userinjfrog.observability.metrics.usernamewith Artifactory username for authentication
Replacejfrog_api_keyinjfrog.observability.metrics.apikeywithArtifactory API Key
Replacecommon_jpd_valueinjfrog.observability.metrics.common_jpd与真实non-kubernetes装置或者本月allations where JPD base URL is same to access both Artifactory and Xray (ex:https://sample_base_url/artifactoryorhttps://sample_base_url/xray). Default value is false
Replace placeholders with yourmasterKeyandjoinKey. To generate each of them, use the commandopenssl rand -hex 32
Replace thenewrelic_licensekeyinnewrelic.licensekeyat the end of the yaml file with License key copied from New Relic inNew Relic Setup
Replacejpd_urlinjfrog.observability.metrics.jpd_urlwith Artifactory JPD URL (note - if deployed on K8s use the localhost and port number combination per sidecar)
Replacejfrog_userinjfrog.observability.metrics.usernamewith Artifactory username for authentication
Replacejfrog_api_keyinjfrog.observability.metrics.apikeywithArtifactory API Key
Replacecommon_jpd_valueinjfrog.observability.metrics.common_jpd与真实non-kubernetes装置或者本月allations where JPD base URL is same to access both Artifactory and Xray (ex:https://sample_base_url/artifactoryorhttps://sample_base_url/xray). Default value is false
Update the following fields in/helm/xray-values.yaml:
Replace thenewrelic_licensekeyinnewrelic.licensekeyat the end of the yaml file with License key copied from New Relic inNew Relic Setup
Replacejpd_urlinjfrog.observability.jpd_urlwith Artifactory JPD URL (note - if deployed on K8s use the localhost and port number combination per sidecar)
Replacejfrog_userinjfrog.observability.usernamewith Artifactory username for authentication
Download and configure the relevant fluentd.conf files for New Relic
Configuration Steps for Artifactory
Download the artifactory fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.rt
Logs data
Override the match directive (jfrog.**) of the downloadedfluent.conf.rtto send logs data to New Relic:
Download the Xray fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.xray
Logs and Violation data
Override the source directive of the downloadedfluent.conf.xrayto pull Xray Violations:
Download the Nginx fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.nginx
覆盖匹配指令(最后一节n) of the downloadedfluent.conf.nginxwith the details given below:
Required:LICENSE_KEYis the License Key from New Relic inNew Relic Setup
Configuration Steps for Mission Control
Download the Mission Control fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.missioncontrol
覆盖匹配指令(最后一节n) of the downloadedfluent.conf.missioncontrolwith the details given below:
Required:LICENSE_KEYis the License Key from New Relic inNew Relic Setup.
Configuration Steps for Distribution
Download the distribution fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.distribution
覆盖匹配指令(最后一节n) of the downloadedfluent.conf.distributionwith the details given below:
Required:LICENSE_KEYis the License Key from New Relic inNew Relic Setup.
Configuration Steps for Pipelines
Download the pipelines fluentd configuration file to a directory the user has permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations discussed above in theEnvironment Configurationsection.
cd $JF_PRODUCT_DATA_INTERNAL wget https://raw.githubusercontent.com/jfrog/log-analytics-newrelic/master/fluent.conf.pipelines
覆盖匹配指令(最后一节n) of the downloadedfluent.conf.pipelineswith the details given below:
Required:LICENSE_KEYis the License Key from New Relic inNew Relic Setup.
Dashboards
Artifactory dashboard
JFrog Artifactory Dashboard is divided into three sections: Application, Audit, Requests and Docker.
Application- This section tracks Log Volume (information about different log sources) and Artifactory Errors over time (bursts of application errors that may otherwise go undetected).
Audit- This section tracks audit logs that help you determine who is accessing your Artifactory instance and from where. These can help you track potentially malicious requests or processes (such as CI jobs) using expired credentials.
Requests- This section tracks HTTP response codes, top 10 IP addresses for uploads and downloads.
Docker- To monitor Dockerhub pull requests users should have a Dockerhub account, either paid or free. Free accounts allow up to 200 pull requests per 6-hour window. Various widgets have been added in the new Docker tab under Artifactory to help monitor your Dockerhub pull requests. An alert is also available to enable, if desired, that will allow you to send emails or add outbound webhooks through configuration to be notified when you exceed the configurable threshold.
Metrics- To gain insights into the system performance, storage consumption, and connection statistics associated with JFrog Artifactory
Xray dashboard
JFrog Xray Dashboard is divided into three sections: Logs, Violations and Metrics
Logs- This dashboard provides a summary of access, service and traffic log volumes associated with Xray. Additionally, customers are also able to track various HTTP response codes, HTTP 500 errors, and log errors for greater operational insight.
Violations- This dashboard provides an aggregated summary of all the license violations and security vulnerabilities found by Xray. Information is segment by watch policies and rules. Trending information is provided on the type and severity of violations over time, as well as, insights on most frequently occurring CVEs, top impacted artifacts and components.
Metrics- To gain insights into the system performance, storage consumption, connection statistics, count and type of artifacts and components scanned by JFrog Xray.
