Spark ETL to extra analytics data from Gerrit Projects.
Job can be launched with the following parameters:
bin/spark-submit \ --conf spark.es.nodes=es.mycompany.com \ --conf spark.es.net.http.auth.user=elastic \ --conf spark.es.net.http.auth.pass=changeme \ $JARS/SparkAnalytics-assembly-1.0.jar \ --since 2000-06-01 \ --aggregate email_hour \ --url http://gerrit.mycompany.com \ -e gerrit/analytics
{"author": "John", "emails": ["john@email.com", "john@anotheremail.com"]} {"author": "David", "emails": ["david.smith@email.com", "david@myemail.com"]}
A docker compose file is provided to spin up an instance of Elastisearch with Kibana locally. Just run docker-compose up
.
Kibana will run on port 5601
and Elastisearch on port 9200
The Elastisearch default user is elastic
and the default password changeme
If Elastisearch dies with exit code 137
you might have to give Docker more memory (check this article for more details)