---
title: Build a Real Time Analytic dashboard with Solr Search and Spark Streaming
author: admin
type: post
date: 2015-05-21T17:59:27+00:00
url: /build-a-real-time-analytic-dashboard-with-solr-search-and-spark-streaming/
sf_thumbnail_type:
- none
sf_thumbnail_link_type:
- link_to_post
sf_detail_type:
- none
sf_page_title:
- 1
sf_page_title_style:
- standard
sf_no_breadcrumbs:
- 1
sf_page_title_bg:
- none
sf_page_title_text_style:
- light
sf_background_image_size:
- cover
sf_social_sharing:
- 1
sf_sidebar_config:
- left-sidebar
sf_left_sidebar:
- Sidebar-2
sf_right_sidebar:
- Sidebar-1
sf_caption_position:
- caption-right
sf_remove_promo_bar:
- 1
categories:
- Querying
- Tutorial
---
Search is a great way to interactively explore your data. The Search App is [continuously improving][1] and now comes with a better support for real time!
In this video, we are collecting tweets with Spark Streaming and directly indexing them into Solr with the [Spark Solr][2] app. Note that we are using a slightly [modified version][3] that adds more [tweet information][4].
You can see the tweets rolling in! Compared to the previous version:
* the dashboard updates its widgets only when the data changes without any page jumping
* the dashboard can refresh itself automatically every N seconds
* a main date filter lets you quickly select a rolling date range for all the dashboard
Tweets coming in
**Instructions**
Download a [nightly Solr 5.x][6], uncompress it and start it:
{{< highlight bash >}}
bin/solr start -cloud
bin/solr create -c tweets
{{< /highlight >}}
Then compile the [Spark Solr app][7].
Enable the analytic widgets in hue.ini:
{{< highlight bash >}}[search]
latest=true
{{< /highlight >}}
**Sum-up**
They are other ways to index data in [near real time][8] but we took this approach as the scenario was working out of the box with just Spark Streaming and the Solr app. Next time, we will preview the new [Analytics Features][9] of Solr 5.2 and show how we can use Python Spark to index some data!
As usual feel free to comment on the [hue-user][10] list or [@gethue][11]!
[1]: https://gethue.com/more-solr-search-dashboards-possibilities/
[2]: https://github.com/LucidWorks/spark-solr
[3]: https://github.com/romainr/spark-solr
[4]: https://github.com/romainr/spark-solr/commits/master
[5]: https://cdn.gethue.com/uploads/2015/05/live-search.png
[6]: https://builds.apache.org/job/Solr-Artifacts-5.x/lastSuccessfulBuild/artifact/solr/package/
[7]: https://github.com/romainr/spark-solr#example-applications
[8]: http://www.cloudera.com/content/cloudera/en/documentation/cloudera-search/v1-latest/Cloudera-Search-User-Guide/csug_flume_nrt_index_ref.html
[9]: http://yonik.com/solr-facet-functions/
[10]: http://groups.google.com/a/cloudera.org/group/hue-user
[11]: https://twitter.com/gethue