Whether you’re a small company selling hand-knit dog sweaters or a powerhouse like Amazon, most modern websites have a way to let their users search for something. The developers of these modern websites take advantage of open source tools like Lucene and Elasticsearch to add powerful search capabilities, like synonyms and fuzzy matching. However, integrating these tools with frameworks like Rails is hard. Fortunately, Gems like Searchkick provide a simpler way to integrate Rails and Elasticsearch, allowing you take full advantage of the robustness that Elasticsearch has to offer.
In the example below, we’ll show you how to easily add powerful search capabilities to your Rails application using Searchkick.
Prerequisites
- Download and install Elasticsearch
- (Optional) Setup a PostgreSQL database
Setup
Since our goal is to quickly bootstrap this application, let’s get started with the Rails app scaffolding!
Local Setup
The following commands set up your rails folder structure, creates a Question model and its associated migration, and runs the migration in your database.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
rails new search_app
rails g scaffold Question text:string answer:string
rake db:migrate[/pcsh]
Now add the following gems to your Gemfile, then run bundle install.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# Gemfile
gem 'searchkick'[/pcsh]
PostgreSQL (optional)
If you’re using PostgreSQL, make sure your database is running and add the following configuration (with the associated values in your secrets.yml file) to database.yml in your rails application before you run your migrations.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
username: <%= Rails.application.secrets[:DATABASE_USERNAME] %>
password: <%= Rails.application.secrets[:DATABASE_PASSWORD] %>
host: <%= Rails.application.secrets[:DATABASE_IP] %>
port: 5432
development:
<<: *default
database: simple_search
test:
<<: *default
database: simple_search_test
production:
<<: *default
database: simple_search_prod
[/pcsh]
Search
ElasticSearch Setup
In order to index objects for searching locally, Elasticsearch will need to be installed and running. You can test this by running curl 'https://localhost:9200/?pretty'. More on that here.
Setup Question Model
With Searchkick and Elasticsearch installed it’s time to tell your Question model to use Searchkick. This will allow Question records to be indexed and searched by Elasticsearch.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# app/models/question.rb
class Question < ActiveRecord::Base
searchkick
end
[/pcsh]
Data for Search
Now that your Question model knows how to interact with Searchkick, let’s create some data that you can use for searching. In many cases, searchable data is created by your users. In this case, you just need test data. To create some test data, fire up your console by typing rails c in the root directory of your Rails app. Then create the following questions:
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# Rails Console
> Question.create(text: "Is Excella hiring?", answer: "Yes")
> Question.create(text: "What is a good programming language?", answer: "Ruby")
> Question.create(text: "Is the food at Excella good?", answer: "Yes")[/pcsh]
Now that you’ve added some data, you can tell Searchkick to index your newly created Question objects.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
> Question.reindex[/pcsh]
Once you have an index for our questions, you can quickly test that the search functionality works in your Rails console:
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
> Question.search('good')
Question Search (16.2ms) curl https://localhost:9200/questions_development/_search?pretty -d '{"query":{"bool":{"must":{"dis_max":{"queries":[{"match":{"_all":{"query":"good","operator":"and","boost":1,"fuzziness":1,"prefix_length":0,"max_expansions":3,"analyzer":"searchkick_search"}}},{"match":{"_all":{"query":"good","operator":"and","boost":1,"fuzziness":1,"prefix_length":0,"max_expansions":3,"analyzer":"searchkick_search2"}}},{"match":{"_all":{"query":"good","operator":"and","boost":10,"analyzer":"searchkick_search"}}},{"match":{"_all":{"query":"good","operator":"and","boost":10,"analyzer":"searchkick_search2"}}}]}},"should":{"nested":{"path":"conversions","score_mode":"total","query":{"function_score":{"boost_mode":"replace","query":{"match":{"query":"good"}},"field_value_factor":{"field":"count"}}}}}}},"size":100000,"from":0,"fields":[]}'
Question Search (16.2ms) curl https://localhost:9200/questions_development/_search?pretty -d '{"query":{"bool":{"must":{"dis_max":{"queries":[{"match":{"_all":{"query":"good","operator":"and","boost":1,"fuzziness":1,"prefix_length":0,"max_expansions":3,"analyzer":"searchkick_search"}}},{"match":{"_all":{"query":"good","operator":"and","boost":1,"fuzziness":1,"prefix_length":0,"max_expansions":3,"analyzer":"searchkick_search2"}}},{"match":{"_all":{"query":"good","operator":"and","boost":10,"analyzer":"searchkick_search"}}},{"match":{"_all":{"query":"good","operator":"and","boost":10,"analyzer":"searchkick_search2"}}}]}},"should":{"nested":{"path":"conversions","score_mode":"total","query":{"function_score":{"boost_mode":"replace","query":{"match":{"query":"good"}},"field_value_factor":{"field":"count"}}}}}}},"size":100000,"from":0,"fields":[]}'
=> #<Searchkick::Results:0x000000026897d0 @klass=Question(id: integer, text: string, slug: string, blurb: string, answer: string, created_at: datetime, updated_at: datetime), @response={"took"=>13, "timed_out"=>false, "_shards"=>{"total"=>5, "successful"=>5, "failed"=>0}, "hits"=>{"total"=>2, "max_score"=>0.03191028, "hits"=>[{"_index"=>"questions_development_20151011160326849", "_type"=>"question", "_id"=>"4", "_score"=>0.03191028}, {"_index"=>"questions_development_20151011160326849", "_type"=>"question", "_id"=>"5", "_score"=>0.03191028}]}}, @options={:page=>1, :per_page=>100000, :padding=>0, :load=>true, :includes=>nil, :json=>false}> [/pcsh]
So what was that? Searchkick just curl'd your Elasticsearch server and returned results! You can see the results of questions that came back below.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
> _.results
Question Load (0.8ms) SELECT "questions".* FROM "questions" WHERE "questions"."id" IN (4, 5)
Question Load (0.8ms) SELECT "questions".* FROM "questions" WHERE "questions"."id" IN (4, 5)
=> [#<Question id: 4, text: "What is a good programming language?", slug: nil, blurb: nil, answer: "Ruby", created_at: "2015-12-14 01:01:25", updated_at: "2015-12-14 01:01:25">, #<Question id: 5, text: "Is the food at Excella good?", slug: nil, blurb: nil, answer: "Yes", created_at: "2015-12-14 01:01:32", updated_at: "2015-12-14 01:01:32">] r[/pcsh]
Using Searchkick in your application
Now let’s create a view and controller for searching the data you’ve set up. Below you will find an example of a controller action that utilizes the search method to render the results in a view.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# app/controllers/search_controller.rb
def results
@query = Question.search(params[:query])
end[/pcsh]
If you want to display search results in a view, you can access them in a template using @query.results.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
# app/views/search/results.html.erb
<% @query.results.each do |result| %>
<div class='result-item'>
<%= link_to("#{result.text}", question_path(result.id)) %>
</div>
<% end %>[/pcsh]
If you are interested in learning more about how this works, check out this Github repository for a working example.
Search on Heroku
Great work! You’ve built an application that lets users search, but now you want to make it available to users over the internet. In other words, you need to deploy your application to a production web server.
Heroku provides a popular platform for quick and simple app deployments. Deploying to Heroku can be a quick way to test out this search functionality in a production environment.
Heroku provides an in-depth guide on setting Rails up for Heroku deployments. You can find that here.
Once your application has been pushed up to Heroku and deployed, go ahead and add the Heroku Elasticsearch addon and set the proper environment variables. You can run these commands in the console from the rails app directory, assuming the Heroku toolbelt is already installed.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
heroku addons:add searchbox:starter
heroku config:add ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL`[/pcsh]
Before you execute a search in your production app, you must index all the questions in production. Assuming you have already migrated our database and added some test data to the questions table, let’s run the reindex command on our Question model.
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
heroku run rake searchkick:reindex CLASS=Question
[/pcsh]
For more details, refer to the Searchkick readme.
TIP: Nightly cron job to reindex
In order for your search results to continually improve, it is good practice to frequently reindex search data. Heroku provides an add-on called the scheduler for this purpose. You can use the scheduler to run a task on a daily, hourly, or 10-minute interval. There is no limit on the number of times your data can be reindexed. The reindexing interval should be determined based on the needs of the application and its users.
To install the scheduler, run:
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
heroku addons:create scheduler:standard[/pcsh]
Open the scheduler application:
[pcsh lang=”ruby” tab_size=”4″ message=”” hl_lines=”” provider=”manual”]
heroku addons:open scheduler[/pcsh]
Click “Add a new job”, and enter the reindex command above (heroku run rake searchkick:reindex CLASS=Question) with an interval of “Daily” to run at midnight (00:00 UTC). If your reindex command is more complex, you can define custom Rake tasks in your application and use the scheduler to execute them.