Ruby/Rails utilizing Solr in Master/Slave setup

Apache Solr is a high-performance enterprise grade search server being interacted with a REST-like API, documents are provided to as xml, json or binary. It extends Lucene search library.

Ruby/Rails communicates with this awesome search server using Sunspot (sunspot, sunspot_rails gem) library, to do a full-text search. Nice tutorial to use Solr in your Rails project using Sunspot.

Solr can perform as a standalone search server and even as master/slave instances collaborating with each other, with slaves polling master to sync-in their data.

Solr instances can be configured like Slave and as a Master.
Configuration file 'solrconfig.xml' needs to be edited with a new RequestHandler for Replication configured:

 

{for Slave} :: to either poll at their Master's machine address at regular intervals

{for Master} :: or commit the changes and clone the mentioned configuration files when Slave asked for it 

for detailed reference:http://wiki.apache.org/solr/SolrReplication 

for optimizaed ways using ssh/rsync based replication:
http://wiki.apache.org/solr/CollectionDistribution

Now, here you can even use a single configuration file with 'Replication' node having fields for both Master and Slave with an 'enable' child-node with possible values 'true|false' set as per requirement for Master & Slave nodes.

Sunspot dealing with a single Solr instance, gets $ProjectRoot/config/sunspot.yml

production:
  solr:
    hostname: standaloneSolr.mydomain.com
    port: 8983
    log_level: WARNING

Sunspot dealing with a master/slave Solr set-up, gets $ProjectRoot/config/sunspot.yml

production:
  solr:
    hostname: slaveSolr.mydomain.com
    port: 8983
    master_hostname: masterSolr.mydomain.com
    master_port: 8983
    log_level: WARNING
  master_solr:
    hostname: masterSolr.mydomain.com
    port: 8983
    log_level: WARNING

If you have more than one slaves, they need to be handled by a load-balancer and the DNS Entry for that Load Balancer comes here in slave's hostname field.

Also, the fields 'master_hostname' and 'master_port' below 'solr:' are not mandatory and supposed to be referred from 'master_solr:' block. But, it has been observed in some cases that mentioning them explicitly avoids non-picking of configuration.

By default, Sunspot configures Ruby/Rails application to Write-Only to Master and Read-Only from Slave.

explicit update SASS in Rails Application

Sass (Syntactically Awesome StyleSheets)?
Available @ http://sass-lang.com/
Is a fine way to produce clean and easy CSS code using a meta-scripting-language. It has feature for developer ease like variables, nesting, mixins, and selector inheritance.

If you are somehow using Sass in your Ruby-On-Rails application and using the default Webrick to serve the web content, it automatically updates the "public/stylesheets/*.css" files from your styling meta-script in "public/stylesheets/sass/*.css".

But, if you are using any other web-server (like, thin)..... in that case you'll have to get your CSS files to be explicitly updated using 'sass --update' action.

$ bundle exec sass --update 
--watch public/stylesheets/sass:public/stylesheets


So, include it in your deploy Rake Tasks and get updated stylesheets everytime without fail.

[dev-ruby] "JRubyOnRails Blames ActiveRecord" *ing: culprit Cucumber, inspector JConsole

I was working on JRubyOnRails project, where we used Cucumber in providing a web-app feature that required functional testing of some web portals.

Scenario: 
     In this Rails application, 'Cucumber' tests for different web-portals are invoked every scheduled interval using 'derailed_cuke' and then the test results are processed and saved in a MySQL database.

Gems Used:
     #for Rails3
          'rails', '3.0.3'
     #for MySQL ActiveRecord
          'activerecord-jdbcmysql-adapter', '1.0.2'
          'activerecord-jdbc-adapter', '1.0.2'
          'jdbc-mysql', '5.0.4'
     #for Cucumber
          'cucumber', '0.10.0'
          'cucumber-rails', '0.3.2'

          'gherkin', '2.3.3'

     #with JRuby version 1.5.5

Exception:

• After starting Rails Server and waiting for some non-specific time, an exception started arising mentioning ActiveRecord and an Heap OutOfMemory Exception being raised.
• I checked for my database services, schema & content; all was as expected and even working fine from 'rails console'.

Action:

• My bad old debug-style... the first thing I did was inserted 'print STATUS_MSG' at all suspected locations for ActiveRecord.
• It worked :), and the location sorted out was the place where cucumber-tests' processed results was saved in database.
• So now, 'Cucumber' was under suspicion radar... I google-d over recent incidents including both 'OutOfMemory Exception' and 'Cucumber', and there were few blogposts about similar problem of 'memory leakage' in Cucumber.

     I had a JConsole window already fired-up and profiling the Memory level changes
     in my JRubyOnRails appication (one of the major benefit of using JRuby).

• So, I started tweaking the Cucumber test's schedule interval to a higher and lower value.
  This clearly showed it's impact over the Memory Profiling by JConsole and the relativity of duration in which the Exception was being raised.

Solution:

     Until the Cucumber flaw is corrected, the work-around used was

        [] pushing in some more RAM
        [] using JVM Parameters to increase the Heap Size used by Rails application
         #jruby -J-Xmx2500m -J-Xms2500m -J-Xmn512m -S rails s
             here,
                  -J-Xmx2500m ~ means maximum Java Heap Size increased to 2500m
                  -J-Xms2500m ~ means initial Java Heap Size increased to 2500m
                  -J-Xmn2500m ~ means eden Java Heap Size increased to 500m
                  for server-side applications, Xmx & Xms is preferred to be kept same