[OpsCode Chef] when chef's changes can be re-edited but un-available to Search

[DevOps:///OpsCode.Chef]

ate a weird meal last evening, chef was angry I think.....

When I created an AWS instance in same way (by swiss-'knife ec2 server create...' toolset) using same old boot-up script to get that insance auto-configured as chef-client; instance got created and was visible in the instance list  but not available to my recipes trying to search for it using its applied role and other tags.

The same procedure has worked successfully for all previous time, and with no change it suddenly started failing.

I logged-in to the freshly created instance and exec 'chef-client --once' again, it had a successful run but still the recipe failed to grab the node by its role and tags.
So it was a perfectly performing instance according to its run_list, but not available to other features trying to grab its information over Chef Search mechanism.

What could be the problem?

The answer to it is hidden in knowing how Chef Search function.

Chef Search uses Solr Indexing service, which an enterprise-grade OpenSource search server based upon Apache Lucene.


It has a system service 'chef-solr' which  acts as a wrapper around Solr to be run for Chef.
Though, this service was fine as I was able to search all earlier created nodes..... just the new additions were escaping the act.

Now, if Chef-Solr was working fine then we need to look its source for data. Obviously, it was working fine for already indexed data but didn't happen to index anything newly added.

Chef-Solr gets data passed on from the message queue of RabbitMQ, placed before to ease the load of several concurrent requests for Solr.
Chef-Exapander is a system-level service that fetches data from RabbitMQ's queue and formats them before passing on to chef-solr.

This combination of chef-exapnder and chef-solr acts as final feature of Chef Indexer.

So, I looked for chef-expander service.
It was in failing mode, restarted the service and bam!
We are back up and the new aditions are visible in Chef Search.

am an MVB..... now, officially

have been walking along-with the technology and blogging about it.....

few days back, I got a mail inviting to become a part of MVB (Most Valued Blogger) program at DZone..... we finished the formalities and now, officially I'm an MVB.

feels good after having a look at some of other members from MVB http://www.dzone.com/page/mvbs

Ruby/Rails utilizing Solr in Master/Slave setup

Apache Solr is a high-performance enterprise grade search server being interacted with a REST-like API, documents are provided to as xml, json or binary. It extends Lucene search library.

Ruby/Rails communicates with this awesome search server using Sunspot (sunspot, sunspot_rails gem) library, to do a full-text search. Nice tutorial to use Solr in your Rails project using Sunspot.

Solr can perform as a standalone search server and even as master/slave instances collaborating with each other, with slaves polling master to sync-in their data.

Solr instances can be configured like Slave and as a Master.
Configuration file 'solrconfig.xml' needs to be edited with a new RequestHandler for Replication configured:

 

{for Slave} :: to either poll at their Master's machine address at regular intervals

{for Master} :: or commit the changes and clone the mentioned configuration files when Slave asked for it 

for detailed reference:http://wiki.apache.org/solr/SolrReplication 

for optimizaed ways using ssh/rsync based replication:
http://wiki.apache.org/solr/CollectionDistribution

Now, here you can even use a single configuration file with 'Replication' node having fields for both Master and Slave with an 'enable' child-node with possible values 'true|false' set as per requirement for Master & Slave nodes.

Sunspot dealing with a single Solr instance, gets $ProjectRoot/config/sunspot.yml

production:
  solr:
    hostname: standaloneSolr.mydomain.com
    port: 8983
    log_level: WARNING

Sunspot dealing with a master/slave Solr set-up, gets $ProjectRoot/config/sunspot.yml

production:
  solr:
    hostname: slaveSolr.mydomain.com
    port: 8983
    master_hostname: masterSolr.mydomain.com
    master_port: 8983
    log_level: WARNING
  master_solr:
    hostname: masterSolr.mydomain.com
    port: 8983
    log_level: WARNING

If you have more than one slaves, they need to be handled by a load-balancer and the DNS Entry for that Load Balancer comes here in slave's hostname field.

Also, the fields 'master_hostname' and 'master_port' below 'solr:' are not mandatory and supposed to be referred from 'master_solr:' block. But, it has been observed in some cases that mentioning them explicitly avoids non-picking of configuration.

By default, Sunspot configures Ruby/Rails application to Write-Only to Master and Read-Only from Slave.