Technology @Walk

Monday, November 21, 2011

[OpsCode Chef] when chef's changes can be re-edited but un-available to Search

[DevOps:///OpsCode.Chef]

ate a weird meal last evening, chef was angry I think.....

When I created an AWS instance in same way (by swiss-'knife ec2 server create...' toolset) using same old boot-up script to get that insance auto-configured as chef-client; instance got created and was visible in the instance list but not available to my recipes trying to search for it using its applied role and other tags.

The same procedure has worked successfully for all previous time, and with no change it suddenly started failing.

I logged-in to the freshly created instance and exec 'chef-client --once' again, it had a successful run but still the recipe failed to grab the node by its role and tags.
So it was a perfectly performing instance according to its run_list, but not available to other features trying to grab its information over Chef Search mechanism.

What could be the problem?

The answer to it is hidden in knowing how Chef Search function.

Chef Search uses Solr Indexing service, which an enterprise-grade OpenSource search server based upon Apache Lucene.

It has a system service 'chef-solr' which acts as a wrapper around Solr to be run for Chef.
Though, this service was fine as I was able to search all earlier created nodes..... just the new additions were escaping the act.

Now, if Chef-Solr was working fine then we need to look its source for data. Obviously, it was working fine for already indexed data but didn't happen to index anything newly added.

Chef-Solr gets data passed on from the message queue of RabbitMQ, placed before to ease the load of several concurrent requests for Solr.
Chef-Exapander is a system-level service that fetches data from RabbitMQ's queue and formats them before passing on to chef-solr.

This combination of chef-exapnder and chef-solr acts as final feature of Chef Indexer.

So, I looked for chef-expander service.
It was in failing mode, restarted the service and bam!
We are back up and the new aditions are visible in Chef Search.

Sunday, November 20, 2011

am an MVB..... now, officially

have been walking along-with the technology and blogging about it.....

few days back, I got a mail inviting to become a part of MVB (Most Valued Blogger) program at DZone..... we finished the formalities and now, officially I'm an MVB.

feels good after having a look at some of other members from MVB http://www.dzone.com/page/mvbs

Wednesday, November 16, 2011

Ruby/Rails utilizing Solr in Master/Slave setup

Apache Solr is a high-performance enterprise grade search server being interacted with a REST-like API, documents are provided to as xml, json or binary. It extends Lucene search library.

Ruby/Rails communicates with this awesome search server using Sunspot (sunspot, sunspot_rails gem) library, to do a full-text search. Nice tutorial to use Solr in your Rails project using Sunspot.

Solr can perform as a standalone search server and even as master/slave instances collaborating with each other, with slaves polling master to sync-in their data.

Solr instances can be configured like Slave and as a Master.
Configuration file 'solrconfig.xml' needs to be edited with a new RequestHandler for Replication configured:

{for Slave} :: to either poll at their Master's machine address at regular intervals

{for Master} :: or commit the changes and clone the mentioned configuration files when Slave asked for it

for detailed reference:http://wiki.apache.org/solr/SolrReplication

for optimizaed ways using ssh/rsync based replication:
http://wiki.apache.org/solr/CollectionDistribution

Now, here you can even use a single configuration file with 'Replication' node having fields for both Master and Slave with an 'enable' child-node with possible values 'true|false' set as per requirement for Master & Slave nodes.

Sunspot dealing with a single Solr instance, gets $ProjectRoot/config/sunspot.yml

production:
solr:
hostname: standaloneSolr.mydomain.com
port: 8983
log_level: WARNING

Sunspot dealing with a master/slave Solr set-up, gets $ProjectRoot/config/sunspot.yml

production:
solr:
hostname: slaveSolr.mydomain.com
port: 8983
master_hostname: masterSolr.mydomain.com
master_port: 8983
log_level: WARNING
master_solr:
hostname: masterSolr.mydomain.com
port: 8983
log_level: WARNING

If you have more than one slaves, they need to be handled by a load-balancer and the DNS Entry for that Load Balancer comes here in slave's hostname field.

Also, the fields 'master_hostname' and 'master_port' below 'solr:' are not mandatory and supposed to be referred from 'master_solr:' block. But, it has been observed in some cases that mentioning them explicitly avoids non-picking of configuration.

By default, Sunspot configures Ruby/Rails application to Write-Only to Master and Read-Only from Slave.

Monday, September 26, 2011

explicit update SASS in Rails Application

Sass (Syntactically Awesome StyleSheets)?
Available @ http://sass-lang.com/
Is a fine way to produce clean and easy CSS code using a meta-scripting-language. It has feature for developer ease like variables, nesting, mixins, and selector inheritance.

If you are somehow using Sass in your Ruby-On-Rails application and using the default Webrick to serve the web content, it automatically updates the "public/stylesheets/*.css" files from your styling meta-script in "public/stylesheets/sass/*.css".

But, if you are using any other web-server (like, thin)..... in that case you'll have to get your CSS files to be explicitly updated using 'sass --update' action.

$ bundle exec sass --update
--watch public/stylesheets/sass:public/stylesheets

So, include it in your deploy Rake Tasks and get updated stylesheets everytime without fail.

Monday, September 5, 2011

mysql-server retained old credentials, got work-around but no solution

While trying re-configuration of an existing mysql-server over CentOS, tried 'yum remove' the mysql-server and then again 'yum install' it.

When I tried setting up a new password for 'root' using 'mysqladmin', it raised an error. Some random troubleshooting showed it still had earlier-installation's root credential working for it.

Trying some more stuff, I manually set 'old_password=0' in '/etc/my.cnf' and then tried re-installing. It still had the earlier password working for it.

For time being, got a work-around fixing the problem...

$ yum erase mysql mysql-server
$ rm -rf /var/lib/mysql
$ yum install mysql mysql-server
$ service mysqld restart

The location of user's information i.e. the user table data resides in '/var/lib/mysql/mysql/user.MDY'; but just removing that only file wouldn't work because it don't get recreated on re-installation if entire directory structure is present.

A service restart is required to create a vanilla copy of '/var/lib/mysql'.

Still... the issue exists of why to do it manually.

The incidence is under discussion at the link below. If you have a non-hackish solution for the problem or could spot the actual mistake, please reply there.

http://stackoverflow.com/questions/6606869/mysql-retains-password-of-earlier-installation-even-after-proper-yum-remove

Sunday, August 21, 2011

cucumber's selenium test failed and firefox ran blank..... OOOPS, version in-compatibility

I faced my cuke test failing a while back, and recently saw some of my colleagues re-checking their environment settings, cucumber installations and tweaking their cuke-features in different ways..... and I thought I missed spreading a major sports-page news for all headline cuke readers which might have been wasting a lot of time of a lot of people just because the word is not well spread.

Newer Versions of Firefox

are not friendly with

in-trend Selenium Drivers

So, if you have direct selenium tests failing, or write cucumber/capybara behavioral tests but seek the cause for failure of correct behavior... lookout for a version downgrade for your firefox from whatever 4.x, 5.x or higher Beta versions you have to any of 3.x versions.
And in case of being the victim of same in-compatibility, you would see your tests GREEN again.

Tuesday, August 9, 2011

Tech-Xpress-Guide so far... from learnings of year 2010