Archive for the 'database' Category

MySQL, Solr and “Communications link failure”

So, I was indexing a 10+ million records in MySQL into Solr and kept coming across the following odd MySQL exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
link failure
Last packet sent to the server was 4467745 ms ago

com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2985) at
com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2871) at

In my code, I loop over a JDBC ResultSet and add the records to Solr per the Solr field schema, mapping [...]

How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data | High Scalability

How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data | High Scalability
Nice article on how the Lucene/Hadoop/Solr stack was used to solve a really big problem.  Someday, I hope (when we have actual code),  they can add Mahout to the equation and do even more interesting things with the data.

Good Math, Bad Math : Databases are hammers; MapReduce is a screwdriver.

Good Math, Bad Math : Databases are hammers; MapReduce is a screwdriver.
Well stated response to a criticism on Map Reduce.  Adding my own two cents, I once used Hadoop, a free open source implementation of Map Reduce (M/R) in a proof of concept implementation, to automatically translate (as in machine translation) a large (in my [...]