Horizontal Scaling -- To scale or Not?

NoSQL is a fast, portable, relational database management system without arbitrary limits, (other than memory and processor speed) that runs under, and interacts with, the UNIX¹ Operating System. It uses the "Operator-Stream Paradigm" described in "Unix Review", March, 1991, page 24, entitled "A 4GL Language". There are a number of "operators" that each perform a unique function on the data. The "stream" is supplied by the UNIX Input/Output redirection mechanism. Therefore each operator processes some data and then passes it along to the next operator via the UNIX pipe function. This is very efficient as UNIX pipes are implemented in memory. NoSQL is compliant with the "Relational Model". -- http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page


Most web architects and developers who manage high volume website experience deadlock when the site can  no longer scale up the demand of connections it can accommodate.  For systems administrators the task are even more daunting.  There are a few items, I will list here that can help you better understand horizontal scaling strategies at least on the systems administrators point of view.

I.  Web Servers

Understanding the core components that run your site is crucial in understanding how you can scale-up.  Apache web servers run most of the most busy website you will find around, but the increasing number of apache run web-site are slowly but surely being challenged by an even more light-weight and faster web server!  Nginx -- to the rescue.  Nginx was taunted as the next generation of web servers that will run the most busiest sites on the web, right now nginx powers a handful of sites found on the net.  With a bit of tinkering, you can marry nginx technology with apache a sort of a high-breed architecture that marries the best of both world.  A good example of this implementation can be seen with most e-commerce websites where it has successfully married both technology to scale-up connections to the web-server wisely.

II. Databases

Regardless of what database system you use it is important that the technology can keep up with your sites requirements.  Most websites found on the internet today uses MySQL to power there sites, but in certain instances mysql faces a greater challenge when demands grow to levels that mysql was originally intended to.  The option was to move to real enterprise class databases systems like Oracle.   Planning is crucial in understanding how you can architect your system to perform beyond what is expected in cases where the database can no longer accommodate such hight demands the problem may lie on the a key design constraint that wasn't there when you designed your database.  Moving the read and write function to separate containers (dbs) will enhance speed.  I know of some implementations where database administrators implemented sphinx technology to optimize indexing of data entries there making key searches available immediately when needed.  This is transparent to the systems since the application will be able to talk this out with the sphinx engine which greatly increases the seek time by several hundredths of  second.  To learn of these technology marriages you will need to look for items that preclude more that just your technical expertise a good read would be to start on the sphinx website itself and see if the technology can help you out with your requirement ..... to be continued...