Big Data Challenges for SaaS applications
The amount of data produced within SaaS applications emerges all the analytics, relationships and BI to become highly available on the customer user interface, this brings up the option of using “Big Data” mechanisms to process really large amount of data. With the usage of these Big Data processing all the possible relationships, hidden patterns and additional information will become available immediately.
Processing these data can be done using Hadoop, NoSQL databases and MapReduce programming model. Reading in the right way the unstructured data can give us better understanding, better insights and better decision-making.
Today’s situation
Yet, current traditional databases that are up and running in the most cases are with millions of rows with complex relationships with other tables, with store procedures, joins, aggregate functions which make the migration process seem a nightmare.
This process needs to be with minimum downtime, as the business can’t handle such service interruptions, which might be translated in a lot of money loosing
We have three patterns that emerge in this situation: the current traditional database, migrate to NoSQL or go for hybrid solution combining both of these technologies.
Known problems with traditional databases such as the lack of scale, the constant change of market requirements, constantly adding servers when the number of users increases and RDBMS were projected for distributed environment.
Key points for the transition
As a first step we should analyze what’s best for our business. There are different noSQL technologies that are suited in different cases. We can use Neo4j graph database, MongoDB or Couchbase document-oriented databases. There are several choices available.
To go for a document data model, we should understand that information in the database is inserted without a predefined schema, and also we have to learn a new way of thinking using those databases, as the learning curve its not an easy task.
The main difficulty business face is modeling tabular data into flexible data with no fixed structure that does not conform a strict defined schema.
Issues include privacy concerns, interpretation, dealing with unstructured data.
As noSQL databases are in their early stages, approaches like combining both technologies may be suited for some business cases, but we should keep it present that processing massive amount of data would be our future and we need to go thru this as soon as we can.