Startups: Scaling your database Thinking big and thinking global from day one



When I started my first business back in 2001, the normal approach to building and growing your company was to first gain traction and market share in your region, then once you had succeeded in building a successful regionally focused business, start looking to replicate and improve that model and expand into new regions or adjacent markets. Back then slow and steady won the race.  You had the luxury of time and therefore the ability to focus on a specific region or market, perfecting your product and/or services and using that as a basis for growth.

As we approach 2020, the reality of ubiquitous connectivity provides us with a new way of thinking when it comes to starting your own company, specifically technology based companies. Not only are new Startups and enterprises increasingly looking to a global market, they are actively encouraged by investors and emboldened to think big from the start. If you are a venture backed Startup, you no longer have the luxury of time.

Go Global from Day 1

You must now think big and globally from day one. It doesn’t take a rocket scientist to therefore deduce that you in turn must be prepared to be big and global from day one. So, growth is no longer steady, growth needs to be rapid. When it comes to technology, rapid growth translates to rapid scale.

One thing that I have learned over the years with customers is that scaling rapidly while maintaining continued availability, is no simple technology feat. There are many areas of complexity and risks that must be effectively managed to ensure that rapid scale with continuous uptime is achieved. In reality, as much as this is about technology, this is just as much about your team and effective architecture and infrastructure planning.

If you are a technology focused Startup, getting your data model and infrastructure right from the start is just as important as creating an engaging and effective app that connects with your target market. You must ask yourself the question, what happens if we hit the mark with our new app and we go viral overnight? Furthermore, have you selected the right data infrastructure that will help you add capacity as rapid as you add new users?

Effective planning

The bottom line is that you must plan effectively from the start. This presentation by Amazon Web Services Solution Architect, Joel Williams at AWS re:Invent 2016, provides a beginner’s guide to scaling to 11 million+ users on Amazon’s AWS.  Joel provides great guidance on using AWS and the associated cloud infrastructure, technologies and orchestration software to scale rapidly.

Joel raises a specific question about selecting the either SQL or NoSQL technology. He believes that the right approach is to go with SQL at the beginning. His evidence and reasoning relates mainly to SQL being established technology and that most developers are not going to break this technology in the first 10 million users.

In myexperience,I have seen plenty of examples of both scenarios. Those customers that have planned on using the NoSQL database Apache Cassandra from the start, and those that have selected an SQL data store to begin their journey. Working with customers that are trying to move quickly from their existing Postgres (or MySQL or Oracle or SQL server, etc) data store because they have hit their scaling limit is not fun for anybody involved.

Architect to be massive

If you do have an incredible vision and plan to be large, why not architect that way from the beginning? I would also argue that NoSQL technology is now well proven, and open source communities are strong and largely behind some of these amazing technologies.

There is evidence to suggest that open source Apache Cassandra is one of the most scalable databases that provides both exceptional performance and the ultimate high availability architecture.

Cost considerations

Finally, cost is something that all good solution architects should be considering. Scaling with proprietary database technologies cannot only become cost prohibitive, it can also cause significant pain with continued technology lock-in that creates continued escalation in costs. provides a good solid independent reference detailing all database technologies that can be considered, both SQL and NoSQL. The website also provides detailed information on key data-related technologies such as search and analytics engines.