Hadoop was seen as a virtual playground full of possibilities for engineers to reinvent aRead More
7 reasons why we need SQL now more than ever
SQL has been around for a long time, and we occasionally see attempts to displace it as the language of choice for analyzing data. You can find a whole page of query languages listed here, for example.
So why is SQL still needed?
1. Wide adoption
That ubiquity tells you there is a large talent pool for any projects involving SQL, and that a lot of organizations are currently using it. SQL has been around for over 40 years, so it is now a battle-hardened solution for data processing. Also, having that vast community of developers along with sites like StackOverflow means there is a lot of shared knowledge that you can access should you encounter issues.
2. Sustained level of interest, and new applicability
Looking at Google trends for SQL over the last 5 years, you can see a slight drop-off in interest over time, but nothing significant. Looking at the trend over the last 12 months you can see that related topics are other areas of interest, such as Apache Hive and Spark (which both have SQL interfaces), as well as AWS to tie into the increasing interest in cloud. So it looks like SQL isn’t going away any time soon.
3. Ease of use
In terms of learning SQL, the core concepts are easily understood, and will stand any user in good stead as they move between different databases. They will need to learn the differences between different products of course, but they can rely on standard SQL to give them an underlying mechanism for creating and populating objects and accessing the data in them, thanks to ANSI SQL standards.
4. Free and open source options
Developers can also take advantage of a number of free, open source SQL engines, such as MySQL and PostgreSQL, when learning SQL. Indeed, many companies use these SQL engines in production nowadays.
5. Wide range of supporting tools
In addition, most tools for manipulating data use SQL to access that data from a variety of SQL engines. So if you have data stored in Hadoop, and want to access it with Tableau, or Power BI, you will end up using SQL to access that data. Bear in mind that a lot of end users have a favorite tool, such as Tableau, and will be insistent on using it should the underlying data be moved to a different technology, so that technology will have to support access via SQL.
6. Let the computer do optimization
SQL also benefits from being a declarative language, meaning you specify what results you want and leave a query optimizer to work out the best way to get those results. This is more straightforward than having to code a solution in an imperative language, and means that you get benefits from optimizer improvements over time without having to rewrite code. If you have performance-critical code which you want to tune, you can often give the optimizer hints where you know more about the data than the optimizer can work out from statistics, but this is an edge case rather than the well-trodden path.
7. Increasing data volumes
Everyone agrees that data volumes are increasing rapidly. For example, this article reports an IDC study indicating IoT will drive a tenfold increase in data volumes to 163 zettabytes by 2025.
In such an environment with your company’s data growing rapidly, you want to be using the same proven technology as larger companies that have trodden that path before you.
As this article notes from Dresner’s 2017 Big Data Analytics Market Study, the vast majority of respondents were using SQL solutions for big data access in their December 2017 report, with 73% considering SparkSQL critical to their analytics strategies.
This article quotes Splice Machine’s CEO as saying, “For it to be actually acceptable by the Global 2000 [companies], it has to be in a higher abstraction or language, and that we believe is SQL, the standard data language.”
This article talks about a former NoSQL user who was forced to migrate back to SQL to get all the benefits of that approach.
In conclusion, with seven strong reasons we need SQL, it is far from being relegated to a dying language.