什么是宫刑

Background

As a national music app with over 220 million monthly active users, QQ Music has created a three-dimensional (“listen, watch, and play”) pan-music entertainment ecosystem, providing a diversified music life experience for over 800 million users. Behind such quality service, trillions of GB of new music content and behavioral data are generated every day and petabyte-level data computing is required.

Challenges & Goals

In the evening of July 14, 2022, the well-known singer Jay Chou's latest album "Greatest Works of Art" was officially released on QQ Music. In terms of traffic data, the album's eponymous song MV garnered over 1.2 million views within only 15 minutes after its release. Within just one hour and 47 minutes, the number of views hit 6 million, the number of shares hit 200 thousand, the number of comments hit 120 thousand, and the MV video obtained the 10-million-level certification on the MV peak list, all of which broke the MV single-day history record on QQ Music.

As a music application, QQ Music boasts a huge amount of data and plenty of business scenarios. The large amount of data means that the data analysis business is subject to higher requirements and that the underlying database will be challenged with high concurrency and fast response during traffic peaks. Broadly speaking, the release of new digital music albums may bring the following challenges for the database:

1.The need for high concurrency and low latency

The event starts with a large number of users accessing the information about the same artist, song, or album at the same time, which poses the challenges of hot row updates, high concurrency, and low latency for the database.

2.The need for rapid scaling

The high instantaneous concurrency requires the database to expand its capacity quickly to support multiples of performance.

3.The need for massive data storage and data security

As order data and log entries spike and zero data loss is required, the database must support the storage of large amount of data while ensuring data security.

The database involved in this album release event is mainly the album order database. During album pre-sale and sale, the database receives a large number of order data writes and updates at the same time, which requires high performance and consistency. Therefore, the database must support high-performance queries, writes, and updates while ensuring no data loss.

In this context, the whole database architecture of QQ Music needs a more secure and stable service model. Tencent Cloud's enterprise-grade distributed database service TDSQL meets the needs of this event.

Tencent Cloud Solution

QQ Music has created a three-dimensional (“listen, watch, and play”) pan-music entertainment ecosystem, providing a diversified music life experience for over 800 million users. Behind such quality service, trillions of GB of new music content and behavioral data are generated every day and petabyte-level data computing is required. After countless technical architecture upgrades and performance optimization efforts by the technical teams of both QQ Music and Tencent Cloud Database, a highly available, high-performance, and secure computing and analysis platform has been gradually formed.

TDSQL supports three synchronization methods: strong sync, semi-sync, and async. The performance of strong sync is basically close to that of the async method. In the scenario of new album release, TDSQL’s strong sync meets the needs well. In addition, TDSQL supports fast primary/standby switchover and fast shard and replica addition, which can quickly add multiple shards and replicas with business transparency and instantly meet the requirements of the event. Although the situation of multiple replicas and shards residing on only a few devices occurred during stress testing, the problem was solved smoothly and quickly through primary/standby switchover and fast data relocation.

With TDSQL’s distributed architecture, QQ Music deploys its database clusters in both the “one primary node and one standby node” mode and the “one primary node and multiple standby nodes” mode. For the core business, QQ Music adopts the global database architecture of Tencent Cloud's native database service TDSQL-C to realize node deployment with cross-region disaster recovery and provide balanced performance, cost-effectiveness, and data security to meet the needs of different business scenarios.

Up until now, QQ Music has been connected to Tencent Cloud databases for three years, and the total volume of data has exceeded 100 TB. With respect to business scenarios, QQ Music features a large number of offline analysis scenarios and some difficult problems related to database performance or component control issues in daily Ops, which the Tencent Cloud database team can respond to and solve in a timely manner.

As regards database management, QQ Music is faced with the following problems:

1.As business data such as logs, records, and orders accumulates, the native MySQL centralized architecture needs to be continuously split into shards and tables, which is a big workload for database administrators (DBAs) and needs to be adapted to the business logic. TDSQL supports automatic horizontal splitting, which can solve this problem well.

2.With the growth of business, the development of more Data Definition Language (DDL) statements is required. The Instant DDL kernel capability provided by Tencent Cloud's native database service TDSQL-C can complete data changes, which originally took dozens of minutes or even hours, within just one second, significantly improving the Ops efficiency of DBAs.

3.To free DBAs from having to deal with various slow log and low-performance queries frequently on a daily basis, TDSQL's DBbrain platform can quickly and accurately identify database performance bottlenecks through comprehensive analysis and diagnosis of various indicators of database instances.

Currently built on a variety of database architectures, QQ Music supports diverse features including real-time activity, latest comments, and stick-to-the-top, provides a millisecond-level cross-city read delay, and supports auto scaling to cope with high-concurrency reads and writes from tens of millions of users. These capabilities make database management easier and more focused on business.

Benefits

At present, QQ Music is in the stage of migration from self-developed IDCs to the cloud. In the future, the main goal is to, with the help of the mature infrastructure and services of Tencent Cloud, eliminate the basic, laborious Ops work and put more effort into business. Moreover, QQ Music will continue to build automated Ops systems and tools to gradually achieve intelligent database Ops.

In this regard, Tencent Cloud's native database service TDSQL-C can meet all kinds of needs of QQ Music in all aspects. Based on the architecture of compute-storage separation, TDSQL-C supports various capabilities including HTAP, extreme auto scaling, and massive distributed storage, and provides standard service solutions such as the intelligent Ops platform and the serverless edition.

Tencent Cloud’s platform for intelligent, centralized database management allows data to flow freely between different engines to better support rapid business development. The platform provides abundant APIs to support flexible calling and one-click operation in various application scenarios. The platform supports an intelligent Ops system that offers second-level diagnosis of 90% of common faults and SQL optimization suggestions to make system Ops much easier. Based on multi-source synchronization tools, the platform implements second-level cross-engine data synchronization and shields engine differences for business. Moreover, the platform supports plug-in load balancing management to further improve availability.

Thanks to the full-stack database service of Tencent Cloud, QQ Music now can cope with all demands in analytical processing (AP) and transaction processing (TP) scenarios, supporting the core business of orders and comments from tens of millions of users. From various links including big data infrastructure, full-link data tool chain, and field data value application, QQ Music and Tencent Cloud have achieved win-win results and unleashed the value of multivariate data.