PALite keeps each table as a separate sqlite3 database (single file on disk). Therefore, each table requires a separate sqlite3 connection object. In addition, to speed up access to tables and reduce contentions between threads, each thread has its own sqlite3 connection.

Example:

• A test created three tables.
• Two working threads (can access any table).

Resulting number of db connections: 3_tables * 2_threads = 6 connections.

For tests with many tables – thousands or even millions – the required number of connections quickly reaches the system limits.

Solution:
Spread pages data across limited number of database files (shards) based on table_id. Each table_id is mapped to a shard via table_id % NUM_SHARDS, so all records for a given table reside in one shard.

This allows for higher concurrency when multiple threads are accessing different page shards simultaneously. At the same time, the overall number of database connections remains low. It is essential for tests that open many tables.