`n

Connection Pooling Setup

Optimize database performance with proper connection pooling. Learn configuration strategies for MySQL, PostgreSQL, MongoDB with Node.js, Python, and Java examples.

Connection Pool Architecture

App
→
Pool
→
DB

Published: September 25, 2024 | Reading time: 18 minutes

🔗 Connection Pooling Benefits

Performance: Reuse connections instead of creating new ones

Resource Management: Control maximum concurrent connections

Scalability: Handle high-traffic applications efficiently

Connection Pool Metrics

50ms
Connection Time
2ms
Pool Get Time
25x
Performance Gain
95%
Resource Savings

Connection Pool Visualization

🏊 Connection Pool State

Active: 8
Idle: 12
Error: 0
Max: 20
Min: 5
Total: 20

Node.js Connection Pooling

🟢 Node.js MySQL Connection Pool

Configure MySQL connection pooling with mysql2 package for optimal performance.

🟢
Node.js MySQL Pool Configuration
const mysql = require('mysql2/promise'); // Connection pool configuration const poolConfig = { host: 'localhost', user: 'app_user', password: 'secure_password', database: 'myapp', // Pool settings connectionLimit: 20, // Maximum connections queueLimit: 0, // Unlimited queue acquireTimeout: 60000, // 60 seconds timeout: 60000, // 60 seconds reconnect: true, // Auto-reconnect // Connection settings charset: 'utf8mb4', timezone: 'Z', // Performance settings multipleStatements: false, dateStrings: false, debug: false }; // Create connection pool const pool = mysql.createPool(poolConfig); // Usage example async function getUserById(id) { const connection = await pool.getConnection(); try { const [rows] = await connection.execute( 'SELECT * FROM users WHERE id = ?', [id] ); return rows[0]; } finally { connection.release(); // Return connection to pool } } // Graceful shutdown process.on('SIGINT', async () => { console.log('Closing connection pool...'); await pool.end(); process.exit(0); });

🟢 Node.js PostgreSQL Connection Pool

Configure PostgreSQL connection pooling with pg-pool for high-performance applications.

🟢
Node.js PostgreSQL Pool Configuration
const { Pool } = require('pg'); // Connection pool configuration const poolConfig = { host: 'localhost', port: 5432, user: 'app_user', password: 'secure_password', database: 'myapp', // Pool settings max: 20, // Maximum connections min: 5, // Minimum connections idleTimeoutMillis: 30000, // Close idle connections after 30s connectionTimeoutMillis: 2000, // Connection timeout // Connection settings ssl: false, application_name: 'myapp', // Performance settings statement_timeout: 0, query_timeout: 0, keepAlive: true, keepAliveInitialDelayMillis: 0 }; // Create connection pool const pool = new Pool(poolConfig); // Usage example async function getUserById(id) { const client = await pool.connect(); try { const result = await client.query( 'SELECT * FROM users WHERE id = $1', [id] ); return result.rows[0]; } finally { client.release(); // Return connection to pool } } // Pool event handlers pool.on('connect', (client) => { console.log('New client connected to pool'); }); pool.on('error', (err) => { console.error('Unexpected error on idle client', err); }); // Graceful shutdown process.on('SIGINT', async () => { console.log('Closing connection pool...'); await pool.end(); process.exit(0); });

Python Connection Pooling

🐍 Python MySQL Connection Pool

Configure MySQL connection pooling with PyMySQL and SQLAlchemy for Python applications.

🐍
Python MySQL Pool Configuration
from sqlalchemy import create_engine from sqlalchemy.pool import QueuePool import pymysql # Connection pool configuration DATABASE_URL = "mysql+pymysql://app_user:secure_password@localhost/myapp" engine = create_engine( DATABASE_URL, # Pool settings pool_size=20, # Maximum connections max_overflow=10, # Additional connections beyond pool_size pool_timeout=30, # Seconds to wait for connection pool_recycle=3600, # Recycle connections after 1 hour pool_pre_ping=True, # Validate connections before use # Connection settings echo=False, # Log SQL statements echo_pool=False, # Log pool events # Performance settings connect_args={ "charset": "utf8mb4", "autocommit": False, "use_unicode": True } ) # Usage example from sqlalchemy.orm import sessionmaker SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) def get_user_by_id(user_id): db = SessionLocal() try: result = db.execute( "SELECT * FROM users WHERE id = %s", (user_id,) ) return result.fetchone() finally: db.close() # Context manager for automatic cleanup from contextlib import contextmanager @contextmanager def get_db(): db = SessionLocal() try: yield db finally: db.close()

Java Connection Pooling

☕ Java HikariCP Configuration

Configure HikariCP connection pool for Java applications with optimal performance settings.

☕
Java HikariCP Pool Configuration
// HikariCP Configuration HikariConfig config = new HikariConfig(); // Database connection settings config.setJdbcUrl("jdbc:mysql://localhost:3306/myapp"); config.setUsername("app_user"); config.setPassword("secure_password"); // Pool settings config.setMaximumPoolSize(20); // Maximum connections config.setMinimumIdle(5); // Minimum idle connections config.setConnectionTimeout(30000); // Connection timeout (30s) config.setIdleTimeout(600000); // Idle timeout (10 minutes) config.setMaxLifetime(1800000); // Connection max lifetime (30 minutes) // Connection settings config.setDriverClassName("com.mysql.cj.jdbc.Driver"); config.addDataSourceProperty("cachePrepStmts", "true"); config.addDataSourceProperty("prepStmtCacheSize", "250"); config.addDataSourceProperty("prepStmtCacheSqlLimit", "2048"); config.addDataSourceProperty("useServerPrepStmts", "true"); config.addDataSourceProperty("useLocalSessionState", "true"); config.addDataSourceProperty("rewriteBatchedStatements", "true"); config.addDataSourceProperty("cacheResultSetMetadata", "true"); config.addDataSourceProperty("cacheServerConfiguration", "true"); config.addDataSourceProperty("elideSetAutoCommits", "true"); config.addDataSourceProperty("maintainTimeStats", "false"); // Create data source HikariDataSource dataSource = new HikariDataSource(config); // Usage example public class UserRepository { private final HikariDataSource dataSource; public UserRepository(HikariDataSource dataSource) { this.dataSource = dataSource; } public User getUserById(int id) throws SQLException { String sql = "SELECT * FROM users WHERE id = ?"; try (Connection connection = dataSource.getConnection(); PreparedStatement statement = connection.prepareStatement(sql)) { statement.setInt(1, id); ResultSet resultSet = statement.executeQuery(); if (resultSet.next()) { return new User( resultSet.getInt("id"), resultSet.getString("name"), resultSet.getString("email") ); } } return null; } }

Connection Pool Configuration

📊 Pool Size Configuration

Configure optimal pool size based on your application needs.
// Pool size calculation // Formula: (CPU cores * 2) + effective_spindle_count // Small application (2 CPU cores) max_connections = 10 min_connections = 2 // Medium application (4 CPU cores) max_connections = 20 min_connections = 5 // Large application (8 CPU cores) max_connections = 40 min_connections = 10

⏱️ Timeout Configuration

Set appropriate timeouts for connection acquisition and idle connections.
// Timeout settings connection_timeout = 30s // Wait for connection idle_timeout = 10m // Close idle connections max_lifetime = 30m // Connection max age query_timeout = 60s // Query execution timeout // Best practices // - Keep connection_timeout short (10-30s) // - Set idle_timeout to 5-10 minutes // - Use max_lifetime to prevent stale connections

🔄 Connection Lifecycle

Manage connection lifecycle for optimal resource usage.
// Connection lifecycle events on_connect: validate_connection() on_checkout: ping_database() on_checkin: reset_connection() on_close: cleanup_resources() // Health checks pre_ping = true // Validate before use test_on_borrow = true // Test on checkout test_on_return = false // Don't test on return test_while_idle = true // Test idle connections

Connection Pool Monitoring

📊 Pool Monitoring Dashboard

Monitor connection pool health and performance metrics.

8
Active Connections
12
Idle Connections
20
Total Pool Size
2ms
Avg Wait Time
📊
Pool Monitoring Script
// Node.js pool monitoring const pool = mysql.createPool(config); // Monitor pool metrics setInterval(() => { const stats = { totalConnections: pool.pool._allConnections.length, freeConnections: pool.pool._freeConnections.length, acquiringConnections: pool.pool._acquiringConnections.length, queuedRequests: pool.pool._connectionQueue.length }; console.log('Pool Stats:', stats); // Alert if pool is exhausted if (stats.freeConnections === 0 && stats.queuedRequests > 0) { console.warn('Connection pool exhausted!'); } }, 5000); // Python pool monitoring from sqlalchemy import event from sqlalchemy.pool import Pool @event.listens_for(Pool, "connect") def receive_connect(dbapi_connection, connection_record): print("New connection created") @event.listens_for(Pool, "checkout") def receive_checkout(dbapi_connection, connection_record, connection_proxy): print("Connection checked out") @event.listens_for(Pool, "checkin") def receive_checkin(dbapi_connection, connection_record): print("Connection checked in") # Monitor pool size def monitor_pool(): pool = engine.pool print(f"Pool size: {pool.size()}") print(f"Checked out: {pool.checkedout()}") print(f"Overflow: {pool.overflow()}") print(f"Checked in: {pool.checkedin()}")

Common Connection Pool Issues

⚠️ Connection Pool Troubleshooting

1

Connection Pool Exhaustion

All connections are in use and requests are queued

Solution: Increase pool size or optimize query performance

2

Connection Leaks

Connections not being returned to the pool

Solution: Use try-finally blocks or connection managers

3

Stale Connections

Database server closes idle connections

Solution: Enable connection validation and shorter timeouts

4

Connection Timeouts

Requests timeout waiting for connections

Solution: Increase connection timeout or pool size

Connection Pool Best Practices

✅ Connection Pool Best Practices

1

Proper Connection Management

Always release connections back to the pool

Use: try-finally blocks or connection managers

2

Optimal Pool Sizing

Size pool based on application load and database capacity

Formula: (CPU cores × 2) + effective_spindle_count

3

Connection Validation

Enable connection validation to detect stale connections

Settings: pre_ping, test_on_borrow, test_while_idle

4

Monitoring and Alerting

Monitor pool metrics and set up alerts for issues

Metrics: Active connections, wait time, pool exhaustion

Performance Comparison

Connection Pool vs Direct Connections

Metric Direct Connections Connection Pool Performance Gain
Connection Time 50ms 2ms 25x faster
Resource Usage High Low 95% reduction
Scalability Limited High Unlimited
Memory Usage High Medium 50% reduction

Summary

Connection pooling is essential for database performance:

  • Configure appropriate pool sizes based on application load
  • Set proper timeouts for connection acquisition and idle connections
  • Enable connection validation to detect stale connections
  • Monitor pool metrics and set up alerting
  • Always release connections back to the pool

Need Connection Pool Help?

Our database performance experts can help you configure optimal connection pooling for your application.

Get Pool Help